Non-Uniform Memory Access (NUMA)

Non-Uniform Memory Access (NUMA) in Windows Server: Configuration, Specifications, Monitoring, and Usage Examples

Non-Uniform Memory Access (NUMA) is a memory architecture used in multi-processor systems, where the memory is divided into regions (or “nodes”) local to each CPU. Each CPU has faster access to its local memory, but accessing memory from another CPU’s node is slower. Windows Server is NUMA-aware, meaning it can optimize resource allocation based on the NUMA layout, which improves performance for applications that rely heavily on memory and CPU resources.

1. Understanding NUMA and its Benefits on Windows Server

NUMA divides a server’s memory into regions (or “nodes”), each associated with a specific CPU or set of CPU cores. This architecture is designed to improve performance on multi-processor systems by reducing the time CPUs spend accessing memory.

Key Benefits of NUMA

Reduced Memory Latency: Each CPU accesses its local memory faster than remote memory, reducing latency for memory-intensive tasks.
Optimized Resource Allocation: NUMA-aware applications can allocate memory and CPU resources from the same node, leading to faster processing.
Better Performance in Virtualized Environments: NUMA is especially beneficial for applications and virtual machines (VMs) that require significant memory and CPU resources, such as databases and virtualization with Hyper-V.

2. Configuring NUMA on Windows Server

NUMA is usually managed at the hardware or BIOS level. However, Windows Server and Hyper-V can automatically detect and use NUMA for optimized performance.

Basic NUMA Configuration Steps

Enable NUMA in BIOS (if applicable):

Many servers have NUMA settings in the BIOS. Consult your server’s documentation to ensure NUMA is enabled, as some BIOS settings allow you to toggle NUMA on or off.

Check NUMA Configuration in Windows Server:

Windows Server automatically detects and configures NUMA nodes if the hardware supports it. You can check the current NUMA configuration using PowerShell:
powershell Get-ComputerInfo | Select-Object CsNumOfNodes
This command displays the number of NUMA nodes detected by Windows Server.

Configure NUMA in Hyper-V (for virtualized environments):

Hyper-V is NUMA-aware and will automatically assign resources based on the NUMA topology. You can adjust NUMA settings for each VM to optimize performance.
- Open Hyper-V Manager, right-click on the VM, and select Settings.
- Under Processor, expand NUMA settings.
- You can set the maximum number of NUMA nodes a VM can use, specify memory per node, and adjust cores per node based on the application requirements.

Consider Application-Specific NUMA Settings:

Some applications (like SQL Server) are NUMA-aware and allow for additional configuration. Check your application’s documentation to ensure it’s optimized for NUMA.

3. NUMA Specifications and Best Practices for Windows Server

NUMA’s effectiveness depends on several hardware and configuration factors:

CPU and Memory Configuration: Each CPU in a NUMA node has faster access to its own memory but slower access to other nodes’ memory. For optimal performance, try to keep a balance between CPU cores and memory for each node.
Hyper-V and VM Placement: For VMs running on Hyper-V, ensure that each VM is allocated resources from a single NUMA node if possible, as cross-node memory access can lead to latency.
NUMA-Aware Applications: Applications like SQL Server, Exchange, and Hyper-V perform better when they can detect and use NUMA topology to allocate resources efficiently.

Best Practices for NUMA on Windows Server:

Minimize Cross-NUMA Node Traffic: Whenever possible, allocate memory and CPU resources from the same NUMA node to reduce latency.
Match Application Workload with NUMA Nodes: For applications that are not NUMA-aware, restrict them to specific nodes to avoid cross-node memory access.
Use Hyper-V NUMA Settings: In virtualized environments, configure VM NUMA settings to match the host server’s NUMA architecture, helping VMs access local resources more efficiently.

4. Monitoring NUMA on Windows Server

Monitoring NUMA is essential to ensure resources are being used effectively. Windows Server provides several tools for monitoring NUMA performance and understanding resource distribution.

PowerShell Commands for Monitoring NUMA

View NUMA Node Information:
To get detailed information about each NUMA node, use the following command:
powershell Get-ComputerInfo | Select-Object CsNumOfNodes, CsProcessors
This command shows the number of nodes and processors, giving you an idea of how resources are structured.

Using Performance Monitor (PerfMon)

Performance Monitor in Windows Server offers specific NUMA counters that help monitor memory and CPU usage across NUMA nodes.

Open Performance Monitor (PerfMon): Run perfmon from the Start menu.
Add NUMA Counters:

In Performance Monitor, click “Add Counters,” and look for NUMA-related counters, especially under the Processor and Memory sections. Relevant counters include:
- Processor Group\% Processor Time per NUMA Node
- Memory\Available MBytes per NUMA Node
- NUMA Node Memory\Local Node Memory Hits
These counters help you monitor CPU and memory usage per NUMA node and check how effectively each node’s local memory is used.

Example of Monitoring Cross-NUMA Node Traffic

Check Local Node Memory Hits and Foreign Node Memory Accesses to understand if applications or VMs are frequently accessing memory from other NUMA nodes, which may impact performance.

5. Working and Usage Examples of NUMA on Windows Server

Example 1: NUMA Optimization for SQL Server

SQL Server is NUMA-aware and can be optimized for NUMA to improve database performance.

Enable NUMA on SQL Server:

SQL Server automatically detects NUMA on Windows Server and allocates resources per NUMA node. You can check NUMA node distribution by opening SQL Server Management Studio (SSMS) and querying server properties.

Monitor SQL Server Performance:

Use Performance Monitor to check CPU and memory usage for each NUMA node. SQL Server should balance queries and data processing across NUMA nodes.

Result: With NUMA, SQL Server can process queries faster and handle more simultaneous connections by minimizing cross-node memory access.

Example 2: Using NUMA in Hyper-V for Virtual Machine Performance

NUMA-aware Hyper-V configuration improves VM performance by allocating resources based on the NUMA layout.

Configure VM NUMA Settings in Hyper-V:

For a VM with high CPU and memory demands (e.g., a database or application server), configure NUMA settings in Hyper-V Manager to ensure the VM accesses resources from a single NUMA node.

Test and Monitor Performance:

After configuration, monitor the VM’s performance in PerfMon. Look for balanced memory usage and CPU load across NUMA nodes.

Expected Outcome: By aligning VM resource allocation with the NUMA topology, the VM experiences reduced latency, improved memory access times, and better overall performance.

Example 3: Troubleshooting NUMA-Related Latency Issues

In some cases, applications may experience latency if they frequently access memory across NUMA nodes.

Identify Cross-Node Access:

Use Performance Monitor to check Foreign Node Memory Accesses. High values indicate frequent access to memory in other NUMA nodes, which can lead to performance degradation.

Reconfigure Application or VM Settings:

Try restricting the application or VM to specific cores and memory on the same NUMA node to minimize cross-node traffic.

Verify Improvements:

After reconfiguration, monitor again to see if Foreign Node Memory Accesses have decreased and check if performance has improved.

NUMA in Windows Server is a powerful architecture for optimizing memory access in multi-processor systems, especially for applications that are memory- and CPU-intensive. By configuring NUMA settings, monitoring resource usage, and aligning workloads with the NUMA layout, you can achieve significant performance gains in virtualized environments, databases, and other high-demand applications.