Huge Pages Explained: Bigger Memory Pages, Fewer TLB Misses
Huge pages map memory in 2 MB or 1 GB chunks instead of 4 KB, cutting TLB misses for memory-heavy workloads. How they work and when to use them.
Huge pages are memory pages much larger than the standard 4 KB page, typically 2 MB or 1 GB on x86-64 systems. By mapping memory in bigger chunks, they let the processor’s address-translation cache cover far more memory, which reduces costly translation misses for workloads that touch large amounts of RAM, such as databases, virtual machines, and in-memory analytics.
They are a classic example of a low-level knob with a real performance payoff, and also with enough sharp edges that the wrong setting can make things worse.
A quick refresher on pages
Modern operating systems give each process its own virtual address space. The hardware translates virtual addresses to physical ones in fixed-size units called pages. The mapping lives in a multi-level data structure called the page table, which the CPU walks to find the physical page for a given virtual address.
Walking the page table on every memory access would be ruinously slow, so CPUs cache recent translations in the translation lookaside buffer, or TLB. A TLB hit is nearly free. A TLB miss forces a page-table walk, which takes several dependent memory reads, though some of those may themselves be served from the CPU caches.
The TLB reach problem
A TLB holds a limited number of entries. The total memory it can cover at once, called TLB reach, is the number of entries multiplied by the page size.
With 4 KB pages, even a TLB with a few thousand entries covers only a handful of megabytes. A database with a buffer pool of tens of gigabytes, accessed in a scattered pattern, will miss the TLB constantly. Each miss adds latency that no amount of CPU speed can hide, which is part of why memory latency rather than bandwidth so often limits these workloads.
Huge pages attack the problem directly. One 2 MB page covers the same memory as 512 standard pages, so a single TLB entry now reaches 512 times as far. A 1 GB page reaches 262,144 times as far. The page-table walk on a miss also gets shorter, because huge pages are mapped at a higher level of the page table, skipping the lowest level entirely.
Page sizes compared
| Standard page | 2 MB huge page | 1 GB huge page | |
|---|---|---|---|
| Page size on x86-64 | 4 KB | 2 MB | 1 GB |
| Memory per TLB entry | 4 KB | 2 MB | 1 GB |
| Page-table levels walked | All | One fewer | Two fewer |
| Allocation flexibility | Highest | Moderate | Low; best reserved at boot |
| Internal waste for small allocations | Minimal | Can be significant | Large |
| Typical use | Everything | Databases, JVMs, VMs | Very large VMs, specialised workloads |
Other architectures differ. ARM64 systems, for example, can use 4 KB, 16 KB, or 64 KB base pages with correspondingly different huge page sizes, so the exact numbers depend on the platform.
Explicit huge pages vs Transparent Huge Pages
Linux offers two distinct mechanisms, and confusing them is a common source of trouble.
Explicit huge pages (hugetlbfs)
The administrator reserves a pool of huge pages, usually at boot or early in the system’s life while physical memory is still unfragmented. Applications then request them explicitly, either by mapping files on the hugetlbfs filesystem or by passing a huge-page flag when allocating memory.
These pages are reserved up front, are never swapped, and are not split back into small pages. Behaviour is predictable, which is why database servers commonly support them through a configuration option. The downside is rigidity: reserved huge pages are unavailable for other uses even if nothing claims them, and the application must be configured to ask for them.
Transparent Huge Pages (THP)
THP tries to deliver the benefit automatically. The kernel attempts to back suitably aligned anonymous memory regions with 2 MB pages when it can, and a background thread called khugepaged later merges runs of small pages into huge ones.
THP typically runs in one of three modes: always, where the kernel uses huge pages wherever possible; madvise, where it only does so for memory regions the application has flagged with madvise(MADV_HUGEPAGE); and never.
THP is convenient, but it brings costs that explicit huge pages avoid:
- Allocation stalls. If no contiguous 2 MB block is free, the kernel may compact memory to create one, and that work can happen in the allocating thread, causing unpredictable latency spikes.
- Memory bloat. An application that touches one byte of a 2 MB region may get the whole 2 MB, inflating its footprint.
- Interaction with fork. Processes that fork and rely on copy-on-write, such as snapshotting in some in-memory stores, may end up copying 2 MB pages when only a few bytes changed.
For these reasons, several latency-sensitive databases and caches have historically advised running with THP set to madvise or never, while explicitly opting into huge pages where they help.
Who benefits most
Huge pages help when a workload has a large working set accessed in a scattered pattern:
- Relational databases with large shared buffer pools.
- Managed runtimes such as the JVM with large heaps.
- Hypervisors backing guest memory, where translation is doubly expensive because guest and host page tables are both involved.
- In-memory analytics, search indexes, and scientific computing over big arrays.
They help little for small processes, workloads with good locality that already hit the TLB, or programs that allocate and free many small, short-lived regions.
On multi-socket servers, huge pages also interact with NUMA placement. A single 1 GB page sits entirely on one memory node, so where it is allocated matters more than it does for scattered small pages.
How to approach tuning
- Measure first. Use hardware performance counters to check whether TLB misses or page walks are actually a significant cost.
- Prefer explicit opt-in. Use the application’s own huge-page support or
madvisemode rather than enabling THP globally. - Reserve early. For explicit huge pages, reserve the pool at boot so fragmentation does not prevent allocation.
- Watch tail latency, not just averages. THP compaction problems show up as occasional slow requests rather than a lower mean.
The takeaway
Huge pages increase the amount of memory each TLB entry covers, cutting translation misses and shortening page-table walks for large, randomly accessed working sets. Explicit huge pages are reserved and predictable; Transparent Huge Pages are automatic but can introduce latency spikes and memory bloat. Measure TLB pressure before tuning, opt in deliberately for the processes that benefit, and judge the result by tail latency as well as throughput.
Keep reading
Chisato · · 6 min read SSD Wear Leveling: How NAND Flash Stays Alive
SSD wear leveling spreads writes across NAND flash so no block wears out early. How the flash translation layer, garbage collection, and TRIM work together.
Chisato · · 4 min read Thermal Interface Materials Explained
Thermal interface material fills microscopic gaps between a chip and its heatsink so heat can actually transfer to the cooler.
Chisato · · 4 min read Clock Speed vs. IPC: What Actually Makes a CPU Fast
Clock speed measures cycles per second; IPC measures work done per cycle. Real CPU performance is the product of both, not either one alone.