Memory layout, alignment and caches

Most of the time C++ hides memory from you. When it matters — performance, binary formats, concurrency, embedded work — you need to know exactly what an object looks like in memory and how the hardware fetches it. This lesson covers:

• what decides the size and layout of a struct;
• how to read and write raw bytes without undefined behaviour;
• how to arrange data so the cache works for you.

Try it live: paste any struct into the Playground → Memory tab. It shows every byte: members coloured, padding hatched, cache-line boundaries marked, and the size you'd get by reordering.

1. Size, alignment and padding

Every type has a size (sizeof) and an alignment (alignof): an object of that type must start at an address that's a multiple of its alignment. Typical numbers on x86-64 (and AArch64 Linux):

Two rules build a struct's layout:

1. Each member is placed at the next offset that's a multiple of its alignment. The gap before it is padding.
2. The struct's alignment is its strictest member's, and its size is rounded up to a multiple of that, so array elements stay aligned. This is tail padding.

#include <cstddef>
#include <cstdint>
#include <print>

struct Reading {            // the order someone wrote it in
    bool valid;
    double value;
    std::uint16_t channel;
    std::uint32_t timestamp;
    char unit;
};

int main() {
    std::println("valid     at {}", offsetof(Reading, valid));
    std::println("value     at {}", offsetof(Reading, value));      // 8: 7 bytes of padding before it
    std::println("channel   at {}", offsetof(Reading, channel));
    std::println("timestamp at {}", offsetof(Reading, timestamp));  // 20: 2 more bytes of padding
    std::println("unit      at {}", offsetof(Reading, unit));
    std::println("sizeof = {}, alignof = {}", sizeof(Reading), alignof(Reading));  // 32, 8
}

That's 16 bytes of data in a 32-byte object. A million readings waste 16 MB, and every cache line carries half as many readings as it could.

2. Worked example: ordering members

Sort members by decreasing alignment. Then every member lands on a naturally aligned offset with nothing in between:

#include <cstdint>
#include <print>

struct Reading {
    double value;             // 0..7
    std::uint32_t timestamp;  // 8..11
    std::uint16_t channel;    // 12..13
    bool valid;               // 14
    char unit;                // 15
};
static_assert(sizeof(Reading) == 16);

int main() { std::println("{}", sizeof(Reading)); }

Halving the size without changing behaviour is common in real code, and free. Caveats:

• Reorder only where layout isn't part of an interface (file formats, network protocols, extern "C" structs).
• Keep members that are used together close together. Locality can matter more than a few bytes.

static_assert(sizeof(T) == N) next to a struct whose size matters makes sure nobody undoes your work by accident.

3. Over-alignment: alignas

alignas(N) raises a type's or member's alignment, never lowers it:

#include <cstdint>
#include <memory>
#include <print>

struct alignas(64) CacheLine {   // starts on a 64-byte boundary; sizeof is a multiple of 64
    long value = 0;
};

int main() {
    std::println("{} {}", sizeof(CacheLine), alignof(CacheLine));     // 64 64
    auto p = std::make_unique<CacheLine>();                            // new respects alignas (C++17)
    std::println("{}", reinterpret_cast<std::uintptr_t>(p.get()) % 64); // 0
}

Before C++17, new ignored over-alignment. Now there are aligned operator new(size_t, std::align_val_t) overloads, used automatically for such types.

False sharing preview. Cores exchange memory with each other in whole cache lines, usually 64 bytes. Two counters on one line, updated by two threads, make that line bounce between cores even though no data is shared. Put each hot per-thread counter on its own line with alignas(64).

std::hardware_destructive_interference_size names that distance. However, its value depends on -mtune, so GCC warns (-Winterference-size) when it ends up in a type's layout in a header, because that's an ABI hazard. A fixed constant like inline constexpr std::size_t cache_line = 64; is the usual fix. Level 8 returns to this.

4. Empty types: EBO and [[no_unique_address]]

An empty class still has sizeof 1, because distinct objects need distinct addresses. As a member, that byte plus padding costs real space. As a base, it can take no space at all: the empty base optimisation (EBO). C++20's [[no_unique_address]] gives members the same treatment:

#include <print>

struct NoLock { void lock() {} void unlock() {} };

template <class Lock>
struct Plain { Lock lock; int value; };

template <class Lock>
struct Lean { [[no_unique_address]] Lock lock; int value; };

int main() {
    std::println("{} {}", sizeof(Plain<NoLock>), sizeof(Lean<NoLock>));  // 8 4
}

This is how the standard library keeps stateless allocators, comparators and deleters free: std::unique_ptr<T> with the default deleter is one pointer wide.

5. Bit-fields

#include <cstdint>

struct Flags {
    std::uint32_t ready : 1;
    std::uint32_t mode  : 3;
    std::uint32_t count : 12;
};
static_assert(sizeof(Flags) == 4);

Bit-fields pack small values into bits. Their exact layout is implementation-defined: bit order, straddling storage units, and padding. So:

• Don't use them to match hardware registers or wire formats portably.
• You can't take a bit-field's address.
• Neighbouring bit-fields share memory, so two threads writing different bit-fields of the same word is a data race.

For portable bit layouts, shift and mask explicitly.

6. Object representation without undefined behaviour

Here's the most common bug in "low-level" C++:

float f = 3.5f;
std::uint32_t bits = *reinterpret_cast<std::uint32_t*>(&f);   // undefined behaviour

The strict-aliasing rule says you may only read an object through its own type, a signed/unsigned variant of it, or char/unsigned char/std::byte.

The optimiser relies on this rule. It assumes a uint32_t* and a float* never point to the same object, and may reorder or delete accesses. The code above "works" at -O0 and can break at -O2. UBSan doesn't detect it.

The correct tools:

#include <bit>
#include <cstdint>
#include <cstring>
#include <print>

int main() {
    float f = 3.5f;
    auto a = std::bit_cast<std::uint32_t>(f);   // C++20: copies the bits, constexpr, checked sizes

    std::uint32_t b;
    std::memcpy(&b, &f, sizeof b);              // the classic way; compiles to the same single move

    std::println("{:#x} {:#x}", a, b);          // 0x40600000 0x40600000
}

std::bit_cast refuses mismatched sizes, and types that aren't trivially copyable, at compile time:

#include <bit>
#include <cstdint>

auto bits = std::bit_cast<std::uint64_t>(1.0f);  // 4 bytes → 8 bytes: ill-formed

Two more tools, briefly:

• std::start_lifetime_as<T>(p) (C++23) says "these bytes already hold a T". You'd use it on a buffer filled by read() or a memory-mapped file, so the implicit-object rules give you a real T without copying. GCC 16 has it. The buffer must still be suitably aligned for T.
• std::launder(p) gets a usable pointer to a new object that was created in storage previously occupied by another object, in some corner cases (const or reference members). You'll rarely need it outside of writing containers.

#include <cstddef>
#include <memory>
#include <print>

struct Point { int x, y; };

int main() {
    alignas(Point) std::byte buf[sizeof(Point)]{};   // e.g. filled by I/O
    Point* p = std::start_lifetime_as<Point>(buf);  // alignment is correct, so this is fine
    std::println("{} {}", p->x, p->y);              // 0 0
}

7. Unaligned data and endianness

Bytes from a file or socket can start at any address. Casting bytes.data() + offset to const std::uint32_t* is a misaligned load (undefined behaviour, which UBSan does report) as well as an aliasing violation. It also silently assumes the machine's byte order matches the format's.

Instead, assemble the value from bytes. Compilers recognise this pattern and emit a single load (plus a bswap if needed):

#include <cstddef>
#include <cstdint>
#include <print>
#include <span>
#include <stdexcept>
#include <vector>

// Little-endian: the byte at the lowest offset is the least significant.
std::uint32_t load_u32_le(std::span<const std::byte> b, std::size_t off) {
    if (b.size() < 4 || off > b.size() - 4)   // not `off + 4 > size`: that can wrap
        throw std::out_of_range("load_u32_le");
    std::uint32_t v = 0;
    for (int i = 3; i >= 0; --i)
        v = (v << 8) | std::to_integer<std::uint32_t>(b[off + i]);
    return v;
}

int main() {
    std::vector<std::byte> buf{std::byte{0xFF}, std::byte{0x78}, std::byte{0x56},
                               std::byte{0x34}, std::byte{0x12}};
    std::println("{:#x}", load_u32_le(buf, 1));   // 0x12345678, from an odd address
}

std::endian::native tells you the platform's byte order, and std::byteswap (C++23) reverses one integer's bytes. With byte-by-byte assembly you don't need either: the code is correct on every platform.

Faded example — your turn

Write load_u16_be (big-endian, 2 bytes): which byte is most significant, and what's the bounds check?

std::uint16_t load_u16_be(std::span<const std::byte> b, std::size_t off) {
    if (/* ??? */) throw std::out_of_range("load_u16_be");
    return static_cast<std::uint16_t>(/* ??? */);
}

8. Caches: why layout decides speed

RAM is ~100× slower than the CPU. Caches hide that by fetching whole 64-byte lines and prefetching the next ones when you walk memory sequentially. So the cost of a loop is often decided by bytes moved per useful byte, not by the operation count.

Measured on this machine (GCC 16, -O2; run it yourself in the Playground with sanitizers off — under ASan at -O0 the numbers mean nothing):

#include <cstddef>
#include <vector>

long sum_rows(const std::vector<int>& m, int n) {   // walks memory in order: every line fully used
    long s = 0;
    for (int r = 0; r < n; ++r)
        for (int c = 0; c < n; ++c) s += m[std::size_t(r) * n + c];
    return s;
}

long sum_cols(const std::vector<int>& m, int n) {   // jumps n*4 bytes each step: one int per line fetched
    long s = 0;
    for (int c = 0; c < n; ++c)
        for (int r = 0; r < n; ++r) s += m[std::size_t(r) * n + c];
    return s;
}

Now watch it happen. A toy cache (4 sets × 2 ways of 16-byte lines, so 128 bytes — a real L1 is 32 KiB of 64-byte lines) runs both loops over an 8×8 int matrix. Press play, or step with ←/→:

Both loops are O(n²). Big-O counts operations; caches charge for memory traffic.

9. Array of structs vs struct of arrays

#include <cstdint>
#include <vector>

struct ParticleAoS { float x, y, z, vx, vy, vz, mass; std::uint32_t flags; };  // 32 bytes
std::vector<ParticleAoS> particles;   // AoS: one particle's fields together

struct ParticlesSoA {                 // SoA: one field's values together
    std::vector<float> x, y, z, vx, vy, vz, mass;
    std::vector<std::uint32_t> flags;
};

• AoS is natural when you use all fields of one object at a time.
• SoA wins when a hot loop touches a few fields of many objects. Every fetched byte is useful, access is sequential, and the loop vectorises (SIMD, Level 10).

Many engines and data systems use SoA, or "AoSoA" chunks, for exactly this reason.

Pointer chasing is the extreme case. A linked list or a tree of separately allocated nodes turns each step into a dependent cache miss, which the prefetcher can't predict. Prefer contiguous storage (vector, flat_map, arrays of indices) on hot paths, and keep node-based structures for when you truly need their guarantees: stable addresses, O(1) splicing.

Summary

• Members sit at multiples of their alignment, and the size rounds up to the struct's alignment. Order members largest-alignment-first, and pin important sizes with static_assert.
• alignas raises alignment (cache lines, SIMD); new honours it since C++17.
• Empty members cost space unless they're bases or [[no_unique_address]].
• Never read an object through a pointer of another type. Use std::bit_cast or memcpy, and assemble integers from bytes when reading external data.
• Memory traffic, not operation count, often decides speed: walk memory in order, keep hot fields contiguous (SoA), avoid pointer chasing, and keep per-thread data on separate cache lines.