Writing functions in assembly

You rarely need to write assembly — compilers are very good, and intrinsics cover most of what's left. But writing a few functions by hand is the fastest way to make the machine model stick: every register choice, every condition code, every byte of stack is your decision. And it's still the right tool for some jobs: context switches, startup code, hand-scheduled kernels, and reading other people's hot loops.

This lesson writes whole functions in GAS Intel syntax inside a C++ file.

Debug it live: run any snippet in the Stepper with Instructions ticked. Step insn executes one instruction; changed registers light up, and flags show as chips.

1. The boilerplate

A top-level asm block is pasted verbatim into the compiler's assembly output. Declare the function in C++ with extern "C" (no name mangling), then define it:

#include <cstdio>

extern "C" long add3(long a, long b, long c);

asm(R"(
    .text                    # make sure we're in the code section
    .intel_syntax noprefix   # Intel operand order: dst, src
    .globl add3              # export the symbol
add3:
    lea rax, [rdi+rsi]
    add rax, rdx
    ret
    .att_syntax prefix       # hand the file back to the compiler's syntax
)");

int main() { std::printf("%ld\n", add3(1, 2, 3)); }

Each line of boilerplate is there for a reason:

• .text — the compiler might have a data section current where your block lands; code in .data isn't executable.
• .intel_syntax noprefix … .att_syntax prefix — GCC emits AT&T syntax by default; switch for your block and switch back.
• .globl — makes the label a symbol other object files can call.
• R"( … )" — a raw string, so backslashes and quotes need no escaping.
• # comments — // is not a comment in x86 GAS.

2. Loops: test, body, advance, jump

Use numeric local labels: 1: can be defined many times, and 1b / 1f mean "the nearest 1: backward / forward". That keeps blocks from clashing with each other.

#include <cstdio>

extern "C" long sum(const long* a, unsigned long n);

asm(R"(
    .text
    .intel_syntax noprefix
    .globl sum
sum:
    xor eax, eax                      # total = 0   (writing eax zeroes rax)
    xor ecx, ecx                      # i = 0
1:
    cmp rcx, rsi                      # i < n ?
    jae 2f                            #   unsigned: n is a size_t
    add rax, QWORD PTR [rdi+rcx*8]    # total += a[i]
    inc rcx
    jmp 1b
2:
    ret
    .att_syntax prefix
)");

int main() {
    long a[] = {1, 2, 3, 4, 5};
    std::printf("%ld %ld\n", sum(a, 5), sum(a, 0));
}

Three habits that prevent most bugs:

1. Test before the first iteration unless the body is always valid at least once. A do-while strlen reads past the end of "".
2. Pick condition codes by signedness. jb/jae/ja/jbe (and cmova…) are unsigned; jl/jge/jg/jle are signed. Sizes and pointers are unsigned; long elements are signed.
3. Match operand sizes to the data. BYTE PTR for char, DWORD PTR for int, QWORD PTR for long and pointers; scale indices by the element size ([rdi+rcx*8]).

Pointer walking

Instead of an index you can advance the pointer itself — one register fewer:

#include <cstdio>

extern "C" unsigned long asm_strlen(const char* s);

asm(R"(
    .text
    .intel_syntax noprefix
    .globl asm_strlen
asm_strlen:
    mov rax, rdi                 # p = s
1:
    cmp BYTE PTR [rax], 0        # *p == 0 ?
    je 2f
    inc rax                      # ++p
    jmp 1b
2:
    sub rax, rdi                 # p - s
    ret
    .att_syntax prefix
)");

int main() { std::printf("%lu %lu\n", asm_strlen("hello"), asm_strlen("")); }

Bytes inside registers

memchr(p, c, n) compares bytes with (unsigned char)c. You don't need to mask anything: the low byte of rsi has its own name, sil.

1:  cmp  BYTE PTR [rdi], sil     ; compare one byte with (unsigned char)c
    je   found
    inc  rdi
    dec  rdx                     ; n counts down …
    jne  1b                      ; … and the loop stops at zero

3. Branch-free selection with cmov

cmovCC dst, src copies only if the condition holds. A running maximum needs no branch at all — but the condition must be the signed one for long:

#include <cstdio>

extern "C" long asm_max(const long* a, unsigned long n);

asm(R"(
    .text
    .intel_syntax noprefix
    .globl asm_max
asm_max:
    mov rax, QWORD PTR [rdi]     # best = a[0]   (n >= 1)
    mov ecx, 1
1:
    cmp rcx, rsi
    jae 2f
    mov rdx, QWORD PTR [rdi+rcx*8]
    cmp rdx, rax
    cmovg rax, rdx               # signed: -1 < 1
    inc rcx
    jmp 1b
2:
    ret
    .att_syntax prefix
)");

int main() {
    long a[] = {-7, -3, -9};
    std::printf("%ld\n", asm_max(a, 3));
}

With cmova instead, -3 would be 0xFFFF…FFFD — bigger than any positive number.

4. Calling back into C++

Once your function calls something, the ABI starts to bite:

• Anything you need after the call must live in a callee-saved register (rbx, rbp, r12–r15) — and you must save those first.
• rsp must be a multiple of 16 at the call. On entry it is 8 off; every push moves it by 8.

#include <cstdio>

extern "C" long map_sum(const long* a, unsigned long n, long (*f)(long));

asm(R"(
    .text
    .intel_syntax noprefix
    .globl map_sum
map_sum:
    push rbx                     # entry: rsp % 16 == 8
    push r12
    push r13
    push r14                     # four pushes: rsp % 16 == 8 again
    sub rsp, 8                   # now aligned for the calls
    mov rbx, rdi                 # cursor
    lea r12, [rdi+rsi*8]         # end
    mov r13, rdx                 # f
    xor r14d, r14d               # sum
1:
    cmp rbx, r12
    jae 2f
    mov rdi, QWORD PTR [rbx]     # argument: *cursor
    call r13                     # f(*cursor) → rax
    add r14, rax
    add rbx, 8
    jmp 1b
2:
    mov rax, r14
    add rsp, 8                   # undo in reverse order
    pop r14
    pop r13
    pop r12
    pop rbx
    ret
    .att_syntax prefix
)");

long square(long x) { return x * x; }

int main() {
    long a[] = {1, 2, 3, 4};
    std::printf("%ld\n", map_sum(a, 4, square));
}

Proving it: crucible/abi.hpp

Tests can't see whether you restored rbx — unless they plant a value there first. crucible::abi::call does exactly that, and crucible::abi::probe is a callback that records rsp so a test can check the alignment:

#include <crucible/abi.hpp>
#include <cstdio>

extern "C" long twice(long (*f)(long), long x);

asm(R"(
    .text
    .intel_syntax noprefix
    .globl twice
twice:
    push rbx                     # one push: rsp is aligned
    mov rbx, rdi
    mov rdi, rsi
    call rbx
    mov rdi, rax
    call rbx
    pop rbx
    ret
    .att_syntax prefix
)");

int main() {
    crucible::abi::reset_probe();
    auto r = crucible::abi::call(twice, crucible::abi::probe, 40L);
    std::printf("ret=%ld clobbered=%zu aligned=%d\n", r.ret, r.clobbered.size(),
                (int)crucible::abi::probe_aligned());
}

5. No safety net

AddressSanitizer and UBSan instrument compiled C++. Your assembly is invisible to them: reading one byte past a buffer won't be reported, it will just produce a wrong answer (or, near a page boundary, a crash). So:

• write the loop invariant as a comment before writing the loop;
• test the boundaries yourself: n == 0, a match exactly at n - 1 and at n, the empty string, negative values;
• step through suspicious code instruction by instruction in the Stepper.

6. When not to write assembly

Compilers already recognise many loops. Clang turns a hand-written strlen loop into a call to libc's strlen, which scans 32 bytes per step with AVX2; GCC turns a zeroing loop into memset. Write assembly to learn, to control something the compiler can't express, or after measuring — and keep a C++ reference implementation to test it against.