C++ ↔ Python boundaries with pybind11

Graphics research happens in Python: NumPy and PyTorch for the maths (py-numpy-geometry, py-torch-autograd), notebooks for the experiments. Production renderers and the hot loops — projection of a million splats, tile binning, a rasteriser — are C++ (or CUDA). A rendering engineer is expected to stand on the line between them: take a research prototype, move its inner loop into C++, and hand it back to Python so that the researchers' code keeps working, only faster.

That line is where the bugs live. A Python caller can pass any dtype, any shape, any memory layout; C++ code must check what it got, read it correctly, never keep a pointer longer than the memory lives, and not hold the interpreter hostage while it computes. This skill is about doing that with pybind11 (version 3 is current), the most widely used binding library. Everything in the snippets runs: the sandbox embeds CPython 3.14 with NumPy, so a C++ program can define a module and call it from Python in the same process.

1. Why bind, and with what

You bind when profiling says the time is in a loop Python can't vectorise well (per-point branching, a sort per tile, a BVH traversal), or when the code already exists in C++ (the production renderer, a camera model) and the research side needs to call the same implementation instead of a re-implementation that drifts.

The mechanisms differ, the questions don't: what does the C++ side accept, who owns each byte, which thread holds the interpreter, and what happens to an error. pybind11 is used for all of it below; section 8 shows the other two serious options in a few lines.

2. A module in twenty lines

PYBIND11_MODULE(name, m) defines an extension module (a .so that Python imports); PYBIND11_EMBEDDED_MODULE(name, m) defines the same module inside a program that embeds the interpreter — that is what the sandbox and the tests here use. The body is identical. m.def binds a function; pybind11 deduces argument and return types from its signature, py::arg names arguments (and enables keywords and defaults), and the string is the docstring:

#include <pybind11/embed.h>
namespace py = pybind11;

float exposure(float value, float stops) { return value * (1 << static_cast<int>(stops)); }

PYBIND11_EMBEDDED_MODULE(demo, m) {
    m.doc() = "a tiny module";
    m.def("exposure", &exposure, py::arg("value"), py::arg("stops") = 1.0f,
          "Scale a linear value by 2**stops (whole stops only).");
    m.def("clamp01", [](double x) { return x < 0 ? 0.0 : x > 1 ? 1.0 : x; });
}

int main() {
    py::scoped_interpreter guard;   // start Python; finalize at the end of main
    py::exec(R"py(
import demo
print(demo.exposure(0.25), demo.exposure(0.25, stops=3), demo.clamp01(1.7))
print(demo.exposure.__doc__.splitlines()[0])
try:
    demo.exposure("bright")
except TypeError as e:
    print("TypeError:", str(e).splitlines()[0])
)py");
}

Note the last line: passing a string where a float is expected never reaches your code. The dispatcher tried every overload, none could cast the arguments, and it raised TypeError listing the signatures it has. That is the first layer of checking you get for free; for arrays you need more.

3. NumPy arrays: the buffer protocol and py::array_t

A NumPy array is a pointer, a dtype, a shape and strides — bytes to step per axis (py-numpy-geometry §6 covers views and copies from the Python side). Python's buffer protocol is the C-level contract that exposes exactly that (Py_buffer: buf, itemsize, format, ndim, shape, strides), so any C++ code can read an array's memory without copying.

pybind11 gives you two handles on it:

• py::array — any NumPy array. Ask it dtype(), ndim(), shape(i), strides(i), data().
• py::array_t<T, Flags> — an array whose dtype is T. As an argument type it also decides what the caster accepts:

Silent conversion is convenient for small inputs (a 3×3 K given as a list) and a trap for big ones: a float64 point cloud gets copied and rounded on every call, and an "in-place" function writes into the temporary copy. For the main data, state the contract and enforce it with a clear message:

#include <pybind11/embed.h>
#include <pybind11/numpy.h>
#include <string>
namespace py = pybind11;

double sum_strict(const py::array& a) {
    if (!a.dtype().is(py::dtype::of<float>()))
        throw py::type_error("expected float32, got " + py::str(a.dtype()).cast<std::string>());
    if (a.ndim() != 1) throw py::value_error("expected a 1-D array");
    auto v = a.unchecked<float, 1>();
    double s = 0;
    for (py::ssize_t i = 0; i < v.shape(0); ++i) s += v(i);
    return s;
}

PYBIND11_EMBEDDED_MODULE(demo, m) {
    m.def("sum_strict", &sum_strict);
    m.def("bump", [](py::array_t<float> a) {           // forcecast: may be a copy
        auto w = a.mutable_unchecked<1>();
        for (py::ssize_t i = 0; i < w.shape(0); ++i) w(i) += 1.0f;
    });
}

int main() {
    py::scoped_interpreter guard;
    py::exec(R"py(
import numpy as np, demo
print(demo.sum_strict(np.arange(4, dtype=np.float32)))
for bad in (np.arange(4.0), np.zeros((2, 2), np.float32)):
    try:
        demo.sum_strict(bad)
    except (TypeError, ValueError) as e:
        print(type(e).__name__, "-", e)

a32, a64 = np.zeros(3, np.float32), np.zeros(3)
demo.bump(a32); demo.bump(a64)
print("float32 bumped:", a32.tolist(), "| float64 bumped:", a64.tolist())
)py");
}

The last line is the forcecast trap in one picture: the float64 array was converted to a new float32 array, the function bumped that, and the copy was thrown away.

4. Shapes and strides

Element (i, j) of a 2-D array lives at data + i·strides[0] + j·strides[1] bytes. Only for a freshly made C-contiguous (N, 3) float32 array is that data + (3i + j)·4. Every view research code makes casually — P.T, P[:, :3] of an (N, 4) homogeneous array, P[::2], an np.asfortranarray, a reversed P[::-1] (negative stride!) — has the same shape and dtype and different strides.

You have three correct options:

1. Honour the strides: a.unchecked<float, 2>() (or array_t::unchecked<2>()) returns a proxy whose r(i, j) does the byte arithmetic; it is as fast as raw indexing for contiguous input. mutable_unchecked for writing.
2. Require contiguity and copy if needed: py::array_t<float, py::array::c_style | py::array::forcecast> — simple, but a hidden copy for every view.
3. Reject non-contiguous input with a ValueError — honest, and annoying for callers.

Option 1 is the default for library code. The proxies do no bounds checks, so check ndim() and the shape before you build them (unchecked<…, 2> on a 1-D array throws, but reading r(i, 2) on an (N, 2) array is out of bounds):

#include <pybind11/embed.h>
#include <pybind11/numpy.h>
namespace py = pybind11;

// Sum of each row's z, the right way and the "it's always contiguous" way.
double z_sum(const py::array_t<float>& a) {
    auto r = a.unchecked<2>();
    double s = 0;
    for (py::ssize_t i = 0; i < r.shape(0); ++i) s += r(i, 2);
    return s;
}
double z_sum_naive(const py::array_t<float>& a) {
    const float* p = a.data();
    double s = 0;
    for (py::ssize_t i = 0; i < a.shape(0); ++i) s += p[i * 3 + 2];
    return s;
}

PYBIND11_EMBEDDED_MODULE(demo, m) {
    m.def("z_sum", &z_sum);
    m.def("z_sum_naive", &z_sum_naive);
}

int main() {
    py::scoped_interpreter guard;
    py::exec(R"py(
import numpy as np, demo
H = np.arange(16, dtype=np.float32).reshape(4, 4)   # homogeneous (x, y, z, w) rows
for name, P in [("H[:, :3]", H[:, :3]), ("contiguous copy", np.ascontiguousarray(H[:, :3]))]:
    print(f"{name:16} strides={P.strides}  numpy={P[:, 2].sum():5.1f}  "
          f"z_sum={demo.z_sum(P):5.1f}  naive={demo.z_sum_naive(P):5.1f}")
)py");
}

5. Errors: exception translation

A C++ exception must never unwind through the interpreter's C frames. pybind11 catches everything that leaves a bound function and raises a Python exception instead:

Pick the exception Python code expects: wrong dtype is a TypeError, wrong shape or value a ValueError, out-of-range index an IndexError. Put the actual dtype or shape in the message — "expected float32 (N, 3), got float64 (N, 4)" saves the caller a debugging session. Your own exception types can be registered so callers can catch them precisely:

#include <pybind11/embed.h>
#include <stdexcept>
namespace py = pybind11;

struct DecodeError : std::runtime_error {
    using std::runtime_error::runtime_error;
};

PYBIND11_EMBEDDED_MODULE(demo, m) {
    py::register_exception<DecodeError>(m, "DecodeError", PyExc_ValueError);
    m.def("pixel", [](int x) { if (x >= 4) throw std::out_of_range("x = " + std::to_string(x) + " >= 4"); return x; });
    m.def("decode", [] { throw DecodeError("truncated PLY header"); });
    m.def("oops", [] { throw std::logic_error("invariant broken"); });
}

int main() {
    py::scoped_interpreter guard;
    py::exec(R"py(
import demo
for f in (lambda: demo.pixel(9), demo.decode, demo.oops):
    try:
        f()
    except Exception as e:
        print(f"{type(e).__name__:12} {e}   (ValueError subclass: {isinstance(e, ValueError)})")
)py");
}

6. Ownership and lifetimes

Every object that crosses the boundary has an owner on one side, and the other side holds a reference into it. Three tools say who owns what.

Return value policies decide what a returned pointer or reference means. The default, automatic, is the source of most binding crashes:

reference_internal returns a reference and keeps self alive while the result lives (it is reference + keep_alive<0, 1>).

py::keep_alive<Nurse, Patient>() ties lifetimes between arguments (1 is self, 0 the return value): a Renderer::set_target(Image*) that stores the pointer needs keep_alive<1, 2>(), or a temporary Image dies right after the call while the renderer still writes into it.

Bases and capsules decide who owns an array's memory. A NumPy array made with py::array_t<T>(shape, strides, ptr, base) is a view over ptr that holds a reference to base; without a base, pybind11 copies (it cannot know how long ptr lives). Two zero-copy patterns cover nearly everything:

• The memory belongs to a bound C++ object (an Image): pass the object's own Python handle as the base. The view keeps the Image alive; deleting the Python name img doesn't free the pixels while a view exists.
• The memory is new (a result computed in C++): put the buffer on the heap and pass a py::capsule whose destructor frees it. NumPy calls it when the last view goes away.

#include <pybind11/embed.h>
#include <pybind11/numpy.h>
#include <vector>
namespace py = pybind11;

struct Image {
    int w, h;
    std::vector<float> px;
    Image(int w, int h) : w(w), h(h), px(static_cast<std::size_t>(w) * h, 0.0f) {}
};

PYBIND11_EMBEDDED_MODULE(demo, m) {
    py::class_<Image>(m, "Image")
        .def(py::init<int, int>())
        .def("total", [](const Image& im) { double s = 0; for (float p : im.px) s += p; return s; })
        .def("pixels", [](py::object self) {               // zero-copy view, owner kept alive
            auto& im = self.cast<Image&>();
            return py::array_t<float>({im.h, im.w}, im.px.data(), self);
        })
        .def("pixels_copy", [](Image& im) {                // no base: pybind11 copies
            return py::array_t<float>({im.h, im.w}, im.px.data());
        });
    m.def("ramp", [](int n) {                             // new memory, owned by a capsule
        auto* buf = new std::vector<float>(n);
        for (int i = 0; i < n; ++i) (*buf)[i] = static_cast<float>(i);
        py::capsule owner(buf, [](void* p) noexcept { delete static_cast<std::vector<float>*>(p); });
        return py::array_t<float>(n, buf->data(), owner);
    });
}

int main() {
    py::scoped_interpreter guard;
    py::exec(R"py(
import gc, weakref, numpy as np, demo
img = demo.Image(4, 2)
v, c = img.pixels(), img.pixels_copy()
v[:] = 1.0; c[:] = 5.0
print("total after writing both:", img.total(), "| v.base is the Image:", type(v.base).__name__)
w = weakref.ref(img)
del img; gc.collect()
print("Image alive while the view exists:", w() is not None, "| v.sum() =", v.sum())
del v; gc.collect()
print("after del v:", w() is None)
r = demo.ramp(5)
print("ramp:", r, "owner:", type(r.base).__name__)
)py");
}

The buffer protocol route does the same for np.asarray(img) and memoryview(img): declare py::class_<Image>(m, "Image", py::buffer_protocol()) and describe the memory with .def_buffer(...) returning a py::buffer_info(ptr, sizeof(float), py::format_descriptor<float>::format(), ndim, shape, strides). Python keeps the exporting object alive for as long as the buffer is in use.

The inverse direction needs the same care. A C++ object that stores a pointer into a NumPy array it was given (py::array::data()) must also store the py::array itself (a reference), or the caller's del frees memory C++ still points at. Copying into a std::vector at the boundary is the safe default; zero-copy is an optimisation you document.

7. The GIL: release it around pure C++

CPython (in its default build) has a global interpreter lock: one thread runs Python code at a time, and every Python C API call — creating an array, changing a reference count, raising an exception object — requires holding it. A bound function is called with the GIL held, and pybind11 does not release it for you. A C++ loop that runs for 200 ms therefore freezes every other Python thread for 200 ms: the UI thread of a viewer, a data loader, a second camera's projection that could run in parallel.

py::gil_scoped_release releases the GIL for its scope and re-acquires it in its destructor. The rule is mechanical: inside the scope, touch no Python object — no allocation of arrays, no py::object copies or destructions, no callbacks. Read and write through unchecked proxies taken before the scope; allocate outputs before it; build the return value after it.

py::array_t<float> uv({n, py::ssize_t{2}});     // Python allocation: GIL held
auto P = points.unchecked<float, 2>();          // raw views: plain pointers + strides
auto UV = uv.mutable_unchecked<2>();
{
    py::gil_scoped_release nogil;               // other Python threads may run now
    for (py::ssize_t i = 0; i < n; ++i) { /* only P(...), UV(...), doubles */ }
}                                               // GIL re-acquired here
return py::make_tuple(uv, valid);               // Python objects again: GIL held

py::call_guard<py::gil_scoped_release>() on m.def does the same for functions whose arguments and results are plain C++ types. To call Python from a released region (a progress callback), take it back for the call with py::gil_scoped_acquire. In debug builds pybind11 checks reference-count changes and aborts with "…is being called while the GIL is either not held or invalid"; treat that message as a lifetime bug, not noise.

The snippet plays exactly that, deterministically: wait_for_signal blocks on a semaphore that only the Python worker thread can release.

#include <pybind11/embed.h>
#include <semaphore>
namespace py = pybind11;

std::binary_semaphore ready{0};

PYBIND11_EMBEDDED_MODULE(demo, m) {
    m.def("signal", [] { ready.release(); });
    m.def("wait_for_signal", [] {
        py::gil_scoped_release nogil;   // without this line: deadlock
        ready.acquire();
    });
}

int main() {
    py::scoped_interpreter guard;
    py::exec(R"py(
import threading, demo
t = threading.Thread(target=demo.signal)
t.start()
demo.wait_for_signal()     # C++ waits with the GIL released; the thread gets to run
t.join()
print("woke up: the worker ran Python while C++ was waiting")
)py");
}

Two more facts worth knowing. Releasing has a cost (a few hundred nanoseconds), so release around loops, not around a 3-element dot product. And CPython 3.13+ has an optional free-threaded build without a GIL; pybind11 3 modules opt in with PYBIND11_MODULE(splatcore, m, py::mod_gil_not_used()), which is a promise that your code is thread-safe without the GIL — the releases stay harmless, the data races become yours.

8. Building a real extension module

In a project, the bindings are a separate target on top of the C++ library, built with CMake (shown as fragments: the sandbox only embeds). pybind11 3 ships a CMake package; find Python first, so pybind11 uses CMake's FindPython:

# python/CMakeLists.txt
find_package(Python 3.10 REQUIRED COMPONENTS Interpreter Development.Module)
find_package(pybind11 3 CONFIG REQUIRED)        # or FetchContent / a submodule

pybind11_add_module(splatcore MODULE bindings.cpp)
target_link_libraries(splatcore PRIVATE splat::core)
install(TARGETS splatcore LIBRARY DESTINATION .)

pybind11_add_module creates a MODULE library named with the interpreter's suffix (splatcore.cpython-314-x86_64-linux-gnu.so), hidden symbol visibility and LTO in release builds, and does not link libpython (the interpreter that imports the module provides those symbols). Development.Module is the component for extensions; Development.Embed is for programs that embed Python, like the tests here. The C++ side:

// python/bindings.cpp
#include <pybind11/numpy.h>
#include <pybind11/pybind11.h>
#include <splat/...>                       // the core library's headers
namespace py = pybind11;

PYBIND11_MODULE(splatcore, m) {           // must match the target name
    m.def("project_points", &project_points, py::arg("points"), py::arg("K"), py::arg("R"), py::arg("t"));
}

cmake --preset release && cmake --build --preset release
PYTHONPATH=build/release/python python -c "import splatcore; print(splatcore.__doc__)"

Failures you will meet, and what they mean:

To ship it as a pip-installable wheel, a pyproject.toml with scikit-build-core drives the same CMakeLists (pip install . builds it):

[build-system]
requires = ["scikit-build-core>=0.10", "pybind11>=3"]
build-backend = "scikit_build_core.build"

[project]
name = "splatcore"
version = "0.1.0"

Test the module from Python with pytest against a NumPy reference — the same shape of test the build task uses — and generate .pyi stubs (pybind11-stubgen splatcore) so editors and type checkers know the signatures.

For comparison, the same function as a nanobind binding, where the dtype, shape and device constraints are part of the parameter type:

#include <nanobind/ndarray.h>
namespace nb = nanobind;
using Points = nb::ndarray<const float, nb::shape<-1, 3>, nb::device::cpu>;  // any strides

NB_MODULE(splatcore, m) {
    m.def("centroid_z", [](Points p) {
        auto v = p.view();                   // strided accessor, like unchecked
        double s = 0;
        for (size_t i = 0; i < v.shape(0); ++i) s += v(i, 2);
        return s / v.shape(0);
    });
}

and as a PyTorch custom operator, the route for a CUDA rasteriser used in a training loop (gfx-differentiable-rendering): the schema is registered with the dispatcher, implementations per device, and the backward is registered from Python with torch.library.register_autograd:

#include <torch/library.h>

std::tuple<at::Tensor, at::Tensor> project_cpu(const at::Tensor& points, const at::Tensor& K,
                                               const at::Tensor& R, const at::Tensor& t);

TORCH_LIBRARY(splat, m) {
    m.def("project(Tensor points, Tensor K, Tensor R, Tensor t) -> (Tensor, Tensor)");
}
TORCH_LIBRARY_IMPL(splat, CPU, m) { m.impl("project", &project_cpu); }
// TORCH_LIBRARY_IMPL(splat, CUDA, m) { m.impl("project", &project_cuda); }
// Python: uv, valid = torch.ops.splat.project(points, K, R, t)

9. A checklist for every bound function

1. Contract: which dtypes, shapes and layouts are accepted, which are converted, which raise — written in the docstring and tested.
2. Read through strides (or require and document contiguity).
3. Ownership: every returned view has a base that owns its memory; every stored pointer has a keep_alive or a stored reference; every returned T&/T* has an explicit policy.
4. GIL: released around the loop, nothing Python inside.
5. Errors: the Python exception a caller would expect, with the actual dtype or shape in the message.
6. Tests from Python against a NumPy reference, including transposed and sliced inputs, the edge cases (N = 0, z = 0), and the error cases.

In an interview

"How would you speed up this NumPy function?" often ends at "move the loop to C++ and bind it". Expect follow-ups on exactly the points above: what happens when someone passes float64 or a transposed array; how a returned array keeps its memory alive (base / capsule) and what reference_internal does; why automatic on a raw pointer can double-free; when you release the GIL and what you may not do while it is released; how C++ exceptions reach Python. Good things to volunteer: you'd choose a strict dtype for the big arrays and forcecast for the small ones; you test with non-contiguous views; you know TORCH_LIBRARY custom ops are the route when the code must live inside a PyTorch training graph (and nanobind for leaner new bindings); and you can name the import errors and what causes them.

In the project

Ungraded practice in projects/scene-viewer/python/ (stage 4); the evidence is your own build and test output.

1. Build and test it: cmake --preset python && cmake --build --preset python && ctest --preset python -L python.
2. Read python/splatpy.cpp: strict float32 checks with the argument's name, stride-aware unchecked reads, the GIL released around C++ work, PlyError registered as a ValueError subclass. Find each rule from this lesson.
3. render returns a copy. Add a zero-copy variant that returns a view of an image object kept alive by the array (py::array with a base object), and test in python/tests/test_splatpy.py that the view survives deleting the owner's name.
4. Build the module in an ASan build and import it with the runtime preloaded (LD_PRELOAD=$(gcc -print-file-name=libasan.so), ASAN_OPTIONS=detect_leaks=0 for the interpreter's allocations). Break the keep-alive on purpose and read the report.