Nonblocking I/O & epoll readiness

Nonblocking I/O makes lack of immediate progress explicit. A readiness mechanism then lets a program wait for useful state changes rather than spin on EAGAIN. Epoll stores an interest set and reports ready registrations, along with an application-supplied data field. Readiness is a state observation, not an exclusive reservation of bytes for one caller.

Separate readiness from progress

Register an fd, wait for an event, then try the nonblocking operation. Keep its actual progress in application state. EAGAIN returns control to the event loop; zero read means EOF; other failures need an explicit error path. A positive short read is still progress and may be followed by another successful read.

#include <cassert>
#include <cstdio>
#include <cstdlib>
#include <fcntl.h>
#include <unistd.h>
#include <sys/epoll.h>

int main() {
    int p[2];assert(::pipe2(p,O_NONBLOCK|O_CLOEXEC)==0);int ep=::epoll_create1(EPOLL_CLOEXEC);assert(ep>=0);
    epoll_event event{};event.events=EPOLLIN;event.data.fd=p[0];assert(::epoll_ctl(ep,EPOLL_CTL_ADD,p[0],&event)==0);
    assert(::write(p[1],"abc",3)==3);epoll_event out[1];int a=::epoll_wait(ep,out,1,0);char bytes[3];assert(::read(p[0],bytes,1)==1);
    int b=::epoll_wait(ep,out,1,0);assert(::read(p[0],bytes,2)==2);int c=::epoll_wait(ep,out,1,0);
    std::printf("%d %d %d\n",a,b,c);::close(ep);::close(p[0]);::close(p[1]);
}

The build uses level-triggered watches for bounded per-turn drains. If the budget expires while bytes remain, another level-triggered turn still observes readiness. Under edge triggering, keep the fd on an application ready queue until it is drained to EAGAIN; stopping early and waiting for another edge can strand bytes. An unbounded busy fd can also starve its neighbors, so readiness semantics and fairness policy need to be designed together.

HUP and data can coexist

A writer can send bytes and then close. The reader may observe readable data and hangup together. Discarding the buffer when HUP arrives loses valid input. Read the remaining bytes and let a zero read establish EOF.

#include <cassert>
#include <cstdio>
#include <cstdlib>
#include <fcntl.h>
#include <unistd.h>
#include <cerrno>

int main() {
    int p[2];assert(::pipe2(p,O_NONBLOCK|O_CLOEXEC)==0);char byte=0;
    auto first=::read(p[0],&byte,1);int blocked=first<0&&errno==EAGAIN;::close(p[1]);auto last=::read(p[0],&byte,1);
    std::printf("%d %ld %ld\n",blocked,long(first),long(last));::close(p[0]);
}

Identity, masks and dispatch

The kernel can reuse descriptor numbers after close. Keep the complete 64-bit application token, and validate the active object's lifetime when dispatching an event from a batch. A generation number can protect against stale application identity; epoll does not invent that policy for you.

The task sorts returned tokens only to make its API deterministic. It makes no claim that the kernel returns readiness in that order, that one batch contains every ready fd, or that readiness means a nonblocking operation cannot report EAGAIN. Preserve all event bits instead of treating the mask as one enum. Only subscribe to writable readiness while output is queued or a connect needs completion, otherwise sockets that are usually writable can create a busy loop.

Before the fill, trace a six-byte buffer with a two-byte budget: first ab, then another ready turn, then cdef and EAGAIN. The build also tests token width, modification/removal, EOF after buffered data and rejection of blocking descriptors. Continue with TCP sockets.