Tasks, scheduling and queues (FreeRTOS)

Choose FreeRTOS & CMSIS (C++) for the RTOS sequence. After the four foundations, continue with software timers and event flags.

The interrupt-driven skills built firmware as a superloop plus ISRs: handlers capture events, the main loop does the work. That stops scaling when several activities have different deadlines — a control loop every 5 ms, a radio protocol with its own timeouts, a logger that may take 20 ms to flush. An RTOS lets you write each activity as its own sequential task that blocks while it waits, and a scheduler decides, at every event, which task runs.

This skill uses FreeRTOS (kernel V11, the ARM_CM4F port), the most common RTOS in firmware job ads and interviews. Everything runs on the emulated Cortex-M4 board (mps2-an386) with the configuration in <crucible/mcu/rtos/FreeRTOSConfig.h>: a 1 kHz tick (1 tick = 1 ms), priorities 0–7, preemption and time slicing on, stack-overflow checking method 2, configASSERT enabled.

About the timing. qemu runs with -icount: one instruction is one nanosecond of virtual time, so every tick count in this lesson is exact and the same on every run — ideal for reasoning about schedules. It is emulated time. It says nothing about a physical board's caches, flash wait states or interrupt latency, it doesn't establish that a real deadline is met, and running here is not hardware-in-the-loop testing.

1. Tasks and their states

A task is a C function that never returns, plus its own stack and a task control block (TCB: saved context, priority, state, list links). xTaskCreate(fn, name, stack_words, arg, priority, &handle) makes one; it starts running when the scheduler picks it.

#include "FreeRTOS.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>

static void blinker(void* arg) {
    const char* name = static_cast<const char*>(arg);
    for (int i = 0; i < 3; ++i) {
        std::printf("tick %3lu: %s\n", (unsigned long)xTaskGetTickCount(), name);
        vTaskDelay(pdMS_TO_TICKS(name[0] == 'f' ? 10 : 25));  // Blocked: costs no CPU
    }
    if (name[0] == 's') std::exit(0);  // end the program (semihosting)
    vTaskDelete(nullptr);              // a task function must never return
}

int main() {
    xTaskCreate(blinker, "fast", 512, (void*)"fast", 1, nullptr);  // 512 words of stack
    xTaskCreate(blinker, "slow", 512, (void*)"slow", 1, nullptr);
    vTaskStartScheduler();  // never returns while the scheduler runs
}

The two tasks interleave by time: fast prints at ticks 0, 10, 20; slow at 0, 25, 50 (and ends the program at tick 75, after its last delay). Almost all of that time both tasks are Blocked and only the idle task runs. Each task is in exactly one state:

• Running — executing. On this single-core M4, exactly one task.
• Ready — able to run; a task of equal or higher priority has the CPU.
• Blocked — waiting for time (vTaskDelay) or an event (a queue, a semaphore, a notification), always with a timeout (portMAX_DELAY means "forever" — avoid it in production code). A Blocked task uses no CPU.
• Suspended — out of scheduling until vTaskResume; no timeout.
• Deleted tasks (vTaskDelete) are gone; a task that deletes itself has its TCB and stack freed later by the idle task.

A task function must never fall off its end: loop forever, or call vTaskDelete(nullptr).

2. Preemptive priority scheduling

The rule is short: the highest-priority Ready task runs. Priorities here are 0 (lowest, the idle task) to 7. Note the direction: a larger FreeRTOS priority number is more important — the opposite of Cortex-M NVIC priorities, where 0 is the most urgent. Mixing those up is a classic interview trap.

With configUSE_PREEMPTION 1 the scheduler runs at every point where the answer could change:

• an API call makes a higher-priority task Ready (create a task, send to a queue, give a semaphore or notification) — the switch happens inside that call, before it returns;
• the running task blocks or suspends;
• the tick interrupt: delayed tasks whose time has come become Ready, and with configUSE_TIME_SLICING 1 the CPU rotates among Ready tasks of the same priority.

Time slicing never gives a lower priority a turn. A task that never blocks starves everything below it — including the idle task, created by vTaskStartScheduler at priority 0. Idle runs when nothing else can; it frees self-deleted tasks' memory and is where a real system goes to sleep. (The other kernel task here is the timer service task, at priority 7, for software timers.)

#include "FreeRTOS.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>

static void work(TickType_t ticks) {  // stay Ready (busy) for `ticks` ticks
    TickType_t t = xTaskGetTickCount();
    while (xTaskGetTickCount() - t < ticks) {}
}

static void control(void*) {          // priority 3: wakes every 4 ticks, works 1
    for (int i = 0; i < 3; ++i) {
        vTaskDelay(4);
        std::printf("[%2lu] control\n", (unsigned long)xTaskGetTickCount());
        work(1);
    }
    std::exit(0);
}

static void logger(void* arg) {       // priority 1, two of them: time-sliced
    for (;;) {
        std::printf("[%2lu] %s\n", (unsigned long)xTaskGetTickCount(), static_cast<const char*>(arg));
        work(1);
    }
}

int main() {
    xTaskCreate(control, "control", 512, nullptr, 3, nullptr);
    xTaskCreate(logger, "logA", 512, (void*)"logA", 1, nullptr);
    xTaskCreate(logger, "logB", 512, (void*)"logB", 1, nullptr);
    vTaskStartScheduler();
}

Output: logA and logB alternate one tick each ([0] logA, [1] logB, …), and every time control wakes it preempts whichever logger is running: [4] control, [9] control, [14] control. Its period is 5, not 4: it works for one tick and then delays 4 — section 3 is about that.

How to pick priorities: by deadline, not by "importance" — the task with the shortest deadline (or period) gets the highest priority (rate-monotonic thinking). Keep high-priority tasks short and blocking; push long work down.

3. vTaskDelay vs xTaskDelayUntil

vTaskDelay(n) blocks for n ticks from now. xTaskDelayUntil(&last, period) blocks until last + period and then advances last by exactly period — absolute deadlines that don't depend on how long the work took:

#include "FreeRTOS.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>

static void work(TickType_t ticks) {
    TickType_t t = xTaskGetTickCount();
    while (xTaskGetTickCount() - t < ticks) {}
}

static void periodic(void*) {
    std::printf("vTaskDelay(10):          ");
    for (int i = 0; i < 4; ++i) {
        work(3);
        vTaskDelay(10);
        std::printf("%3lu", (unsigned long)xTaskGetTickCount());
    }
    TickType_t last = xTaskGetTickCount();
    TickType_t base = last;
    std::printf("\nxTaskDelayUntil(&last, 10): ");
    for (int i = 0; i < 4; ++i) {
        work(3);
        xTaskDelayUntil(&last, 10);
        std::printf("%3lu", (unsigned long)(xTaskGetTickCount() - base));
    }
    std::printf("   (ticks since the loop started)\n");
    std::exit(0);
}

int main() {
    xTaskCreate(periodic, "periodic", 512, nullptr, 1, nullptr);
    vTaskStartScheduler();
}

It prints 13 26 39 52 for vTaskDelay — each cycle is 3 ticks of work plus 10 of sleep, and the error accumulates — and 10 20 30 40 for xTaskDelayUntil.

Details interviewers like:

• Tick granularity. A delay ends at a tick boundary, so vTaskDelay(1) called just before a tick may sleep only a sliver of a millisecond. "At least N ms" needs N + 1 ticks; pdMS_TO_TICKS(ms) converts (and rounds down).
• Overruns. If the work takes longer than the period, xTaskDelayUntil doesn't block and returns pdFALSE — count that; it's your overrun detector. The task then runs back to back and starves lower priorities.
• Jitter. Even with absolute wake-ups, a task starts late when a higher priority is running at its wake-up tick. The wake time is exact; the start time is not.

4. Stacks: sizing and high-water marks

Every task has its own stack, sized at creation in words (StackType_t, 4 bytes on the M4): 512 is 2 KiB. Local variables, call depth, and the context the hardware and kernel save at every switch (including FPU registers for a task that used floating point) all live there. Exceptions on Cortex-M run on the separate main stack (MSP), but the exception's entry frame is still pushed onto the interrupted task's stack.

There's no MMU to catch an overflow: the task writes over whatever is below its stack — usually another task's TCB or the heap. FreeRTOS gives you two tools:

• Measure. New stacks are filled with 0xa5. uxTaskGetStackHighWaterMark(handle) scans for the deepest overwritten word and returns how many words were never used — the minimum free stack since the task started.
• Detect. configCHECK_FOR_STACK_OVERFLOW 1 checks the saved stack pointer at each context switch; 2 (used here) also checks that the last 16 bytes still hold the fill pattern. Both run only at a switch, after the damage, and miss an overflow that jumps past the checked bytes or corrupts something fatal first. Detection is a safety net, not a sizing method.

#include "FreeRTOS.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>
#include <cstring>

static void checksum_block() {         // a function with a big local buffer
    volatile unsigned char block[1200];
    std::memset((unsigned char*)block, 0x5a, sizeof block);
}

static void report(const char* when) {
    // Words never touched since the task started: the minimum free stack so far.
    std::printf("%-24s %lu words free at worst\n", when,
                (unsigned long)uxTaskGetStackHighWaterMark(nullptr));
}

static void task(void*) {
    UBaseType_t fresh = uxTaskGetStackHighWaterMark(nullptr);
    std::printf("%-24s %lu words free at worst\n", "at start:", (unsigned long)fresh);
    report("after a printf:");
    checksum_block();
    report("after a 1200-byte local:");
    std::exit(0);
}

int main() {
    xTaskCreate(task, "task", 512, nullptr, 1, nullptr);  // 512 words = 2 KiB
    vTaskStartScheduler();
}

The run prints 501 words free at start, 346 after one printf (newlib's printf alone takes about 155 words — 600 bytes!) and 201 after the 1200-byte local. Lessons for budgets:

• Measure the high-water mark after a soak test that drives the worst-case path (error handling, the deepest formatting call), then add a margin (20–50 % is common).
• Big local arrays and heavy library calls (printf, floating-point formatting) are the usual culprits; keep buffers static or in a pool, and keep a dedicated logging task if you need printf.
• Interrupts don't use task stacks for their own frames on Cortex-M, but every task must still have room for an exception entry frame on top of its deepest point.

5. Queues: copies, blocking and timeouts

A queue is a fixed-length FIFO of fixed-size items: xQueueCreate(length, item_size). xQueueSend copies item_size bytes from the address you pass into the queue's storage; xQueueReceive copies them out. Both block with a timeout when the queue is full or empty, and both are the kernel's synchronisation too: a receive that takes an item wakes a sender blocked on the full queue (the highest-priority waiter first).

#include "FreeRTOS.h"
#include "queue.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>

struct Sample { unsigned seq; int millivolts; };   // copied into the queue: 8 bytes

static QueueHandle_t samples;

static void adc_task(void*) {                       // priority 2
    for (unsigned i = 0; i < 6; ++i) {
        Sample s{i, 1000 + 10 * int(i)};
        if (xQueueSend(samples, &s, pdMS_TO_TICKS(5)) != pdPASS)
            std::printf("[%2lu] queue full, dropped %u\n", (unsigned long)xTaskGetTickCount(), i);
        vTaskDelay(1);
    }
    vTaskDelete(nullptr);
}

static void filter_task(void*) {                    // priority 1, slow
    Sample s;
    while (xQueueReceive(samples, &s, pdMS_TO_TICKS(20)) == pdPASS) {
        std::printf("[%2lu] got #%u = %d mV\n", (unsigned long)xTaskGetTickCount(), s.seq, s.millivolts);
        vTaskDelay(8);                              // takes 8 ticks per sample
    }
    std::printf("[%2lu] no data for 20 ticks: done\n", (unsigned long)xTaskGetTickCount());
    std::exit(0);
}

int main() {
    samples = xQueueCreate(2, sizeof(Sample));      // 2 items of 8 bytes, copied in and out
    xTaskCreate(adc_task, "adc", 512, nullptr, 2, nullptr);
    xTaskCreate(filter_task, "filter", 512, nullptr, 1, nullptr);
    vTaskStartScheduler();
}

The filter takes 8 ticks per sample and the queue holds 2, so the ADC task meets backpressure: its send blocks for up to 5 ticks and then gives up. The trace is exact here:

[ 0] got #0 = 1000 mV
[ 8] queue full, dropped 3
[ 8] got #1 = 1010 mV
[15] queue full, dropped 5
[16] got #2 = 1020 mV
[24] got #4 = 1040 mV
[52] no data for 20 ticks: done

Every blocking call has a timeout and an outcome the program handles — here "count and drop", elsewhere "retry", "report an error", or "go to a safe state". That's the habit firmware reviewers look for.

Passing pointers: ownership

Copying is simple and safe for small items. For large ones (a 512-byte frame) you queue a pointer, and the queue copies just 4 bytes — the item size is sizeof(Frame*) and you pass &ptr. Now two rules carry all the weight:

• One owner at a time. After a successful send the sender must not touch the buffer; the receiver owns it until it frees it or returns it to a pool.
• The failure path keeps ownership. If the send times out, nobody else saw the buffer: the sender still owns it and must free it. Forgetting that is the classic slow leak.

xQueueSend is xQueueSendToBack; xQueueSendToFront jumps the queue (for an urgent message, never for a stream). xQueueOverwrite on a length-1 queue makes a "latest value" mailbox. ISRs use the …FromISR variants, which never block — the subject of the ISR hand-off skill.

6. Task notifications

Each task carries a small array of 32-bit notification values (2 here). Another task or an ISR can increment, set bits in, or overwrite one, and the owner blocks on it — no separate kernel object to create, and FreeRTOS documents it as faster and smaller than a semaphore. Used as a counting semaphore:

#include "FreeRTOS.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>

static TaskHandle_t worker_handle;

static void worker(void*) {
    for (;;) {
        // Block until notified (or 50 ticks pass); take every pending event at once.
        uint32_t events = ulTaskNotifyTake(pdTRUE, pdMS_TO_TICKS(50));
        if (events == 0) {
            std::printf("[%2lu] worker: quiet for 50 ticks, stopping\n", (unsigned long)xTaskGetTickCount());
            std::exit(0);
        }
        std::printf("[%2lu] worker: %lu event(s)\n", (unsigned long)xTaskGetTickCount(), (unsigned long)events);
    }
}

static void source(void*) {
    xTaskNotifyGive(worker_handle);   // one event
    vTaskDelay(5);
    xTaskNotifyGive(worker_handle);   // three quick events...
    xTaskNotifyGive(worker_handle);
    xTaskNotifyGive(worker_handle);   // ...arrive before the worker runs
    vTaskDelete(nullptr);
}

int main() {
    xTaskCreate(worker, "worker", 512, nullptr, 1, &worker_handle);
    xTaskCreate(source, "source", 512, nullptr, 2, nullptr);
    vTaskStartScheduler();
}

[ 0] worker: 1 event(s)
[ 5] worker: 3 event(s)
[55] worker: quiet for 50 ticks, stopping

ulTaskNotifyTake(pdTRUE, timeout) returns the count before clearing it, so three quick gives arrive as one wake-up reporting 3 — handle them all. pdFALSE decrements by one instead (one wake-up per give). The other forms: xTaskNotify(handle, bits, eSetBits) for event flags, eSetValueWithOverwrite as a one-word mailbox.

Limits: a notification has one receiver (the task that owns it), holds one value (not a stream), and a sender can't block waiting for room. When several tasks wait, or you need buffering, use a semaphore, an event group, or a queue.

7. Static vs dynamic allocation

xTaskCreate and xQueueCreate take their memory from the FreeRTOS heap (pvPortMalloc; here heap_3, newlib's malloc made task-safe — other ports use heap_1…heap_5). The …Static variants take memory you provide:

#include "FreeRTOS.h"
#include "queue.h"
#include "task.h"
#include <cstdio>
#include <cstdlib>
#include <malloc.h>

// Memory for a statically allocated task and queue: placed by the linker.
static StaticTask_t main_tcb;
static StackType_t main_stack[512];
static StaticQueue_t events_cb;
static uint8_t events_storage[8 * sizeof(uint32_t)];

static void main_task(void*) {
    size_t before = mallinfo().uordblks;              // bytes in use on the heap
    QueueHandle_t a = xQueueCreateStatic(8, sizeof(uint32_t), events_storage, &events_cb);
    size_t after_static = mallinfo().uordblks;
    QueueHandle_t b = xQueueCreate(8, sizeof(uint32_t));
    size_t after_dynamic = mallinfo().uordblks;
    std::printf("xQueueCreateStatic: %u heap bytes\n", unsigned(after_static - before));
    std::printf("xQueueCreate:       %u heap bytes\n", unsigned(after_dynamic - after_static));
    vQueueDelete(b);
    vQueueDelete(a);
    std::exit(0);
}

int main() {
    xTaskCreateStatic(main_task, "main", 512, nullptr, 1, main_stack, &main_tcb);
    vTaskStartScheduler();
}

xQueueCreateStatic: 0 heap bytes
xQueueCreate:       120 heap bytes

Static allocation fixes memory use at link time (the map file shows it), can't fail at run time, and can't fragment — which is why many firmware teams allow the heap only during initialisation, or not at all. The cost: you size every stack and queue yourself, and the memory must outlive the object (never a local array that goes out of scope). A good compromise is the pattern of the build task: a fixed pool of buffers created once, handed around by pointer.

8. Choosing a mechanism

The build task combines the first two: a pool of fixed buffers whose free list is itself a queue of pointers, and a bounded pipe that moves buffers from a producer to a consumer, with timeouts, backpressure statistics, no heap, and a stack budget the tests measure.

On real hardware

What changes when the same code runs on a physical Cortex-M board:

• The tick is a real timer. SysTick runs from the core clock, so configCPU_CLOCK_HZ must match the clock you actually configured; tick timing then depends on that oscillator's accuracy. Nothing fast-forwards: an idle board really waits, which is why real systems put the CPU to sleep in idle (an idle hook with wfi, or tickless idle).
• Execution times vary. Caches, flash wait states, bus contention and interrupt load change how long the same code takes; a schedule that is exact here becomes a distribution there. Measure worst cases on the target.
• A hardware watchdog resets the chip if the firmware stops kicking it; the usual pattern kicks only after checking that every task reported progress (the fault-recovery skill builds a software version).
• Tracing. Tools such as SEGGER SystemView or Percepio Tracealyzer record task switches, queue operations and interrupts from the running target and draw the timelines this lesson drew by hand.
• Measuring. Toggle a GPIO at the start and end of a task's work and watch it on a logic analyser or scope: real periods, jitter and latencies, independent of the software's own idea of time.
• Stack high-water marks and overflow checks work the same way — but measure them on the target build, with its compiler flags and libraries.