m³OS docs GitHub

Phase 40

Threading Primitives

The OS supports kernel-level threads via `clone()` with `CLONE_THREAD` and `CLONE_VM`. Multiple threads share an address space and can synchronize via `futex()`. Thread-local storage works via `arch_prctl(ARCH_SET_FS)`. This enables multi-threaded C programs and improves musl libc compatibility (which uses threads internally even for single-threaded programs).

Complete

Learning goal

Understand the relationship between processes and threads at the kernel level: threads are tasks that share an address space.

Category

foundations

Why this phase matters

The OS supports kernel-level threads via clone() with CLONE_THREAD and CLONE_VM. Multiple threads share an address space and can synchronize via futex(). Thread-local storage works via arch_prctl(ARCH_SET_FS). This enables multi-threaded C programs and improves musl libc compatibility (which uses threads internally even for single-threaded programs).

How it works in m3OS

The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.

How it builds on earlier phases

This phase builds on Unix Domain Sockets and turns that foundation into Threading Primitives.

Implementation outline

  • Add tid, tgid, and clear_child_tid fields to Process; implement gettid().
  • Implement ThreadGroup struct and membership tracking.
  • Extend clone() to accept CLONE_VM | CLONE_THREAD | CLONE_FILES | CLONE_SIGHAND.
  • Allocate per-thread kernel stacks; share page table root with CLONE_VM.
  • Implement CLONE_SETTLS (set FS base for new thread).
  • Implement CLONE_CHILD_CLEARTID and set_tid_address().
  • Implement FUTEX_WAIT and FUTEX_WAKE with proper wait queues.
  • Implement FUTEX_WAIT_BITSET and FUTEX_WAKE_BITSET.
  • Update exit() for thread-only exit vs exit_group() for group exit.
  • Update signal delivery for thread groups (tkill, process-directed signals).
  • Build userspace thread test program.
  • Validate with musl pthread_create / pthread_join if possible.

Task progress snapshot

103 of 103 tracked tasks are complete.

Documentation targets

  • Documentation deliverables have not been listed yet.

Deferred follow-on work

  • Robust futexes (auto-release on death)
  • Priority inheritance (PI) futexes
  • clone3() syscall
  • Thread naming (prctl PR_SET_NAME)
  • Per-thread CPU affinity (extend existing per-process affinity)
  • POSIX thread cancellation
  • Per-thread signal masks
  • User-level threading (M:N model)
  • FUTEX_REQUEUE / FUTEX_CMP_REQUEUE / FUTEX_WAKE_OP

Source materials

Important code

Code spotlight cards

Code spotlight

Add `tid` and `tgid` fields to `Process`

kernel/src/process/mod.rs L510-L533

of thread identity. Every thread-aware syscall (gettid, set_tid_address,. Why it matters: The current Process struct has only pid and no concept. Focus on `tid` and `tgid` in `kernel/src/process/mod.rs` while you trace the snippet.

/// Descriptor for a single userspace process.
pub struct Process {
    /// This process's unique identifier.
    pub pid: Pid,
    pub tid: u32,
    pub tgid: u32,
    pub clear_child_tid: u64,
    /// Parent PID.  0 means no parent (init or an orphan).
    pub ppid: Pid,
    /// Current lifecycle state.
    pub state: ProcessState,
    /// Root physical address of this process's page table.
    ///
    /// `None` means the process has not yet been assigned a dedicated
    /// address space and shares the kernel's mappings (pre-Phase 12).
    pub page_table_root: Option<x86_64::PhysAddr>,
    /// Top of this process's kernel-mode stack (virtual address).
    pub kernel_stack_top: u64,
    /// Userspace entry-point virtual address.
    pub entry_point: u64,
    /// Initial userspace RSP: the virtual address of `argc` on the ABI stack,
    /// as returned by `setup_abi_stack`.  This is **not** the raw top of the
    /// stack allocation — it points into the stack, below the argv/envp data.
    pub user_stack_top: u64,

Code spotlight

Implement `gettid()` as a distinct syscall

kernel/src/arch/x86_64/syscall.rs L1446-L1454

gettid() to return their unique TID so musl can track thread identity for. Why it matters: Syscall 186 currently aliases to sys_getpid(). Threads need. Focus on `gettid` and `sys_gettid` in `kernel/src/arch/x86_64/syscall.rs` while you trace the snippet.

/// `gettid()` — return the calling thread's unique thread ID.
fn sys_gettid() -> u64 {
    let pid = crate::process::current_pid();
    crate::process::PROCESS_TABLE
        .lock()
        .find(pid)
        .map(|p| p.tid as u64)
        .unwrap_or(pid as u64)
}

Code spotlight

Define futex wait queue table

kernel/src/process/futex.rs L46-L48

synchronization. Every pthread_mutex_lock that contends, every pthread_cond_wait,. Why it matters: The futex table is the core kernel data structure for thread. Focus on `FUTEX_TABLE` and `FutexWaiter` in `kernel/src/process/futex.rs` while you trace the snippet.

/// Global table of futex wait queues, keyed by `(page_table_root, vaddr)`.
pub static FUTEX_TABLE: Lazy<Mutex<BTreeMap<FutexKey, Vec<FutexWaiter>>>> =
    Lazy::new(|| Mutex::new(BTreeMap::new()));

Key files

  • kernel/src/process/mod.rs — The Process struct in kernel/src/process/mod.rs gains tid and tgid fields.
  • kernel/src/task/mod.rs — The Task struct in kernel/src/task/mod.rs is unchanged — it already has pid

How real OSes differ

  • **NPTL** (Native POSIX Threads Library) -- musl and glibc implement pthreads using `clone()` + `futex()`, exactly the mechanism we implement.
  • **Robust futexes** -- automatically release on thread death; we defer this.
  • **Priority inheritance futexes** -- prevent priority inversion; we defer this.
  • **Thread-specific signal masks** via per-thread `rt_sigprocmask`; we use process-wide masks initially.
  • Linux uses `clone3()` (syscall 435) as the modern replacement for `clone()`; we use the classic `clone()`.
  • Real kernels use a hash table with bucket locks for the futex table; we use a single `BTreeMap` under one lock.

Success criteria

  • `clone(CLONE_VM | CLONE_THREAD | ...)` creates a thread sharing the parent's address space.
  • `gettid()` returns different values for threads in the same group.
  • `getpid()` returns the same value for all threads in a group.
  • `futex(FUTEX_WAIT)` sleeps and `futex(FUTEX_WAKE)` wakes the sleeper.
  • `pthread_mutex_lock/unlock` works (built on futex).
  • Thread-local variables work via per-thread FS base.
  • `set_tid_address()` stores the clear_child_tid pointer and wakes futex on thread exit.
  • `exit()` (syscall 60) terminates only the calling thread.
  • `exit_group()` (syscall 231) terminates all threads in the group.
  • Multiple threads on different cores can run simultaneously.
  • All existing tests pass (single-threaded programs unaffected).

Related phases

Keep exploring the roadmap