m³OS docs GitHub

Phase 37

I/O Multiplexing

Programs can wait on multiple file descriptors simultaneously without busy-waiting. `select()`, an improved `poll()`, and `epoll` provide scalable I/O readiness notification. Non-blocking I/O (`O_NONBLOCK`) works for sockets, pipes, and PTYs. Event-driven servers become possible.

Complete

Learning goal

Understand the I/O multiplexing problem: one thread, many connections.

Category

infrastructure

Why this phase matters

Programs can wait on multiple file descriptors simultaneously without busy-waiting. select(), an improved poll(), and epoll provide scalable I/O readiness notification. Non-blocking I/O (O_NONBLOCK) works for sockets, pipes, and PTYs. Event-driven servers become possible.

How it works in m3OS

The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.

How it builds on earlier phases

This phase builds on Expanded Memory and turns that foundation into I/O Multiplexing.

Implementation outline

  • Add nonblock flag to FdEntry and wire fcntl(F_GETFL/F_SETFL).
  • Implement O_NONBLOCK for pipes (simplest case).
  • Implement O_NONBLOCK for sockets.
  • Implement O_NONBLOCK for PTYs and TTY/stdin.
  • Add per-FD wait queues to pipes, sockets, PTYs, and stdin.
  • Rewrite poll() to use wait queues instead of busy-wait loop.
  • Implement select() on top of the improved poll infrastructure.
  • Implement epoll_create1, epoll_ctl, epoll_wait.
  • Implement accept4().
  • Write test programs: echo server using poll, epoll-based multi-client server.
  • Verify the telnet server works with improved poll (no busy-wait).

Task progress snapshot

122 of 122 tracked tasks are complete.

Documentation targets

  • Documentation deliverables have not been listed yet.

Deferred follow-on work

  • io_uring (submission/completion ring)
  • Edge-triggered epoll (EPOLLET) — complex semantics
  • signalfd, timerfd, eventfd
  • splice/tee/vmsplice (zero-copy)
  • POSIX AIO (aio_read, aio_write)
  • kqueue compatibility

Source materials

Important code

Code spotlight cards

Code spotlight

Add `nonblock` field to `FdEntry`

kernel/src/process/mod.rs L142-L156

FD is non-blocking. A per-FD flag is the simplest way to track this across all. Why it matters: Every read/write syscall path must be able to check whether the. Focus on `nonblock` and `FdEntry` in `kernel/src/process/mod.rs` while you trace the snippet.

/// A single open-file entry in the per-process FD table.
#[derive(Clone)]
pub struct FdEntry {
    pub backend: FdBackend,
    pub offset: usize,
    /// True if the file was opened for reading.
    pub readable: bool,
    /// True if the file was opened for writing.
    pub writable: bool,
    /// Close-on-exec flag (FD_CLOEXEC).
    pub cloexec: bool,
    /// Non-blocking I/O flag (O_NONBLOCK). When set, read/write return
    /// `-EAGAIN` instead of blocking when no data is available (Phase 37).
    pub nonblock: bool,
}

Code spotlight

Implement `fcntl(F_GETFL)` and `fcntl(F_SETFL)`

kernel/src/arch/x86_64/syscall.rs L7267-L7290

Userspace relies on fcntl(fd, F_SETFL, O_NONBLOCK) to toggle non-blocking mode,. Why it matters: The current F_GETFL/F_SETFL handlers are stubs that return 0. Focus on `fcntl(F_GETFL)` and `fcntl(F_SETFL)` in `kernel/src/arch/x86_64/syscall.rs` while you trace the snippet.

/// Minimal fcntl: F_DUPFD, F_GETFD, F_SETFD, F_GETFL, F_SETFL.
fn sys_fcntl(fd: u64, cmd: u64, arg: u64) -> u64 {
    const F_DUPFD: u64 = 0;
    const F_GETFD: u64 = 1;
    const F_SETFD: u64 = 2;
    const F_GETFL: u64 = 3;
    const F_SETFL: u64 = 4;
    const F_DUPFD_CLOEXEC: u64 = 1030;

    match cmd {
        F_DUPFD | F_DUPFD_CLOEXEC => {
            // Find the next free fd >= arg, duplicate oldfd into it.
            let set_cloexec = cmd == F_DUPFD_CLOEXEC;
            let oldfd = fd as usize;
            let min_fd = arg as usize;
            if oldfd >= MAX_FDS {
                return NEG_EBADF;
            }
            if min_fd >= MAX_FDS {
                return NEG_EINVAL;
            }
            let mut entry = match current_fd_entry(oldfd) {
                Some(e) => e,
                None => return NEG_EBADF,

Code spotlight

Add wait queue to pipe table

kernel/src/pipe.rs L149-L172

blocked in poll on the read end must be woken. A per-pipe wait queue provides. Why it matters: When data is written to a pipe (or a writer closes), tasks. Focus on `PIPES` and `WaitQueue` in `kernel/src/pipe.rs` while you trace the snippet.

/// Write to a pipe. Returns:
///   - `Ok(n)` — n bytes written
///   - `Err(false)` — reader closed (EPIPE)
///   - `Err(true)` — buffer full but reader still open (would block)
pub fn pipe_write(pipe_id: usize, src: &[u8]) -> Result<usize, bool> {
    let mut table = PIPE_TABLE.lock();
    let pipe = match table.get_mut(pipe_id).and_then(|p| p.as_mut()) {
        Some(p) => p,
        None => return Err(false),
    };

    if !pipe.has_reader() {
        return Err(false); // EPIPE
    }

    if pipe.is_full() {
        return Err(true); // would block
    }

    let n = pipe.write(src);
    drop(table);
    // Wake readers that may be waiting for data.
    wake_pipe(pipe_id);
    Ok(n)

Key files

  • kernel/src/process/mod.rs — The FdEntry struct in kernel/src/process/mod.rs gains a nonblock: bool field.

How real OSes differ

  • Linux has **io_uring** — the newest, highest-performance async I/O interface using submission/completion rings shared between kernel and userspace.
  • Linux's **epoll** is the standard for network servers (nginx, Node.js, etc.) and supports edge-triggered mode, `EPOLLONESHOT`, and `EPOLLEXCLUSIVE`.
  • **signalfd, timerfd, eventfd** unify signals, timers, and inter-thread events as regular file descriptors that work with epoll.
  • **splice/tee/vmsplice** enable zero-copy data movement between fds.
  • BSD uses **kqueue** instead of epoll — similar concept, different API, arguably cleaner design.
  • Real implementations handle thousands of concurrent fds; our MAX_FDS is 32.

Success criteria

  • `O_NONBLOCK` read on an empty pipe returns `EAGAIN` instead of blocking.
  • `poll()` blocks efficiently (no CPU spin) and wakes when data arrives.
  • `select()` works for read/write readiness on pipes and sockets.
  • `epoll_wait()` returns ready events for registered fds.
  • An epoll-based echo server handles 10+ simultaneous connections.
  • `accept4()` with `SOCK_NONBLOCK` creates non-blocking accepted sockets.
  • `fcntl(F_SETFL, O_NONBLOCK)` makes an existing fd non-blocking.
  • CPU usage is near zero when waiting for I/O (no busy-wait).
  • All existing tests pass without regression.

Related phases

Keep exploring the roadmap