Phase 37
I/O Multiplexing
Programs can wait on multiple file descriptors simultaneously without busy-waiting. `select()`, an improved `poll()`, and `epoll` provide scalable I/O readiness notification. Non-blocking I/O (`O_NONBLOCK`) works for sockets, pipes, and PTYs. Event-driven servers become possible.
Learning goal
Understand the I/O multiplexing problem: one thread, many connections.
Category
infrastructure
Components touched
Builds on
Unlocked by this phase
Why this phase matters
Programs can wait on multiple file descriptors simultaneously without busy-waiting. select(), an improved poll(), and epoll provide scalable I/O readiness notification. Non-blocking I/O (O_NONBLOCK) works for sockets, pipes, and PTYs. Event-driven servers become possible.
How it works in m3OS
The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.
How it builds on earlier phases
This phase builds on Expanded Memory and turns that foundation into I/O Multiplexing.
Implementation outline
- Add
nonblockflag toFdEntryand wirefcntl(F_GETFL/F_SETFL). - Implement
O_NONBLOCKfor pipes (simplest case). - Implement
O_NONBLOCKfor sockets. - Implement
O_NONBLOCKfor PTYs and TTY/stdin. - Add per-FD wait queues to pipes, sockets, PTYs, and stdin.
- Rewrite
poll()to use wait queues instead of busy-wait loop. - Implement
select()on top of the improved poll infrastructure. - Implement
epoll_create1,epoll_ctl,epoll_wait. - Implement
accept4(). - Write test programs: echo server using poll, epoll-based multi-client server.
- Verify the telnet server works with improved poll (no busy-wait).
Task progress snapshot
122 of 122 tracked tasks are complete.
Documentation targets
- Documentation deliverables have not been listed yet.
Deferred follow-on work
- io_uring (submission/completion ring)
- Edge-triggered epoll (EPOLLET) — complex semantics
- signalfd, timerfd, eventfd
- splice/tee/vmsplice (zero-copy)
- POSIX AIO (aio_read, aio_write)
- kqueue compatibility
Source materials
Important code
Code spotlight cards
Code spotlight
Add `nonblock` field to `FdEntry`
FD is non-blocking. A per-FD flag is the simplest way to track this across all. Why it matters: Every read/write syscall path must be able to check whether the. Focus on `nonblock` and `FdEntry` in `kernel/src/process/mod.rs` while you trace the snippet.
/// A single open-file entry in the per-process FD table.
#[derive(Clone)]
pub struct FdEntry {
pub backend: FdBackend,
pub offset: usize,
/// True if the file was opened for reading.
pub readable: bool,
/// True if the file was opened for writing.
pub writable: bool,
/// Close-on-exec flag (FD_CLOEXEC).
pub cloexec: bool,
/// Non-blocking I/O flag (O_NONBLOCK). When set, read/write return
/// `-EAGAIN` instead of blocking when no data is available (Phase 37).
pub nonblock: bool,
} Code spotlight
Implement `fcntl(F_GETFL)` and `fcntl(F_SETFL)`
Userspace relies on fcntl(fd, F_SETFL, O_NONBLOCK) to toggle non-blocking mode,. Why it matters: The current F_GETFL/F_SETFL handlers are stubs that return 0. Focus on `fcntl(F_GETFL)` and `fcntl(F_SETFL)` in `kernel/src/arch/x86_64/syscall.rs` while you trace the snippet.
/// Minimal fcntl: F_DUPFD, F_GETFD, F_SETFD, F_GETFL, F_SETFL.
fn sys_fcntl(fd: u64, cmd: u64, arg: u64) -> u64 {
const F_DUPFD: u64 = 0;
const F_GETFD: u64 = 1;
const F_SETFD: u64 = 2;
const F_GETFL: u64 = 3;
const F_SETFL: u64 = 4;
const F_DUPFD_CLOEXEC: u64 = 1030;
match cmd {
F_DUPFD | F_DUPFD_CLOEXEC => {
// Find the next free fd >= arg, duplicate oldfd into it.
let set_cloexec = cmd == F_DUPFD_CLOEXEC;
let oldfd = fd as usize;
let min_fd = arg as usize;
if oldfd >= MAX_FDS {
return NEG_EBADF;
}
if min_fd >= MAX_FDS {
return NEG_EINVAL;
}
let mut entry = match current_fd_entry(oldfd) {
Some(e) => e,
None => return NEG_EBADF, Code spotlight
Add wait queue to pipe table
blocked in poll on the read end must be woken. A per-pipe wait queue provides. Why it matters: When data is written to a pipe (or a writer closes), tasks. Focus on `PIPES` and `WaitQueue` in `kernel/src/pipe.rs` while you trace the snippet.
/// Write to a pipe. Returns:
/// - `Ok(n)` — n bytes written
/// - `Err(false)` — reader closed (EPIPE)
/// - `Err(true)` — buffer full but reader still open (would block)
pub fn pipe_write(pipe_id: usize, src: &[u8]) -> Result<usize, bool> {
let mut table = PIPE_TABLE.lock();
let pipe = match table.get_mut(pipe_id).and_then(|p| p.as_mut()) {
Some(p) => p,
None => return Err(false),
};
if !pipe.has_reader() {
return Err(false); // EPIPE
}
if pipe.is_full() {
return Err(true); // would block
}
let n = pipe.write(src);
drop(table);
// Wake readers that may be waiting for data.
wake_pipe(pipe_id);
Ok(n) Key files
- kernel/src/process/mod.rs — The FdEntry struct in kernel/src/process/mod.rs gains a nonblock: bool field.
How real OSes differ
- Linux has **io_uring** — the newest, highest-performance async I/O interface using submission/completion rings shared between kernel and userspace.
- Linux's **epoll** is the standard for network servers (nginx, Node.js, etc.) and supports edge-triggered mode, `EPOLLONESHOT`, and `EPOLLEXCLUSIVE`.
- **signalfd, timerfd, eventfd** unify signals, timers, and inter-thread events as regular file descriptors that work with epoll.
- **splice/tee/vmsplice** enable zero-copy data movement between fds.
- BSD uses **kqueue** instead of epoll — similar concept, different API, arguably cleaner design.
- Real implementations handle thousands of concurrent fds; our MAX_FDS is 32.
Success criteria
- `O_NONBLOCK` read on an empty pipe returns `EAGAIN` instead of blocking.
- `poll()` blocks efficiently (no CPU spin) and wakes when data arrives.
- `select()` works for read/write readiness on pipes and sockets.
- `epoll_wait()` returns ready events for registered fds.
- An epoll-based echo server handles 10+ simultaneous connections.
- `accept4()` with `SOCK_NONBLOCK` creates non-blocking accepted sockets.
- `fcntl(F_SETFL, O_NONBLOCK)` makes an existing fd non-blocking.
- CPU usage is near zero when waiting for I/O (no busy-wait).
- All existing tests pass without regression.
Related phases
Keep exploring the roadmap
Phase 36
CompleteExpanded Memory
The kernel supports demand paging, large `mmap()` regions, and `mprotect()`. Virtual pages are mapped lazily — physical frames are allocated on first access via the page fault handler. This unlocks running large cross-compiled binaries (Clang, Python, Node.js, git) that allocate hundreds of megabytes via `mmap()` but only touch a fraction of the mapped pages. QEMU RAM is increased to support these workloads.
Phase 38
CompleteFilesystem Enhancements
The filesystem supports symlinks, hard links, `umask`-backed creation defaults, the remaining Unix permission semantics needed for symlink-aware metadata and directory listing, a `/proc` filesystem for process introspection, and basic device nodes (`/dev/null`, `/dev/zero`, `/dev/urandom`, `/dev/full`). These are the filesystem primitives that Unix tools and build systems assume exist.