Phase 14
Shell And Tools
Turn the minimal shell from Phase 9 into a genuinely useful interactive environment: pipes, I/O redirection, job control, environment variables, and a small set of core utilities that exercise the filesystem and process model.
Learning goal
Understand how a shell implements pipes using file descriptors and `fork`.
Category
productivity
Components touched
Builds on
Unlocked by this phase
Why this phase matters
Turn the minimal shell from Phase 9 into a genuinely useful interactive environment: pipes, I/O redirection, job control, environment variables, and a small set of core utilities that exercise the filesystem and process model.
How it works in m3OS
The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.
How it builds on earlier phases
This phase builds on Writable FS and turns that foundation into Shell And Tools.
Implementation outline
- Add file descriptor table per process: integers mapping to open file descriptions.
- Implement
pipesyscall: allocate a kernel pipe buffer, return two fds. - Implement
dup2syscall: redirect one fd to another beforeexecve. - Add signal infrastructure:
SIGINT,SIGTSTP,SIGCONT,SIGCHLDat minimum. - Wire
Ctrl-CandCtrl-Zin the keyboard server to deliver signals to the foreground process group. - Implement process groups so the shell can address foreground and background jobs.
- Extend the shell parser to handle
|,>,<,>>,&, and variable expansion. - Compile each utility as a standalone static ELF and add to the disk image.
Task progress snapshot
90 of 90 tracked tasks are complete.
Documentation targets
- Documentation deliverables have not been listed yet.
Deferred follow-on work
- stderr redirection and
2>&1 - pipelines longer than two stages
- subshells
$(...)and backtick expansion - here-documents
trapbuilt-in- shell scripting (loops, conditionals, functions)
- tab completion
Source materials
Important code
Code spotlight cards
Code spotlight
Add fd_table field to Process struct
Add fd_table field to Process struct. Why it matters: Per-process file descriptors are required for correct POSIX semantics where each process has independent FD state. Focus on `Process::fd_table` and `Process` in `kernel/src/process/mod.rs` while you trace the snippet.
/// Descriptor for a single userspace process.
pub struct Process {
/// This process's unique identifier.
pub pid: Pid,
/// Parent PID. 0 means no parent (init or an orphan).
pub ppid: Pid,
/// Current lifecycle state.
pub state: ProcessState,
/// Root physical address of this process's page table.
///
/// `None` means the process has not yet been assigned a dedicated
/// address space and shares the kernel's mappings (pre-Phase 12).
pub page_table_root: Option<x86_64::PhysAddr>,
/// Top of this process's kernel-mode stack (virtual address).
pub kernel_stack_top: u64,
/// Userspace entry-point virtual address.
pub entry_point: u64,
/// Initial userspace RSP: the virtual address of `argc` on the ABI stack,
/// as returned by `setup_abi_stack`. This is **not** the raw top of the
/// stack allocation — it points into the stack, below the argv/envp data.
pub user_stack_top: u64,
/// Exit code written when the process transitions to [`ProcessState::Zombie`].
pub exit_code: Option<i32>,
/// Signal that caused the process to stop (set when transitioning to Stopped). Code spotlight
Deep-clone fd_table in sys_fork
Deep-clone fd_table in sys_fork. Why it matters: Forked children must inherit the parent's open file descriptors so pipes and redirections work across fork boundaries. Focus on `sys_fork` and `fd_table` in `kernel/src/arch/x86_64/syscall.rs` while you trace the snippet.
/// `fork()` — create a child process that resumes after the syscall with rax=0.
///
/// Allocates a fresh page table for the child (eager copy of user pages),
/// registers the child in the process table, and spawns a kernel task whose
/// entry function enters ring 3 at `user_rip` with `user_rsp` and rax=0.
///
/// Returns the child PID to the parent.
fn sys_fork(user_rip: u64, user_rsp: u64) -> u64 {
let parent_pid = crate::process::CURRENT_PID.load(core::sync::atomic::Ordering::Relaxed);
log::info!("[p{}] fork()", parent_pid);
// Allocate a new page table for the child, copying kernel entries.
let child_cr3 = match crate::mm::new_process_page_table() {
Some(f) => f,
None => {
log::warn!("[fork] out of frames for child page table");
return u64::MAX;
}
};
// Copy all user-accessible pages from parent into child's page table.
let phys_off = crate::mm::phys_offset();
{
// SAFETY: child_cr3 was just allocated; no other mapper over it exists. Code spotlight
Define Pipe struct
Define Pipe struct. Why it matters: The kernel pipe buffer is the foundation for inter-process communication via the shell pipeline operator. Focus on `Pipe` in `kernel/src/pipe.rs` while you trace the snippet.
/// A kernel pipe: ring buffer with ref-counted reader/writer ends.
pub struct Pipe {
buf: [u8; PIPE_BUF_SIZE],
/// Read position in the ring buffer.
read_pos: usize,
/// Number of valid bytes in the buffer (0 = empty, PIPE_BUF_SIZE = full).
count: usize,
/// Number of open read-end references (FDs pointing to PipeRead for this pipe).
pub reader_count: u32,
/// Number of open write-end references (FDs pointing to PipeWrite for this pipe).
pub writer_count: u32,
} Key files
- userspace/coreutils/ — 14 standalone musl-linked static ELF binaries in userspace/coreutils/:
- kernel/src/pipe.rs — kernel/src/pipe.rs — kernel pipe implementation
- kernel/src/stdin.rs — kernel/src/stdin.rs — kernel stdin buffer with line editing
- userspace/coreutils/*.c — userspace/coreutils/*.c — 14 core utility C programs
- kernel/src/process/mod.rs — kernel/src/process/mod.rs — FdEntry/FdBackend types, fd_table, pgid,
- kernel/src/arch/x86_64/syscall.rs — kernel/src/arch/x86_64/syscall.rs — per-process FD access, pipe/dup2,
- kernel/src/mm/elf.rs — kernel/src/mm/elf.rs — setup_abi_stack_with_envp (envp support)
- kernel/src/main.rs — kernel/src/main.rs — shell rewrite, stdin_feeder_task
- kernel/src/fs/ramdisk.rs — kernel/src/fs/ramdisk.rs — 14 new ELF entries
- xtask/src/main.rs — xtask/src/main.rs — 14 new musl build entries
How real OSes differ
- Real shells (bash, zsh) handle dozens of redirection forms, subshells, here-documents, arithmetic expansion, glob expansion, and complex job control edge cases built up over decades. POSIX specifies the full signal and job control semantics in fine detail. This phase implements the minimum needed to make the shell feel real. Edge cases — redirecting stderr, pipelines with more than two stages, `trap` built-ins — are intentionally deferred.
Success criteria
- `ls | grep txt` produces the filtered listing.
- `cat file.txt > copy.txt` creates a copy via redirection.
- `Ctrl-C` kills the foreground command without killing the shell.
- `sleep 10 &` runs in the background; the shell remains responsive.
- `fg` brings the background job back to the foreground.
- `export PATH=/bin` and then running `ls` without a full path works.
- All listed utility binaries run and produce correct output.
Related phases
Keep exploring the roadmap
Phase 13
CompleteWritable FS
Allow processes to create, write, and delete files so the OS has persistent and ephemeral mutable storage. A RAM-backed tmpfs provides scratch space; FAT32 gains write support for persistence across reboots.
Phase 15
CompleteHardware Discovery
Replace the hardcoded, single-core interrupt model with proper hardware discovery via ACPI and PCI, and switch from the legacy 8259 PIC to the APIC. This unlocks multi-core support and arbitrary device discovery in later phases.