m³OS docs GitHub

Phase 14

Shell And Tools

Turn the minimal shell from Phase 9 into a genuinely useful interactive environment: pipes, I/O redirection, job control, environment variables, and a small set of core utilities that exercise the filesystem and process model.

Complete

Learning goal

Understand how a shell implements pipes using file descriptors and `fork`.

Category

productivity

Why this phase matters

Turn the minimal shell from Phase 9 into a genuinely useful interactive environment: pipes, I/O redirection, job control, environment variables, and a small set of core utilities that exercise the filesystem and process model.

How it works in m3OS

The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.

How it builds on earlier phases

This phase builds on Writable FS and turns that foundation into Shell And Tools.

Implementation outline

  • Add file descriptor table per process: integers mapping to open file descriptions.
  • Implement pipe syscall: allocate a kernel pipe buffer, return two fds.
  • Implement dup2 syscall: redirect one fd to another before execve.
  • Add signal infrastructure: SIGINT, SIGTSTP, SIGCONT, SIGCHLD at minimum.
  • Wire Ctrl-C and Ctrl-Z in the keyboard server to deliver signals to the foreground process group.
  • Implement process groups so the shell can address foreground and background jobs.
  • Extend the shell parser to handle |, >, <, >>, &, and variable expansion.
  • Compile each utility as a standalone static ELF and add to the disk image.

Task progress snapshot

90 of 90 tracked tasks are complete.

Documentation targets

  • Documentation deliverables have not been listed yet.

Deferred follow-on work

  • stderr redirection and 2>&1
  • pipelines longer than two stages
  • subshells $(...) and backtick expansion
  • here-documents
  • trap built-in
  • shell scripting (loops, conditionals, functions)
  • tab completion

Source materials

Important code

Code spotlight cards

Code spotlight

Add fd_table field to Process struct

kernel/src/process/mod.rs L252-L275

Add fd_table field to Process struct. Why it matters: Per-process file descriptors are required for correct POSIX semantics where each process has independent FD state. Focus on `Process::fd_table` and `Process` in `kernel/src/process/mod.rs` while you trace the snippet.

/// Descriptor for a single userspace process.
pub struct Process {
    /// This process's unique identifier.
    pub pid: Pid,
    /// Parent PID.  0 means no parent (init or an orphan).
    pub ppid: Pid,
    /// Current lifecycle state.
    pub state: ProcessState,
    /// Root physical address of this process's page table.
    ///
    /// `None` means the process has not yet been assigned a dedicated
    /// address space and shares the kernel's mappings (pre-Phase 12).
    pub page_table_root: Option<x86_64::PhysAddr>,
    /// Top of this process's kernel-mode stack (virtual address).
    pub kernel_stack_top: u64,
    /// Userspace entry-point virtual address.
    pub entry_point: u64,
    /// Initial userspace RSP: the virtual address of `argc` on the ABI stack,
    /// as returned by `setup_abi_stack`.  This is **not** the raw top of the
    /// stack allocation — it points into the stack, below the argv/envp data.
    pub user_stack_top: u64,
    /// Exit code written when the process transitions to [`ProcessState::Zombie`].
    pub exit_code: Option<i32>,
    /// Signal that caused the process to stop (set when transitioning to Stopped).

Code spotlight

Deep-clone fd_table in sys_fork

kernel/src/arch/x86_64/syscall.rs L715-L738

Deep-clone fd_table in sys_fork. Why it matters: Forked children must inherit the parent's open file descriptors so pipes and redirections work across fork boundaries. Focus on `sys_fork` and `fd_table` in `kernel/src/arch/x86_64/syscall.rs` while you trace the snippet.

/// `fork()` — create a child process that resumes after the syscall with rax=0.
///
/// Allocates a fresh page table for the child (eager copy of user pages),
/// registers the child in the process table, and spawns a kernel task whose
/// entry function enters ring 3 at `user_rip` with `user_rsp` and rax=0.
///
/// Returns the child PID to the parent.
fn sys_fork(user_rip: u64, user_rsp: u64) -> u64 {
    let parent_pid = crate::process::CURRENT_PID.load(core::sync::atomic::Ordering::Relaxed);
    log::info!("[p{}] fork()", parent_pid);

    // Allocate a new page table for the child, copying kernel entries.
    let child_cr3 = match crate::mm::new_process_page_table() {
        Some(f) => f,
        None => {
            log::warn!("[fork] out of frames for child page table");
            return u64::MAX;
        }
    };

    // Copy all user-accessible pages from parent into child's page table.
    let phys_off = crate::mm::phys_offset();
    {
        // SAFETY: child_cr3 was just allocated; no other mapper over it exists.

Code spotlight

Define Pipe struct

kernel/src/pipe.rs L19-L30

Define Pipe struct. Why it matters: The kernel pipe buffer is the foundation for inter-process communication via the shell pipeline operator. Focus on `Pipe` in `kernel/src/pipe.rs` while you trace the snippet.

/// A kernel pipe: ring buffer with ref-counted reader/writer ends.
pub struct Pipe {
    buf: [u8; PIPE_BUF_SIZE],
    /// Read position in the ring buffer.
    read_pos: usize,
    /// Number of valid bytes in the buffer (0 = empty, PIPE_BUF_SIZE = full).
    count: usize,
    /// Number of open read-end references (FDs pointing to PipeRead for this pipe).
    pub reader_count: u32,
    /// Number of open write-end references (FDs pointing to PipeWrite for this pipe).
    pub writer_count: u32,
}

Key files

  • userspace/coreutils/ — 14 standalone musl-linked static ELF binaries in userspace/coreutils/:
  • kernel/src/pipe.rs — kernel/src/pipe.rs — kernel pipe implementation
  • kernel/src/stdin.rs — kernel/src/stdin.rs — kernel stdin buffer with line editing
  • userspace/coreutils/*.c — userspace/coreutils/*.c — 14 core utility C programs
  • kernel/src/process/mod.rs — kernel/src/process/mod.rs — FdEntry/FdBackend types, fd_table, pgid,
  • kernel/src/arch/x86_64/syscall.rs — kernel/src/arch/x86_64/syscall.rs — per-process FD access, pipe/dup2,
  • kernel/src/mm/elf.rs — kernel/src/mm/elf.rs — setup_abi_stack_with_envp (envp support)
  • kernel/src/main.rs — kernel/src/main.rs — shell rewrite, stdin_feeder_task
  • kernel/src/fs/ramdisk.rs — kernel/src/fs/ramdisk.rs — 14 new ELF entries
  • xtask/src/main.rs — xtask/src/main.rs — 14 new musl build entries

How real OSes differ

  • Real shells (bash, zsh) handle dozens of redirection forms, subshells, here-documents, arithmetic expansion, glob expansion, and complex job control edge cases built up over decades. POSIX specifies the full signal and job control semantics in fine detail. This phase implements the minimum needed to make the shell feel real. Edge cases — redirecting stderr, pipelines with more than two stages, `trap` built-ins — are intentionally deferred.

Success criteria

  • `ls | grep txt` produces the filtered listing.
  • `cat file.txt > copy.txt` creates a copy via redirection.
  • `Ctrl-C` kills the foreground command without killing the shell.
  • `sleep 10 &` runs in the background; the shell remains responsive.
  • `fg` brings the background job back to the foreground.
  • `export PATH=/bin` and then running `ls` without a full path works.
  • All listed utility binaries run and produce correct output.

Related phases

Keep exploring the roadmap