m³OS docs GitHub

Phase 22

TTY PTY

Give userspace programs a real terminal abstraction. The kernel's raw stdin buffer and hardcoded `stdin_feeder_task` are replaced by a proper TTY layer with a configurable line discipline. Programs can switch between cooked and raw mode via `tcgetattr` / `tcsetattr`, query the window size via `TIOCGWINSZ`, and detect whether they are attached to a terminal via `isatty`. The phase also lays the infrastructure for a future PTY master/slave pair.

Complete

Learning goal

Understand what a line discipline is and why it lives between the keyboard driver and userspace.

Category

userspace

Why this phase matters

Give userspace programs a real terminal abstraction. The kernel’s raw stdin buffer and hardcoded stdin_feeder_task are replaced by a proper TTY layer with a configurable line discipline. Programs can switch between cooked and raw mode via tcgetattr / tcsetattr, query the window size via TIOCGWINSZ, and detect whether they are attached to a terminal via isatty. The phase also lays the infrastructure for a future PTY master/slave pair.

How it works in m3OS

The roadmap entry defines the learning scope for this phase; a deeper implementation doc has not been matched yet.

How it builds on earlier phases

This phase builds on Ion Shell and turns that foundation into TTY PTY.

Implementation outline

  • Define the Termios struct in a shared userspace library crate matching the Linux layout (c_iflag, c_oflag, c_cflag, c_lflag, c_cc[NCCS]). Use the same field sizes and byte order as the Linux x86_64 ABI so that musl-compiled code can pass the struct directly through ioctl.
  • Create tty_server as a ring-3 server binary. Add it to the xtask image build so it is included in the disk image. Have init spawn it before the shell and wire stdin/stdout fds through it via capability grants.
  • Implement the cooked-mode line discipline in tty_server: read raw bytes from kbd_server, maintain an edit buffer, apply erase (^H / DEL), kill (^U), and word-erase (^W) processing, and deliver a complete line to the read buffer when \n or ^D (EOF) is received.
  • Implement echo: in cooked mode, each accepted character is written back to console_server. Echo is suppressed when ECHO is clear in c_lflag.
  • Implement raw mode: when ICANON is clear in c_lflag, bypass line buffering and deliver bytes to the read buffer immediately; respect VMIN and VTIME from c_cc to control minimum read size and timeout.
  • Implement cbreak mode: ICANON clear but ISIG set, so ^C and ^Z still generate signals while reads return individual bytes.
  • Add ioctl dispatch in the syscall gate for TCGETS (0x5401), TCSETS (0x5402), TCSETSW (0x5403), TCSETSF (0x5404); route each to tty_server via a synchronous IPC call and copy the Termios struct through a shared page.
  • Implement TIOCGWINSZ (0x5413) and TIOCSWINSZ (0x5414) storage in tty_server. When TIOCSWINSZ is called with new dimensions, send SIGWINCH to the foreground process group.
  • Fix isatty path: ioctl(fd, TCGETS, NULL) on a non-TTY fd must return ENOTTY (-25). Ensure the fd table distinguishes TTY fds from plain file fds so the syscall gate returns the right error without forwarding to tty_server.
  • Add PTY allocation stubs: posix_openpt opens /dev/ptmx and allocates a PtyPair in tty_server; grantpt / unlockpt are no-ops; ptsname returns a /dev/pts/N path string. Opening the slave path returns a second fd backed by the same PtyPair. Full bidirectional data routing between master and slave is deferred.
  • Regression-test the shell: readline-style editing, Ctrl-C, Ctrl-Z, Ctrl-D (EOF), and piped input all work correctly through the new layer without behavioral changes compared to Phase 20.

Task progress snapshot

No checklist tasks were discovered for this phase yet.

Documentation targets

  • Documentation deliverables have not been listed yet.

Deferred follow-on work

  • Full PTY data path between master and slave (needed for terminal multiplexers, SSH, script)
  • /dev/pts devpts filesystem (dynamic slave device nodes)
  • VSTART / VSTOP (XON/XOFF) software flow control
  • Modem control lines (DTR, RTS, CTS) and TIOCMGET / TIOCMSET
  • Session leaders and controlling terminals (TIOCSCTTY, setsid)
  • Packet mode (TIOCPKT) for terminal multiplexers
  • Multiple simultaneous TTY instances beyond the single console TTY
  • openpty libc helper (depends on full PTY data path)

Source materials

Important code

Code spotlight cards

Code spotlight

Host-testable CSI state machine in kernel-core/src/fb.rs

kernel-core/src/fb.rs See file

Host-testable CSI state machine in kernel-core/src/fb.rs

Code spotlight

kernel/src/tty.rs — TTY0 global state

kernel/src/tty.rs See file

kernel/src/tty.rs — TTY0 global state

Code spotlight

kernel-core/src/tty.rs — Termios, Winsize, EditBuffer, constants

kernel-core/src/tty.rs See file

kernel-core/src/tty.rs — Termios, Winsize, EditBuffer, constants

Key files

  • kernel-core/src/fb.rs — Host-testable CSI state machine in kernel-core/src/fb.rs
  • kernel/src/tty.rs — kernel/src/tty.rs — TTY0 global state
  • kernel-core/src/tty.rs — kernel-core/src/tty.rs — Termios, Winsize, EditBuffer, constants
  • kernel/src/pty.rs — kernel/src/pty.rs — PTY table, lifecycle, wait queues
  • kernel-core/src/pty.rs — kernel-core/src/pty.rs — PtyRingBuffer, PtyPairState
  • kernel/src/arch/x86_64/interrupts.rs — kernel/src/arch/x86_64/interrupts.rs — Keyboard IRQ handler, scancode buffers
  • kernel-core/src/input.rs — kernel-core/src/input.rs — ScancodeRouter
  • kernel/src/stdin.rs — kernel/src/stdin.rs — Kernel stdin buffer
  • kernel/src/serial.rs — kernel/src/serial.rs — UART 16550 driver, serial RX ring buffer
  • kernel/src/fb/mod.rs — kernel/src/fb/mod.rs — Framebuffer console
  • kernel/src/main.rs — kernel/src/main.rs — serial_stdin_feeder_task (kernel-space line discipline)
  • userspace/stdin_feeder/src/main.rs — userspace/stdin_feeder/src/main.rs — Keyboard line discipline (userspace)
  • userspace/kbd_server/src/main.rs — userspace/kbd_server/src/main.rs — Keyboard IPC server
  • kernel/src/main.rs:538-700 — kernel/src/main.rs:538-700 (serial) and userspace/stdin_feeder/src/main.rs (keyboard) — two independent implementations of ICRNL, ISIG, canonical editing, echo.
  • userspace/stdin_feeder/src/main.rs:220 — userspace/stdin_feeder/src/main.rs:220 — tcgetattr(0) called once at startup. c_cc array never refreshed.
  • kernel-core/src/pty.rs:15 — MAX_PTYS = 16 in kernel-core/src/pty.rs:15. Fixed array, no dynamic growth.
  • kernel/src/arch/x86_64/syscall/mod.rs — kernel/src/arch/x86_64/syscall/mod.rs — TCSETSW treated identically to TCSETS.

How real OSes differ

  • Linux's TTY subsystem is one of the oldest and most complex parts of the kernel. The line discipline (`N_TTY` and others) runs in ring 0, not in a userspace server.
  • The PTY implementation in `drivers/tty/pty.c` handles hundreds of `ioctl` codes, flow control via `VSTART`/`VSTOP` (XON/XOFF), modem control lines, packet mode, and the `/dev/pts` devpts filesystem.
  • Real terminal emulators (xterm, alacritty) open `/dev/ptmx`, fork a child with the slave as its controlling terminal, and process VT100/ANSI escape sequences.
  • This phase implements only the subset needed for an interactive shell: termios cooked/raw switching, window size, and skeleton PTY allocation. A full escape-sequence processor and devpts filesystem are deferred.

Success criteria

  • The shell's line editing (`^H`, `^U`, `^W`, `^C`, `^D`) works identically to before, now routed through `tty_server`.
  • A program that calls `tcgetattr` receives a valid `termios` struct; calling `tcsetattr(TCSANOW, &raw)` switches to raw mode and subsequent reads return individual bytes without echo.
  • Calling `tcsetattr` to restore the saved `termios` returns the terminal to cooked mode.
  • `ioctl(STDOUT_FILENO, TIOCGWINSZ, &ws)` returns the correct `ws_col` and `ws_row` values.
  • `isatty(0)` returns 1 when stdin is the TTY; `isatty(3)` on a plain open file returns 0.
  • `SIGWINCH` is delivered to the shell when `TIOCSWINSZ` is called with new dimensions.
  • The existing shell and all Phase 14 utilities pass their acceptance criteria without regression.

Related phases

Keep exploring the roadmap