m³OS docs GitHub

Phase 29

PTY Subsystem

Implement real pseudo-terminal (PTY) pairs so that multiple independent terminal sessions can run simultaneously. A PTY master/slave pair connects a terminal emulator (or network server) to a shell process, enabling remote access (telnet, SSH) and local terminal multiplexing.

Complete

Learning goal

Understand the Unix PTY model: master side (application) and slave side (shell).

Category

userspace

Why this phase matters

Implement real pseudo-terminal (PTY) pairs so that multiple independent terminal sessions can run simultaneously. A PTY master/slave pair connects a terminal emulator (or network server) to a shell process, enabling remote access (telnet, SSH) and local terminal multiplexing.

How it works in m3OS

Phase 29 implements pseudo-terminal (PTY) pairs, enabling multiple independent terminal sessions. A PTY master/slave pair connects a terminal emulator (or network server) to a shell process — this is the mechanism behind SSH, screen, tmux, and xterm. The implementation provides a fixed pool of 16 PTY pairs with bidirectional ring buffers, per-PTY terminal settings, line discipline processing on the slave side, session management (setsid/getsid), and controlling terminal association (TIOCSCTTY/TIOCNOTTY).

How it builds on earlier phases

This phase builds on Ext2 Filesystem and turns that foundation into PTY Subsystem.

Implementation outline

  • Implement the PTY allocator: a table of PTY pairs, each with a ring buffer connecting master and slave.
  • Implement /dev/ptmx as a special device that allocates a new PTY pair.
  • Connect the slave side to the termios line discipline from Phase 22.
  • Implement setsid() syscall for creating new sessions.
  • Implement TIOCSCTTY ioctl to assign a controlling terminal.
  • Modify fork/exec in login/shell to open a PTY slave as stdin/stdout/stderr for new sessions.
  • Write a simple test program that spawns a shell on a PTY and relays I/O.
  • Verify that terminal features (raw mode, ANSI escapes, signals) work through PTYs.

Task progress snapshot

No checklist tasks were discovered for this phase yet.

Documentation targets

  • Documentation deliverables have not been listed yet.

Deferred follow-on work

  • Packet mode (for flow control signaling)
  • Window size change propagation (SIGWINCH through PTY)
  • Terminal multiplexer (screen/tmux-style)
  • Dynamic PTY allocation beyond the fixed pool
  • /dev/tty (controlling terminal device node)

Source materials

Important code

Code spotlight cards

Code spotlight

Pty

kernel-core/src/pty.rs See file

Each PTY pair uses ring buffers (implemented in kernel-core/src/pty.rs) to pass data

View on GitHub

Code spotlight

Added pub mod pty

kernel-core/src/lib.rs See file

Added pub mod pty

View on GitHub

Code spotlight

**New** — PTY_TABLE, alloc/free/close lifecycle

kernel/src/pty.rs See file

**New** — PTY_TABLE, alloc/free/close lifecycle

View on GitHub

Key files

  • kernel-core/src/pty.rs — Each PTY pair uses ring buffers (implemented in kernel-core/src/pty.rs) to pass data
  • kernel-core/src/lib.rs — Added pub mod pty
  • kernel/src/pty.rs — **New** — PTY_TABLE, alloc/free/close lifecycle
  • kernel/src/main.rs — Added mod pty
  • kernel/src/tty.rs — Removed old alloc_pty skeleton
  • kernel/src/process/mod.rs — Added session_id, controlling_tty, ControllingTty enum; PTY cleanup in close/cloexec
  • kernel/src/arch/x86_64/syscall.rs — PTY read/write/close/open/ioctl, setsid/getsid, fork session inheritance
  • userspace/syscall-lib/src/lib.rs — openpty(), setsid(), getsid() wrappers
  • userspace/pty-test/ — **New** — PTY acceptance test program
  • kernel/Cargo.toml — Version bump to 0.29.0

How real OSes differ

  • Legacy BSD PTYs (`/dev/ptyXX` + `/dev/ttyXX` pairs, fixed count)
  • Unix98 PTYs (`/dev/ptmx` + `/dev/pts/N`, dynamically allocated) — this is what we implement
  • The PTY layer in Linux is deeply integrated with the TTY subsystem, supporting multiple line disciplines, packet mode, and window size propagation.

Success criteria

  • Opening `/dev/ptmx` returns a master fd and allocates a `/dev/pts/N` slave.
  • Writing to the master fd appears as input on the slave fd.
  • Writing to the slave fd appears as output on the master fd.
  • termios settings (raw mode, echo, line discipline) work on the slave side.
  • A shell spawned on a PTY slave behaves identically to the console shell.
  • `setsid()` creates a new session; `TIOCSCTTY` assigns the controlling terminal.
  • `SIGHUP` is delivered to the foreground process group when the master fd is closed.
  • At least 8 simultaneous PTY pairs can be allocated.

Related phases

Keep exploring the roadmap

Phase 28

Complete

Ext2 Filesystem

Replace FAT32 as the primary persistent filesystem with ext2, which natively stores Unix ownership (uid/gid), permission modes, and timestamps in every inode. After this phase, file permissions set via `chmod`/`chown` survive reboots and the FAT32 permissions index file workaround from Phase 27 is no longer needed.

infrastructure 1 prerequisite(s)

Phase 30

Complete

Telnet Server

A telnet server runs inside the OS, allowing remote users to log in over the network and get an interactive shell session. This is the first demonstration of the OS as a networked, multi-user system — a machine you can connect to from another computer (or another terminal on the host) and do real work.

infrastructure 1 prerequisite(s)