Phase 52
Kernel Reliability Fixes
All four confirmed kernel bugs from the Phase 52 bug investigations are fixed, with targeted regression tests proving each fix.
Learning goal
Understand why per-core mutable scratch for syscall return state is fragile
Category
infrastructure
Components touched
Builds on
Unlocked by this phase
Why this phase matters
All four confirmed kernel bugs from the Phase 52 bug investigations are fixed, with targeted regression tests proving each fix.
How it works in m3OS
Phase 52a addresses four confirmed kernel bugs discovered during the Phase 52 service extraction work. These are not design improvements — they are direct correctness fixes for stale IPC return state, a vendored SSH library waker bug, missing POSIX clear_child_tid on thread exit, and missing signal action reset on exec. All four bugs blocked either the copy_to_user reliability investigation or the SSHD post-authentication hang.
How it builds on earlier phases
This phase builds on First Service Extractions and turns that foundation into Kernel Reliability Fixes.
Implementation outline
- Add
restore_caller_contexttokernel/src/ipc/mod.rsdispatch function - Fix
Channel::wake_write()insunset-local/src/channel.rs - Add
clear_child_tidcleanup to thread exit inkernel/src/arch/x86_64/syscall/mod.rs - Add signal action reset to
sys_execveinkernel/src/arch/x86_64/syscall/mod.rs - Write targeted regression tests for each fix
Task progress snapshot
17 of 18 tracked tasks are complete.
Documentation targets
- Documentation deliverables have not been listed yet.
Deferred follow-on work
- Moving
syscall_user_rspto task-owned state (Phase 52b structural hardening) - Typed
UserBufferwrappers to makecopy_to_userauditable (Phase 52b) - AddressSpace object for the underlying mapping divergence (Phase 52b)
Source materials
Important code
Code spotlight cards
Code spotlight
Add `restore_caller_context` to IPC dispatch
Add `restore_caller_context` to IPC dispatch. Why it matters: All seven IPC blocking syscalls (recv, call, reply_recv, notify_wait, call_buf, recv_msg, reply_recv_msg) return with a stale per-core syscall_user_rsp after a context switch. This is the confirmed root cause of wrong user RSP on return from blocking IPC. Focus on `restore_caller_context` and `dispatch` in `kernel/src/ipc/mod.rs` while you trace the snippet.
/// IPC syscall dispatcher, called from `arch::x86_64::syscall::syscall_handler`.
///
/// Userspace syscall numbers `0x1100`–`0x110F` are translated to internal
/// dispatch numbers 1–16 by the flat dispatch table in `arch/x86_64/syscall/mod.rs`.
///
/// | Internal | Userspace | Operation | Args (SysV: rdi=arg0, rsi=arg1, rdx=arg2) |
/// |---|---|---|---|
/// | 1 | 0x1100 | `ipc_recv(ep_cap)` | `arg0 = ep_cap_handle` |
/// | 2 | 0x1101 | `ipc_send(ep_cap, label, data0)` | `arg0..2` |
/// | 3 | 0x1102 | `ipc_call(ep_cap, label, data0)` | `arg0..2` |
/// | 4 | 0x1103 | `ipc_reply(reply_cap, label, data0)` | `arg0..2` |
/// | 5 | 0x1104 | `ipc_reply_recv(reply_cap, label, ep_cap)` | `arg0..2` — ep_cap in arg2 |
/// | 6 | 0x1105 | `sys_cap_grant(source_handle, target_task_id)` | `arg0, arg1` |
/// | 7 | 0x1106 | `notify_wait(notif_cap)` | `arg0 = notif_cap_handle` |
/// | 8 | 0x1107 | `notify_signal(notif_cap, bits)` | `arg0, arg1` |
/// | 9 | 0x1108 | `ipc_register_service(ep_cap, name_ptr, name_len)` | `arg0..2` |
/// | 10 | 0x1109 | `ipc_lookup_service(name_ptr, name_len)` | `arg0, arg1` → new CapHandle |
/// | 11 | 0x110A | `create_irq_notification(irq)` | `arg0 = IRQ number` → new CapHandle |
/// | 12 | 0x110B | `create_endpoint()` | — → new CapHandle |
/// | 13 | 0x110C | `ipc_send_buf(ep_cap, label, data0, buf_ptr, buf_len)` | `arg0..4` |
/// | 14 | 0x110D | `ipc_call_buf(ep_cap, label, data0, buf_ptr, buf_len)` | `arg0..4` → label |
/// | 15 | 0x110E | `ipc_recv_msg(ep_cap, msg_ptr, buf_ptr, buf_len)` | `arg0..3` → label |
/// | 16 | 0x110F | `ipc_reply_recv_msg(reply_cap, label, ep_cap, msg_ptr, buf_ptr)` | `arg0..4` → label |
/// Code spotlight
Add futex WAIT restore (same pattern)
Add futex WAIT restore (same pattern). Why it matters: The futex WAIT path has the same structural hole — it blocks via block_current_on_futex_unless_woken() and returns without an adjacent restore_caller_context(). This was identified in the copy_to_user investigation as a parallel defect. Focus on `sys_futex` and `block_current_on_futex_unless_woken` in `kernel/src/arch/x86_64/syscall/mod.rs` while you trace the snippet.
/// Futex wait/wake implementation for thread synchronization.
/// Supports WAIT, WAKE, WAIT_BITSET, WAKE_BITSET with real blocking queues.
///
/// Supports `FUTEX_WAIT`, `FUTEX_WAKE`, `FUTEX_WAIT_BITSET`, and
/// `FUTEX_WAKE_BITSET` operations with the `FUTEX_PRIVATE_FLAG`.
pub(super) fn sys_futex(uaddr: u64, op: u64, val: u64, val3: u64) -> u64 {
const FUTEX_WAIT: u64 = 0;
const FUTEX_WAKE: u64 = 1;
const FUTEX_WAIT_BITSET: u64 = 9;
const FUTEX_WAKE_BITSET: u64 = 10;
const FUTEX_PRIVATE_FLAG: u64 = 128;
use crate::process::futex::{FUTEX_BITSET_MATCH_ANY, FUTEX_TABLE, FutexWaiter};
let is_private = (op & FUTEX_PRIVATE_FLAG) != 0;
let cmd = op & !(FUTEX_PRIVATE_FLAG);
// Build the futex key: (addr_space pml4_phys, uaddr).
// Private futexes use 0 as root; shared futexes use the real CR3.
let futex_root = if is_private {
0u64
} else {
let pid = crate::process::current_pid();
match crate::process::PROCESS_TABLE Code spotlight
Verify relay task write waker registration
> **Note:** B.2 was already implemented. The should_wait block at line 874-880 checks pty_pending_len > 0 and registers set_channel_write_waker before sleeping. Why it matters: The relay task must register set_channel_write_waker() when pty_pending_len > 0. This was previously missing but may have been partially fixed. Verify the registration is correct and complete. Focus on `channel_relay_task` and `set_channel_write_waker` in `userspace/sshd/src/session.rs` while you trace the snippet.
async fn channel_relay_task(
sock_fd: i32,
pty_fd: i32,
runner: SharedRunner,
state: SharedState,
chan: SharedChan,
progress_notify: SharedNotify,
session_notify: SharedNotify,
output_lock: SharedOutputLock,
) {
let mut chan_buf = [0u8; 4096];
let mut pty_buf = [0u8; 4096];
let mut pty_pending = [0u8; 4096];
let mut pty_pending_len: usize = 0;
// Pending data from channel -> PTY that couldn't be written (EAGAIN).
let mut pty_write_pending = [0u8; 4096];
let mut pty_write_pending_len: usize = 0;
// The relay task wakes on:
// - PTY readable (shell produced output) — via reactor
// - Channel data available (client typed something) — via channel_read_waker
// - Channel write capacity returned (backpressure cleared) — via channel_write_waker
loop {
if state.borrow().session_done { Key files
- kernel/src/ipc/mod.rs — IPC dispatch with restore_caller_context at the boundary
- userspace/sshd/src/session.rs — SSH relay task that depends on correct waker behavior
- kernel/src/arch/x86_64/syscall/mod.rs — clear_child_tid cleanup and signal reset on exec
- sunset-local/src/channel.rs — Write waker fix in vendored SSH library
How real OSes differ
- Linux's syscall return path restores all user state from the `pt_regs` structure on the kernel stack — there is no per-core mutable scratch that needs manual restoration
- Linux's `exit_mm()` and `exit_signal()` properly handle `clear_child_tid` and signal reset as standard parts of the exit/exec code paths
- Production SSH libraries (OpenSSH, libssh) use multi-process or multi-threaded models rather than cooperative async, avoiding the single-threaded waker coordination problem entirely
Success criteria
- IPC blocking syscalls (recv, call, reply_recv, notify_wait) return with correct user RSP on SMP4
- SSH session displays shell prompt without requiring a client keystroke after authentication
- musl `pthread_join` completes (does not hang) when a CLONE_CHILD_CLEARTID thread exits
- exec'd program does not inherit `Handler` signal dispositions from its predecessor
- `cargo xtask check` passes with all fixes
- Each fix has a corresponding entry in the `copy_to_user` or SSHD hang investigation doc showing it resolves the identified issue
Related phases
Keep exploring the roadmap
Phase 52
CompleteFirst Service Extractions
m3OS proves that its microkernel direction is real by extracting at least the first visible core services out of the kernel address space. Console and input-related functionality become supervised ring-3 services, and the system demonstrates that they can fail and be restarted without rebooting the machine.
Phase 52
CompleteKernel Structural Hardening
The kernel's memory management, context management, and TLB coherence subsystems move onto the structurally hardened path defined by this phase. In the checked-in tree, the AddressSpace object, typed user buffers, batched TLB work, and frame zeroing landed, and Phase 52d completed the remaining closure work so the task-owned return-state and generation-tracking story now matches the shipped implementation end to end.