Phase 52
Kernel Structural Hardening
The kernel's memory management, context management, and TLB coherence subsystems move onto the structurally hardened path defined by this phase. In the checked-in tree, the AddressSpace object, typed user buffers, batched TLB work, and frame zeroing landed, and Phase 52d completed the remaining closure work so the task-owned return-state and generation-tracking story now matches the shipped implementation end to end.
Learning goal
Understand why address-space identity should be a first-class kernel object (Redox: `AddrSpaceWrapper`, Zircon: `VmAspace`)
Category
infrastructure
Components touched
Builds on
Unlocked by this phase
Why this phase matters
The kernel’s memory management, context management, and TLB coherence subsystems move onto the structurally hardened path defined by this phase. In the checked-in tree, the AddressSpace object, typed user buffers, batched TLB work, and frame zeroing landed, and Phase 52d completed the remaining closure work so the task-owned return-state and generation-tracking story now matches the shipped implementation end to end.
How it works in m3OS
Phase 52b replaces the fragile kernel patterns that made the Phase 52 bugs possible. Where Phase 52a patched the immediate symptoms, this phase introduces structural defenses: a first-class AddressSpace object with per-core tracking, typed UserSliceRo/UserSliceWo wrappers at the syscall boundary, task-owned UserReturnState that eliminates ~40 manual restore call sites, batch TLB shootdown with address-space targeting, and frame zeroing on free. These patterns are drawn from Redox, seL4, and Zircon architecture analysis.
How it builds on earlier phases
This phase builds on Kernel Reliability Fixes and turns that foundation into Kernel Structural Hardening.
Implementation outline
- Define
AddressSpacestruct, wrap existingpage_table_root+ VMAs - Update
Processto useArc<AddressSpace>instead ofOption<PhysAddr> - Update scheduler dispatch to call
activate_on_core/deactivate_on_core - Add
current_addrspacepointer toPerCoreData - Implement batch
tlb_shootdown_rangeusingactive_on_cores - Define
UserReturnState, add toTask - Save user state to
UserReturnStateat syscall entry - Restore from
UserReturnStatein scheduler dispatch - Remove all
restore_caller_contextcalls - Define
UserSliceRo/UserSliceWo/UserSliceRw, convert syscall handlers - Add frame zeroing to
free_frame - Add generation counter assertions around
copy_to_user
Task progress snapshot
0 of 52 tracked tasks are complete.
Documentation targets
- Documentation deliverables have not been listed yet.
Deferred follow-on work
- VMA tree (BTreeMap or interval tree) — deferred to Phase 52c
- Per-core scheduler with work-stealing — deferred to Phase 52c
- Dynamic IPC resource pools — deferred to Phase 52c
- ISR-direct notification wakeup — deferred to Phase 52c
Source materials
Important code
Code spotlight cards
Code spotlight
Define AddressSpace struct
Define AddressSpace struct. Why it matters: The kernel currently has no address-space abstraction. A process's address space is a raw PhysAddr with no metadata, no refcount, and no per-CPU tracking. This makes targeted TLB shootdown, stale-mapping detection, and thread address-space sharing impossible. Focus on `AddressSpace` and `PhysAddr` in `kernel/src/mm/mod.rs` while you trace the snippet.
/// A process's virtual address space descriptor.
///
/// Wraps the PML4 physical address with metadata for TLB shootdown
/// optimization (generation counter) and multi-core tracking.
pub struct AddressSpace {
pml4_phys: PhysAddr,
generation: AtomicU64,
active_on_cores: AtomicU64,
page_table_lock: Mutex<()>,
} Code spotlight
Update Process to use Arc<AddressSpace>
Update Process to use Arc<AddressSpace>. Why it matters: Replacing the raw Option<PhysAddr> with Arc<AddressSpace> enables shared address spaces for CLONE_VM threads and provides the foundation for all other Track A/B tasks. Focus on `Process::page_table_root` and `Process::addr_space` in `kernel/src/process/mod.rs` while you trace the snippet.
let table = PROCESS_TABLE.lock();
table.find(pid).and_then(|p| {
p.addr_space
.as_ref()
.map(|addr_space| addr_space.as_ref() as *const AddressSpace)
})
}?;
// Before per-core state exists the kernel is still effectively single-core,
// so the current process entry cannot be concurrently reaped before callers
// immediately dereference this pointer in a tightly-scoped unsafe block.
NonNull::new(ptr as *mut AddressSpace) Code spotlight
Add per-core `current_addrspace` tracking
Add per-core `current_addrspace` tracking. Why it matters: Each CPU needs to know which address space is currently loaded so that TLB shootdowns can be targeted. This is the Redox PercpuBlock.current_addrsp equivalent. Focus on `current_addrspace` and `PerCoreData::current_addrspace` in `kernel/src/smp/mod.rs` while you trace the snippet.
/// Per-core state block.
///
/// Each core has one of these, initialized during BSP init or AP bootstrap.
/// The `gs_base` MSR points to this struct so that `per_core()` can retrieve
/// it in O(1) without MMIO.
#[repr(C)]
pub struct PerCoreData {
/// Self-pointer at offset 0 — reserved for future `gs:[0]` access.
/// Currently unused: `per_core()` reads `IA32_GS_BASE` via `rdmsr`.
self_ptr: *const PerCoreData,
/// Logical core index (0 = BSP, 1..n = APs in MADT order).
pub core_id: u8,
/// LAPIC ID from the MADT.
pub apic_id: u8,
/// Set to `true` once this core has completed initialization.
pub is_online: AtomicBool,
/// Pointer to this core's TSS (for runtime RSP0 updates).
tss_ptr: *mut TaskStateSegment,
/// Pointer to this core's GDT (pre-allocated on BSP, loaded on AP).
gdt_ptr: *const GlobalDescriptorTable,
/// Segment selectors for this core's GDT.
gdt_code: SegmentSelector,
gdt_data: SegmentSelector,
gdt_tss: SegmentSelector, Key files
- kernel/src/mm/mod.rs — AddressSpace struct with generation and active-core tracking
- kernel/src/mm/user_mem.rs — UserSliceRo, UserSliceWo typed wrappers
- kernel/src/task/mod.rs — UserReturnState in the Task struct
- kernel/src/smp/tlb.rs — Batch TLB shootdown with address-space targeting
- kernel/src/mm/frame_allocator.rs — Zero-on-free for user-accessible frames
- kernel/src/process/mod.rs — Process uses Arc<AddressSpace>
How real OSes differ
- Redox uses `Arc<AddrSpaceWrapper>` with `used_by: LogicalCpuSet` and `tlb_ack: AtomicU32` for the same purpose. Source: `src/context/memory.rs`.
- Zircon's `VmAspace` is independently ref-counted with `active_cpus_` bitmap and PCID-deferred TLB flush. Source: `zircon/kernel/vm/include/vm/vm_aspace.h`.
- seL4 embeds a TLB bitmap in unused PML4 entries (`tlb_bitmap.h`) — a clever space-saving trick m3OS could consider.
- Linux uses `mm_struct` with `cpu_bitmap` for lazy TLB mode and targeted shootdown.
Success criteria
- No `restore_caller_context` calls remain in the codebase
- `UserSliceRo`/`UserSliceWo` are used at all syscall boundary copy sites
- `AddressSpace.active_on_cores` correctly tracks which CPUs have each AS loaded
- TLB shootdown for `munmap(ptr, 1 MiB)` sends 1 IPI (not 256)
- Freed frames read as all-zeros when accessed through a stale mapping
- `copy_to_user` reliability reproducer (real DeviceTTY/keyboard-IRQ path on SMP4) no longer fails
- `cargo xtask check` passes
- `cargo xtask test` passes
Related phases
Keep exploring the roadmap
Phase 52
CompleteKernel Reliability Fixes
All four confirmed kernel bugs from the Phase 52 bug investigations are fixed, with targeted regression tests proving each fix.
Phase 52
CompleteKernel Architecture Evolution
Phase 52c established the architecture direction for per-core scheduling, growable IPC resources, a unified kernel-side line discipline, O(log n) VMA lookup, and ISR wakeup improvements. In the checked-in tree, the VMA tree, growable endpoint/capability/service tables, `LineDiscipline`, the live `push_raw_input` keyboard path, and the ISR wake queue all landed. Phase 52d then recorded the remaining honest limits: the scheduler hot path still relies on the global `SCHEDULER` lock, and notifications remain fixed-size for ISR safety.