Operating System
Design, Processes & Threads

Ask someone to describe a "good" operating system and they'll say it's fast. That's true, but incomplete enough to be dangerous. An OS is good for a purpose: the same design that makes a pacemaker's OS good (never fail, ever) would make a gaming laptop's OS terrible (nothing would ever run fast). Before you can judge whether Linux's choices or Windows's choices are "right," you need a vocabulary for what each choice is being traded against.

The six goals

Every OS design decision — down to something as small as how WriteWell saves a file — is a vote for some of these six goals at the expense of others.

Why you can't max out all six

Design goals aren't a checklist you tick off independently; they sit in tension. Three tensions come up constantly enough to name:

• Performance vs. security. Every bounds check, every permission lookup, every sandbox boundary is CPU cycles spent not doing the user's work. A kernel that trusted every request would be blazingly fast and catastrophically insecure.
• Performance vs. portability. A hardware abstraction layer (HAL) lets the same OS run on different CPUs by inserting an indirection between your code and the hardware. The indirection is exactly what costs the performance.
• Scalability and reliability often help each other — the same redundancy (replicated state, multiple workers) that lets a system scale out also gives it something to fail over to. This is the one pairing on this list that isn't purely a trade-off, and it's worth remembering precisely because the other pairs train you to expect conflict everywhere.

Figure 1 · the same six goals, two designs

Practice 1.1

1. An OS is being chosen for a pacemaker's embedded controller. Rank the six goals for this deployment and justify your top two.
   Show solution

   Reliability and security dominate, and in that order. A pacemaker that fails "gracefully" is still a failure with a heartbeat attached to it, so reliability (surviving hardware faults, brownouts, radiation-induced bit flips) outranks everything. Security is a close second now that pacemakers are wirelessly programmable — an adversarial firmware update is a real threat model, not a hypothetical one. Performance matters only insofar as it must meet a hard real-time deadline (Unit 9 covers this); beyond that deadline, more speed buys nothing. Portability and interoperability are nearly irrelevant — the device runs on one known chip for its entire service life.

2. Now rank the same six goals for a high-frequency trading system that must react to market data in microseconds.
   Show solution

   Performance dominates so heavily that some trading systems deliberately weaken portability (hand-tuned for one exact CPU model, sometimes one exact silicon stepping) and even reliability in the "survive-any-fault" sense — they'd rather crash fast and restart than pay for the redundancy that would let them limp on. Security still matters (this is still money) but is often pushed to the network edge rather than paid for on every hot-path instruction. This is a useful contrast with the pacemaker: same six goals, almost the opposite ranking.

3. WriteWell's team disables the spell-checker's sandbox to shave 40 ms off startup time. Which goal was traded for which, and what's the concrete risk?
   Show solution

   Security was traded for performance. The concrete risk: the spell-checker parses dictionary and document content that could, in principle, be crafted to exploit a parser bug. Sandboxed, a compromised spell-checker thread is contained to its own restricted resources; unsandboxed, it runs with the same privileges as T_ui and T_save, so a single parser bug becomes a compromise of the whole process — including the document being edited and whatever files that process's user can access.

4. Synthesis. Explain, with a concrete OS mechanism, why scalability and reliability can reinforce each other even though performance and security almost never do.
   Show solution

   Take replicated storage: keeping three copies of a file across three machines is originally a reliability mechanism (one disk dying doesn't lose data). The same three copies let you serve three times the read traffic by routing requests to whichever replica is least busy — a scalability win, for free, from a mechanism you built for a different reason. Performance and security don't get this kind of accident: a security check is extra work on the same critical path as the performance you're trying to protect, so there's no mechanism that happens to serve both at once the way redundancy serves both scalability and reliability.

When T_save calls write(), that request has to pass through filesystem code, then a device driver, before a single byte reaches the disk. Where that code lives — all bundled into one privileged program, or split across several separately-protected ones — is the single biggest structural decision an OS designer makes, and it decides almost everything else in this course: how fast a syscall is, how much a driver bug can damage, how portable the design is to new hardware.

Three ways to organise the same code

Picture a company that has to handle "write this file." A monolithic kernel is one department that does everything in-house — filesystem, drivers, memory manager — all reporting to the same manager, talking to each other with direct function calls. A microkernel keeps only the absolute essentials in-house (scheduling, basic memory protection, message delivery) and outsources everything else — the filesystem, the device drivers — to separate contractors who can only be reached by sending a message and waiting for a reply. A hybrid kernel keeps almost everything in-house like the monolithic design, for speed, but organises it into cleanly separated internal modules so a change to one doesn't require rebuilding the whole thing.

Formal picture: what actually crosses a boundary

Two costs distinguish these designs, and it's worth being precise about which is which:

• Mode transitions. A trap from user mode into kernel mode (and back) — needed for any syscall, in every architecture. Cheap: on modern x86, tens to low hundreds of cycles.
• Address-space (context) switches. Needed only when execution must move into a different process's address space — which happens on every IPC hop between separate servers in a microkernel, but not when a monolithic or hybrid kernel calls between its own internal functions, because those functions all live in the one kernel address space. Section 1.4 puts an exact number of microseconds on this.

This is the whole trade-off in one sentence: microkernels pay address-space switches for isolation; monolithic and hybrid kernels pay potential blast radius for speed.

Figure 2 · tracing T_save's write() across three architectures

Practice 1.2

1. QNX is used to run the infotainment and instrument-cluster software in some cars, where a crashed graphics driver must not be able to take down the rest of the system. Explain why a microkernel is a deliberate fit here, in terms of the goals from 1.1.
   Show solution

   Reliability and, to a lesser extent, security dominate this deployment (a stalled instrument cluster is a safety issue, not an inconvenience). In QNX's microkernel design, the graphics driver is a separate user-space server; if it crashes, the microkernel and the other servers (the ones actually driving the engine-management display or warning lights) are unaffected and the driver can be restarted. In a monolithic design, a bug in that same driver runs in kernel mode and can corrupt kernel memory used by everything else. The performance cost of the extra IPC hops is judged acceptable because the workload isn't latency-critical at microsecond scale.

2. Give one reason Linux, despite decades of microkernel research existing, remains monolithic for general-purpose desktop and server use.
   Show solution

   Raw syscall and I/O throughput matters enormously for general-purpose workloads (web servers, databases, compilers), and a monolithic design avoids paying an address-space switch on every filesystem or network operation. Linux also mitigates the isolation downside with loadable kernel modules, kernel-space sandboxing technologies (eBPF verification, for instance), and aggressive testing/fuzzing of driver code — reducing (not eliminating) the blast-radius risk without paying the microkernel's per-call cost.

3. Interpretation. A vendor claims "our hybrid kernel gives us microkernel-level isolation with monolithic-level speed." Is this a fair description of what a hybrid kernel like the Windows NT kernel actually does?
   Show solution

   Not quite, and this is a common marketing overstatement. A hybrid kernel's core services (I/O manager, memory manager, the filesystem driver stack) still run in one shared kernel address space — a bug in a kernel-mode filesystem filter driver can still corrupt kernel memory, exactly as in a monolithic design. What the hybrid design actually buys is engineering modularity (drivers are separately loadable, layered, easier to develop and reason about) and the option to push some services to user mode when isolation is worth the cost — not the hard, mechanism-enforced isolation a true microkernel gives every component by default.

The scheduler is about to take T_ui off the CPU and put T_save on. A moment later it will put T_ui back — and T_ui needs to resume exactly where it left off, mid-instruction if necessary, with no idea that time passed at all. Something has to remember precisely enough for that illusion to hold.

The bookmark, and the note pinned to it

Think of pausing a thread like being interrupted mid-recipe. A bookmark alone (which step you were on) isn't enough — you also need a note of what's in each mixing bowl, which oven you were using, and where the ingredients are. That note is what a control block is: enough state, saved outside the CPU, to resume an execution exactly where it stopped.

An OS keeps two levels of this note. The Process Control Block (PCB) holds what's shared by an entire process — the "kitchen" everyone in it uses. The Thread Control Block (TCB) holds what's private to one thread of execution within that process — one cook's exact position in the recipe.

Formal contents

Figure 3 · PCB and two TCBs for process P (WriteWell)

The real implementations diverge from the textbook picture

The PCB/TCB split above is the conceptual model every textbook teaches. Real kernels implement it two genuinely different ways, and the difference is worth knowing because it's a favourite "explain the internals" exam question.

Practice 1.3

1. A process has three threads. At this instant, one is RUNNING, one is READY, and one is BLOCKED on disk I/O. For each, say whether its PC and SP are "live in the CPU" or "saved in its TCB."
   Show solution

   Only the RUNNING thread's PC and SP are live in the CPU registers. Both the READY thread and the BLOCKED thread have their PC and SP saved in their own TCBs — from the kernel's point of view, "not currently executing" is the only fact that matters for where the register state lives, regardless of why it isn't executing.

2. Variation. Contrast what fork() duplicates versus what pthread_create() shares, in terms of the CLONE_* flags.
   Show solution

   fork() calls clone() with none of CLONE_VM, CLONE_FILES, or CLONE_SIGHAND set (the child gets copy-on-write copies of the address space and its own duplicated file descriptor table, though the underlying open file objects are still shared). pthread_create(), underneath, calls clone() with CLONE_VM | CLONE_FILES | CLONE_SIGHAND | CLONE_THREAD — the new task shares the address space, file table, and signal handlers outright, which is precisely what makes it "a thread" rather than "a process" in the textbook sense.

3. Interpretation. A crash dump shows two Linux task_structs with different pid values but the same mm pointer. What does that tell you, and what would you expect their tgid values to be?
   Show solution

   They're two threads of the same process — sharing an mm_struct means they share the address space, which is exactly the definition of "same process, different thread" in the PCB/TCB model. Their tgid (thread group ID, the value user space sees as "the PID") should be identical, even though each has its own distinct pid for the kernel's internal scheduling purposes.

4. Synthesis. In a microkernel (1.2), the filesystem server is itself a user-space process that might run several of its own worker threads. Are those workers' TCBs "part of the operating system" the same way a monolithic kernel's TCBs are?
   Show solution

   Not in the same sense. In a monolithic kernel, every TCB (Linux's task_struct) is kernel data, maintained and trusted by the kernel itself. In a microkernel, the filesystem server's worker threads have TCBs maintained by whatever user-space threading library the server uses — the microkernel only knows about the server process's own kernel-level schedulable entities (however many the microkernel itself schedules for that server), not about threads the server manages internally. This is precisely the kernel-level vs. user-level distinction that Section 1.5 makes precise.

Getting interrupted mid-task costs more than the interruption itself — you also lose the moment it takes to reload what you were doing. A CPU has exactly the same problem when the scheduler switches it from T_ui to T_save, just measured in microseconds instead of minutes. The question this section answers precisely: how much, and when does it start to matter?

Direct cost and indirect cost

A switch has two kinds of cost, and only one of them shows up if you just count instructions.

A thread switch (T_ui → T_save, same process) pays only the direct cost: registers and stack pointer change, but the page tables underneath don't, so every entry already sitting in the TLB is still valid. A process switch pays direct cost and indirect cost: the new process's page table root has to be loaded, which (on most hardware, most of the time) invalidates the TLB, so the first several memory accesses after the switch each cost a full page-table walk instead of one fast lookup.

Worked numbers

Published measurements of context-switch cost vary by roughly an order of magnitude depending on hardware and workload — direct costs in the low single-digit microseconds are typical on modern x86, with indirect costs from tens to hundreds of microseconds depending on how much of the working set has to be re-cached. The figures below sit inside that published range and are what we'll use for every calculation in this section.

A process switch costs roughly 2× a thread switch here — and the gap grows with the working set, since only the process switch's indirect term scales with it (Practice 1.4 asks you to show this).

Figure 4 · does the overhead actually matter?

Practice 1.4

1. Using the same per-switch costs, compute the overhead percentage for both switch types at a 5 ms quantum.
   Show solution

   overhead% = cost ÷ (cost + q). For q = 5000 µs: thread switch = 1.8 ÷ 5001.8 ≈ 0.036%; process switch = 3.7 ÷ 5003.7 ≈ 0.074%. Both are still small, but note the process-switch figure is already double the thread-switch figure at every quantum — the ratio between them doesn't depend on q at all, only their absolute values do.

2. Variation. Suppose T_save's working set right after a switch-in is 100 pages instead of 30. Recompute the process switch's total cost and its ratio to a thread switch.
   Show solution

   Indirect cost = 100 × 0.05 µs = 5.0 µs. Total process switch = 1.8 + 0.4 + 5.0 = 7.2 µs, which is 4.0× the 1.8 µs thread switch — double the earlier 2× ratio, entirely because the indirect term is the only one that scales with working set. This is exactly why "process switches cost roughly 2× a thread switch" is a working-set-dependent statement, not a hardware constant.

3. Interpretation. A profiler reports that a server spends 4% of its CPU time "in context switch overhead" at a 200 µs quantum. Is that figure alarming on its own?
   Show solution

   Not on its own — it matches almost exactly what this section's numbers predict for process switches at a pathologically short 200 µs quantum (1.82% for thread switches, and up to several times that for process switches with a larger working set), so the right response isn't "the kernel is broken," it's "this quantum is too short for this workload." The fix under consideration should be the scheduling quantum or the process/thread mix, not the context switch implementation itself.

4. Synthesis. Explain, using 1.2's vocabulary, why a microkernel's IPC-heavy design is especially sensitive to the numbers in this section.
   Show solution

   Every IPC round trip between two separate microkernel servers is, at the hardware level, an address-space switch — it pays the process-switch cost from this section's table, indirect term included, not the cheaper thread-switch cost. A monolithic kernel's internal function calls between "filesystem code" and "driver code" pay neither, because they never leave one address space. This is exactly the mechanism behind seL4's engineering effort in 1.2: shrinking that per-IPC direct-plus-indirect cost is the whole game for making a microkernel design competitive.

Imagine a manager who only tracks "departments," never individual staff. If one worker in a department is on a long phone call, does the manager notice the other three workers in that department are free to keep working — or does the manager just see "that department is busy" and route no new work to any of them? Whether an OS's scheduler can see T_ui and T_save as two separate, independently-schedulable things, or only as one thing called "process P," is exactly this question.

Two ways to implement "a thread"

A kernel-level thread (KLT), the 1:1 model, is created with a syscall and gets its own kernel-visible TCB (Section 1.3) — the kernel schedules it directly, exactly like any other thread. A user-level thread (ULT), the N:1 model, is implemented entirely by a library running in user space: the library keeps its own tiny scheduler, its own per-thread stacks and saved registers, and multiplexes many ULTs onto a single kernel-level entity that the OS is the only one aware of. From the kernel's point of view, an N:1 process with fifty user-level threads looks exactly like a process with one thread.

The trade-off that actually matters

User-level threads are cheap: creating one, or switching between two, never involves a syscall or the context-switch costs from Section 1.4 — it's a library-managed stack swap, often under a hundred nanoseconds. Kernel-level threads are expensive to create and switch by comparison, but they buy two things a user-level thread can never get on its own: genuine parallelism across multiple cores (the kernel can put two KLTs on two cores at once; two ULTs multiplexed onto one KLT never run simultaneously, no matter how many cores the machine has), and independence when one thread blocks — which the rest of this section makes concrete.

Figure 5 · T_save's autosave blocks — who else stalls?

T_save occasionally saves to a slow, cloud-synced folder; that write blocks for 250 ms while it happens — well past the ~100 ms threshold at which a delay reads as a perceptible stall rather than instant response. Step through what each threading model does with that 250 ms.

Run the numbers on a full hour of typing: autosave fires every 30 s (120 times/hour), and suppose about one save in six hits the slow, cloud-synced path. Under 1:1, none of that reaches T_ui — the kernel keeps scheduling it independently the whole time. Under N:1, the cumulative freeze is 20 slow saves × 250 ms ≈ 5.0 seconds of frozen keystrokes, every hour, entirely invisible to any profiler that only measures CPU time, because the CPU isn't busy during a block — it's the user who's waiting.

Practice 1.5

1. A process has one KLT hosting four ULTs. One ULT calls a blocking read(). What happens to the other three?
   Show solution

   Under the plain N:1 model, all three block along with it — the kernel only knows about the single KLT, and that KLT is now waiting in the kernel for the read to complete, so none of the process's user-level threads can run, regardless of what they were doing.

2. Variation. Same setup, but the runtime is M:N with two KLTs backing those four ULTs. Does the answer change?
   Show solution

   Only partially. The ULT that made the blocking call takes its KLT down with it, but the other KLT is still schedulable, so up to one of the remaining three ULTs can keep running if the runtime has migrated it there. The other two are still stuck waiting for a free KLT — M:N reduces the blast radius of one blocking call, it doesn't eliminate it, which is part of why the engineering complexity in the depth box above was judged not worth it.

3. Interpretation. A four-core machine runs a process with eight user-level threads multiplexed onto one kernel thread, all doing CPU-bound work with no blocking calls at all. Why is this design guaranteed to leave three cores idle no matter how the user-level scheduler is tuned?
   Show solution

   The kernel can only place kernel-visible schedulable entities onto cores, and this process exposes exactly one. No amount of cleverness in the user-space scheduler changes how many kernel-visible entities exist; it can only decide which of the eight ULTs currently occupies that one KLT. True parallelism requires enough KLTs for the cores you want to use — a purely N:1 design is architecturally capped at using one core, permanently.

4. Synthesis. Connect this section back to 1.3: what has to be true about a threading library's internal data structures for it to implement ULTs at all, given that the kernel provides no TCB for them?
   Show solution

   The library must maintain its own control-block equivalent for each ULT entirely in user memory — a saved program counter, stack pointer, register set, and stack region, precisely mirroring the TCB fields from 1.3, just outside the kernel's view and outside its protection. This is also exactly why a crash or infinite loop in the user-space scheduler can wedge every ULT in the process with no kernel-level recourse: there's no kernel-level record of them to fall back on.