Kernel Mode, User Mode & OS Structure

By the end of this lesson you will be able to explain why a CPU has two modes, list instructions only the kernel may run, and compare monolithic, microkernel and hybrid OS designs.

In the last lesson you saw that the OS protects programs from each other. But the OS is also just code running on the same CPU. So how does it stop a buggy app from simply switching off the protection? The answer is built into the CPU itself.

Two modes: kernel and user

Modern CPUs have at least two modes of operation:

• Kernel mode (also called supervisor mode or privileged mode): the code can do anything, including talking to devices and changing memory settings.
• User mode: the code can only do safe things, such as arithmetic and reading its own memory.

The CPU keeps track of the current mode with a mode bit. Your apps always run in user mode. Only the kernel runs in kernel mode.

A bank branch is a good picture. Customers (user mode) can fill forms and wait at the counter. Only staff behind the counter (kernel mode) can open the cash vault. A customer who walks into the vault room is stopped at once.

On x86 processors these levels are called rings. Ring 0 is the most privileged and is used by the kernel. Ring 3 is used for apps. Rings 1 and 2 exist but mainstream systems hardly use them.

Privileged instructions

A privileged instruction is a CPU instruction that is allowed only in kernel mode. If user-mode code tries one, the CPU does not run it. Instead it raises a trap: an automatic jump into the kernel, which then decides what to do (usually stopping the program).

Step 1: run this small simulation of a CPU. It runs a list of instructions and checks the mode before each privileged one.

PRIVILEGED = {"disable_interrupts", "set_timer", "write_disk"}

def run(program, mode):
    for instr in program:
        if instr in PRIVILEGED and mode == "user":
            print(f"  TRAP: '{instr}' not allowed in user mode, program stopped")
            return
        print(f"  ok: {instr}")

app = ["add", "multiply", "disable_interrupts", "add"]
print("App in user mode:")
run(app, "user")
print("Same code in kernel mode:")
run(app, "kernel")

App in user mode:
  ok: add
  ok: multiply
  TRAP: 'disable_interrupts' not allowed in user mode, program stopped
Same code in kernel mode:
  ok: add
  ok: multiply
  ok: disable_interrupts
  ok: add

Step 2: notice that the last add never runs in user mode. The real CPU behaves the same way: the trap hands control to the kernel, and on Linux the program usually ends with a signal such as SIGSEGV or SIGILL.

How do we get into kernel mode?

If apps can never run privileged instructions, how do they save a file or print a page? They ask the kernel. There are three ways the CPU switches from user mode to kernel mode:

1. System call: the program deliberately asks the kernel for a service. You will study these in System Calls.
2. Interrupt: a device needs attention, for example a key was pressed or the timer ran out.
3. Exception: the program did something wrong, such as dividing by zero or touching memory it does not own.

In all three cases the CPU switches to kernel mode and jumps to a fixed kernel address that the kernel set up at boot. When the kernel is finished, it switches back to user mode and the program continues (or is stopped).

The timer interrupt is the key to fairness. The kernel sets a timer before it lets a program run. Even if the program loops forever, the timer fires and the kernel gets control back.

How an OS is structured

Kernel designers must decide how much code runs in kernel mode. More code in the kernel is faster, but a bug there can crash everything.

Monolithic kernel

In a monolithic kernel, almost all OS services (scheduling, memory, file systems, device drivers, networking) run together in kernel mode as one large program. Parts call each other directly, which is fast.

Linux is monolithic, but it is modular: drivers can be loaded and removed while the system runs as loadable kernel modules. You can list them:

lsmod | head -4

Module                  Size  Used by
snd_hda_intel          61440  2
nvme                   61440  3
ext4                 1110016  1

The exact list depends on your hardware.

Microkernel

A microkernel keeps only the bare minimum in kernel mode: basic scheduling, memory setup and a way for programs to send messages to each other. File systems and drivers run as normal user-mode programs called servers.

If a driver crashes, only that server dies and can be restarted. The cost is speed, because work now needs extra messages and mode switches. QNX (used in many car systems), MINIX 3 and the formally verified seL4 are microkernels.

Hybrid kernel

A hybrid kernel starts from microkernel ideas but keeps many services in kernel mode for speed. The Windows NT kernel used by Windows 10 and 11, and Apple's XNU kernel used by macOS and iOS, are usually described as hybrid.

Layered design

A layered OS is built like floors of a building. Each layer uses only the layer below it, which makes testing easier. It is a teaching model more than a modern product design, but its idea of clean interfaces lives on inside every kernel.

Common mistakes

• Thinking "admin" or "root" means kernel mode. An administrator's programs still run in user mode. Root only gets more permission from the kernel.
• Believing user mode is a software check only. The CPU hardware enforces it, which is why apps cannot simply skip it.
• Saying Linux is a microkernel because it has modules. Modules still run in kernel mode, so Linux is monolithic and modular.

Key points

• The CPU has kernel mode (everything allowed) and user mode (safe instructions only), tracked by a mode bit.
• Privileged instructions in user mode cause a trap into the kernel.
• System calls, interrupts and exceptions are the three ways into kernel mode.
• The timer interrupt guarantees the kernel always gets the CPU back.
• Monolithic kernels are fast; microkernels isolate faults; hybrid kernels mix both.
• Linux is monolithic with loadable modules; Windows NT and XNU are hybrid.

PRIVILEGED = {"disable_interrupts", "set_timer", "write_disk", "halt"}

def run(program, mode):
    done = 0
    for instr in program:
        if instr in PRIVILEGED and mode == "user":
            print(f"  TRAP on {instr}")
            break
        done += 1
    print(f"  {mode} mode finished {done} instruction(s)")

scheduler = ["set_timer", "add", "halt"]
run(scheduler, "user")
run(scheduler, "kernel")

TRAP on set_timer
user mode finished 0 instruction(s)
kernel mode finished 3 instruction(s)