Monday, January 5, 2026

What Is an Operating System?

Happy New Year! I wanted to start off the new year by getting back to basics with a topic that many of my students frequently ask about: operating systems. Understanding how operating systems work provides the foundation for learning networking, cybersecurity, troubleshooting, and nearly every other area of modern computing.

An Operating System (OS) is system software that manages computer hardware, software resources, and provides services for computer programs. It acts as an intermediary between applications and the hardware, ensuring that programs run correctly and efficiently while enforcing rules for resource sharing, security, and stability.


Major Components of an Operating System

An OS typically includes the following components:

  1. Kernel – The core part of the OS that directly manages hardware and system resources.
  2. Device Drivers – Specialized modules that allow the OS to communicate with hardware devices.
  3. Process Management – Controls program execution, scheduling, and multitasking.
  4. Memory Management – Allocates RAM to applications and ensures isolation between processes.
  5. File System Management – Manages how data is stored, retrieved, and organized on storage devices.
  6. I/O System – Handles communication between the OS and peripherals like keyboards, displays, printers, and storage.
  7. Security & Access Control – Ensures user authentication, permission enforcement, and system integrity.
  8. User Interface – Provides a way for humans to interact with the system (Command Line Interfaces, GUIs, APIs).

The Kernel in Detail

The kernel is the foundation of the OS. It runs in privileged mode (kernel mode), giving it unrestricted access to system resources. Applications, on the other hand, run in user mode with restricted privileges.

Core Responsibilities of the Kernel:

  1. Process Management – Creates, schedules, and terminates processes. It decides which process gets CPU time and for how long.
  2. Memory Management – Allocates memory to processes, tracks usage, handles virtual memory, and manages swapping between RAM and disk.
  3. Device Management – Interfaces with device drivers to control hardware (e.g., disk drives, network interfaces).
  4. File System Access – Provides APIs for file operations like open, read, write, and close.
  5. System Calls – Defines the interface between user applications and the OS kernel (e.g., open(), fork(), read() in Linux).
  6. Security & Protection – Enforces privilege levels, process isolation, and prevents unauthorized memory access.
  7. Networking – Manages network protocols, sockets, and packet transmission.

 

Anatomy of Linux system call in ARM64 | East River Village

 

Kernel Interaction with Other Components

  • Applications request services via system calls.
  • Kernel processes the request and interacts with hardware (via device drivers).
  • Results are returned to the application in user space.

Types of Kernels

  1. Monolithic Kernel – Large kernel where all OS services (memory, file systems, device drivers) run in kernel mode (e.g., Linux).
  2. Microkernel – Minimal kernel with only essential functions (process and memory management), while drivers and services run in user space (e.g., Minix, QNX).
  3. Hybrid Kernel – A mix of both, providing microkernel architecture but with some monolithic elements for performance (e.g., Windows NT, macOS XNU).

Kernel Comparison: Windows vs. macOS vs. Linux

1. Windows (NT Kernel)

  • Type: Hybrid kernel.
  • Architecture: Combines microkernel principles with monolithic performance.
  • Features:
    • Handles subsystems for POSIX, Win32, and .NET.
    • Uses the Hardware Abstraction Layer (HAL) for portability.
    • Strong focus on backward compatibility with legacy software.
  • Strengths: Rich driver support, backward compatibility, strong enterprise integration.
  • Weaknesses: Larger attack surface due to complexity; more closed-source.

2. macOS (XNU Kernel)

  • Type: Hybrid kernel (XNU = “X is Not Unix”).
  • Architecture: Combination of Mach (microkernel) + BSD (Unix subsystem) + Apple I/O Kit.
  • Features:
    • Mach provides inter-process communication (IPC), memory management.
    • BSD layer provides POSIX compliance, file systems, and networking.
    • I/O Kit supports modular device drivers.
  • Strengths: Unix-based stability, good memory protection, modern design.
  • Weaknesses: Limited hardware support compared to Linux/Windows.

3. Linux (Monolithic Kernel with Modular Extensions)

  • Type: Monolithic (but modular).
  • Architecture:
    • Device drivers, file systems, networking stacks all run in kernel mode.
    • Supports dynamically loadable kernel modules.
  • Features:
    • Rich system call interface.
    • Broad hardware and platform support (phones, servers, supercomputers).
    • Open source—highly customizable.
  • Strengths: High performance, flexibility, large community, transparency.
  • Weaknesses: Complexity for beginners, fragmentation across distributions.

 

A diagram of a computer system

AI-generated content may be incorrect.

 


Similarities Between the Kernels

  • All manage processes, memory, file systems, devices, and networking.
  • All provide APIs/system calls for applications.
  • All enforce security boundaries between user mode and kernel mode.
  • All support multitasking and multiprocessor hardware.

Differences Between the Kernels

  • Design Philosophy:
    • Windows/macOS → hybrid (balance performance with modularity).
    • Linux → monolithic but modular (simplicity + performance).
  • Openness:
    • Windows/macOS → closed source.
    • Linux → open source.
  • Portability:
    • Linux → runs on everything from phones to supercomputers.
    • Windows/macOS → limited to vendor-supported hardware.
  • Driver Models:
    • Linux → community-driven and modular.
    • Windows → vendor-driven and highly standardized.
    • macOS → tightly controlled Apple ecosystem.

In summary:
The kernel is the “core manager” of an operating system, bridging software and hardware. While Windows NT and macOS XNU use hybrid kernel designs for flexibility and modularity, Linux uses a powerful monolithic but modular kernel for performance and openness. Despite different philosophies, they all share the same essential role: controlling hardware resources, enforcing security, and enabling applications to run smoothly.