[HOME]

ECE1176 System Software
Electrical and Computer Engineering Department
University of Pittsburgh

System software is an interface between software and hardware.

This includes:

  • Operating System (OS) & Kernel: The central manager of the computer that controls hardware resources like the CPU, memory, and storage, while running a core kernel to handle low-level operations.
  • Process Management & Scheduling: The OS creates, schedules, switches, and terminates processes or threads to allow multitasking, using schedulers and context-switching mechanisms to share CPU time.
  • Memory Management: Controls both physical RAM and virtual memory, handling address translation (paging/segmentation), allocation, and swapping to keep applications isolated and running smoothly.
  • Hardware Abstraction: Hides complex hardware details behind clean, standardized interfaces (Hardware Abstraction Layer HAL) so application software does not need to know specific chip architectures or peripheral wiring.
  • Device Drivers: Specialized translation programs that allow the operating system and hardware peripherals (like graphics cards, printers, or Wi-Fi chips) to communicate.

    System Software concepts that apply to embedded systems

    This includes:

  • Hardware Abstraction Layers (HAL) and Device Drivers: Code layers that translate generic software calls into specific hardware register manipulations, making software more portable across different chips.
  • Interrupt Handling and ISRs: Interrupt Service Routines that temporarily pause the main CPU execution to respond immediately to hardware events (like a timer tick, sensor signal, or communication packet).
  • Memory-Mapped I/O (MMIO) and Direct Memory Access (DMA): Mapping peripheral control registers directly into the processor's memory address space, and using DMA to transfer data between peripherals and RAM without burdening the CPU.

    System Software concepts for high-performance AI co-processors (such as NPUs, TPUs, or GPUs).

    This includes:

    Runtime Management and Workload Orchestration

  • Heterogeneous Task Scheduling: Coordinating work between the host CPU (handling control logic, file I/O, and pre/post-processing) and the AI co-processor (handling heavy math).
  • Asynchronous Execution and Queuing: Using non-blocking command queues so the CPU can submit batches of work to the co-processor without stalling the main application thread.
  • Stream/Context Management: Isolating independent execution streams on the co-processor to run multiple small inference requests or pipeline tasks concurrently.

    Memory and Data Movement Subsystems

  • Zero-Copy and Unified Memory: Minimizing redundant data copies between host system RAM and co-processor high-bandwidth memory (HBM) or local SRAM.
  • Memory Layout Transformations (Data Formatting): Dynamically re-arranging tensor memory layouts (e.g., from standard NCHW to specialized block-channel layouts) to maximize cache locality and memory controller efficiency.
  • Low-Precision Quantization Support: Software management of dynamic or static bit-width reduction (moving from FP32/FP16 down to INT8 or INT4) to shrink memory footprints and accelerate throughput.

    Drivers and Hardware Abstraction Layers (HAL)

  • Hardware Abstraction: Exposing unified driver APIs (such as CUDA, OpenVINO, or vendor-agnostic runtimes like ONNX Runtime and WebNN) so machine learning frameworks (PyTorch, TensorFlow) can target the co-processor transparently.
  • Low-Overhead Dispatch: Reducing driver-level software latency for small, real-time inference calls (crucial for edge AI and low-latency voice or vision pipelines).