Real Time In Embedded Linux

What is real time?

A task is a real-time task if it has to complete before a certain point in time, known as the deadline.

Real-time systems can be classified into:

• Soft real-time: Missing a deadline is acceptable but may degrade the overall quality of the system.
• Hard real-time: Missing a deadline can lead to severe consequences. Hard real-time systems can be further divided into mission-critical systems and safety-critical systems.

Identifying sources of non-determinism:

The main goal of building a real-time system is to reduce variability in response time, ensuring higher reliability that deadlines will not be missed - that is, making the system more deterministic. Below are some areas that can cause non-determinism:

• Scheduling: Real-time threads must be scheduled before other threads and must follow a real-time policy (e.g., `SCHED_FIFO` or `SCHED_RR`) with decreasing priority order according to Rate Monotonic Analysis (RMA).
• Scheduling latency: The kernel must be able to reschedule immediately when an event occurs (e.g., an interrupt or timer) without unbounded delay.
• Priority inversion: Happens when a high-priority thread is blocked by a mutex held by a lower-priority thread.
• Accurate timers: Necessary for managing deadlines at millisecond or microsecond precision.
• Page faults: A page fault during execution of critical code can impact all timing estimates.
• Interrupts: Occur at unpredictable times and may introduce unexpected processing overhead.
• Processor caches: Introduce non-determinism, especially in multi-core devices.
• Memory bus contention: When peripherals access memory via DMA, they use memory bus bandwidth, slowing down CPU access.
• Scheduling Latency: This is the delay from the moment a wake-up event occurs (e.g., an interrupt or system timer) to the moment the thread actually starts executing.

PREEMPT_RT and Xenomai in Embedded Linux

PREEMPT_RT and Xenomai are two prominent solutions for achieving real-time capabilities in embedded Linux systems. Each approach has its own characteristics and is suitable for different requirements in real-time embedded system development.

PREEMPT_RT

PREEMPT_RT is a set of patches for the Linux kernel that transforms it into a fully preemptible real-time system. As of September 20, 2024, PREEMPT_RT has been fully integrated into the mainline Linux kernel, with version 6.12 being the first to have built-in real-time capabilities on supported architectures including x86, x86_64, RISC-V, and ARM64.

Working Principles

PREEMPT_RT converts the standard Linux kernel into a fully preemptible kernel, allowing higher-priority tasks to execute before lower-priority ones without being blocked by non-preemptible code sections. Specifically:

1. Fully Preemptible Kernel: Almost all kernel code can be interrupted to execute higher-priority tasks.
2. Threaded Interrupt Handling: In PREEMPT_RT, most interrupt handlers are converted into threads, allowing them to be preempted by higher-priority tasks.
3. Low Latency: Minimizes latency in task execution, allowing high-priority tasks to preempt lower-priority ones.

Advantages of PREEMPT_RT

1. Compatibility with Existing Applications: If an application runs on a standard Linux kernel, it will also run on PREEMPT_RT without modifications.
2. Wide Hardware Support: Leverages the diverse hardware support of Linux.
3. Familiar API: Allows the use of POSIX/Linux APIs, which are widely known and easy to use.
4. Easy Deployment: Simply install a kernel with the PREEMPT_RT patch and run your program.
5. Flexible Prioritization: Can adjust the priority of tasks and interrupts, even setting user tasks to have higher priority than interrupts.

Disadvantages and Limitations

1. Not for Lowest Latency: The goal of PREEMPT_RT is to make the kernel deterministic and preemptible, not to achieve the lowest possible latency.
2. Performance Impact: Using PREEMPT_RT mode impacts overall system performance.
3. Hardware Not Designed for RT: The hardware that Linux typically runs on is not designed with real-time purposes in mind.

Installing and Using PREEMPT_RT

Since kernel version 6.12, applying patches is no longer necessary on x86, arm64, and RiscV architectures. For other architectures or older kernels, patches are still required.

The basic installation process includes:

1. Download the kernel source code and PREEMPT_RT patch
2. Apply the patch to the kernel
3. Compile the kernel with the "Fully Preemptible Kernel (Real-Time)" option
4. Install the new kernel and reboot the system

Xenomai

Xenomai is a real-time development framework designed for Linux-based systems, allowing the creation of applications with strict timing requirements. Unlike PREEMPT_RT, Xenomai uses a dual-kernel architecture.

Dual-Kernel Architecture

Xenomai operates based on a separate real-time microkernel running alongside the Linux kernel:

1. Xenomai nucleus: A real-time microkernel running alongside the Linux kernel, handling real-time tasks.
2. pipe (Interrupt Pipeline): Technology that allows Xenomai and the Linux kernel to coexist. I-pipe intercepts and processes interrupts before they reach the Linux kernel.
3. Skins: Xenomai provides various "skins" (interfaces), allowing applications to use APIs from different RTOSes.

Xenomai 3 and Options

Xenomai 3 supports both dual-kernel and single-kernel configurations:

1. Cobalt core: The dual-kernel version, similar to Xenomai 2. A small extension is integrated into the Linux kernel, handling all time-critical operations such as interrupt handling and real-time thread scheduling.
2. Mercury core: Relies on the real-time capabilities of the native Linux kernel. Usually requires PREEMPT-RT to meet strict response time requirements.

Xenomai 4 with EVL

The latest version, Xenomai 4 with EVL (Emulated Virtualization Layer), continues to use the dual-kernel architecture but with a new virtualization layer:

• EVL: Virtualizes real-time services within the Linux kernel context, providing a lightweight and efficient real-time execution environment.

Advantages of Xenomai

1. High Determinism: Xenomai provides strict determinism and precise timing for real-time tasks.
2. Efficient Interrupt Handling: Interrupts are efficiently managed in the Xenomai environment to minimize interrupt latency.
3. Linux Compatibility: Xenomai is designed to be compatible with the Linux ecosystem, allowing real-time tasks to interact seamlessly with Linux services and drivers.
4. Multiple API Support: Provides various skins, making it easy to transition from other RTOSes.

Disadvantages and Limitations

1. Requires Application Modifications: Applications need to be recompiled to use Xenomai libraries.
2. Limited Hardware Support: Xenomai only supports certain architectures and kernel versions.
3. More Complex Deployment: Compared to PREEMPT_RT, Xenomai requires more changes to the kernel and applications.

Installing and Using Xenomai

The Xenomai installation process includes:

1. Download Xenomai source code and a supported Linux version
2. Use the prepare-kernel.sh script to integrate Xenomai support into the Linux source
3. Compile the kernel and Xenomai user libraries
4. Install and configure the system

Comparison of PREEMPT_RT and Xenomai

Reference link here

Reference link here