Software impacts on design for multi-core processors in single chip

Paul Boughton

Arun Subbarao looks at the issues for operating systems using multi-core devices in embedded systems.

Recent innovations in processor architecture have been focused on creating multi-core processors with two or more processing cores in a single chip, giving operating systems and applications increased computing power. The benefits are obvious; applications that were designed around uni-processor configurations can replace uni-processor cores with dual or quad core processors. The computing power of these configurations increases dramatically with no appreciable change in its physical configuration.

The software impact of multi-core processors is fairly immediate on operating systems design. The OS has to adapt to support symmetric multi-processing (SMP) or asymmetric multi-processing (AMP), two major approaches for support of multi-core processors. The operating system design has to adapt in the areas of scheduling, interrupt handling, synchronisation and load balancing. Application programs can also be affected by multi-core processors based on the ability of the OS to provide fine-grained control of process scheduling to applications. For example, an application can make a request to execute on a specific processor core only. However, the increase in compute power through multi-core processors can be better harnessed through another trend in OS design - Virtualisation.

Virtualisation is a technique used to create an execution environment for software that is similar than the one it was originally designed for, but on a different hardware or operating system. This can be achieved usually at two levels, in the operating system or in hardware.

Operating System Virtualisation is done using binary compatibility layers that run on heterogeneous operating system environments, while presenting an interface similar to the original OS environment. This is most often done to achieve migration and execution of applications across multiple heterogeneous operating system environments.

Hardware Virtualisation involves the emulation of the underlying hardware capabilities to allow operating systems themselves to run in a hardware environment different from its original environment. These software programs that emulate the underlying hardware capabilities are called virtual machines (VM) or virtual machine monitors (VMM). A VM abstracts the capabilities of hardware and makes it available in environments different from the original hardware.

Hardware Virtualisation can also be extended to allow multiple heterogeneous operating systems to execute on single physical machine. The ample computing resources of modern multi-core processors make this extension possible. However, these multiple instances of heterogeneous operating systems need to execute in a resource isolated environment, with no functional impact to other instances of operating systems. This is essential since they will be sharing computing resources.

Enabling multiple instances of heterogeneous operating systems on a single machine involves solving technical challenges in virtualisation and resource isolation, while retaining complete binary compatibility and acceptable level of performance.

Virtualising multiple instances of an operating system can be done using either full or partial virtualisation. This provides the illusion of real hardware for the operating systems executing on this virtual machine. However, both full and partial virtualisations have some key differences in their overall architecture, leading to a different set of trade-offs.

Full virtualisation of the underlying hardware requires virtualising all the capabilities of the processor and board. This involves complex manipulations of memory management and privilege levels that are computation intensive on commodity processors. This leads to performance overheads that are much higher than the non-virtualised versions of the OS. However, the biggest benefit of full virtualisation is to allow operating systems to run unmodified, although at the cost of a significant performance overhead.

Partial or Para-Virtualisation is usually a technique where the underlying hardware is not completely simulated in software. This architecture allows commodity operating systems to be easily virtualised on commodity processors, although, with the requirement that the virtualised operating system requires code modifications to adhere to the partially virtualised architecture.

The other key requirement for running multiple operating systems in the context of a virtual machine is the ability to isolate the physical resources of a computer. This is achieved by time-space partitioning, a concept used extensively in safety-critical and secure systems.

In a time-space partitioned system, the virtual machine sub-divides two key computing resources: CPU time and physical memory. Here, the physical memory is divided into unique, non-overlapping ranges, and assigned to individual heterogeneous virtualised operating systems.

The time scheduler allocates periods of CPU time to each virtualised OS that is usually fixed and cyclic. This gives the illusion of exclusive access to computing resources for the virtualised operating systems. The ability of the virtual machine to support time-space partitioning is a basic pre-requisite for the execution of multiple virtualised operating systems on a single machine.

Both full and partial Virtualisations support 100 per cent binary compatibility with the stand-alone version of the operating system. It also allows the ability to retain the benefits of multiple address spaces within a single operating system instance. One significant difference between a stand-alone operating system and a virtualised version is that the virtualised OS runs in a less privileged, user mode. This is necessary since the virtual machine that provides the virtualised architecture is the sole entity that is running at highest, supervisor, level.

The MILS (Multiple Independent Levels of Security/Safety) architecture adopts the best principles of security and safety-critical design to define a hard real-time, secure embedded OS that can be evaluated to the highest levels of security (EAL7) and safety assurance (DO178B), while preserving the flexibility to support diverse security policies. The architecture identifies four key security policies: Information Flow, Data Isolation, Residual Information Protection and Damage Limitation.

This uses a small partitioning kernel (RTOS) that runs in supervisor mode and provides brick-wall partitioning of memory, time and I/O resources. The partitioning kernel only provides the basic functionality needed to support the underlying hardware. Within each partition, the traditional OS functionality executes in user mode completely isolated from other partitions. The middleware and applications make up the rest of the components that may execute in a single partition.

An example architecture that exemplifies the principles of virtualisation, real-time and security on multi-core processors is LynxSecure. This combines time-space partitioning and virtualisation to allow multiple, heterogeneous operating systems to execute in a robust, highly secure environment on 64-bit, multi-core processors. It allows safety-critical and secure operating systems to function alongside non-secure operating systems without compromising the entire system's security, reliability and data integrity.

Arun Subbarao is VP Engineering at LynuxWorks, San Jose, CA, USA.