Installing Xenomai 3.x ====================== Introduction ------------ Xenomai 3 is the new architecture of the Xenomai real-time framework, which can run seamlessly side-by-side Linux as a co-kernel system, or natively over mainline Linux kernels. In the latter case, the mainline kernel can be supplemented by the https://www.kernel.org/pub/linux/kernel/projects/rt/[PREEMPT-RT patch] to meet stricter response time requirements than standard kernel preemption would bring. One of the two available real-time cores is selected at build time. The dual kernel core is codenamed _Cobalt_, the native Linux implementation is called _Mercury_. [NOTE] If you are looking for detailed information about installing a legacy Xenomai 2.x release, please refer to link:installing-xenomai-2.x[this document] instead. Please note that Xenomai 2.x is discontinued and not maintained anymore. Installation steps ------------------ Xenomai follows a split source model, decoupling the kernel space support from the user-space libraries. To this end, kernel and user-space Xenomai components are respectively available under the `kernel/` and `lib/` sub-trees. Other top-level directories, such as `scripts/`, `testsuite/` and `utils/`, provide additional scripts and programs to be used on either the build host, or the runtime target. The `kernel/` sub-tree which implements the in-kernel support code is seen as a built-in extension of the Linux kernel. Therefore, the standard Linux kernel configuration process should be used to define the various settings for the Xenomai kernel components. All of the kernel code Xenomai currently introduces implements the _Cobalt_ core (i.e. dual kernel configuration). As of today, the _Mercury_ core needs no Xenomai-specific code in kernel space. The `lib/` sub-tree contains the various user-space libraries exported by the Xenomai framework to the applications. This tree is built separately from the kernel support. Libraries are built in order to support the selected core, either _Cobalt_ or _Mercury_. [[cobalt-core-install]] Installing the _Cobalt_ core ---------------------------- Preparing the _Cobalt_ kernel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _Xenomai/cobalt_ provides a real-time extension kernel seamlessly integrated to Linux, therefore the first step is to build it as part of the target kernel. To this end, `scripts/prepare-kernel.sh` is a shell script which sets up the target kernel properly. The syntax is as follows: ------------------------------------------------------------------------------ $ scripts/prepare-kernel.sh [--linux=] [--ipipe=] [--arch=] ------------------------------------------------------------------------------ `--linux`:: specifies the path of the target kernel source tree. Such kernel tree may be already configured or not, indifferently. This path defaults to $PWD. `--ipipe`:: specifies the path of the interrupt pipeline (aka I-pipe) patch to apply against the kernel tree. Suitable patches are available from the project's link:/downloads/ipipe/[download area]. This parameter can be omitted if the I-pipe has already been patched in, or the script shall suggest an appropriate one. The script will detect whether the interrupt pipeline code is already present into the kernel tree, and skip this operation if so. `--arch`:: tells the script about the target architecture. If unspecified, the build host architecture suggested as a reasonable default. For instance, the following command would prepare the Linux tree located at `/home/me/linux-3.10-ipipe` in order to patch the Xenomai support in: ------------------------------------------------------------------------------ $ cd xenomai-3 $ scripts/prepare-kernel.sh --linux=/home/me/linux-3.10 ------------------------------------------------------------------------------ Note: The script will infer the location of the Xenomai kernel code from its own location within the Xenomai source tree. For instance, if `/home/me/xenomai-3/scripts/prepare-kernel.sh` is executing, then the Xenomai kernel code available from `/home/me/xenomai-3/kernel/cobalt` will be patched in the target Linux kernel. Configuring and compiling the _Cobalt_ kernel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once prepared, the target kernel can be configured as usual. All Xenomai configuration options are available from the "Xenomai" toplevel Kconfig menu. There are several important kernel configuration options, documented in the link:troubleshooting-a-dual-kernel-configuration#kconf[TROUBLESHOOTING] guide. Once configured, the kernel can be compiled as usual. If you want several different configs/builds at hand, you may reuse the same source by adding `O=../build-` to each make invocation. In order to cross-compile the Linux kernel, pass an ARCH and CROSS_COMPILE variable on make command line. See sections <>, <>, <>, for examples. [[cobalt-kernel-parameters]] _Cobalt_ kernel parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~ The Cobalt kernel accepts the following set of parameters, which should be passed on the kernel command line by the boot loader. [options="header",grid="cols",frame="topbot",cols="2,3,1"] |============================================================================ ^|NAME ^|DESCRIPTION ^|DEFAULT |xenomai.allowed_group= | Enable non-root access to Xenomai services from user-space. is the ID of the Linux user group whose members should be allowed such access by the Cobalt core. | None |xenomai.sysheap_size= | Set the size of the memory heap used internally by the Cobalt core to allocate runtime objects. This value is expressed in kilo-bytes. | 256 |xenomai.state= | Set the initial state of the Cobalt core at boot up, which may be _enabled_, _stopped_ or _disabled_. See the documentation about the link:../documentation/xenomai-3/html/man1/corectl/index.html[corectl(1)] utility for a description of these states. | enabled |xenomai.smi= | *x86-specific*: Set the state of the SMI workaround. The possible values are _disabled_, _detect_ and _enabled_. See the discussion about link:dealing-with-x86-SMI[SMIs] for a description of these states.| detect |xenomai.smi_mask= | *x86-specific*: Set of bits to mask in the SMI control register. | 1 (=global disable) |============================================================================ [[cobalt-build-examples]] Examples of building the _Cobalt_ kernel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The examples in following sections use the following conventions: `$linux_tree`:: path to the target kernel sources `$xenomai_root`:: path to the Xenomai sources [[cobalt-core-x86]] Building a _Cobalt/x86_ kernel (32/64bit) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Building _Xenomai/cobalt_ for x86 is almost the same for 32bit and 64bit platforms. You should note, however, that it is not possible to run Xenomai libraries compiled for x86_32 on a kernel compiled for x86_64, and conversely. Assuming that you want to build natively for a x86_64 system (x86_32 cross-build options from x86_64 appear between brackets), you would typically run: ------------------------------------------------------------------------------ $ cd $linux_tree $ $xenomai_root/scripts/prepare-kernel.sh --arch=x86 \ --ipipe=ipipe-core-X.Y.Z-x86-NN.patch $ make [ARCH=i386] xconfig/gconfig/menuconfig ------------------------------------------------------------------------------ ...configure the kernel (see also the recommended settings link:configuring-for-x86-based-dual-kernels[here]). Enable Xenomai options, then build with: ------------------------------------------------------------------------------ $ make [ARCH=i386] bzImage modules ------------------------------------------------------------------------------ Now, let's say that you really want to build Xenomai for a Pentium-based x86 32bit platform, using the native host toolchain; the typical steps would be as follows: ------------------------------------------------------------------------------ $ cd $linux_tree $ $xenomai_root/scripts/prepare-kernel.sh --arch=i386 \ --ipipe=ipipe-core-X.Y.Z-x86-NN.patch $ make xconfig/gconfig/menuconfig ------------------------------------------------------------------------------ ...configure the kernel (see also the recommended settings link:configuring-for-x86-based-dual-kernels[here]). Enable Xenomai options, then build with: ------------------------------------------------------------------------------ $ make bzImage modules ------------------------------------------------------------------------------ Similarly, for a 64bit platform, you would use: ------------------------------------------------------------------------------ $ cd $linux_tree $ $xenomai_root/scripts/prepare-kernel.sh --arch=x86_64 \ --ipipe=ipipe-core-X.Y.Z-x86-NN.patch $ make xconfig/gconfig/menuconfig ------------------------------------------------------------------------------ ...configure the kernel (see also the recommended settings link:configuring-for-x86-based-dual-kernels[here]). Enable Xenomai options, then build with: ------------------------------------------------------------------------------ $ make bzImage modules ------------------------------------------------------------------------------ The remaining examples illustrate how to cross-compile a _Cobalt_-enabled kernel for various architectures. Of course, you would have to install the proper cross-compilation toolchain for the target system first. [[cobalt-core-powerpc]] Building a _Cobalt/powerpc_ kernel ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A typical cross-compilation setup, in order to build Xenomai for a ppc-6xx architecture running a 3.10.32 kernel. We use the DENX ELDK cross-compiler: ------------------------------------------------------------------------------ $ cd $linux_tree $ $xenomai_root/scripts/prepare-kernel.sh --arch=powerpc \ --ipipe=ipipe-core-3.10.32-powerpc-1.patch $ make ARCH=powerpc CROSS_COMPILE=ppc_6xx- xconfig/gconfig/menuconfig ------------------------------------------------------------------------------ ...select the kernel and Xenomai options, save the configuration ------------------------------------------------------------------------------ $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux- uImage modules ------------------------------------------------------------------------------ ...manually install the kernel image and modules to the proper location [[cobalt-core-arm]] Building _Cobalt/arm_ kernel ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Using codesourcery toolchain named `arm-none-linux-gnueabi-gcc` and compiling for a CSB637 board (AT91RM9200 based), a typical compilation will look like: ------------------------------------------------------------------------------ $ cd $linux_tree $ $xenomai_root/scripts/prepare-kernel.sh --arch=arm \ --ipipe=ipipe-core-X.Y.Z-x86-NN.patch $ mkdir -p $build_root/linux $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- O=$build_root/linux \ csb637_defconfig $ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- O=$build_root/linux \ bzImage modules ------------------------------------------------------------------------------ ...manually install the kernel image, system map and modules to the proper location [[mercury-core-install]] Installing the _Mercury_ core ----------------------------- For _Mercury_, you need no Xenomai-specific kernel support so far, beyond what your host Linux kernel already provides. Your kernel should at least provide high resolution timer support (`CONFIG_HIGH_RES_TIMERS`), and likely complete preemption (_PREEMPT_RT_) if your application requires short and bounded latencies. Kernels with no real-time support can be used too, likely for basic debugging tasks, and/or running applications which do not have strict response time requirements. Therefore, unlike with _Cobalt_, there is no additional steps for preparing and/or configuring the kernel for _Mercury_. [[library-install]] Installing the Xenomai libraries and tools ------------------------------------------ Prerequisites ~~~~~~~~~~~~~ Generic requirements (both cores) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - GCC must have support for legacy atomic builtins (__sync form). - GCC should have a (sane/working) support for TLS preferably, although this is not mandatory if building with `--disable-tls`. - If you plan to enable the user-space registry support (i.e. +--enable-registry+), then CONFIG_FUSE_FS must be enabled in the target kernel running the real-time applications. In addition, the FUSE development libraries must be available from the toolchain. - If you plan to build from the sources available from the Xenomai GIT tree (git.xenomai.org), the autoconf (>= 2.62), automake and libtool packages must be available on your build system. This is not required when building from a source tree extracted from a link:/downloads/xenomai/[release tarball]. _Cobalt_-specific requirements ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - The kernel version must be 3.10 or better. - An interrupt pipeline (I-pipe) patch must be available for your target kernel. You can find the official patches issued by the Xenomai project link:/downloads/ipipe/[there]. Only patches from the *ipipe-core* series are appropriate, legacy patches from the *adeos-ipipe* series are not. - A timestamp counter (TSC) is required from running on a x86_32 hardware. Unlike with Xenomai 2.x, TSC-emulation using a PIT register is not available. _Mercury_-specific requirement ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - There is no particular requirement for Mercury setups, although using a NPTL-based glibc or uClibc is recommended. Configuring ~~~~~~~~~~~ If building the source obtained from the Xenomai GIT tree (git.xenomai.org), the `configure` script and Makefiles must be generated in the Xenomai source tree. The recommended way is to run the automatic reconfiguration script shipped, from the top of the source tree: --------------------- $ ./scripts/bootstrap --------------------- If building from a link:/downloads/xenomai/[release tarball], a set of autoconf-generated file will be readily available from the extracted source tree, and therefore reconfiguring will not be required. When run, the generated `configure` script prepares for building the libraries and programs, for both the _Cobalt_ and _Mercury_ cores. The core-specific code which may be needed internally is automatically and transparently selected at compilation-time by the build process. The options listed below can be passed to this script. Generic configuration options (both cores) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [horizontal] *--with=core=*:: Indicates which real-time core you want to build the support libraries for, namely _cobalt_ or _mercury_. This option defaults to _cobalt_. *--prefix=*:: Specifies the root installation path for libraries, include files, scripts and executables. Running `$ make install` installs these files to `$DESTDIR/`. This directory defaults to /usr/xenomai. *--enable-debug[=partial]*:: This switch controls the debug level. Three levels are available, with varying overhead: - _symbols_ enables debug symbols to be compiled in the libraries and executables, still turning on the optimizer (-O2). This option has no overhead, it is useful to get meaningful backtraces using gdb while running the application at nominal speed. - _partial_ includes _symbols_, and also turns on internal consistency checks within the Xenomai code (mostly present in the Copperplate layer). The `CONFIG_XENO_DEBUG` macro is defined, for both the Xenomai libraries and the applications getting their C compilation flags from the `xeno-config` script (i.e. `xeno-config --cflags`). The partial debug mode implicitly turns on `--enable-assert`. A measurable overhead is introduced by this level. This is the default level when `--enable-debug` is mentioned with no level specification. - _full_ includes _partial_ settings, but the optimizer is disabled (-O0), and even more consistency checks may be performed. In addition to `__XENO_DEBUG__`, the macro `CONFIG_XENO_DEBUG_FULL` is defined. This level introduces the most overhead, which may triple the worst-case latency, or even more. [normal] Over the _Mercury_ core, enabling _partial_ or _full_ debug modes also causes the standard malloc interface to be used internally instead of a fast real-time allocator (TLSF). This allows debugging memory-related issues with the help of _Valgrind_ or other dynamic memory analysers. *--disable-debug*:: Fully turns off all consistency checks and assertions, turns on the optimizer and disables debug symbol generation. *--enable-assert*:: A number of debug assertion statements are present into the Xenomai libraries, checking the internal consistency of the runtime system dynamically (see _man assert(3)_). Passing `--disable-assert` to the _configure_ script disables built-in assertions unconditionally. By default, assertions are enabled in partial or full debug modes, disabled otherwise. *--enable-pshared*:: Enable shared multi-processing. When enabled, this option allows multiple processes to share real-time objects (e.g. tasks, semaphores). *--enable-registry[=/registry-root-path]*:: Xenomai APIs can export their internal state through a pseudo-filesystem, which files may be read to obtain information about the existing real-time objects, such as tasks, semaphores, message queues and so on. This feature is supported by http://fuse.sourceforge.net/[FUSE], which must be available on the target system. Building the Xenomai libraries with the registry support requires the FUSE development libraries to available from the toolchain. In addition, CONFIG_FUSE_FS must be enabled in the target kernel. [normal] When this option is enabled, the system creates a file hierachy at `//` under the registry root path, where you can access the internal state of the active real-time objects. The session label is obtained from the --session runtime switch. If no session name is specified, `anon@` will be used. E.g. looking at the properties of a VxWorks task could be done as follows: If not specified in the configuration switch, the registry root path will be +/var/run/xenomai+. -------------------------------------------------------------------- $ cat /var/run/xenomai/root/anon@12656/12656/vxworks/tasks/windTask name = windTask errno = 0 status = ready priority = 70 lock_depth = 0 -------------------------------------------------------------------- [normal] You may override the default root of the registry hierarchy either statically at build time by passing the desired root path to the --enable-registry configuration switch, or dynamically by using the `--registry-root` runtime option passed to the application. [NOTE] When running over _Xenomai/cobalt_, the `/proc/xenomai` interface is also available for inspecting the core system state. *--enable-lores-clock*:: Enables support for low resolution clocks. By default, libraries are built with no support for tick-based timing. If you need such support (e.g. for pSOS (TM) or VxWorks (TM) APIs), then you can turn it on using this option. [NOTE] The POSIX API does not support tick-based timing. Alchemy may use it optionally. *--enable-clock-monotonic-raw*:: The Xenomai libraries requires a monotonic clock to be available from the underlying POSIX interface. When `CLOCK_MONOTONIC_RAW` is available on your system, you may want to pass this switch, otherwise `CLOCK_MONOTONIC` will be used by default. [NOTE] The _Cobalt_ core implements `CLOCK_MONOTONIC_RAW`, so this switch is turned on by default when building with `--with-core=cobalt`. On the contrary, this option is turned off by default when building for the _Mercury_ core, since we don't know in advance whether this feature does exist on the target kernel. *--enable-tls*:: Xenomai can use GCC's thread local storage extension (TLS) to speed up the retrieval of the per-thread information it uses internally. This switch enables TLS, use the converse `--disable-tls` to prevent this. [normal] Due to GCC bugs regarding this feature with some release,architecture combinations, whether TLS is turned on by default is a per-architecture decision. Currently, this feature is enabled for x86 and powerpc by default, other architectures will require `--enable-tls` to be passed to the _configure_ script explicitly. [normal] Unless `--enable-dlopen-libs` is present, the _initial-exec_ TLS model is selected. [normal] When TLS is disabled, POSIX's thread-specific data management services are used internally (i.e. pthread_set/getspecific()). *--enable-dlopen-libs*:: This switch allows programs to load Xenomai-based libraries dynamically, using the `dlopen(3)` routine. Enabling dynamic loading introduces some overhead in TLS accesses when enabled (see `--enable-tls`), which might be noticeable depending on the architecture. [normal] To support dynamic loading when `--enable-tls` is turned on, the _global-dynamic_ TLS model is automatically selected. [normal] Dynamic loading of Xenomai-based libraries is disabled by default. *--enable-async-cancel*:: Enables fully asynchronous cancellation of Xenomai threads created by the real-time APIs, making provision to protect the Xenomai implementation code accordingly. [normal] When disabled, Xenomai assumes that threads may exit due to cancellation requests only when they reach cancellation points (like system calls). Asynchronous cancellation is disabled by default. [CAUTION] Fully asynchronous cancellation can easily lead to resource leakage, silent corruption, safety issues and all sorts of rampant bugs. The only reason to turn this feature on would be aimed at cancelling threads which run significantly long, syscall-less busy loops with no explicit exit condition, which should probably be revisited anyway. *--enable-smp*:: Turns on SMP support for Xenomai libraries. [CAUTION] SMP support must be enabled in Xenomai libraries when the client applications are running over a SMP-capable kernel. *--disable-sanity*:: Turns off the sanity checks performed at application startup by the Xenomai libraries. This option sets a default, which can later be overriden using the --[no-]sanity options passed to a Copperplate-based Xenomai application. Sanity checks are enabled by default when configuring. *--enable-fortify*:: Enables `_FORTIFY_SOURCE` when building the Xenomai code unless --enable-debug=full is also given on the command line, in which case --enable-fortify is silently ignored. *--disable-valgrind-client*:: Turns off the Valgrind client support, forcing `CONFIG_XENO_VALGRIND_API` off in the Xenomai configuration header. *--enable-doc-build*:: Causes the inline Xenomai documentation based on the http://doxygen.org[Doxygen markup language] to be produced as PDF and HTML documents. Additional documentation like manpages based on the http://asciidoc.org/[Asciidoc markup language] is produced too. _Cobalt_-specific configuration options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [options="header",grid="cols",frame="topbot",cols="m,2*d"] |============================================================================ ^|NAME ^|DESCRIPTION ^|DEFAULT |--enable-x86-vsyscall |Use the x86/vsyscall interface for issuing syscalls. If disabled, the legacy 0x80 vector will be used. Turning on this option requires NPTL. |enabled |--enable-arm-tsc |Enable ARM TSC emulation. footnote:[In the unusual situation where Xenomai does not support the kuser generic emulation for the target SOC, use this option to specify another tsc emulation method. See `--help` for a list of valid values.] |kuser |--enable-arm-quirks |Enable quirks for specific ARM SOCs Currently sa1100 and xscale3 are supported. |disabled |============================================================================ _Mercury_-specific configuration options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [options="header",grid="cols",frame="topbot",cols="m,2*d"] |============================================================================ ^|NAME ^|DESCRIPTION ^|DEFAULT |--enable-condvar-workaround | Enable workaround for broken priority inheritance with condition variables in glibc. This option adds some overhead to RTOS API emulators. |disabled |============================================================================ footnoteref:[disable,Each option enabled by default can be forcibly disabled by passing `--disable-