DIAPM-RTAI
(Dipartimento di Ingegneria
Aerospaziale, Politecnico
di Milano - Real
Time Application
Interface)
A Hard Real Time support for LINUX
History and motivation for a Real Time Application Interface (RTAI)
It all started with a variant to NMT_RTLinux that grew out
of the need of having a periodic scheduler to enhance efficiency in control
applications, when one can work with a basic period and integer multiples
of it. In fact the original RTLinux release had just a oneshot timer that,
while very flexible in getting close to microsecond timing resolutions,
required much overhead in keeping the clock time and in being reprogrammed.
That is related to the use of the 8254 timer, which is connected to the
ISA bus, that is becoming more and more a dragging weight in relation to
the CPU power of today's PCs. Moreover the initial release had no real
time support functions, i.e. semaphore, timing functions, messages etc.,
and that made impractical the implementation of relatively complex control
applications requiring a few cooperating tasks.
Ironically at some time we found ourselves in the need
of using oneshot precise heterogeneous timers to implement PWM control
systems at medium frequencies that, if done within the RTLinux tasks, would
have allowed much more flexibility than an hardwired implementation. Again
however the original release maintained its architecture and its overhead
remained excessive. So we implemented the oneshot timers in a different
way by using the CPU TSC (Time Stamp Clock) with far great efficiency,
but with earlier than Pentium machines, and compatibles, no more usable.
Another important reason for a variant was the fact it
solved the problem of a bug free FPU support that , for some time, has
been lacking in the official release.
Right now the official RTLinux has solved most of these
problems and has a periodic timer too, along with semaphores and mailboxes.
However the oneshot timing is still less efficient.
With the releases for Linux 2.2.xx we went back to what
we were thinking before RTLinux appeared, i.e. develop Real Time
Hardware Abstraction Layer (RTHAL) onto which a Real Time Application
Interface (RTAI) could be mounted to make Linux usable for hard real time
applications.
Unfortunately kernel 2.0.25, on which we were working
at the very beginning of the all story was to much unclean in design, for
the idea we had in mind. Thus we were lucky that people at
NMT introduced RTLinux, as it gave us the possibility of gradually learning
about the kernel and its relation to the hardware.
So when 2.2.xx appeared to have a cleaner interface to
the hardware we could go back to our original idea, but with a somewhat
more deeper understanding of what was behind.
What has come out is a comprehensive Real Time Application
Interface (RTAI), usable both for uniprocessors (UP) and for symmetric
multi processors (SMP), that allows the use Linux kernel 2.2.xx for
many hard real time applications.
SMP tasks are defaulted to work on any cpu but you can
assign them to any subset, or even to a single
cpu, by using the function "rt_set_runnable_on_cpus". It
is also possible to assign any real time interrupt service to a specific
cpu by using "rt_assign_irq_to_cpu" and "rt_reset_irq_to_sym_mode".
Thus a user can statically optimize his/her application
if he/she believes that it can be better done than
by using a symmetric load distribution. The
possibility of forcing any interrupts to a specific cpu is clearly not
related to the smpscheduler and can be used also with interrupt
handlers alone.
Note that only the real time interrupt handling is forced
to a specific cpu. That means that if you check
this feature by using "cat /proc/interrupts" for a
real time interrupt that is chained to Linux, e.g the timer when rtl_sched
is installed, you can still see some interrupts
distributed to all the cpus, even if they are
mostly on the assigned one. That is because Linux interrupts are
kept symmetric by the RTAI dispatcher of Linux irqs.
The schedulers allow to chose between a periodic and
a oneshot timer, not to be used together. The periodic
ticking is less flexible but, with the usual PC hardware
much more efficient. So it is up to you which one to choose in
relation to the applications at hand.
It should be noted that in the oneshot mode the time
is measured on the base of the CPU time stamp clock
(TSC) and not on the 8254 chip, which is used only to
generate oneshot interrupts. The periodic mode is instead timed by the
8254 chip only.
In this way slow I/Os to the ISA bus are limited as much
as possible with a sizeable gain in efficiency. The
oneshot mode has just about 15-20% more overhead
than the periodic one. It is likely that local APIC timers could lead
to a further improvement.
Right now local APIC timers are hard disabled on
UPs and a preliminary experience with with a
single SMP local APIC timer, to be released soon for SMP, shows that
there is no performance improvement for a periodic
scheduling while the oneshot case gain is sizeable,
but not so large with respect to the already available solution.
In fact by using the TSC just two outb are required to
reprogram the 8254, i.e 2.5 us approximately,
against almost nothing for the APIC timer. However
you have to broadcast a message to all the cpus in any case, and that
is about 2 us, the APIC bus is an open drain 2 wires one
and is not lightning like. Note
that the performance loss of the 8254 is just a fraction of the
overall task switching procedure, which is always substantially
heavier in the oneshot case than in periodic mode.
Since the TSC is not available on 486 machines, for them
we use a form of emulation of the "read time
stamp clock" (rdtsc) assembler instruction based on counter2
of the 8254. So you can use RTAI also on such machines. Be warned that
the oneshot timer on 486 is a performance overkill because
of the need of reading the tsc, i.e. 8254 counter2
in this case, twice. That can take 6-8 us, i.e.
more than it takes for a full switch among many tasks while using a
periodic timer. Thus only a few Khz period is viable for
real time tasks if you want to keep Linux alive.
No similar problems exist for the periodic timer that
needs not to use any TSC at all. So,
compared to the 20% cited above, the real time performance ratio of the
oneshot/periodic timer efficiency ratio can be very low on
486 machines. Moreover it will produce far worse
jitters than those caused on Pentiums and upward machines.
If you really need a oneshot timer buy a Pentium, at least.
Instead, for a periodic timing, 486s can be still more
than adequate for many applications.
A feature of our RTAI implementation is that interrupt
handlers preambles take care natively of the
task switched (TS) flag. Thus you can freely use floating point
operations in your interrupt handlers, without causing a trap fault
whatever thing Linux is doing.
RTAI is thus very suitable to trap interrupts without
minding of Linux so that you can effectively interact
with the bare PC hardware much in the same way
I, and maybe many of you, liked the old good time of DOS.
With RTAI you have the added advantage that Linux maintains
all of its features untouched so that you can
pass to it whatever you get from your handler for logging,
displaying and postprocessing, by using fifos and/or shared memory.
Imagine a remote controller at 10 Khz, +-5 us average
interrupt uncertainty, connected through the
internet, with all the bells and whistles of X and its
applications. It is an application we simulated easily.
See the "examples/timer" directory in examples for some clues,
without remote control however.
Note that RTAI has also some very useful system services,
including: timings, semaphores, messages and remote procedure
calls (RPC). That makes it easier to develop
complex real time applications. RPCs are a limited form
of QNX messages as they pass either just an unsigned integer or a pointer
to an unsigned integer for reason of efficiency. They can
be easily changed to be fully compatible with
QNX if you'd like.
There are many significant examples that allow you to
experiment with forcing tasks and timer interrupts
to any cpu, a summary of cpu usage is printed at examples
module rmmoding.
Many warm thanks are due to Emanuele Bianchi, Davide Martini,
Lorenzo Dozio, Maurizio Quadrio and Daniele
Danlugli for their cooperation, suggestions and precious
help in various jobs using Linux for hard real time control systems.
Emanuele, Davide and Lorenzo did a nice work to verify
and test our RTLinux variant of NMT-RTLinux
in production work for their thesis and following research
works at DIAPM.
This helpful html documentation is almost entirely due
to Emanuele, Davide and Lorenzo.
Lorenzo played also a very important role in retrofitting
our efficient oneshot timers to the then purely periodic
timing control, because of the need to use them to
easily pulse width modulate air jets emulating on-off control rockets.
Maurizio is one of the top Linux experts at DIAPM and
has been our "great saver" during some
deadly disk losses occurred while developing and using RTL and RTAI
Daniele has been very fundamental in the past scheduler
development, especially in relation to fpu support.
He was the first one to point out unbearable
jitters when the fpu was used in RTLinux and in its DIAPM variant.
Without his convincing cries many people, including ourselves,
would never have known that they were using
the fpu just to generate troubles in their applications.
It is because of that that the fpu support is included natively
also for RTAI interrupt handlers.
For other information on RTAI see the files INSTALL and
README in the rtai distribution.