	Getting started with Real-Time Linux

	Michael Barabanov (baraban@fsmlabs.com)
	Copyright VJY Associates L.L.C. 1999. All rights reserved.


	Introduction

Real-Time Linux is a hard real-time operating system that can coexist
with the Linux OS on the same computer. With Real-Time Linux, it is
possible to create real-time POSIX.1b threads that will run at precisely
specified moments of time. The white paper in rtl/doc/design.pdf
explains the basic architecture in more detail.

This document provides a quick starting guide for new users of Real-Time Linux (RTL).
Other documents useful in this context are:

- The Real-Time Linux whitepaper (URL)
- The Single UNIX specification (http://www.opengroup.org/onlinepubs/7908799/index.html)
- The LinuxThreads library documentation (included with glibc2). You can try
running man 3 pthread_create to see if it is installed on your system
- Getting Started With POSIX Threads (by Thomas Wagner and Don Towsley)
http://centaurus.cs.umass.edu/~wagner/threads_html/tutorial.html
- Other documents or books describing POSIX threads.

The Real-Time Linux distribution contains UNIX manual pages for
functions specific to Real-Time Linux in rtl/man directory.
You can modify the MANPATH environment variable accordingly.

The html/man directory contains manual pages converted to HTML.

The best way to get started with Real-Time Linux is to browse through
the included example programs. In particular, examples/measurements
and examples/fp are recommended.

Be sure to also read the Frequently Asked Questions file included with
the distribution.


	General Notes

Since Real-Time programs in RTL are executed in the kernel space, special care
must be taken when programming real-time tasks. Programming errors may easily bring the
system down. The future versions of RTL will provide ways to debug
RT-threads in user space, and then gradually switch to the hard real-time mode.


	Linux Modules

This version of Real-Time Linux supports loading RT-programs in the form
of Linux modules. A Linux module source file is an ordinary C language
file, with the main() function replaced by a pair of init/cleanup
functions:

int init_module();

This function is called when the module is loaded in the kernel. It
should return 0, on failure, a negative value.

void cleanup_module();

This function is called when the module is unloaded.

The compiled module is loaded in the kernel with the insmod(1) command.
See also the Linux Kernel-HOWTO.


	Writing Real-Time Linux POSIX threads

Threads are light-weight processes sharing a common address space.
Conceptually, the Linux kernel threads of control (one for each CPU in
the system) are also RTL threads.
In RTL, all threads share the Linux kernel address space.

To create a new Real-Time thread, use pthread_create (3) function.

#include <pthread.h>

int  pthread_create(pthread_t  *  thread, pthread_attr_t *
      attr, void * (*start_routine)(void *), void * arg);

Note. This function must only be called from the Linux kernel thread (e.g.,
from init_module()).

The thread is created using the attributes specified in the "attr" thread attributes object.
If attr is NULL, default attributes are used. See POSIX functions
pthread_attr_init(3), pthread_attr_setschedparam(3), 
pthread_attr_getschedparam(3), and RTL-specific functions
pthread_attr_getcpu_np(3), pthread_attr_setcpu_np(3)

The id of the created thread is stored in the location pointed to by
"thread". The function pointed to by start_routine is taken to be
the thread code. It is passed the "arg" argument.

To kill a thread, the (non-portable) pthread_kill_np RTL function 
can be used:

int pthread_delete_np (pthread_t thread);
The thread stops its execution immediately.


	Scheduling threads

RTL provides a way to request the thread code to be run at particular
moments of time (scheduling). Real-Time Linux provides several clocks that can
be used as a reference for thread scheduling. To obtain the current
clock reading, use the clock_gettime(3) function.

int clock_gettime(clockid_t clock_id, struct timespec *ts);

clock_id is the clock to be read. ts is the location to store
the read value.

struct timespec {
	time_t  tv_sec; /*     seconds */
	long    tv_nsec /*  nanoseconds */
};

Note. The currently supported clocks are:

CLOCK_UST. This stands for "Unadjusted System Time". This clock is never
adjusted or reset. It provides a nice source of timestamps.

CLOCK_REALTIME. This is the standard POSIX realtime clock. Currently 
it is the same as CLOCK_UST. It is planned that this clock
will give the world time in future versions of RTL.

CLOCK_8254. Used on non-SMP machines for scheduling.

CLOCK_APIC. Used on SMP machines. CLOCK_APIC corresponds to the local
APIC clock of the processor that executes clock_gettime. You can not
read or set the APIC clock of other processors.

You can get the clock that the scheduler uses for task scheduling
by calling the rtl_getschedclock(3) function.


Real-time Linux provides a pure priority-driven scheduler.
In this scheduler, the highest priority ready thread is always
chosen to run. If the two threads have the same priority, which
one is chosen is undefined. The thread priority can be modified
at thread creation time using pthread_attr_setschedparam(3)
or afterwards using pthread_setschedparam(3).

int pthread_setschedparam(pthread_t thread, int policy,
    const struct sched_param *param);

The policy argument is currently not used in RTL, but should be specified
as SCHED_FIFO for compatibility with future versions.
The struct sched_param contains the sched_priority member.
Higher values correspond to higher priorities. Use
sched_get_priority_max(3) and sched_get_priority_min(3) to
determine possible values of sched_priority.

To make a real-time thread execute periodically, the following
function can be used:

int pthread_setperiod_np(pthread_t thread, const struct itimerspec *its);

struct itimerspec {
	struct timespec  it_interval; /*  timer period */
	struct timespec  it_value;    /*  timer expiration */
};

pthread_setperiod_np marks the thread as periodic. The timing is
specified by the "its" parameter. The it_value member of the passed
struct itimerspec specified the time of the first invocation;
the it_interval is the thread period. The period can be 0,
which corresponds to a one-time task invocation.

The actual execution timing is done with the pthread_wait_np(3) function

int pthread_wait_np(void);

This function suspends the execution of the calling thread until
the time specified by pthread_setperiod_np().


	The simplest "Hello World" RTL program


#include <rtl.h>
#include <time.h>
#include <pthread.h>

pthread_t thread;

void * start_routine(void *arg)
{
	struct sched_param p;
	p . sched_priority = 1;
	pthread_setschedparam (pthread_self(), SCHED_FIFO, &p);

	pthread_make_periodic_np (pthread_self(), gethrtime(), 500000000);

	while (1) {
		pthread_wait_np ();
		rtl_printf("I'm here; my arg is %x\n", (unsigned) arg);
	}
	return 0;
}

int init_module(void) {
	return pthread_create (&thread, NULL, start_routine, 0);
}

void cleanup_module(void) {
	pthread_delete_np (thread);
}

Write this program to a file called hello.c, copy any rtl.mk
file from the distribution to the same directory and compile
the program with make -f rtl.mk hello.o

Note. You should have the RTL compiled and installed for this
to succeed.

Note. This program is also found in examples/hello

Once you have the timer (rtl_time.o) and the scheduler (rtl_sched.o)
modules loaded, you can use the following command to start
the thread:
insmod hello.o

You should see the thread printing messages. To stop the program, run
rmmod hello.


	Using Real-Time FIFOs

The general philosophy of Real-Time Linux is that the real-time part
of an application should be fast, small, and simple. You should try
to split your application in a way where most work is done in user
space. This approach makes for easier debugging and better understanding
of the real-time part of the system.

Several communication mechanisms exist to allow real-time threads
and user space Linux processes to communicate. The most important
ones are Real-Time FIFOs and shared memory.

Real-Time FIFOs are First-In-First-Out queues that can be read and
written by Linux processes and Real-Time threads. FIFOs are
uni-directional. You can use a pair of FIFOs for bi-directional data exchange.
To use the FIFOs, system/rtl_posixio.o and fifos/rtl_fifo.o Linux
modules must be loaded in the kernel.

RT-FIFOs are Linux character devices with the major number of 150.
The device file names are /dev/rtf0, /dev/rtf1, etc through /dev/rtf63
(the maximum number of RT-FIFOs in the system is configurable
during system compilation). The device entries in /dev are created
during system installation.

Before a real-time FIFO can be used, it needs to be initialized.

#include <rtl_fifo.h>

int rtf_create(unsigned int fifo, int size);
int rtf_destroy(unsigned int fifo);

rtf_create allocates the buffer of the specified size for 
the fifo buffer. The fifo argument corresponds to the minor
number of the device.

rtf_destroy deallocates the FIFO.

Note. These functions must only be called from the Linux kernel
thread (e.g., from init_module()).

After the FIFO is created, the following calls can be used
to access it from Real-Time threads: open(2), read(2), write(2),
close(2). The support is planned for all of the stdio functions
support.

Note. The current implementation requires the FIFOs to be opened
in the non-blocking mode (O_NONBLOCK) by Real-Time threads.

Linux processes can use UNIX file IO functions without restrictions.

See the examples/measurement/rt_process.c example program to
see how RT-FIFOs can be used.


        Using shared memory

For the shared memory, you can you the excellent mbuff driver
by Tomasz Motylewski <motyl@stan.chemie.unibas.ch>. It is included
in the distribution and is located in the drivers/mbuff directory.
The manual is included with the package. Here I just briefly
describe the basic mode of operation. 

First, the mbuff.o module must be loaded in the kernel.
Two functions are used to allocate blocks of shared memory,
connect to them and eventually deallocate them.

#include <mbuff.h>

void * mbuff_alloc(const char *name, int size);
void mbuff_free(const char *name, void * mbuf);

The first time mbuff_alloc is called with the given name,
a shared memory block of the specified size is allocated. The
reference count for this block is set to 1. On success,
the pointer to the newly allocated block is returned. NULL
is returned on failure.
If the block with the specified name already exists,
this function returns the pointer that can be used to access
this block and increases the reference count.

mbuff_free deassociates mbuf from the specified buffer.
The reference count is decreased by 1. When it reaches 0,
the buffer is deallocated.

These functions are available for use in both Linux processes
and the Linux kernel threads.

Note. mbuff_alloc and mbuff_free can not be used from Real-Time
threads. You should call them from init_module and cleanup_module
only.


	Accessing physical memory and IO ports from Real-Time threads


These capabilities are essential for programming hardware devices in the
computer.

RTL, just like ordinary Linux, supports the /dev/mem device (man 4 mem)
for accessing physical memory from RTL threads. The rtl_posixio.o module
must be loaded. The program opens /dev/mem, mmaps it,
and then proceeds to read and write the mapped area.
The program in examples/mmap shows how to do that.

The IO port access functions are described specifically for the x86
architecture

#include <asm/io.h>

void outb(unsigned int value, unsigned short port); /* output a byte to a port */
void outw(unsigned int value, unsigned short port); /* output a word to a port */
char inb(unsigned short port); /* read a byte from a port */
short inw(unsigned short port); /* read a word from a port */

The functions with the _p suffix (e.g. outb_p) provide a small delay
after reading or writing the port. This delay is needed for some slow
ISA devices on fast machines.
(See also Linux I/O port programming mini-HOWTO).

Look in examples/sound to see how some of these functions are used to
program the PC Real-Time clock and the speaker.


	Using Floating Point Operations in RTL POSIX threads

By default, the use of floating-point operations in RTL POSIX threads
is prohibited. You can use the following RTL-specific function
to change the status floating-point operations:

int pthread_setfp_np (pthread_t thread, int flag);

To enable FP operations in the thread, use the flag of 1. To disable
FP operations, pass 0.

The examples/fp directory contains several examples of tasks using
floating point and the math library.


	Symmetric Multi-Processing Considerations

From the point of view of thread scheduling, the Real-Time Linux
implements a separate UNIX process for each active CPU in the system.
In general, thread control functions can only be used 
for threads running on the local CPU. The notable exceptions are:

int pthread_wakeup_np (pthread_t thread); /* wake up suspended thread */
int pthread_delete_np (pthread_t thread); /* destroy the thread */


	Interfacing RTL components to Linux

(RTL white paper discusses this topic in more detail).
RTL threads, sharing the common address space with the Linux
kernel, in principle can call the Linux kernel functions. However,
in general, this is not safe to do, because RTL threads may
run even while Linux has interrupts disabled.
Only functions that do not modify Linux kernel data structures
(e.g., vsprintf), may be called from RTL threads.

RTL provides two mechanisms of delayed execution to overcome this
limitation: soft interrupts and task queues.


	Using Interrupts. Soft interrupts and shared hard interrupts

There are two types of interrupts in Real-Time Linux: hard and soft.
The soft interrupts are normal Linux kernel interrupts. They
have an advantage that some Linux kernel functions can safely be called
from them (which ones, depends on the kernel version). However, for
many tasks, they do not provide hard real-time performance: they
may be delayed for considerable periods of time.

Hard interrupts (or real-time interrupts), on the other hand,
have much lesser latency. However, just like for real-time threads,
only a very limited set of kernel functions may be called from the
hard interrupt handlers. The solution is to split the functionality
between hard and soft interrupt handlers.

#include <rtl_core.h>

int rtl_request_irq(unsigned int irq, 
		unsigned int (*handler)(unsigned int, struct pt_regs *));
int rtl_free_irq(unsigned int irq); 

These two functions are used for installing and uninstalling hard interrupt 
handlers for specific interrupts. The manual pages describe
their operation in detail.

Interrupt-driven RTL threads can be created using thread wakeup/suspend
functions:

int pthread_wakeup_np (pthread_t thread);
int pthread_suspend_np (void);

An interrupt-driven thread calls pthread_suspend_np and blocks.
Later the interrupt handler calls pthread_wakeup_np for this thread.
The thread will run until the next call to pthread_wakeup_np.


	Soft interrupts

int rtl_get_soft_irq( void (*handler)(int, void *, struct pt_regs *),
		const char * devname);

This function allocates a virtual irq number and installs the handler
function for it. This virtual interrupt can later be triggered
using rtl_global_pend_irq(3).

void rtl_global_pend_irq(int ix);

rtl_global_pend_irq is safe to use from real-time threads
and real-time interrupts.

void free_irq(unsigned int irq, void *dev_id);

free_irq is the Linux kernel function that uninstalls the handler
for the interrupt "irq". For the interrupts requested with
rtl_get_soft_irq the dev_id should be 0.

Soft interrupts are used in the Real-Time Linux FIFO implementation
(fifos/rtl_fifo.c).


	RTL Task Queues

TODO

	Writing RTLinux device drivers. The POSIXIO module

TODO

	RT-Linux serial driver (rt_com)

rt_com is a driver for 8250 and 16550 families of UARTs commonly used in
PCs (COM1, COM2, etc).

Please refer to drivers/rt_com for examples of using serial IO from
real-time programs.


	Writing RTLinux schedulers and modifying the standard scheduler

This is an advanced topic to be written later. Here are some points:
the scheduler is implemented in the scheduler/rtl_sched.c and
architecture-dependent files in include/arch-i386 and scheduler/i386.
The scheduling decision is taken in the rtl_schedule() function.
By modifying this function, it is possible to change the scheduling policy.

