Patch the kernel, in the directory of the kernel run this line:
patch -p1 -b < ../../rtai-3.9/base/arch/x86/patches/hal-linux-2.6.38.8-x86-2.11-02.patch
When I tryed to install RTAI 3.9 in Ubuntu 12.04 with kernel 2.6.38.8 I have those problems:
apt-get install gcc-multilib
cd /usr/include/ mv gnu gnu.sav ln -fs i386-linux-gnu/gnu gnu
RTAI kernel patch does not work properly. When the rtai_hal.ko RTAI module is loaded with IsolCpusMask (e.g. insmod rtai_hal.ko IsolCpusMask=0x02). It crashes with error message (using dmesg):
BUG: unable to handle kernel NULL pointer dereference at 0000000000000030 IP: [<ffffffff8101b932>] __ipipe_set_irq_affinity+0x11/0x61
To solve the problem a solution is to modify the kernel file ./arch/x86/kernel/ipipe.c adding the code:
if (irq_to_desc(irq) == NULL) return CPU_MASK_NONE;
in the function __ipipe_set_irq_affinity(), just after the declaration cpusmask_t oldmask;. The the kernel must be recompiled.