S-Link to PCI driver for Linux 2.2/2.4 and RTAI
by Sidik Isani, Canada-France-Hawaii Telescope
2000 June 14th
Last revised: 2001 October 31th (v1.8)
Version 1.8 2003-03-04 - Makefile hacking and removed sys/types include from sspci.h.
Version 1.7 2001-07-26 - Added separate irq handlers for realtime/non-real-time.
Version 1.6 2001-07-18 - Fixed race condition for very short (<1Kb) DMA packets.
Version 1.5 2001-06-11 - Added new section for SLIDAS and updated slidas-example.
Version 1.4 2001-05-30 - Now compiles with 2.2.14 to 2.2.19 and 2.4.5 with/without devfs.
Version 1.3 2001-05-23 - Minor fix for 2.4 kernel and libc5. Documentation updated.
Version 1.2 2001-03-18 - Himemfb+sspci both work with devfs and 2.4 kernel now. Added calls to libspio for DMA transfers (see spio.h).
Added byte-swapping ioctls (see sspci.h).
The slidas-example now does everything through libspio.
Version 1.0.3 2000-10-23 - Fixed Makefiles and added preliminary libspio.
- S-Link
receiver ROMB/LDC devices only.
- Linux 2.2 or 2.4 Kernel
module (tested with 2.2.16 and 2.4.4)
- RTAI Real-Time interrupt handler option (tested with 2.2.16+rtai-1.3 and 1.6)
- Multiple cards supported (select MAX at compile time)
- Interrupt sharing between identical S-Link cards on the same IRQ
- Interrupt sharing with other devices (except with RTAI)
- read() system call with standard behavior for slow transfers
- Blocking read
- Non-blocking read
- Interruptible
- DMA read using ioctl's (or libspio) for fast transfers to reserved, physically linear RAM
- Contiguous read of up to 4GB without user intervention
- Ring buffer mode
- Continuous ring buffer mode
- Special ring buffer mode for descrambling block data
- select() support to notify user programs when DMA or FIFO data
is ready to be read.
- libspio library (included) to control most S-Link features.
- ioctl()'s to control some other low level features.
- /proc/sspci interface shows some useful statistics.
- /dev/himemfb for memory reservation (works without patching your kernel)
- Compilation into kernel not tested. (Compile as module for now.)
- No ``software scatter-gather'' mode; DMA requires ``himemfb'' driver (included).
- Need to add locking/synchronization for ring-buffer mode.
- Need to add ioctl's to get current buffer.
- Need to add write() interface as in other slink driver?
- Need to implement read timeout ioctl
- Download the latest copy of the driver - currently sspci-1.8.tgz
from http://software.cfht.hawaii.edu/sspci.
- Choose an appropriate kernel. Probably most 2.2.x and 2.4.x kernels
will be fine. To have real-time features, a 2.2.x+rtai real-time kernel is required.
We use 2.2.16 and have done limited testing with 2.4.4.
- Reserve some memory for the DMA buffer area. This is done at boot
time using the ``mem=XXX'' kernel command line option. For example,
to reserve 16 MB for DMA transfer buffer on a machine with 128 MB,
boot the kernel with ``mem=112M''. If you use LILO, this can be
permanently added to /etc/lilo.conf by putting the line
append="mem=112M"
in the appropriate section(s). Remember to change ``112M'' to an
appropriate value for your system.
- Compile and install the ``himemfb'' driver. (Change to the himemfb/
directory, type ``make'' and then ``insmod himemfb.o'' to load
the driver into your kernel. Make sure /usr/src/linux points
to the same kernel source tree as the one you are running.)
This driver will detect
the memory set aside by the mem=XXX option and make it available
to the user through mmap(). NOTE: applications must call mmap on
/dev/himemfb, not /dev/sspci with this driver. The
recommended approach is to use the calls in libspio, provided with
this driver, to get access to the DMA buffer. See the header
file of this library libspio/spio.h, and
slidas-example/slidas.c for more help.
- Compile and install the ``sspci'' S-Link driver. (Change to the
sspci directory, type ``make'' and then ``insmod sspci.o''
to load the driver into your kernel.) Use the -DRTAI option
only if RTAI is installed. (See sspci/Makefile for access to this
compile-time option.) Normally, the driver will build without
RTAI support unless it finds rtai.h in your normal kernel
include directory.
- Type ``cat /proc/sspci'' to see if all cards were properly initialized.
TIP: The ``sspci-watch'' script makes a simple text status display.
The library libspio should make it easy for an application
to access the features of the himemfb and sspci drivers. An
example application, designed to work with a SLIDAS board, has
been included with the driver. If you actually have
SLIDAS hardware
with which to test your S-Link receivers, here is how we configured
our test machine:
- The machine has 1.5 GB of RAM
- The kernel boot option mem=768M was given to to reserve
768M for Linux (leaving the other 768M for himemfb to find.)
If you have less memory to commit to your test, the slidas-example
program should still work. The himemfb driver will grab whatever
Linux isn't using, and slidas-example should use the full amount.
A few MB should be enough for a good test.
- We have a pair of SLIDAS boards in our test machine.
If slidas-example finds the pair, it will try to do both
simultaneous and separate reads. If only a single card is
found, the tests with ``one board'' and ``all boards'' are
equivalent. Just like the memory size, slidas-example should
detect the number of S-Links automatically.
- A cable can be installed which connects URL bits from
connector J1 to the LEMO connector. The slidas-example program
assumes this cable is in place, and will toggle the URL bits
when it wants to trigger data. Without this cable, it should be
possible to start the flow of data by pressing the Run/Step
button manually after slidas-example resets the link.
(To run this way, I recommend compiling slidas.c
with undef COMPLETE_TESTS. I'm not sure the FIFO tests
will wait long enough for you to press the Run/Step button
manually. The DMA tests should, however.)
- The slidas-example contains a C function
which calculates the same pseudo-random number sequence as
the SLIDAS.
``SW1'' must be turned to position 6 to generate
the expected pseudo-random data with no control words.
(Remember, control words are not implemented in the driver.
We do not use them for our application.)
- ``SW2'' should be in position 0 for ``continuous''
data when the Run/Step button is pressed.
- ``SW3'' should be in position 1 to generate data words
(in position 0 it would generate only control words.)
- ``SW4'' is in position 2 for most of our tests, because
a 10 MHz LCLK frequency best simulates our real application.
The slidas-example assumes this clock frequency is selected.
You can select another frequency, but some of the statistics
generated by slidas-example might be bogus.
- ``SW5'' is set to ``WORD''.
- ``SW6'' and ``SW7'' are all ``OFF'' because the driver does
not handle errors.
- ``SW8'' must be set to ``LDC''.
- ``SW9'' (LDOWN) is set to ``DOWN'' in our tests.
Q: Why doesn't the install procedure mention creating /dev/sspci*?
A: We use devfs, so it doesn't affect us.
If you don't use devfs
you can still create the device nodes with the following commands:
mknod /dev/himemfb c 122 0
mknod /dev/sspci0 c 120 0
mknod /dev/sspci1 c 120 1
However, as long as your application uses libspio, things should
work even if you don't have the proper entries in /dev/ since
the library creates temporary ones in /tmp/ in that case.
Q: What causes the ``driver reply queue overrun'' errors?
A: These are normal when slidas-example is doing the FIFO-mode
tests, since the SLIDAS is set to send data continuously.
Q: The slidas-example detects mismatches (data errors). What's the problem?
A: The slidas-example could fail on any of the tests if your
S-Link receiver boards have an old version of the AMCC chip.
These old chips are known to have problems with newer PCI
chipsets.
Normally, slidas-example will do FIFO tests first, and then DMA.
If slidas-example is failing before it gets to the DMA
tests, try editing slidas.c and undef'ing COMPLETE_TESTS.
This causes it to skip directly to a DMA ring buffer test.
Sidik Isani
2003-06-16