Patches

The patches mentioned in the table and examples below can be downloaded from the PATCHES subdirectory. Do not install the nfs_tcp patch!

List of 2.2 Linux Kernels we've used

Feature Patch Kernel version
2.2.20-si 2.2.19-cfht 2.2.16nfs3 22.16nfs3rt 2.2.16-cfht 16-raid-nfs3
devfs support devfs-patch-v99.16 PATCHv99.21 PATCHv99.20 PATCHED PATCHED PATCHED ---
nfs3 client nfsv3-0.21.3.dif SET SET PATCHED PATCHED PATCHED PATCHED
nfs3 server dhiggen-over-0.21.3 SET SET PATCHED PATCHED PATCHED PATCHED
nfs over TCP (broken!) nfs_tcp.dif NO(ok) NO(ok) DOH!!! NO(ok) NO(ok) NO(ok)
syskonnect ethernet --- SET SET NOTSET NOTSET SET NOTSET
aic7xxx SCSI support --- SET SET --- PATCHED PATCHED ---
aic7xxx SCSI for rtai aic7xxx-5.1.32 SET SET --- PATCHED PATCHED ---
3C905C support (420s) 3c59x-24JUN00 SET SET PATCHED ??? PATCHED SET(old ok?)
promise ultra/66 supp. si-ide.20000825 --- SET=ok? --- --- PATCHED ide.20000805
DVD support SET SET-ok? SET-ok? SET-ok? SET ???
new raid (0.9) support -raid-A0 --- 2.2.19-A1 --- --- PATCHED PATCHED
cramfs support si-cramfs PATCHED PATCHED --- --- PATCHED ---
Misc CFHT patch si-nfs3/over-raid-A0(cosmetic) (cosmetic) si-nfs3 si-nfs3 over-raid-A0
rtai real-time linux si-rthal3 (rtai1.6) --- rtai1.7(SMP!)--- rtai1.3 PATCHED ---
2+GB file size (LFS) YES(1) YES(1) --- --- NO (2GB) ---
USB support YES YES --- --- NO (USB) ---
(1) Requires C library upgrade.
SET: Standard support compiled in from unpatched kernel.
NOTSET: Standard support exists but was not turned on in this build.
PATCHED: A patch was added to add proper support.
KERNELVER=linux-2.2.16nfs3    # ... oops, with tcp, but this is what we're running:
PATCHLIST="devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 nfs_tcp.dif 3c59x-24JUN00.patch si-nfs3.patch"

KERNELVER=linux-2.2.16nfs3new # should have been this way:
PATCHLIST="devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 3c59x-24JUN00.patch si-nfs3.patch"

KERNELVER=linux-2.2.16nfs3rt  # RTAI-1.3
PATCHLIST="devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 aic7xxx-5.1.32.patch 3c59x-24JUN00.patch si-nfs3.patch si-rtai-1.3.patch"

KERNELVER=linux-2.2.16nfs3rthal3 # RTAI-1.6 (you can turn off RTAI in "make menuconfig" now...)
PATCHLIST="devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 aic7xxx-5.1.32.patch 3c59x-24JUN00.patch si-ide.20000825.patch si-cramfs.patch si-nfs3.patch si-rthal3.patch"

KERNELVER=linux-2.2.16-cfht
PATCHLIST="devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 aic7xxx-5.1.32.patch 3c59x-24JUN00.patch si-ide.20000825.patch raid-A0.patch si-cramfs.patch si-over-raid-A0.patch si-rthal3.patch"

KERNELVER=linux-2.2.20-si # Does NOT support: ultra66, RTAI, or XFS.
PATCHLIST="devfs-v99.21.patch si-apmblank.patch si-cramfs.patch si-iopener.patch si-misc.patch si-cosmetic.patch si-makefile.patch raid-A0-over-devfs.patch"

devfs-patch-v99.16 nfsv3-0.21.3.dif dhiggen-over-0.21.3 aic7xxx-5.1.32.patch 3c59x-24JUN00.patch si-ide.20000825.patch raid-A0.patch si-cramfs.patch si-over-raid-A0.patch si-rthal3.patch"

# Example command for generating a patched kernel tree:
# tar xzf linux-2.2.16.tar.gz
# mv linux $KERNELVER
# cd $KERNELVER
# for i in $PATCHLIST \
  ; do zcat ../PATCHES/linux-2.2.16-$i.gz | patch -p1 || break ; done
# find -name '*.rej'

Harddrive Speed Tests:

5400RPM Maxtor drives w/2MB cache:
model capac platform timedsk hdprm -t hdprm -T
98196H8 80 22 (pono /dev/hda1,/dev/hdf1)
98196H8 11x80 6 (pono:/local/data)
98196H8 80 610 10 23 122 (dethost2)
98196H8 3x80 610 17 (dethost2)
94098U8 40 410 17 17 124 (buttle)
94098U8 40 ultra33 23 25 138 (elixir2 /tmp /dev/hdh1)
94098U8 7x40 410 32 32 132 (elixir2,druid /dev/md0)
7200RPM Maxtor drives:
model capac platform timedsk hdprm -t hdprm -T
5T040H4 40 20 23 128
5T040H4 3x40 20 27 137
5T040H4 40/66 Ultra 28 (with the normal IDE driver in 2.4.2)
5T040H4 40/66 Ultra 35 (with the ultra66 IDE driver in 2.4.2)
10KRPM SCSI Quantum drive:
model capac platform timedsk hdprm -t hdprm -T
ATLAS 9GB 610 20

100MBit Ethernet Tests

proto r/wsize MTU from to MB/sec
nfs3 8192 1500 2.2.16 2.4.2(40/66)8

Gigabit Ethernet Tests

proto r/wsize MTU from to MB/sec
ftp n/a 1500 2.2.16(scsi)2.4.2(ramfs)37
ftp n/a 9000 2.2.16(scsi)2.4.2(ramfs)55
nfs3 8192 1500 2.2.16 2.2.16(scsi)12
nfs3 8192 9000 2.2.16 2.2.16(scsi)12
nfs2 8192 9000 2.2.16 2.2.16(scsi)24
nfs2 4096 9000 2.2.16 2.2.16(scsi)24
nfs3 8192 1500 2.2.16 2.2.16(ramdisk)20
nfs3 8192 9000 2.2.16 2.2.16(ramdisk)25
nfs2 8192 9000 2.2.16 2.2.16(ramdisk)30
nfs3 8192 9000 2.2.16 2.4.2(40/66)26
nfs2 8192 9000 2.2.16 2.4.2(40/66)26
nfs2 4096 9000 2.2.16 2.4.2(40/66)20
nfs3 8192 9000 2.4.2 2.2.16(scsi)7
nfs2 8192 9000 2.4.2 2.2.16(scsi)20
Legend:
Nothing in ()'s in the from field means data came from "timedisk".
(ide) means the data was read from or written to a 40/60/80GB Maxtor IDE drive.
(raid) means the data was read from or written to a striped IDE software RAID.
(scsi) means the the data was read from or written to a 9GB Quantum Atlas.
(ramfs) means the data was read from or written to a ramdisk or ramfs.

Linux notes from July 5, 2000

 
New root fileserver.
================
 There is now a new root filesystem server for all the Linux machines.
 It holds the operating system and configuration for each machine
 which it serves at bootime. This task used to be handled by Druid
 but is now handled by this dedicated server. For reliability,
 don't use this host for anything other than its intended use.

syslog
======
  All NFS-root clients now log system messages to their console and to
  their boot server.  This means all messages are now going to
  server:/var/log/messages.  A lot of the "noisy" messages
  from syslog and NFS have been fixed too.

kernel
======
  The kernel version is stock 2.2.16 (not the RedHat frankenmonster 2.2.16
  kernel!)  Inevitably, some patches were applied though:

   - devfs v99.16 (a great help to NFS root machines)
   - the latest 3c59x Ethernet driver from the 2.2.17pre tree.
   - a small fix to the RAID-0 stuff from the 2.2.17pre tree.
   - Three knfs patches from oss.sgi.com
     1. NFS3 client
     2. NFS3 server
     3. nfs_tcp (don't use this last mode... it doesn't seem well tested.)

  An identically configured kernel plus RTAI realtime-linux capabilities
  is also available for testing.  It seems to have all the functionality
  of the regular kernel but with realtime response to interrupts, and
  programmable timers.  (For many applications, this seems like a good
  alternative to VxWorks!)  Both kernel images should run fine on any
  e-machine, 210, 410, or 420 workstation that we have.  (I have ordered
  some 3c59x ethernet cards for the e-machines, so they will look the
  same as the 410s.)

gcc-2.7.2
=========
  The kernel (and anything "root" compiles) is automatically built by
  gcc-2.7.2, not the "egcs" compiler in /usr/bin.  If you need to try
  gcc-2.7.2 for your own stuff, put /usr/local/gcc272/bin at the front
  of your PATH.  This is already the default for root.

NFSv3 client
============
  Linux boxes are now reading and writing Sun disks using NFS3
  instead of NFS2.

Writing to a Sun disk:    5 MB/sec  (was < 0.5 MB/sec with synchronous NFS2!)
Reading from a Sun disk: 11 MB/sec  (was 9.5 with NFS2, and CPU load during a transfer is lower also.)

  If you try your own read tests, be aware that you could be reading
  from a cache.  On a Sun, you can see rates around 200 MB/sec in
  this case!  On Linux the cached read speed is around 100 MB/sec,
  or maybe higher on a dual CPU machine.

NFSv3 server
============
  Linux boxes are now allowing Suns to access the Linux disks with
  the NFS3 protocol also.  Here we see a slight hit in performance,
  but the gain should be in reliability.

Writing to a Linux disk:   8.0 MB/sec (was around 10 MB/sec but unreliable)
Reading from a Linux disk: 8.5 MB/sec (was around 8.5 MB/sec)

  The numbers for NFS3 from Linux->Linux are a little different:

Writing from another Linux machine: 6.5 MB/sec (was around 7.5)
Reading from another Linux machine: 10 MB/sec (was around 7.5)

  So reading is improved but writing is slightly slower.  Writing
  across NFS from Linux->Linux also incurs a higher CPU load on
  the server than it did with NFS2.  In general, the switch to NFS3
  seems to have helped us though.

  *** File locking is untested *** (Eugene may test this.)

 . . .