Kernel changes - November 2013
These tests were in response to NTP bug 2314, where I reported that the Raspberry Pi sometimes reported NTP jitter as zero in loopstats, making it somewhat more difficult to see whether any change had brought about an improvement or not. They involve recompiling the kernel, which is a long-winded process, but one I have documented here (with notes and help from Rob Windgassen).
There are two results presented here, one is a Raspberry Pi card which was using the user-mode PPS option from Folkert van Heusden, where the switch from user-mode to kernel mode is expected to reduce jitter and offset, and the other is a Raspberry Pi which was using the kernel-mode PPS from Hauke Lampe. Here, the improvements are due to the configuration changes made to the kernel and suggested by Rob Windgassen.
Using the improved kernel configuration from Rob Windgassen
The RMS offset has been reduced from about 0.6 Ás to less than 0.3 Ás, and the jitter is now being recorded (proving that bug 2314 was due to the kernel configuration, and not a fault in NTP).
However, looking at the frequency reported by the loopstats it seems that it is now being adjusted in steps rather than smoothly, and perhaps this needs further investigation.
Switching from user-mode to kernel-mode
The RMS offset has been reduced from over 1.5 Ás to a much lower value (perhaps the 0.3 Ás seen above), and the averaged jitter has been reduced from 3.8 Ás to about 1.2 Ás. Once again we see the frequency adjustment is no longer smooth, but stepped.
Some thoughts from Jorge Amaral, Portugal
My hardware is composed of a raspi model b and a Adafruit ultimate v3 GPS modem.
While I was copying your work, i decided to use driver 20 for ntp (ntp speaks directly with the gps modem, no more
shm). You can forget running the gpsd. The only changes that I have
./configure --disable-all-clocks --disable-parse-clocks --without-lineeditlibs --enable-NMEA --enable-LOCAL-CLOCK --enable-SHM --enable-linuxcaps --enable-ATOM
Then after ntp compiled and installed i create a symlink for ntp to read the gps and the pps devices:
cd /dev ln -sf /dev/ttyAMA0 /dev/gps0 ln -sf /dev/pps0 /dev/gpspps0
and the only thing missing is the ntp.conf slightly changed:
Comment from Pete Stephenson:
Two other things that I have changed were the cpu_scaling and the serial latency:
echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor setserial /dev/ttyAMA0 low_latency
As you can see, it's working with GPS and PPS support, and precision is 2^-20 (1Ás)
Once again thank you very much for your work and I hope that this can help you too :-)
I was delighted to receive this note from Philip Gladstone about reducing the Ethernet latency on the Raspberry Pi.
It appears that the Ethernet latency on the RPi is fairly large, but it can be reduced at a slight cost in performance. To do this, edit your:
along with all the other parameters that are being set.
DJT: I was able to put this to the test recently when I was doing some NTP measurements, and from a Raspberry Pi which had been running for a while, I made the change to cmdline.txt and rebooted, with the following effect on NTP delay as reported by a remote PPS-locked server:
The delay has been reduced from ~0.51 to ~0.35 milliseconds. Linux ping times showed a similar improvement:
For comparison, here is a similar plot from the BeagleBone Black, which does not have the Ethernet via USB configuration:
Matthew George NG7M in Syracuse Utah sent the following notes:
Thanks for all your effort and details on your website. I did want to pass along some info that stumped me for weeks now. I started into the NTPD and GPSDO obsession after the 3.18.7+ Raspbian kernel source was out... so it was all about device tree and the modules if you are compiling your own version of the kernel.
I have been stumped as to why I could never get modules loading when I compiled my own version of 3.18.13 on a Raspberry Pi 2. So there are a couple of things you might want to add to your website if someone is hell bent on getting kernel level PPS going with the newer kernel source and device tree etc.
Here is the magic trick which maybe you already know about if you are using the
RPi 2. After you compile and link the kernel image, modules will not load unless you run another script that is part of the Linux source distribution to enable Device Tree modules in the kernel.
$ ../../../scripts/mkknlimg Image kernel7.img
Then copy the new kernel7.img to your /boot directory as root... Bingo! Shazam... Finally, I have kernel PPS working with a couple of different GPSDO devices... a Trimble Thunderbolt and a Motorola Oncore UT+ in my case.
Before I was lucky to get consistent microsecond offsets and now I'm consistent with sub microseconds and clearly in the land of nanosecond offsets! Here is some output from my loopstats file for example with kernel PPS running with 3.18.13: (ntpd up maybe 20 minutes):
57165 85453.930 -0.000000019 -6.178 0.000001206 0.001265 3 57165 85461.930 0.000000198 -6.178 0.000001418 0.001184 3 57165 85469.930 0.000000820 -6.177 0.000002364 0.001117 3 57165 85477.930 0.000000200 -6.177 0.000001489 0.001045 3 57165 85485.930 0.000000625 -6.177 0.000001315 0.000983 3 57165 85493.930 0.000000490 -6.177 0.000002125 0.000924 3 57165 85501.930 0.000000559 -6.177 0.000001567 0.000870 3 57165 85509.930 0.000000860 -6.176 0.000002316 0.000826 3 57165 85517.930 0.000000269 -6.176 0.000002903 0.000775 3 57165 85525.930 -0.000000412 -6.176 0.000002248 0.000728 3 57165 85533.930 -0.000000479 -6.176 0.000002118 0.000686 3 57165 85541.930 0.000000179 -6.176 0.000001374 0.000643 3 57165 85549.930 0.000000644 -6.176 0.000001699 0.000612 3 57165 85557.930 0.000000009 -6.176 0.000001122 0.000572 3 57165 85565.930 -0.000000176 -6.176 0.000000947 0.000536 3 57165 85573.930 -0.000000222 -6.153 0.000002364 0.008075 3
This is with a maxpoll at 3 as you can see... not much difference with maxpoll 4 with PPS now in my experience.
Anyway Google: *mkknlimg* and you will find the posts about needing to run this script on your new 3.18.7+ kernel for the raspi 2 to get modules working / Device Tree working. My first RPi was an RPi 2, so no experience with the previous RPi models.
I hope this might help someone else that comes along. Your website is a fantastic resource and I hope you find my details above worth posting... :)
Best Regards David.