Windows Serial Port GPS/PPS reference clock for NTP

I took renewed interest in using a Windows box instead of my FreeBSD box as a local stratum 1 reference clock when I became aware of some work done by Dave Hart.  The Windows port of NTP has long supported using an RS-232 GPS device with just a NMEA serial data, but the software has not supported a PPS (pulse per second) feed to the CD (carrier detect) pin.  The old timeout-based serial data handling gave an unreasonably large jitter, meaning that a local serial clock could be less accurate than an Internet source!  Windows has had the problem of poor timestamp resolution preventing its use as a highly accurate reference clock.  However, Dave has done some work on the interpolation algorithm used, which has enabled much better timekeeping from Windows.  Ntpd long ago overcame the OS clock precision using interpolation, Dave Hart's changes have been to improve that interpolation, most substantially removing about 1 ms systemic jitter.

He has also done work on an update to the serial device driver (serial.sys) with a view to making the driver provide the timestamps rather than the user program, which should enable even greater precision to be obtained.  It's not clear to me at the moment how this driver will fit in with NTP - perhaps it will be supplied with the NTP distribution and will be used by the common NTP PPSAPI Atom driver?

Dave Hart writes: The official NTP distribution is source-only.  The patch or the entire source for serialpps.sys might become part of the distribution if Microsoft's DDK license allows, but the binary wouldn't be.  On the other hand, I wouldn't be surprised to see serialpps.sys included by Meinberg with their binary distribution, though the quality of installer (i.e. Dave Hart's installer) is laughable by comparison, so they might opt to let people download it separately.  Either way, I would expect the PPSAPI implementation for serialpps.sys to be included in the ntp distribution sources and Meinberg binaries.

Note that if you are using a non-standard COM port, or wish to avoid having to install non-standard drivers, you can use the user-mode loopback-ppsapi-provider DLL instead.  Thanks to Meinberg for providing this!  More details.
 

For the impatient - brief details

• Install the Meinberg NTP port for Windows - download here: http://www.meinbergglobal.com/english/sw/ntp.htm
• Installation instructions are here.
• For Windows Vista and Windows-7 I recommend installing to C:\Tools\ rather than C:\Program Files\
• Download the latest development binaries for Windows here.
• Stop the NTP service, and overwrite the files in C:\Tools\NTP\bin\ with those from the download.
• Restart the NTP service and make sure it all works - your Firewall might need to be told about changed program versions.
• Set your GPS so that it only sends $GPRMC sentences only at 4800 baud.  Too many sentences won't allow one per second.  This setting will depend on what your GPS receiver allows.
• Attach your GPS to a free COM port, and note the COM port number (e.g. COM4).
• Add a line to your ntpd.conf like:
    server 127.127.20.4 minpoll 4 prefer # NMEA serial port
  The fourth number in the IP-address like field (127.127.20.4) should be the COM port number - 4 in this example.
• Connect your GPS to your PC, and follow any steps to remove the "Microsoft Serial BallPoint Mouse" which has appeared in the Device Manager - details http://www.gpsmap.net/GarminHints.html#GPSR_ComputerMouse
• Reboot, and check that NTP is working correctly.  An "ntpq -p" should include the line:  *GPS_NMEA(4)  where the port number is in brackets.  
• If running on Windows Vista or Windows-7, try setting the system environment variable NTPD_USE_INTERP_DANGEROUS = 1, restart NTP, and see whether it improves the timekeeping performance.
• For 64-bit systems, where unsigned drivers are not allowed, see here: NTP on Vista and Windows-7

There is a step-by-step guide here from Jeremy W. Langston who followed the guide on this page.  I would recommend that you use the latest version available rather than the specific version listed by Jeremy.
 

Hardware details

The hardware is essentially unchanged from my earlier FreeBSD stratum-1 clock, with a small box to interface a Garmin GPS 18 LVC puck to a serial port on a PC.  I did build a second version of the hardware where I use the more recent GPS 18x LVC puck which is much more sensitive and allows indoor operation.  This version used a spare USB port for the 5V power required by the puck.  Please note some issues with the 18x firmware, which mean that you are only using the 18x for precise within-the-second sync, and using another source (e.g. Internet) for the nearest-second sync.

Environment Variables for NTP

NTP can use a number of system environment variables to control its operation.  For a typical end-user, client-only install of NTP, none of these settings will be required.  For a stratum-1 NTP server install, you may only need the PPSAPI_DLLs variable, and even then only if you want the lower jitter of the kernel-mode PPS driver.  Note: you will need to restart the NTP service for these changes to take effect.

• NTPD_CPU - Processor mask to which to wire thread.
  E.g. NTPD_CPU=1
   
• NTPD_INT_INT - Timer Interval override.  When using NTPD interpolation, the performance may benefit from a value between 20 and 50 milliseconds.
  E.g. NTPD_INT_INT=29
   
• NTPD_PCC - PPC is a generalised term meaning "processor cycle counter", set to any value to force the use of the PCC if the frequency is also set.
   
• NTPD_PCC_FREQ - Force use of RDTSC with the frequency specified.  No longer recommended.
  E.g. NTPD_PCC_FREQ=498250000  (for a 500 MHz nominal CPU).
    
• NTPD_TICKADJ_PPM - For specifying baseline clock slew on Microsoft Windows.  If your ntp.drift was limiting at its positive extreme add +500, or if your ntp.drift was limiting at its negative extreme add -500.  You may need to iterate to find the best setting for your own system.  Note that most modern PCs are within 100 ppm, so an extreme value in ntp.drift may indicate other problems.
  e.g. NTPD_TICKADJ_PPM=500
   
• NTPD_USE_INTERP_DANGEROUS - If set, forces the NTP time interpolation to be used, disabling the system clock.  Value does not matter.
  e.g. NTPD_USE_INTERP_DANGEROUS=1
   
• NTPD_USE_SYSTEM_CLOCK - If set, forces the Windows clock to be used, disabling interpolation.  Value does not matter.
  e.g. NTPD_USE_SYSTEM_CLOCK=Yes
   
• PPSAPI_DLLS - Points to the PPSAPI provider DLL.
  e.g. PPSAPI_DLLS=C:\serialpps\serialpps-ppsapi-provider\x86\serialpps-ppsapi-provider.dll
  e.g. PPSAPI_DLLS=C:\Tools\NTP\PPS\serialpps-ppsapi-provider.dll

How to set a system environment variable in Windows XP and Windows-7.
 

Note from Dave Hart - How to update to local ref-clock

[With slight edits from DJT]  Windows 2000 should work based on reports from testers (DJT - it does).

• Download the latest version, dated 20090226 or later, from http://davehart.net/ntp/refclock/ Replacing only ntpd.exe is fine.
    
• Edit the configuration file to remove the Internet sources or add minpoll to each to keep from hitting them every 16-64s.
    
• Add:

server 127.127.20.1 minpoll 4 prefer

If your GPS is sending $GPRMC each second at 4800 baud it should just work.  If it's configured to send only a different NMEA sentence, we'll need a mode option on that server line.  You could check Pixie's ntp.conf, if it's using a mode option on the server 127.127.20.x line copy it.  [DJT: in my case, the GPS was already sending the correct $GPRMC sentences.]

Using real-time priority

For potentially greater precision, you can allow ntpd.exe to raise itself to the  real-time priority level.  You can do this in one of two ways.  If security is not of the greatest concern, use method 2, otherwise use method 1.  Dave Hart writes:

• Method 1 -

  net localgroup administrators ntp /de

  Next, open Local Security Policy. There are different approaches, but Start/Run secpol.msc is easiest for me to type :)  Drill down to Local Policies then User Rights Assignment.  Scroll down the right side to see "Increase Scheduling Priority".  Double-click on it to edit and in the resulting dialog box you'll likely see only Administrators listed.  Click the "Add user or group" button and type "ntp" and hit enter.  If you used a different user name when installing ntp, type that instead of "ntp".  Click OK twice, close the Local Security Policy window, and restart ntpd to verify it worked.

• Method 2 - add the ntp user account to the local administrators group, so that ntpd.exe will be able to raise itself to the  real-time priority level.  You can do that with this command-line entry:

  net localgroup administrators ntp /add

  Then you should be able to restart the ntp service and verify it's working.

Look in Task Manager on the Processes tab for ntpd.exe.  If you don't see it, make sure "Show processes for all users" is enabled.  Show the priority by choosing View / Select columns... from the menu, and in that box turn on the check mark next to "Base priority".  Click OK and you should see ntpd.exe at Realtime base priority.  If it's High, the privilege wasn't available.

If you're running one of my test releases, a message is written to the event log or configured log file at startup indicating whether High or Realtime priority is being used.

Real-time priority in practice

Please look at the graphs below which show the value of jitter as reported by "ntpq -c rv" versus time.  The priority of the ntpd.exe process was changed from high to real-time during the period of these graphs.  Can you see when?

It was actually at 06:20 on Sunday, just to the left of centre in the top graph.  It may have made a slight difference, but not a lot.  On this system, at least, the offset is much greater than the jitter (possibly due to the relatively large temperature variations to which the system is subject).  It's in a room in a domestic environment, and not in a temperature regulated computer room.

A Gotcha!

The gotcha is making sure you expect to lose the serial port on next reboot to the BallPoint Serial Mouse driver.  Avoid a nasty surprise the next time Microsoft pushes out an off-cycle automatic update by planning to reboot, disable the BallPoint Serial Mouse device in Device Manager, and reboot again before too long after you move the GPS to the Windows box.  But it doesn't have to be right away, it's a gamble you'll win most nights :)

[DJT] My Windows 2000 Server system also saw a Microsoft Serial Mouse, which I also had to disable and reboot yet again.  Seeing the reach at 200 (meaning that just one value had been read from the serial port) was the clue.  You disable the spurious mouse by going to Device Manager and selecting the Serial BallPoint Mouse.  Right-click, and select Disable.  Don't uninstall.

[DJT] I received an e-mail noting that Microsoft had a COMDisable tool available.  See: KB819036.  

"Although it has been a year or so since I used it, I think I recall that it was able to resolve all of our issues with Windows detecting an NMEA GPS as a serial mouse at boot time.  This may actually do the same thing as your disabling things in Device Manager, but I think it is likely a better solution."

Implementation Notes

Hardware

As I don't have a second GPS receiver, I simply made up a lead with a parallel connection to the data and carrier detect signals from the GPS.  It seems to work in my environment, with signal levels still around 4 volts even with one "RS-232" transmitter driving two "RS-232" receivers.  I saw no detrimental effect to the accuracy of the existing FreeBSD system on PC Pixie.  Of course, this is outside the "official" RS-232 standards, and for a proper installation multiple line drivers should be used.  I later added the more sensitive GPS 18x LVC for testing, powered off the 5V USB port rather than a separate power supply.

In the early stages of testing, you can use my Serial Port LEDs monitor program to check that you are seeing the DCD pulse one per second.  You need to stop NTP while using this program as access to the serial port cannot be shared.  Please note: If you are using Windows for NTP with a PPS signal, the DCD line must flash briefly on, not be mostly on flashing briefly off as inverting the PPS signal is not supported by the Windows NTP port (i.e. fudge <ref-clock> flag2 1 isn't supported).  Mike reported that after a few pulses the DCD appeared to stop flashing.  While I can't explain that, Mike noted that Realterm included an similar facility and echoed the NMEA data as well!  You can get Realterm here.

Software

A new version of ntpd.exe is required to support GPS/PPS operation over a serial line with Windows.  I installed this version (actually just overwriting the existing ntpd.exe file was sufficient in my case), and looking at the Event Log I found the following entry, which shows that the PPS line on the CD pin is being detected:

Using user-mode PPS timestamp

When I first changed to the new ref-clock I got an unexpectedly large amount of jitter, and a few 20ms steps in the offset.  I had seen no steps when operating this PC just with Internet and LAN-based servers.  Dave Hart noticed from the Event Log entry:

HZ 99.856 using 43 msec timer 23.256 Hz 64 deep 

that I had a system with a 100Hz clock (quite unusual), and following a brief test, he provided me with an ntpd.exe which had a 26 millisecond sampling interval for his additions rather than the default 43ms.  To check that the correct 26msec interval has been used, check for the line:

HZ 99.856 using 26 msec timer 38.462 Hz 64 deep 

The initial results from this were much better, with jitter from ntpd -c rv being reported in the 200µsec region, so I therefore tried his serialpps.sys driver which adds timestamps to the received data (better timestamps, timestamps for more events?).  To know that the new driver is working, look for this line in the event log:

Using serialpps.sys timestamps

Please note that later versions of the drivers from Dave Hart change this, so that the serialpps.sys talks to the ATOM ref-clock driver, and not the GPS/NMEA ref-clock, which reverts to using user-mode timestamps.  You need to run both ref-clocks to get the kernel-mode timestamps.

Results

The results were not as good as Dave Hart had expected, although they were certainly better than just using Internet sources.  There was an issue with this particular PC, as it was discovered to be using a 100Hz timing interrupt (10msec), and the code which Dave had written to enhance the interpolation does not work as well with 100Hz.  We did try playing the value of the sampling, and found that 26 or 27 were better values for a 100Hz system.  However, we could not get the low jitter values which Dave expected.

To set the sampling value, you can set the environment variable NTPD_INT_INT to the value 26, using the Control Panel, System, Advanced page.  Be sure to set the System variables, and not the User variables!

One Windows 2000 issue which we encountered was that just using restart in the Services control application was that the environment variables weren't read afresh when a Restart operation was requested.  To overcome this, I made a small batch file to run NTPD.exe interactively.  For example:

net stop ntp
SET NTPD_INT_INT=27
CD C:\Tools\NTP\bin\
ntpd.exe -n -g -c "C:\WINNT\system32\drivers\etc\ntp.conf" -M

To stop ntpd, issue a Ctrl-C into the command window.

Results from different values of NTPD_INT_INT

Using the interactive start of ntpd.exe, I ran for a couple of minutes first (waiting for a 377 in the Reach field), then sampled the reported Jitter values from an "ntpq -p" at about half-minute intervals, with each INT value.  Jitter values are in milliseconds.

A Second System

We decided to leave further experiments on the 100 Hz system, as I was considering scrapping that system in any case.  I then moved the serial/CD line to another PC (Feenix) which has a 1.9GHz single-core processor (compared with 550MHz) and which runs Windows XP.

Sample NTP configuration file:

# Use drift file driftfile "C:\Tools\NTP\etc\ntp.drift" # ref-clock drivers server 127.127.20.1 minpoll 4 prefer server 127.127.22.1 minpoll 4 # Note, "Atom" driver # Use specific NTP servers server 0.uk.pool.ntp.org minpoll 10 server 1.uk.pool.ntp.org minpoll 10 server 2.uk.pool.ntp.org minpoll 10 # Statistics collection enable stats statsdir "C:\Tools\NTP\etc\" statistics loopstats

Notes:

• The Atom driver can only be used if the modified serialpps.sys has been installed.
   
• The Internet servers should not have iburst specified, as this causes a transient on starting where NTP momentarily syncs to the Internet server as the GPS/PPS clock pipeline is filling up.
   
• If you are serving time to the Internet, be sure to add the required "restrict" lines to restrict control access to your local network.

# Use drift file driftfile "C:\Tools\NTP\etc\ntp.drift" # ref-clock drivers server 127.127.20.1 minpoll 4 prefer server 127.127.22.1 minpoll 4 # Note, "Atom" driver # Use specific NTP servers server 0.uk.pool.ntp.org minpoll 10 server 1.uk.pool.ntp.org minpoll 10 server 2.uk.pool.ntp.org minpoll 10 # Statistics collection enable stats statsdir "C:\Tools\NTP\etc\" statistics loopstats # Suggestions for NTP restrictions restrict source notrap nomodify nopeer noquery restrict 127.0.0.1 restrict ::1 restrict 192.168.0.0 mask 255.255.255.0 peer

Note that you will need to replace the 192.168.0.0 in the restrict line above with the address for your network, such as 192.168.1.0 or 192.168.10.0.

Results from the second system

It is likely that PC Feenix will replace PC Pixie as my main stratum-1 server, so I am interested to know how it compares with the existing stratum-1 PC, Pixie, which runs a homebuilt FreeBSD system.  I wrote a simple data collection process in Perl which collects the output from an "ntpq -c rv" command, and filters the data of interest into a text file, which can hen be analysed offline with a program written in Delphi.  The main criterion, for me, was how close to UTC the system would run - the offset field.

With the data plotted on the same scale, the Windows system appears a lot worse than the FreeBSD system, although it still offers performance typically well inside 200µs, and any system syncing to Feenix will know about the offset and allow for that when setting its own time.  

[Dave Hart comments: Actually no, ntpd tries to simply bring its time into alignment with the selected source (actually with the average of the survivors) but does not attempt to compensate for the current offsets the sources have to their sources.] 

It's rather revealing, though, to take what is basically the same data and scale the axes differently, as I do on the MRTG plot below:

Although there is more relative variation on the FreeBSD system (the jitter is higher relative to the offset), the general shape of the offset plots is the same, being consistently lower early thing in the morning when the room temperature is at its lowest.  Although it's difficult to judge, the Windows system appears to have some 20 - 50 times as great a variation.  (This is running with the user-mode PPS, though).  It's also interesting to look at the frequency error (in parts per million [ppm]) as reported by NTP.  In the plot below, only the change in frequency offset from the mean value is shown, to allow for the fact that we may be looking for differences less than 1 ppm whereas the two systems could have a difference in mean error of several hundred ppm.

Here, the similarity shape of the two curves is even more striking, albeit that the daily deviation of the Dell 4400 (Feenix) is twice that of the older Viglen PC (Pixie).  Two completely different PCs, so it's hardly surprising that they have two different temperature coefficients.  The glitches on the green line (Feenix) are due to the NTP on that PC being stopped and restarted to change the ntpd.exe process priority.  There seems to be slightly more "noise" on the FreeBSD system, but as that's using a version of NTP which is some four years older, it's not surprising that there may have been algorithm or accuracy improvements in the intervening time.

Here is another offset comparison showing March 20 and 21 of 2009.  I notice that there are fewer transients in the Windows offset compared with the FreeBSD offset.  Perhaps there were improvements in the GPS algorithm between the older 4.2.0 working on the FreeBSD system and the newer 4.2.4 on the Windows system?

Some current monitoring graphs

• Comparison of FreeBSD (pixie) and Windows (Feenix) PCs
• CPU, memory use etc. for PC Feenix

Kernel-mode versus User-mode

One option which has been investigated during these tests, but which has unfortunately not made it into either the release or development branches in the use of kernel-mode rather than user-mode for timestamping the transition of the PPS line.  This has the unfortunate effect of making the jitter quite a lot worse, and subject to the effects of other activity on the system.  However, the effects on frequency error and offset are not too great.

Using Kernel Mode

On 2009 June 05 Dave Hart wrote to me:

[After version 4.52.5p180] there's a new .dll there which ntpd currently finds by way of environment variable PPSAPI_DLLS.  If you put it in the same directory as ntpd.exe, this should work

PPSAPI_DLLS=serialpps-ppsapi-provider.dll

you can also use a path:

PPSAPI_DLLS=c:\serialpps\serialpps-ppsapi-provider.dll

Keeping it out of the ntpd.exe directory avoids having it stepped on by running the Meinberg installer.

He also wrote a fuller explanation on comp.protocols.time.ntp on 2009 Dec 01:

To have full PPSAPI capability with ntpd on Windows you need serialpps.sys, serialpps-ppsapi-provider.dll, and a recent-enough ntpd.exe.  You need a serial port which can be driven by the stock serial.sys (which includes a huge variety including some simple multiport designs).  serialpps.sys must be "installed" (if you can call it that, the file must be in place and pointed to by the serial.sys image path registry entry).  serialpps-ppsapi-provider.dll must be accessible and pointed to by environment variable PPSAPI_DLLS visible to ntpd.exe (typically set system-wide), such as:

C:\>set
...
PPSAPI_DLLS=c:\serialpps\serialpps-ppsapi-provider.dll
...

You can find both serialpps.sys and serialpps-ppsapi-provider.dll in:

http://people.ntp.org/burnicki/windows/serialpps-20120321-signed.zip

There are more releases of serialpps .zip files in that directory than there are different versions of serialpps.sys within.  The most recent changes, adding serialpps-ppsapi-provider.dll, simply shuffled code previously hard-wired into ntpd.exe off into a per-provider DLL, but did not change the ioctl interface or serialpps.sys.
 

How do I know kernel-mode is working?

Following a discussion on the comp.protocols.time.ntp newsgroup, it became apparent that the current release of NTP (4.2.6) logs that it is using user-mode timestamps, even though kernel-mode timestamps are actually in use.  This caused some head-scratching!  It seems that because one policy is minimal messages, and because of the architecture of the Windows solution with a device driver and a DLL, and because of the timing of the startup, it is not easy to log that the kernel-mode is in use.
In fact, it's actually much easier to check, and requires no visits to the Event Viewer.  From a command-prompt, issue the "ntpq -p" command against the server running the GPS/PPS/Atom software.  You should see an output something like the one below:

C:> ntpq -p feenix
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
oPPS(1)          .PPS.            0 l   15   16  377    0.000    0.087   0.003
*GPS_NMEA(1)     .GPS.            0 l   14   16  377    0.000    0.052   0.002
+stamsund        .PPS.            1 u   46   64  377    0.555   -0.038   1.203
-paddington.ragg 145.238.203.14   2 u 1008 1024  377   23.190    3.053   0.504
-dnscache-london 140.203.16.5     2 u 1047 1024  377   20.872    2.892   1.258
-utserv.mcc.ac.u 193.62.22.98     2 u  955 1024  377   28.712    4.269   1.949
+linnaeus.inf.ed 129.215.64.232   3 u  338 1024  377   35.373    1.102   1.066

Of course, your server names will be different, but the two lines with the tally-codes (first column) * and o should both be present.  The * tally-code indicates the normal NTPD "I am using this server" state, and in the above example it shows that it is using the GPS_NMEA local reference clock, so the basic serial-port functionality is present and correct.  The o tally-code indicates that the kernel-mode functionality has been detected by the Atom driver, and that it is in use.  The (1) indicates the COM port number being used for the serial input.
 

How well does it work?

Here is a graph of switching from a test release of NTP with the kernel-mode code to a development snapshot with user-mode, and therefore without the benefits of the ATOM driver.

Jitter

Frequency variation range

Offset

Any side effects of using serialpps.sys?

Dave Hart writes:  The effect (on CPU etc.) compared to using serial.sys is quite similar.  serialpps.sys adds a new "ioctl" function which software other than ntpd won't know about or use.  Until the first time that ioctl function is called by a given process, serial.sys behaves identically.  After the first call to that ioctl function, the interrupt handler for "modem status" line changes (including DCD) squirrels away a system timestamp and cycle counter value at each DCD clear->asserted transition, retrievable via the same ioctl function.

serialpps.sys timekeeping performance compared to the "user-mode PPS" hack is relatively marginal under optimal circumstances, as David Taylor has demonstrated [above - DJT].  The more loaded the box, the more likely serialpps.sys's interrupt-time timestamping of the DCD transitions will be substantially more accurate than ntpd doing the same thing from user mode.

And that "User-mode" message?

Dave Hart writes:  I should have said the fact that a message about user-mode PPS is logged does not imply PPSAPI is not available or even being used.  You could see the user-mode PPS message (which is only seen with the Windows ntpd, for those who are baffled by this user-mode PPS discussion) during startup of NMEA configured to use PPSAPI directly (which implies not configuring PPS/atom as a separate source).  The user-mode PPS message is coming from the common reference clock serial support code, and occurs if a PPS signal is detected on the DCD line of a serial port being read by a refclock.  In that case, the timestamp of each PPS event is saved and replaces the normal end-of-line timestamp of the following line of serial input read by the clock driver, which is the reason you must configure only one sentence (line of serial input) per second for the user-mode PPS hack to work as intended.

There is an issue with the messages coming from entirely separate components.  Moreover, as I have just said, the presence of user-mode PPS timestamps (replacing end-of-line timestamps on serial input) does not imply the lack of PPSAPI timestamps on the same port.  If I configure the NMEA driver to use PPSAPI directly, user-mode PPS timestamps remain active and functional from the perspective of the refclock serial support in ntpd on Windows. The fact that higher-level code in the driver only uses those timestamps briefly during startup before switching to PPSAPI-provided timestamps is invisible to the code that logs the "using user-mode PPS" message.

Similarly, the PPSAPI client code used by NMEA and atom has no way of knowing if the same port is also being used for serial timecode input, and if so, if the end of line timestamps have been tinkered.

Finally, the PPSAPI client code in ntpd tends to follow the model of being quiet unless something goes wrong, so it's the absence of error messages with a PPSAPI-enabled driver (whether atom or another with PPSAPI integrated and enabled) that "logs" the success. 
 

Using a Serial - USB converter

Many PCs today do not have a serial port, but they have plenty of USB ports.  I wanted to try NTP via a USB port, and I already had a serial converter which gave my PC COM4.  This converter supports the DCD line, and this is critical to getting good results, as the NMEA output from many GPS devices is sent at low priority, resulting in drifts on 100 ms or more.  However, it appears that NTP will only support COM ports up to COM3, so I was initially unable to test.  I have a built-in modem on COM3, and these are the results from trying to use a serial NMEA reference clock (GPS 18 LVC).
Tested with Dave Hart's: ntpd 4.2.5p161@1.1825-o Apr 02 11:57:05.06 (UTC-00:00) 2009 (9)

After setting the USB device to COM2 in the Device Manager, the first restart of NTP gave an error, and subsequent restarts gave "NT_COM: Device COM2: CreateFile error: Input/output error".  Time for a reboot?  Working OK now (had to disable the Microsoft Serial BallPoint device in the Device Manager, as expected).  Postscript:  NTP has since been updated and can now support serial port numbers up to COM255.

Performance - approximate only: compare Narvik (direct connect USB/NMEA/DCD) with Feenix & Stamsund (direct-connect serial GPS reference clock) Bacchus (ref-clock over LAN) here.  Please note the different offset scales on these graphs!  PC Narvik was changed from a LAN connection to USB ref-clock just before 16:00 in the middle day graphs below, where you can see the transient caused by the multiple reboots.  

• The variation of offset on Narvik was much reduced after the change, but note that before the change the minpoll was 64 s, but after the change the minpoll was 16s (ref-clock only).
• The hiccup around 22:00-22:30 on Narvik was caused by an indexing program using over 70% CPU during that period, and hence warming the system clock crystal.
• The step up at 05:00 near the right-hand side of these graphs is caused by the central heating starting up, and you can see that in the direct-connect PCs it causes a step of (very approximately) 150 µs (Feenix) and at least 600 µs (Stamsund), and about 400 µs on Narvik (USB-connect).
• The minimum average jitter value on Narvik was 110 - 140 µs with the LAN connection, and 45 µs with the USB connection.  For comparison, the two directly connected PC had minimum average jitter values of 2.3 µs (Feenix) and 2.7-3 µs (Stamsund), and the LAN-connected PC had about 80 µs (Bacchus).
• PC Feenix is a completely non-interactive system, with a very stable load, whereas the USB-connected system (PC Narvik) is an interactive use system with a rather variable load.

	PC Feenix Direct-connect Serial NMEA/DCD/PPS
	PC Stamsund Direct-connect Serial NMEA/DCD/PPS
	PC Narvik USB-connect Serial NMEA/DCD/PPS
	PC Bacchus LAN to GPS ref-clock

And just for interest, here is the transition back from using the USB source to using a LAN connection to a GPS-fed stratum-1 server at about 1600 UTC on the middle day:

showing that a direct-connected GPS-serial-USB reference clock is better than LAN-connected stratum-1 server, at least in my setup where the serial-USB adapter carries the DCD signal.  Jitter increased from about 45µs to about 160µs when reverting to the LAN-alone references.  However, it may also be that the jitter reduction was due in part to the shorter polling interval used for the ref-clock than for LAN operation.  

Loopstats

Loopstats are available on request.
 

NTPplotter - a program to plot loopstats

Although Meinberg's NTP Time Server Monitor is a very handy and comprehensive program, it does not plot jitter, so I wrote a simple NTP Plotter program.  This program will take one or more loopstats files on its command-line, or you can drag-and-drop one or more loopstats files for display.  In addition, you can have either on the command-line or via drag-and-drop a directory specification and all the loopstats files in that directory will be processed.  This is a free and unsupported tool, but your comments are welcome.

Jitter

On the plot above, you can see the effect of changing from the user-mode serial-port time-stamping with the default serial driver, to the kernel-mode time-stamping offered by the serialpps.sys port driver (provided by Dave Hart) and the ATOM ref-clock driver (part of NTPD).  The jitter has been more than halved.  The large glitch was caused by having to reboot (twice) to install the new serial driver and remove the phantom Microsoft Ball Mouse which is how the GPS serial stream is detected.

Frequency variation range

Offset

The two graphs of frequency error (i.e. the variation in the frequency from its mean value), and the offset graph, show quite clearly the effects of temperature variation.  During the night (0000-0600 UTC and local) the room cools down, until the heating comes on automatically at 0600.  At that time, the offset starts to increase and the frequency increases to try and reduce the offset.  By about 1100 the offset has been reduced to near zero, and the frequency stabilises.  At 2300, the cycle repeats.  There are three days in the above plots, and the three cycles are clear to see.  Interestingly, there is little diurnal variation in the jitter value, and while using kernel-mode time-stamps reduces the jitter, it does nothing to reduce the offset value, this being much more a function of the NTP algorithms and their interaction with the temperature-induced oscillator drift.

• Download the NTPplotter program

Early comments from Dave Hart

1. The latest binary is now actually 20090301 with refclock_atom support, but if you're not using refclock_atom 20090226 should be equivalent.

2.  I'd suggest just put mode 7 on the server line for 127.127.20.1 since that covers all sentences and should really be what default/0 means.  As it stands 0 and 1 both mean $GPRMC only, 2 and 4 enable only $GPGLL and $GPGGA (forget which is which), so 7 (1+2+4) enables all three sentences and is the easy starting point.

3.  It would be nice to mention NTPD_INT_INT=26 in system env vars as workaround at least until I post a binary with that default (which I'd like to wait on to make sure it works well enough for 64 Hz machines).

4.  Nitpicking but RS-232 is nominally +/- 12V ish, with any positive voltage theoretically 1 on the receiver side and any negative 0.  We're using TTL-level RS232-compatible signalling and lucky our PCs serial line drivers work well with TTL (even shared TTL for you).

5.  You've spent a lot of time and I appreciate it, writing it up is a great way to leverage the work and save us answering the same questions repeatedly in the future.  I am sure you have other responsibilities and priorities, but when you want to put some more time into ntp, consider that you can run just a single pair of wires carrying serial signal ground and CD to another PC and with serialpps.sys installed configure 127.127.22.1 and have the same sort of precision with just slightly slower acquisition time as it ensure the seconds are right.  At some point you might need to use a proper level converter to drive many ports off one TTL sender, but I applaud your trying it the easy way first and blazing a trail :)

11 April 2009 - comments from Dave Hart

David, feel free to quote anything here as you feel appropriate in email to others or on your web pages.  As David indicated, the disabling of the code which reported "using serialpps.sys timestamps" was intentional.  That was part of the process of preparing my "QPC" ntp-stable source code tree, loaded with all sorts of fixes, to have related changes pulled over into either both ntp-stable and ntp-dev (for bugs) or into only ntp-dev (for enhancements).  There are three major pieces I was thinking about:

1. Improvements to the Windows serial I/O code to come closer to the ntpd/refclock expectation of at least so-called TTY line discipline, where only complete lines trigger a read of serial input to return to the application.  Note this is where some PPS time-stamping fun stuff was added in my QPC tree.
2. Once I had a PPS solution working, I saw a 1 ms error most of the time (but not always, tantalizing me).  After a week or so of head-scratching I saw that during a relatively steep slew after starting with tens of milliseconds offset, the 1 ms error went away, only to return as the offset stabilized near zero. This hint led me to suspect the error wasn't in the PPS time-stamping process, but in the synthetic timeline ntpd constructs on Windows using a processor cycle counter (also known as a performance counter on Windows) on top of the system clock, which takes relatively large steps of up to 15.6 ms each time it changes.  That led to some serious head-scratching and experimentation and finally a new approach for that synthetic timeline which knocked the vast majority of that +/- 500 µs error away.
3. Support for my hacked serial driver, serialpps.sys, which is a very slightly modified serial.sys that is loaded instead of serial.sys with its "installer" (not guaranteed to survive service packs).  This was originally done only via the disabled "Using serialpps.sys timestamps" code, but later moved to the much-preferred PPSAPI and atom/PPS refclock driver, once I took the time to wrap the trivial bit of functionality up in the full PPSAPI interface.

Keep in mind the need to decide whether something goes into -stable and -dev at the same time, normally bugs only, and depending on how close the next new -stable release is, only more serious bugs would be put in -stable because a new -stable version means throwing away the old -stable code and making the current -dev (for an instant) the new -stable.  Changes which can't be classified as bugs, or aren't considered serious enough, go in -dev only and don't make it to -stable until the next non-patch stable release.

Point 1 above (serial I/O improvements) is clearly a bug and as a result was the first part I managed to peel off and integrate into the official NTP release, as of ntp-stable 4.2.4p7RC1 and ntp-dev 4.2.5p160.  That includes the "user-mode PPS timestamp" part but not the "serialpps.sys timestamps" part disabled in the latest QPC private release of mine.

Point 2 above (synthetic timeline or "interpolation" 1 ms error) is harder to call a bug.  The old code has more error, but either approach is an estimate with some inherent error expected compared to the preferred approach of the OS presenting a smoothly-increasing timescale.  Add into consideration that it has been over two years since the last non-patch ntp-stable release (4.2.4) and two years of divergence has accumulated between the ntp-dev and ntp-stable trees, far too much for everyone's comfort.  That means a 4.2.6 release should be coming very soon, and everyone believes it will happen any month now (sigh).  On balance I decided to only attempt to integrate the new interpolation scheme into ntp-dev, and that has only just happened in the last few days with the release of 4.2.5p162.  4.2.5p163 is identical to 4.2.5p162, the only reason for the bump was a missed step of pulling 4.2.4p7RC2 ntp-stable final version number changes into ntp-dev, where they were then replaced as usual with the ntp-dev version number, bumped without changing anything else, to 4.2.5p163.  This was a big relief to me as these two pieces are the real meat of my private releases' patches.

Point 3 above (serialpps.sys support) is a bit of a can of worms by comparison, and the benefit (while real) isn't huge compared to using only NMEA + user-mode timestamps possible with any Windows serial port, using serial.sys or not.  It is not forgotten, but to be honest I'm taking a bit of a breather and helping with some other NTP project issues and haven't done anything to move that forward into the official NTP code yet.  The biggest issue in my mind is licensing.  As I read the Windows Driver Kit license agreement and related documentation, I have no right to distribute serial.sys object code (binary file such as serialpps.sys).  You may have noticed serialpps.sys contains only a patch file showing my additions to the WDK serial.sys source code, and doesn't actually include the source code, you have to get it from the WDK.  That was motivated by the same licensing concern.  I have not pulled down my serialpps.sys binaries (in fact I cross-compiled x64 and Itanium versions and posted them as well since I dug into the licensing) because I can't imagine Microsoft would even bother to send me a cease and desist letter, let alone take more serious legal action.  Largely, of course, the binaries I provide are only useful to those using Windows.  Here it's important to note that the NTP distribution is as close to public domain software as you could hope.  The license is a BSD-style do what you will and don't hold us liable or use our name to promote your product boilerplate.  Dr. David L. Mills, the major author, is adamant about keeping NTP freely usable by anyone any way, in particular including in commercial products.  There are at least potential emotional, if not legal, conflicts in including support for a serialpps.sys that we can't include in official distributions in source or binary form, and would have be obtained, arguably illegally, from me or others who have mirrored serialpps .zip files from my website.

After all that damned detail and history, I still haven't explained why I disabled the "using serialpps.sys timestamps" code path from the Windows serial I/O code in ntpd in my latest binaries.  Basically, knowing the serialpps.sys changes would be the last major bit I'd try to move into the official code, I looked at my two parallel implementations in the Windows serial I/O code (providing timestamps via NMEA or another serial-based refclock driver) and in PPSAPI (providing timestamps via the atom/PPS refclock driver) and decided the earlier implementation in the serial I/O code definitely wouldn't make it in with the rest of the serial I/O changes going into both -stable and -dev, and also were contrary to Dr. Mills' expressed desire to concentrate PPS functionality in the atom/PPS driver.  Also, I did not want to risk my sneaky slipping of the user-mode PPS support into the Windows serial I/O code as part of a general redesign and cleanup of that code to be potentially derailed by the original serialpps.sys support code and its licensing and Mills issues, when the same precision serialpps.sys timekeeping works fine via Mills' preferred PPSAPI/atom solution.  So while you are free to keep it alive in local patches, I consider the serialpps.sys support in the Windows I/O code-path a fun bit of history that I have no plans to integrate into the official code at this point (because of no benefit vs. PPSAPI approach and the concerns).

Phew.

Even with the ugly licensing situation, I think the PPSAPI/refclock_atom approach to serialpps.sys support will make it into ntp-dev soon.  That may not be before 4.2.6 is released, however.  In the meantime you can continue using my QPC releases and if there's good reason, I can always fix something in that tree and make another release.

[My sincere thanks to Dave Hart for his work in Windows, and for his permission to include this most illuminating commentary of his significant enhancements to NTP, and the development processes involved in an open-source project].

 

Copyright © David Taylor, Edinburgh

@gm8arv on Twitter

Last modified: 2024 Dec 12 at 15:16