NTP 4.2.4 vs. 4.2.5
Home Page Up Setup on Windows Using NTP Windows LAN tips Performance Events Cable modem notes Monitoring with MRTG GPS 18 + FreeBSD GPS 18 + Windows GPS 18x firmware GPS 18x waveforms NTP 4.2.4 vs. 4.2.5 NTP 4.2.7p241 Rapco 1804M notes Raspberry Pi RPi - quick-start RPi - notes RPi - cross-compile RPi vs BBBlack Sure GPS board Timestamp issues TSC Interpolation Vista & Windows-7/8 Wi-Fi warning Windows after reboot Win-7/8 & Internet Win-7 to Win-10 gains New versions

 

NTP 4.2.4 compared with NTP 4.2.5/6

It's possible that the issues discussed on this Web page have been resolved by the excellent work done by Dave Hart (and very much appreciated!) on NTP 4.2.7p241, so I have created a new NTP 4.2.7p241 Web page devoted to tests on that version.
 

March 2010 - comparison on a Windows Vista system

This system is wireless synced to a stratum-1 server on my LAN.  Using wireless may well increase the jitter of the link between the stratum-1 server and the client PC.  It isn't run continuously - just during the day.  I plotted the offset on two different scales - the top graph has a range of +/- 100 milliseconds, and the lower graph a range of +/- 3 milliseconds.

Here I compare ntpd 4.2.6 on the first day, with ntpd 4.2.4 on the second day.  From the upper graph, it's obvious that there is much less variation of the offset with 4.2.4 (273) than with the more recent version.  Event the transient event is reduced from some 60ms to 30ms (although that could be just by chance of exactly what the event was).  As ever, the question is why is there so much difference between versions, and the disappointment is that the more recent version is the one which is significantly worse with Windows Vista and Windows-7?

Version numbers:
  First day - 4.2.6 - ntpd 4.2.6-o Dec 09 11:48:30.27 (UTC-00:00) 2009 (1)
  Second day - 4.2.4 - ntpd 4.2.4p6@DLH-QPC-o May 30 3:58:32.88 (UTC) 2009 (273)
 

December 2009 - comparison on a Windows Vista system

I was able to get limited access to LAN-synced Windows Vista system and was interested to test it, as my existing Vista system is rather badly behaved due, I believe, to having a satellite data feed with a particular USB DVB card and driver.  Here is the +/- 100ms scale plot.

Puffin's NTP

The monitoring was started at point A on the graph above, and the changes were:

  1. Monitoring started - ntpd 4.2.4p6@DLH-QPC (230) in use
  2. Changed to 4.2.4p6@DLH-QPC (273)
  3. Changed to ntpd 4.2.5p250-RC
  4. Reverted to 4.2.4p6@DLH-QPC (273)

It's obvious by inspection that the variation of offset was substantially greater while running NTP 4.2.5 that while running NTP 4.2.4.  The system was substantially unused during the period 21:00-07:00, so if anything the offset should have been more stable than less.  These results serve to confirm what has already been found, and which is reported in more detail below.
 

September 2009 - comparison with 4.2.5p212 and 4.2.4p6

The experiments were repeated over September 16-17, using a more recent NTP, but without the "tick 1".  However, the results were very similar to those below.  This newer NTP seemed no worse on a Windows XP system (Narvik), so it was left running there.
  

September 2009 - experiments with "tick 1"

In September 2009 Dave Hart suggested that using NTP 4.2.5 with the "tick 1" (for ~1000us reported Windows clock precision) might produce comparable performance to the 4.2.4 which I had been using on both Vista and Windows-7 to get acceptable performance.  I have two systems I could test, one with Windows Vista and one with Windows-7.  As you can see from the quick-look graphs below, the Vista system wasn't made a lot worse, but the the performance of the Windows-7 system was rather upset.  The experiment was stopped on the morning of September 08, as can be seen on the larger graphs below.

Windows Vista system - PC Gemini

Please note: this system is not well behaved!  It has a data stream fed from satellite with a DVB receiver connected to the PC over a USB port.  It was switched from NTP 4.2.4 to 4.2.5 with "tick 1" at 01:00, about one-fifth of the way across the graphs below.  The character of the offset has changed from almost all positive excursions with some relatively stable intervals, to a more symmetrical variation but missing the stable intervals.  In general terms, I would have said the amount of offset deviation was similar, neither better nor worse.  Near the centre of the graph, at 18:30, it was switched to "tick 0.3" which made little change, and later to "tick 0.1" at 05:00 but again with no obvious improvement.  It was switched back to NTP 4.2.4 at 08:00 about forth-fifths of the way across the graph.

The jitter as reported by NTP has changed from a rather high variability with peaks of 20ms and an averaged value around 4ms, to a somewhat steadier value averaging around 14ms.

Windows-7 system - PC Hydra

This system was less successful with "tick 1", although there was one glitch as well which is still to be resolved.  NTP 4.2.5p186 was installed on this system at 00:50 - about one fifth of the way across the graphs below.  It was immediately obvious that the performance was nothing like what was expected, and checks showed that NTP has measured the Windows clock precision as twice the correct value, so NTP was restarted just after 05:00.  However, in spite of the measurement now being correct, the poor performance continued.  Offset is now highly variable, being of a much greater amplitude than before.  The spectrum of the offset is different, though, have a much greater low-frequency component.  In the interests of experimentation, at 05:23 near the end of the graph it was switch to "tick 10", with no obvious change.  At 08:00, four-fifths across the graph, the earlier NTP 4.2.4 was restored.

Frequency offset is now wondering all over the place, having previously had a quiet stable value around 4.8 ppm.

Jitter has increased from an average value of less than 2ms to around 10-11ms.  This average value is not greatly different to the Vista system, possibly suggesting that in both cases an extra jitter of some 10ms has been added (I understood the intention was to add less jitter than this, but I could be wrong).

 

2009 Jun 09

After running another 4.2.5 version of ntp [ntpd 4.2.5p181-o Jun 06 15:21:08.65 (UTC-00:00) 2009 (1)], I re-installed ntpd 4.2.4 [4.2.4p6@DLH-QPC-o Mar 10 15:23:14.36 (UTC) 2009 (230)].  As expected, the offset was much reduced for most of the time, although the transient peaks remained.  I've also shown the results from my NTP Plotter program.  Note that at 05:00 UTC at the end of the graph the system was being prepared for the Vista SP2 update, and hence the hour of higher offset.

I would be really grateful if someone could look into this issue.

The graphs below show three distinct symptoms:

  • With NTP 4.2.4 the offset is stable for periods of up to a couple of hours, but then shows positive spikes of up to about 50ms, whereas 4.2.5 appears to have a much more symmetrical, but a much greater, offset value.
  • With 4.2.4, the slope of the frequency error seems less than with 4.2.5, almost a sawtooth shape, with leading positive slopes corresponding to the spikes in the offset.
  • The jitter is much less with 4.2.4, being some 3ms average versus 15ms average with 4.2.5.

 

2009 May 15

Comparison between "ntpd 4.2.4p6@DLH-QPC-o (230)" before 0800 on the middle day shown, versus "ntpd 4.2.5p175@1.1858-o May 12 15:20:06.38 (UTC-00:00) 2009 (1)" after about 0800.  Top system, Windows Vista Ultimate (with a rather bad USB driver, most likely).  Lower system: Windows 7 RC (7100).

Jitter comparison

The variation in offset on the Vista Ultimate system (with the problematic USB device) becomes more symmetrical, but much greater in amplitude, and the large 50+ms transients are still present. The average jitter from the NTP loopstats is rather difficult to judge, as using my "normal" 6-hour exponential average it's still rather variable.  With the NTP 4.2.4 the average jitter reaches as low as 1.5ms, with a mean value around 4.5ms.  With 4.2.5, the value is around 15ms, some three times worse.

The Windows-7 PC shows a change from an offset variation almost too small to see at the plotted scale to a series of transients around 25ms amplitude.  The average jitter changes from 1.2-1.8ms with NTP 4.2.4 to around 10ms.  The frequency error changes from having a variation of less than 0.2 ppm to a variation of 1.5ppm, and appears some 3.7 ppm greater.

Here is a plot of PC Hydra with an expanded vertical scale:

 

2009 March 28

On 28 March 2009, I updated NTP from Dave Hart's (239) which was from -stable (4.2.4) to build "4.2.5p158@1.1823-o" of the -dev version (4.2.5).  The PCs were updated at:

  • 09:00-09:10 UTC on PCs Stamsund, Gemini and Narvik
  • 09:40 on PC Bacchus

From the graphs of offset below, and checks on the loopstats I note:

  • Ref-clock PC Stamsund
    • The performance is similar, although there may be a little more variability in the offset values (on the 5-minute timescale).
    • From the loopstats, the jitter on PC Stamsund remains unchanged at 3-3.5 microseconds.
  • Windows 2000 PC Bacchus
    • From the loopstats, the jitter on Bacchus may have been reduced slightly from 150-200 microseconds to 100-150 microseconds.  Although the earlier version did also reach 100 microseconds jitter, the newer version may do do more consistently.
  • Vista PC Gemini
    • The Vista PC Gemini shows significantly worse performance, although the mostly positive spikes in offset (red in the alternative presentation below) have been replaced by a more symmetrical variation.
    • From the loopstats, the jitter appears to have increased from some 2-5 milliseconds, to some 10-20 milliseconds.
  • Interactive PC Narvik
    • The performance is similar, although there is noticeably more variability in the offset values (on the 5-minute timescale).
    • From the loopstats, the jitter appears in the same region, with odd periods around 140 microseconds, but mostly 70-80 microseconds.

I have so far had not answer to a newsgroup question as to whether there have been algorithm changes between these two versions which might account for the observed differences.

Apart from the Windows Vista PC, the performance appears broadly similar, although the offset shows a greater variability.
  

Alternative presentation

Normal presentation

 

Differences observed on another Vista PC (Dave Hart's VistaMC)

The jitter has increased from about 1.5 milliseconds to 25 milliseconds.

 

 

 
Copyright © David Taylor, Edinburgh   Last modified: 2015 Jan 18 at 09:32