Using a RAMdisk for the Basic Service

With current versions of the TelliCast client, the usage of temporary files has changed.  There are no longer .FSY files containing the received data packets all jumbled up, but each file is assembled in a temporary directory, and up to 256 of these temporary directories are automatically created by the TelliCast software.  In the cast-client.ini file, you will have a line in the [parameters] section pointing to a tmp_directory path on the root of the hard disk (or even worse, on the root of an SSD!):

[parameters]
tmp_directory=\EUMETCast\tmp\bas

Unchanged lines have been omitted.  Change this to point to your RAMdisk, for example:

[parameters]
tmp_directory=Z:\EUMETCast\tmp\bas

I've found that when taking most of the image data, but not the IASI spectroscopy data, that a 1 GB (1000 MB) RAMdisk works well, although from the MRTG plots you might be tempted by 300-500 MB (peaks sampled at 5-minute intervals shown in pink).  Click on a graph for the usual day/week/month/year plots.  The pink graphs below show the peak values - quite a bit different from the 2-hour averages!

	1000 MB RAMdisk on PC Alta
	4000 MB RAMdisk on PC Harstad
	1000 MB RAMdisk on PC Stamsund

Using a RAMdisk for HVS-1 and HVS-2

Experimental, bit in use on two PCs here!

As files transmitted on the HVS-1 (and likely HVS-2) tend to be much larger and much smaller in number, you may get away with simply using a fast hard disk dedicated to HVS-1 reception.  Not the main system disk, nor a separate partition on a disk, but a physically separate hard disk dedicated to HVS-1 reception.

However if this approach doesn't work for you it may be worth trying a very large RAMdisk on which both the TelliCast temporary files and the \received\ directory tree live.  Of course, the received files must be moved off RAMdisk onto HD as soon as they are ready, otherwise the RAMdisk would quickly fill with received data!  Ernst Lobsiger has pioneered this approach with his MVMSG script for Linux, subsequently adapted for Windows.  It's in use also by Arne van Belle and another person whose name I have forgotten.  I've experimented with a Windows version of the script using my TrimTree program, and Windows Robocopy command.  For this to work, it is required that the RAMdisk be formatted in NTFS.  I am now using this on a PC receiving the Basic Service data alone, with an 4 GB RAMdisk using FAT32 (so why did I think NTFS was needed?).

I have one script which is run once every ten minutes, and two other scripts which are run at start-up - one for HVS-1 and one for HVS-2.  As final HVS-2 data has yet to arrive, and as the HVS-1 data I get on this PC is not regularly processed, please experiment with these scripts and report back how they work for you!

1. The first script cleans out any temporary files which have been closed by TelliCast.  TrimTree will not delete temporary files which are being held open, i.e. receiving data.  In periods of bad weather or when other interruptions occur, it's possible that these files will stay open for a long while, even though the data they contain has been corrupted e.g. by lost packets.  This script deletes such files to try and prevent them filling the RAMdisk.

Ernst Lobsiger notes: "Scheduled every 10 minutes means that a tmp can get 25m (BAS), 16m (HVS-1) and 16m (HVS-2) old since last write.  I schedule mvmsg [an alternative script] every 2 minutes and delete HVS tmps not modified for 6m.  So my maximum tmp age is 8m unmodified.  The 16m for HVS is certainly safe but asks for very BIG RAM-disks e.g. in case of broken Sentinel-3X 700MB files."

If you are not processing the files, you may want to add commands to clear out the copied files too:

 
2. The second script copies files which are complete (i.e. TelliCast has received all the data and has renamed the temporary file to its final name) to their final destination.  Rather than re-invent the wheel, I've used Microsoft's robust copy command: Robocopy.  It starts by ensuring that the RAMdisk source directory exists, otherwise the subsequent Robocopy command would fail.  Robocopy is then used to copy the directory tree from the RAMdisk (drive Z:) to the destination disk (drive D:) from which subsequent processing can take place.
As the command line is quite long, I have split it with the DOS continuation character, a caret "^".  The parameters to the Robocopy command are:

• The source directory:  Z:\EUMETCast\received\hvs-1
• The destination directory:  D:\Tools\EUMETCast\received
• The file names to copy:  *
• Copy subdirectories:  /S
• Move rather than copy, i.e. delete the source file:  /MOV
• The number of retries on failed copies (3):  /R:3
• The time to wait between retries (10 seconds):  /W:10
• Monitor for changes every 1 minute:  /MOT:1
• Don't log file names:  /NFL

Of course, your directory names and needs will differ from mine, so please treat these scripts as a guide.

Using a RAMdisk for the earlier TelliCast Client

To tell TelliCast to use the new RAMdisk, you need to edit the recv.ini file.  Using Windows Explorer, select the recv.ini file from the location where you installed the TelliCast software, and double-click on it:

Using Notepad, add the file_database_directory line to the [locations] section, and comment out any existing file_database_directory line by placing a hash (#) character at the from of the line.  I do this so that I can see what I have changed, and so that I can revert if a change doesn't work!

Next, in the [parameters] section add a file_database_size line, setting the value to 83,000,000 (without the commas!).  This for an 80 MB RAMdisk.  Be sure to enter the correct number of zeros!  For the Ayecka SR1 enter the address of the LAN adapter on your PC which is connected to the SR1 for the interface_address paramter.

You will need to reboot your PC for the new RAMdisk size to become effective, and this will also restart TelliCast which is required to make it re-read the changed recv.ini file.

What value should file_database_size be?

• For an 80 MB RAMdisk mentioned above, use 83,000,000
• For a 300 MB RAMdisk which is now recommended, use 313,900,000
• For a 500 MB RAMdisk which is now recommended if taking Metop data, use 524,000,000
   

• The next steps
• Back to the main Ayecka page

 

Copyright © David Taylor, Edinburgh

@gm8arv on Twitter

Last modified: 2021 Oct 17 at 08:31