Setting
up Split-PC EUMETCast reception
Arne van Belle has developed an alternative way of using two PCs for EUMETCast reception, and I asked him to write it up following the interest shown after his excellent presentation at the 2006 GEO Symposium at Leicester. Thanks for providing this information, Arne. David Taylor.
Objectives
- Run
an older, less powerful, PC to receive all data 24hours a day.
- Run
the MSG Data Manager on this PC to decode and store
all images on disk.
- A
RAM
disk is recommended to buffer the TelliCast File database.
How to do this
All decoded images and data are moved over the network to a second PC. This PC is only switched on when you actually need it and can be used for lots of other demanding tasks like photo or movie editing, surfing and so on. On this PC you can view the received images, run MSG Animator, GeoSatSignal and do many more.
To make this setup work we need to move all decoded data from the receive PC to the second PC without interrupting the reception. And we want it done quickly and fully automated. Microsoft offers the robocopy command line utility. This is part of the Microsoft Windows resource kit and can be downloaded for free. The resource kit has to be installed on Windows XP or Windows Server 2003 first. Once installed the robocopy command can be copied and also works on Windows 2000 and NT4.
To
automatically invoke this script I use MSG Data Manager's option to invoke a
script named MSGnotify.bat at the end of every 15 minute cycle.
The script I wrote checks if the second PC is ready to receive and then moves
all decoded images and data to the second PC.
If the second PC is switched off or if a previous transfer is still in
progress the script will end.
Folders are created on the fly to create exactly the same folder structure on
your process PC as if MSG Data manager were running on this machine.
You can “Taylor” this script to your needs. Another useful option could be to synchronise the receive PC clock with your second PC that has internet connection for timekeeping. I also use it to move received AVHRR data and delete other data that I do not process.
Setting up
the script
There are two
versions; a basic and a full-options
version. These scripts are given in a format you can copy at the end of
this Web page.
Before using the script you have to rename the script to MSGnotify.bat and place
it in the MSG Data Manager folder. Edit in notepad and set the source and
destination hostnames/paths once. You need to create a share on the second PC /
drive where the files need to be moved to. Make sure the receive PC has
privileges to write to this drive / folder.
Put the required tools pslist.exe, robocopy.exe, in C:\WNNT or C:\WINDOWS or
another folder included in your path. The
full-options version also requires TrimTree and sleep.exe
Don't
forget to activate option "Notify scan end" in MSG Data Manager's
setup.
Sources for additional tools
robocopy.exe from Microsoft Windows 2003 Resource Kit downloads: http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en
pslist.exe
from Sysinternals, download from: http://www.sysinternals.com/Files/PsList.zip
sleep.exe
also comes from the Microsoft
Windows Resource Kit 2003/XP
Trimtree.exe
by David Taylor http://www.david-taylor.myby.co.uk/software/disk.html
The Scripts
You can download the scripts from this Zip file. To prevent you executing the scripts without checking them first, and to bypass any clever software which doesn't allow scripts in Zip files or downloads, the file extensions have been renamed from .BAT to .TXT. If you copy and paste the scripts, please be careful with any line-wrap which results!
Basic Script
----------------------------------------------------
@ECHO Off
SETLOCAL
:: MsgNotify.bat: Move processed images and data from Eumetcast receive/decode PC
:: to processing/archiving PC.
:: Arne van Belle V4basic May 2006
SET _isource=C:\msg\images
SET _idest=\\process-computername\share-name\msg\images
:: Set selection and options for Robocopy, see Robocopy.txt for syntax
SET _what=/S /Z /COPYALL /MOV /XX
SET _options=/R:10 /W:30 /NJH
:: Check if MsgNotify was called with parameter MSGSCANEND
IF %1==MSGSCANEND (
ECHO Scan completed, checking if processing PC is online and ready to receive data ...
) ELSE (
GOTO NOSCANEND
)
:: Check if processing PC is on-line
IF NOT EXIST %_idest%\moveimg.txt GOTO NOMOVE
ECHO Check if a previous move is still running ...
pslist robocopy | findstr " CPU Time" > nul
if errorlevel 1 goto START
if errorlevel 0 goto FOUND
:START
ECHO Cycle %5%6 complete, moving decoded images to processing PC ...
ROBOCOPY %_isource% %_idest% %_what% %_options%
ECHO Move completed.
GOTO END
:NOMOVE
ECHO Move option is disabled or processing PC is off-line,
ECHO Create empty file moveimg.txt in %_idest%
GOTO END
:FOUND
ECHO Previous Robocopy command is still running, will retry next cycle
GOTO END
:NOSCANEND
IF %1==MSGDATALOSS (echo M S G D A T A L O S S !!! ) ELSE echo %1
:END
----------------------------------------------------
Full-options script
----------------------------------------------------
@ECHO Off
SETLOCAL
:: MsgNotify.bat: Move processed images and data from Eumetcast receive/decode PC
:: to processing/archiving PC.
:: Arne van Belle V4 May 2006
:: Parameters: <message text file name>
:: MSGDATALOSS
:: MSGSCANEND YYYYY MM DD HH MM
:: %1 %2 %3 %4 %5 %6
:: Optional: sync time from other pc: net time \\computername /set /y
:: This script also requires:
:: Trimtree.exe by David Taylor www.satsignal.net
:: pslist.exe from Sysinternals www.sysinternals.com/Files/PsList.zip
:: sleep.exe and robocopy.exe from Microsoft Windows Resource Kit 2003/XP
:: Copy these files in c:\winnt or c:\windows or other folder in path.
::
:: Place this script in the c:\Program Files\David J Taylor\MSG Data Manager folder and
:: activate option "Notify scan end" in MDM setup.
:: Set source and destination paths for decoded Images and raw Avhrr data
:: You have to set these once to match your configuration !
:: Use " " if there are spaces in the path
:: _asource and _adest are only required if you want to process AVHRR files on the processing PC !
:: some examples _isource=C:\msg\images _idest=\\P4\P4_d\msg\images
:: _asource="C:\Program Files\T-Systems\BusinessTV-IP\received" _adest=\\P4\P4_d\received
SET _isource=C:\msg\images
SET _idest=\\process-computername\share-name\msg\images
SET _asource="C:\Program Files\T-Systems\BusinessTV-IP\received"
SET _adest=\\process-computername\share-name\received
:: Set selection and options for Robocopy, see Robocopy.txt for syntax
SET _what=/S /Z /COPYALL /MOV /XX
SET _options=/R:10 /W:30 /NJH
::SET _options=/R:10 /W:30 /NJH /IPG:100 set this line to slow down transferrate
:: Check if MsgNotify was called with parameter MSGSCANEND
IF %1==MSGSCANEND (
ECHO Scan completed, checking if processing PC is online and ready to receive data ...
) ELSE (
GOTO NOSCANEND
)
ECHO Deleting some unused data first ...
:: Optional: Delete the GIF and JPEG RSS data (1800x800 Europe only)
:: The FRS*OPEN_MTP data has less artifacts and wider coverage (5000x1666)
TrimTree -1 %_isource%\RSS\ EUR_*.bz2
:: Optional: Delete EARS-ATOVS data if you don't process it
:: If you do not process EARS-AVHRR it is better to turn off PID500
:: Delete the received files (if CH1 ATOVS unmanaged in MDM)
::TrimTree -1 %_asource%\ amsua*.* amsub*.* hirs*.* mhs*.*
:: or Delete MDM managed files (if you have CH1 ATOVS managed in MDM)
TrimTree -1 %_isource%\ATOVS\ amsua*.* amsub*.* hirs*.* mhs*.*
:: Optional: Delete SAF data that SeaIceViewer does not process
TrimTree -1 %_isource%\SAF\ *-LMLDLI_*.* *-LMLSSI_*.* *-LMLSST_*.* *-MAPDLI_*.* *-MAPSSI_*.* *-NARSST*.*
:: Check if processing PC is on-line
:: Put (empty) file moveimg.txt on processing PC in MSG\IMAGES to activate script
IF NOT EXIST %_idest%\moveimg.txt GOTO NOMOVE
ECHO Check if a previous move is still running ...
pslist robocopy | findstr " CPU Time" > nul
if errorlevel 1 goto START
if errorlevel 0 goto FOUND
:START
ECHO Cycle %5%6 complete, moving decoded images to processing PC ...
:: Move most recent channel 12north images first
ROBOCOPY %_isource%\HRIT %_idest%\HRIT *ch12n* %_what% %_options% /MAXAGE:1
ROBOCOPY %_isource% %_idest% %_what% %_options%
:: Move EARS-AVHRR raw data to processing PC using Robocopy
ROBOCOPY %_asource% %_adest% avhrr_* %_what% %_options%
ECHO Move completed.
sleep 30
GOTO END
:NOMOVE
ECHO Move option is disabled or processing PC is off-line,
ECHO Create empty file moveimg.txt in %_idest%
sleep 30
GOTO END
:FOUND
ECHO Previous Robocopy command is still running, will retry next cycle
sleep 30
GOTO END
:NOSCANEND
IF %1==MSGDATALOSS (echo M S G D A T A L O S S !!! ) ELSE echo %1
sleep 30
:END
----------------------------------------------------
Information kindly supplied by:
Arne
van Belle
May
2006