|
|
The MSG Data ManagerWhat is MSG?Meteosat-8 (or MSG-1 as it was known during trials) is the first of the new European geostationary weather satellites. MSG is an acronym for Meteosat Second Generation. Meteosat-8 (MSG-1) served from 2003 to 2007 AND is now providing Indian Ocean Data Coverage (IODC), MSG-9 is now the standby Rapid Scan satellite, with Meteosat-10 providing Rapid Scan, and MSG-11 as the prime zero degree satellite since January 2013. Meteosat-8/9/10/11 have twelve sensor channels rather than three, higher spatial resolution, much higher visible channel resolution, better thermal accuracy, and double the frequency of image delivery than previous Meteosat satellites; all of which necessitates a change to the data transmission method. Meteosat-11 (MSG-4) has replaced Meteosat 7, and the WEFAX and PDUS transmissions have now ceased. The data from MSG is purely digital, and is delivered as a set of a number of compressed files per image. As there are now twelve channels delivered four times an hour, the total data rate is some 1.5GB per hour, 15000 files per day, requiring significant management on your computer. Please note that you will need a licence from EUMETSAT to receive some of this data, but UK amateurs should see the GEO Quarterly for more details. They are unlikely to have to pay licence charges. In addition to Meteosat-9 data and other geostationary weather satellite data, you will get access to the upper-atmosphere ATOVS data and Sea-Ice data. These are processed from polar orbiting satellites and added to the data stream.
What is the MSG Data Manager?The MSG Data Manager is a Windows program that will process
the files received from Meteosat-8/9/10/11. These files might be received over the
Internet in the future, but today they are easily received with a Digital Video Broadcasting
(DVB) card installed in your PC, and a standard domestic satellite antenna, LNB,
and cable. A description of my own
system is here. The program can also process data received from the
EUMETSAT FTP service (binary transfers, my FTP
Helper program). The data manager will automatically process the files into
images, and optionally delete the large amounts of source data that would
otherwise accumulate. The program allows you to choose which of the twelve
channels you process to images, and also allows you to keep some raw data in the
processing PC should you wish. An Award!My demonstration of Receiving and Processing Meteosat-8 Data with the MSG Toolset Plus suite (which comprises the MSG Data Manager, MSG Animator and GeoSatSignal) was voted the best of conference at the 2004 EUMETSAT Meteorological Satellite Conference in Prague. The judgement was made based upon scientific content, presentation and overall interest and relevance. Who uses the MSG Data Manager?This software has been supplied to the Finnish Meteorological Institute, Meteo-Swiss, Universities, Museums, and other professional and amateur users throughout Europe.
What PC configurations are supported?You can use a single PC for receiving and processing data if the PC is powerful enough, or a two-PC network is supported, with the lower-spec PC as the Reception PC with the DVB card, and the higher-specification PC as the Processing PC, grabbing files off the Receiver PC over the network. You can ask about any particular configuration on the MSG-1 self-help Group, where hundreds users with working systems can help, and there is a discussion about PC requirements here. Windows-7 or 8 is required for support - older versions are not supported. I would expect XP/32 and Vista/32 still to work but I cannot support these versions as I no longer have PCs running XP or Vista. Data VolumesEUMETSAT state that the expected data volume is around 7000 files per hour (I think that's too high) totalling 540 MBytes. If you take EARS data and RSS data that increases by 44 MB. Therefore a full day's data would be about 15GB. If you keep all 12 MSG channels, but save the images as JPEG not PNG, the saved images amount to 1.1GB per day in 1152 files. Of course, if you take all the FSD (Foreign Satellite Data) as well the volumes can increase massively, up to a total of some 500 GB/day.
What functions does the MSG Data Manager have?
For an extra charge, you can also get the MSG Data Manager PRO version. This is aimed primarily at those doing research, where it is important to be able to access the full 10-bit data, rather than have the data converted to 8-bit images. If you are writing your own software for processing MSG data, the PRO version is probably what you need. Extra functions in the PRO version include:
If you wish to process the images further, for example to make false-colour combinations, remap to standard map projections, or to animate the images, you could consider my GeoSatSignal program. For purpose-built real-time animations, without colour or rectification to standard mapping, my MSG Animator program is the tool of choice.
What data does the MSG Data Manager process and display?
MSG Data Manager OperationThe program first copies data received by the DVB system from the
Receiver PC to the Processing PC, and optionally deletes it from the receiver PC.
You can select which channels are to be deleted, and which copied for processing. As the satellite scan proceeds, you will see the images build
up from south to north, and once an image is complete it will be saved to disk. Visual channel images are JPEG compressed, and thermal channel images
PNG compressed (lossless).
MSG Data Manager Screen ShotsHere are some screen grabs from a recent version of the MSG Data Manager. The first thing you need to do on getting the program is to
tell it where to look for the files from EUMETCast, where on the processing PC the
raw data is to be stored, and where the processed images should live. Note
that both of these latter directories will use an addition year/month/day naming
convention for the folder names so that the number of files per directory is
kept at a reasonable level.
Once setup is complete, you press the Run button and switch to the
MSG HRIT tab, where data from all the selected channels builds up. During the operation of the program, you will see the scan
build from South to North. Any gaps at the top of the images are there
because that part of the earth has not yet been scanned. Watching the
pictures build like this is fun! The channel spectral region will pop-up as your
mouse moves over the thumbnail. Note that the last image, that of channel 12, is not
square like the others, but has a 2:1 aspect ratio. It is over 11000
pixels high and over 5000 pixels wide! The imager on the satellite can be
adjusted to cover different parts of the Earth on the north and south parts of
the scan. Here you see it almost centred on Europe in the northern part of
the scan, but centred on Africa for the majority of its scan. The program
centres all the data to make the composite
picture. The scan position is altered from time to time, as shown in this 2MB
animation from EUMETSAT.
From the Detailed Image tabsOn 2004 May 27 at 08:15 UTC I selected channel 12 by clicking on the image, and then moved the view to the are around Sicily and mount Etna. The snow-covered peak of Etna and the darker volcanic soil around Etna are both easily visible. The islands of Gozo and Malta are visible to the south of Sicily. All these images are from EUMETSAT data.
More recent versions of the MSG Data Manager allow multiple Detail Image pages, and allow the detail images to be coloured with your choice of visible or thermal palettes (Colour Lookup Tables or CLUTs). On the two images below, note how the low fog-like cloud off Norway and the southern coast of the UK stands out well in the visible channel, but not in the thermal channel with the particular LUTs in use. (I show the PRO version of the MSG Data Manager below. This has eight detailed displays versus four with the Standard version).
On May 18th, 2004, much the UK and Ireland was clear of cloud. Here is an HRV (channel 12) image from Meteosat-8, rectified to orthographic projection in GeoSatSignal. False-colour is achieved by using data from the 10.8µm thermal channel.
Rapid Scanning ServiceIn May 2008 EUMETSAT started a Rapid Scanning Service which uses the MSG-1 (now MSG-2) satellite now located at 9.5° East to provide a new image of North Africa and Europe every five minutes. This provides considerable improvements for "nowcasting" and of course can produce superbly smooth animations! To achieve the fast scan, only the northern-most part of the globe is scanned, resulting in just three image segments being transmitted (and nine for channel 12 HRV). To distinguish the rapid scan mode of operation, and to provide a slightly more detailed view of Europe, the MSG Data Manager will provide a stretched display of the thumbnails when operating in rapid Scan mode. You can read about running two copies of the MSG Data Manager, one for normal scan and one for rapid scan, in the FAQ. The screen display will appear as shown below. Do you like this mode? Of course, the Detail Images and the saved images retain their correct shape and can be processed as normal - it is only the thumbnail display which has this distinctive appearance to show that RSS is being received.
How do I get the MSG Data Manager?
This software has been supplied to the Finnish Meteorological Institute, Meteo-Swiss, Swiss, Italian and Spanish TV, Weather Services, Airports, Universities, Museums, and professional and amateur users throughout Europe. Downloads
Beta version: This program is under continual development - check here for the latest update. The RSS Holiday software renames MSG-2 data to MSG-3
during RSS breaks
- download here. Important: The MSG Data Manager includes a 30-day trial licence. Please wait until you have data to test before installing the program, as the 30-day countdown starts from the first time the program is run! This program requires a display size of at least 800 x 700 pixels.
If you are interested in MSG data, did you know that with the same EUMETCast reception hardware you can now get high resolution AVHRR data from the polar orbiting satellites such as NOAA-19 and Metop-A and Metop-B I have software for these EUMETSAT options in my AVHRR Manager, DWDSAT HRPT Viewer, and my Metop Manager.
MSG AnimatorThe MSG Animator provides real-time animation of images from Meteosat Second Generation and other weather satellites received with the MSG Data Manager. It is dedicated to EUMETCast data. If you want false-colour animations with geographic re-projection and weather chart overlay, please look at my GeoSatSignal program. Program highlights
Sample animationsHere are some sample still screen-shots showing the MSG Animator in action. You can use thermal channel, water-vapour channel or visible channel data as the source of the animation and, by using colour look-up tables, this date can be made easier to understand.
Getting the MSG Animator
Downloads
Beta version: This program is under continual development - check here for the latest update. Visit the SatSignal self-help group to discuss this software with other users
Specialised softwareIn addition to the software listed on this page, I have a number of other software packages for processing Meteosat-8/9/10/11 data including MSG to GeoTIFF for converting the data to radiance, brightness-temperature and reflectance and saving the results as GeoTIFF files, and the MSG Fire Tracker for detecting fire hot-spots using a multi-channel algorithm and reporting the results up to an FTP server. Please contact me if you are interested in this software. I can also supply readers for the EARS ATOVS data, the Sea-Ice and Sea-Surface Temperature (SST) data, and the GRIB data from EUMETCast MPEF and DWDSAT data broadcast over the DWDSAT service. Fire detection with Meteosat-8/9/10For example, here is a sample screenshot from the MSG Fire Tracker program. Using Meteosat-8 data from 07 August 2005, fires have been detected in North Africa, Spain, Portugal and the south of France. The detected fire pixels are highlighted by yellow rectangles in the image below.
MSG timestamp programSome data sources such the EUMETSAT Archive (MARF) provide MSG HRIT segment files which can be processed with the MSG Data Manager, but the timestamp on these files is either incorrect or becomes lost when transferring between the source and your PC. The MSG timestamp program is a very simple, free utility which will timestamp MSG HRIT according to their file names, so that they will process in their natural order. This is useful to allow the MSG Data Manager to process these files, and may help other manufacturer's software as well. This is a free utility.
Sample images from Meteosat-8 (MSG-1)Guidelines on publishing EUMETSAT data UK Snow - January 2004This image was from January 29th 2004 after extensive snow-falls in the UK. The high-resolution visible channel (HRV) has been combined in GeoSatSignal with thermal information from the 10.8µm thermal channel to make a false-colour image, which has then been rectified to a polar-stereographic projection. Image copyright EUMETSAT 2004.
Tropical cyclone Manou - May 2003Here is an image taken during the trials of EUMETSAT's MSG-1. It show tropical cyclone Manou off Madagascar. It was processed by the MSG Data Manager and Paint Shop Pro. Image taken 2003 May 08 at 0744 UTC. Click for a full resolution version (130KB). Note that the image is near the edge of the field of view, and there appears slightly squashed. The HRV channel provides images in the visible waveband, 11136 pixels by 5568 pixels. Image copyright EUMETSAT 2003.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
