Project

General

Profile

Using a container based on netCDF or HDF5 for grib

Added by Luis Kornblueh over 13 years ago

We recently had a discussion on improving performance on access to files by using container for small record streaming data. The later could be for example GRIB. Getting a fast access to GRIB data require in general some type of index. These are usually provided by self describing formats like netCDF or HDF5. It could improve random access in any type of post-processing. This would be an interesting point to discuss. One advantage of this would be that the GRIB records do not get compromised.

By the way HDF5 does contain undocumented (maybe I'm not the best manual reader) the GRIB encoding algorithm as compression type. This would give another possibility to think on.
It might be hard, because WMO has only recently adopted GRIB2 and the introduction is still a very slow process.