Project
ncarnate — Legacy Scientific Data to Modern netCDF4
I rebuilt ncarnate to convert supported HDF4 and netCDF data into netCDF4, verify stored values against the source, and reconstruct coordinates for supported satellite grids and swaths. The Python CLI and library are published on PyPI and conda-forge.
Code ↗ (opens in a new tab)Article →Docs & install ↗ (opens in a new tab)
Inside a conversion
Keep the measurements. Add the coordinates.
Explore the sea-ice file behind the article: compare stored values, inspect a grid cell, and see how ncarnate organizes the output.
964,416 values compared · all equal
Full Arctic and Antarctic daily concentration arrays, checked against the original HDF4 file.

Inspect a cell
The measurements stay the same. ncarnate makes their locations explicit with x, y, latitude, and longitude arrays.
Click a cell or use the arrow keys. This grid samples every eighth row and column; values are not averaged.
Selected cell
0–100 means percent ice cover. Codes 110 and 120 mark missing data and land; they are not percentages.
- Stored type
- Full array
- Values checked
Added in the output
- Latitude
- Longitude
- Projected x
- Projected y
The source defines a polar stereographic projection in its grid metadata. ncarnate reconstructs cell-center coordinates and writes CF coordinate references.
Coordinates here are rounded for display. They are calculated positions, not new satellite measurements.
HDF4 source field
netCDF4 group excerpt
The output field references lon lat and a polar_stereographic grid mapping.
Row and column numbers start at zero. The sample includes every eighth row and column; selections snap to the nearest sampled cell.
About this conversion
This inspector reads a recorded conversion made with ncarnate 2.3.0. The Python converter does not run in your browser. The two complete daily concentration arrays were compared before extracting this compact sample.
Data: NASA/NSIDC AE_SI12, from The HDF Group’s historical sample. The original and converted files are the same ones used for the article’s figure.
Download the original HDF4 file · Download inspector data · Reproduce the conversion
AMSR_E_L3_SeaIce12km_B02_20020619.hdf
Input: 60.2 MB · Output: 35.5 MB. These are whole-file sizes; the inspector displays two fields.
See both poles on the geographic maps

Maps use the converted x/y coordinates and CF projection. Coastlines: Natural Earth. View full-size image.
Scientific data outlives the tools that made it. A satellite granule written in 2007 still holds real measurements, but it arrives in an aging container format with its geography left implicit in a block of metadata that general-purpose tools don’t reconstruct on their own. ncarnate converts supported files to modern netCDF4 and reconstructs coordinates for supported satellite grids and swaths, so compatible tools such as xarray, QGIS, or Panoply can locate the measurements.
For a hands-on walkthrough with an AMSR-E sea-ice granule, read Introducing ncarnate: converting HDF4 and netCDF to netCDF4, or browse the ncarnate documentation and installation guide.
Context
ncarnate began as a 2020 utility of mine, netcdf4_recompressor — a tool to
rewrite a netCDF file with different compression. Two things were wrong with it.
Its command line had been broken since it was written: a stray debugging line
made it silently do nothing. And it only ever handled netCDF, never the
HDF4/HDF-EOS2 files its own documentation claimed to support. In 2026 I rebuilt
it around a larger goal than compression — read the legacy earth-science formats
and emit netCDF4 that today’s tools can use directly.
The problem
There are two hard parts, and they pull in different directions.
The first is fidelity. Recompressing or converting a scientific file must change how it is stored and nothing about what it says. Packed integers, fill values, and scale factors have to survive; a conversion that quietly re-quantizes a measurement has corrupted the data it claims to preserve.
The second is geolocation. HDF-EOS2 doesn’t store latitude and longitude for every pixel. A grid is described by projection parameters; a swath carries coarse geolocation plus rules for interpolating it. Human-readable it is not, and until those coordinates are made explicit and standard, the file won’t line up on a map in a general-purpose tool.
Approach
The fidelity side is a contract enforced mechanically: converting changes storage, never data. Stored values are preserved value-for-value — integer and packed values bit-for-bit, while floating-point checks treat NaN representations and signed zero as equivalent. Fill values and scale factors are carried across as declarations rather than applied, and every output is checked value-for-value against its source before it is allowed to replace anything — the original is never destroyed on a failed run.
The geolocation side is the harder part. ncarnate parses the
HDF-EOS2 StructMetadata and reconstructs
CF-convention coordinates. Grid projections
(polar-stereographic, geographic, and the EASE-Grid equal-area projection) become
CF grid mappings plus one-dimensional x/y and two-dimensional lat/lon,
inverse-projected through PROJ via pyproj. Swath
geolocation is attached as CF coordinates, and where a swath stores its geography
at coarser resolution than its data (a 5 km grid under a 1 km field),
the missing coordinates are interpolated — through three-dimensional Cartesian
space rather than raw latitude and longitude, so the result stays correct across
the antimeridian and near the poles, which is exactly where polar-orbiting
satellites spend their time.
To check the reconstruction against something other than itself, I compared its output for an AMSR-E sea-ice granule against The HDF Group’s own independent conversion of the same file; the coordinates agree to about a hundred-thousandth of a degree.
Outcome
ncarnate is live on PyPI under the MIT
license — pip install ncarnate, CLI and library both. It converts supported
HDF4/HDF-EOS2 and netCDF files to recompressed netCDF4 and adds CF coordinates
for supported satellite grids and swaths. The format-support notes
describe the supported cases and limitations.
It’s also on conda-forge —
conda install -c conda-forge ncarnate. Conda-forge supplies the native HDF4,
netCDF, and PROJ libraries together, and the recipe went through a conda-forge
maintainer’s review before it was merged. With pip, no separate native-library
installation is needed when binary wheels are available for your platform and
Python version; if pip needs to build pyhdf from source, you will need the
HDF4 library and build tools — see the installation guidance.