satrain.input

The satrain.input module provides configuration classes to represent and configure the retrieval input data for the SPR dataset. The currently supported input datasets are GMI observations, ATMS observations, ancillary data, geostationary observations, and geostationary IR observations. These retrieval inputs are represented by the classes Ancillary, GMI, ATMS, Geo, GeoIR. Each of these classes provides configuration options for the data that is actually loaded.

Usage

The above-named input config classes can be used everywhere that retrieval input data is specified, most notably in the satrain.evaluation.Evaluator and the dataset classes.

Alternatively, the retrieval input datasets can be specified using strings. In this case, the strings gmi, atms, ancillary geo, geo_ir map to the classes Ancillary, GMI, ATMS, Geo, GeoIR, which will be instantiated using their default configuration.

Finally, it is also possible to specify an input using a dictionary. The dictionary must in this case have an entry name, which should correspond the string representation of the input. The remaining key-value pairs in the dictionary will then be passed to the constructor of the corresponding input config class.

As an example, the following three ways of repsenting GMI retrieval input are equivalent:

retrieval_input = GMI(channels=None, include_angles=True, normalize=None, nan=None)
retrieval_input = ["gmi"]
retrieval_input = {
    "name": "gmi",
    "channels": None,
    "include_angles": True,
    "normalize": None,
    "nan": None
}

Members

class ATMS(channels: List[int] | None = None, include_angles: bool = True, normalize: str | None = None, nan: str | None = None)

Retrieval input data from the Advanced Technology Microwave Sounder (ATMS).

The ATMS class represents observations from the Advanced Technology Microwave Sounder as retrieval input data. It allows for selecting subsets of the available ATMS channels and including or excluding the earth-incidence angles in the input data.

The ATMS input will load tensors ‘obs_atms’ containing the ATMS passive microwave observations and, if ‘include_angles’ is set to ‘True’, ‘eia_atms’ containing the earth incidence angles corresponding to the observations in ‘obs_atms’.

Parameters:

• channels – An optional list of zero-based indices identifying channels to load. If ‘None’, all channels will be loaded.

• include_angles – Wether or not to include the earth-incidence angles of the observations in the input.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping the input names from the ATMS input to the corresponding number of channels.

property name: str

String representation of the input.

class Ancillary(variables: List[str] | None = None, normalize: str | None = None, nan: float | None = None)

This InputConfig class will load ancillary data as retrieval input. The class allows for configuration, which variables will be loaded.

Including the ‘Ancillary’ input config in the list of retrieval inputs will load the ancillary data and include it in the retrieval input data as a variable named ‘ancillary’.

Parameters:

• variable – A list of strings specifying the ancillary data to load.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the ancillary data input class to the corresponding number of features.

load_data(ancillary_data_file: Path, target_time: DataArray) → Dataset

Load ancillary data from NetCDF file.

Parameters:

• ancillary_data_file – A Path object pointing to the file from which to load the input data.

• targete_time – Not used.

Returns:

A dicitonary mapping the single key ‘ancillary’ to an array containing the data from all ancillary variables stacked along the first axis.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class GMI(channels: List[int] | None = None, include_angles: bool = True, normalize: str | None = None, nan: str | None = None)

Retrieval input data from the GPM Microwave Imager (GMI).

The GMI class represents observations from the GPM Microwave Imager (GMI) as retrieval input data. It allows for selecting subsets of the available GMI channels and including or excluding the earth-incidence angles in the input data.

The GMI input will load tensors ‘obs_gmi’ containing the GMI passive microwave observations and, if ‘include_angles’ is set to ‘True’, ‘eia_gmi’ containing the earth incidence angles corresponding to the observations in ‘obs_gmi’.

Parameters:

• channels – An optional list of zero-based indices identifying channels to load. If ‘None’, all channels will be loaded.

• include_angles – Wether or not to include the earth-incidence angles of the observations in the input.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping the input names from the GMI input to the corresponding number of channels.

property name: str

String representation of the input.

class Geo(channels: List[int] | None = None, normalize: str | None = None, nan: float | None = None)

The Geo class represents GOES-16 ABI observations in the retrieval input. The full IR input comprises 2 10-minute observations before the median overpass time and 2 after the median overpass time.

Parameters:

• channels – Optional list of zero-based indices identifying the GOES channels to load.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the Geo input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO observation from the desired time steps. The returned array will have the time and channel dimensions along the leading axes of the array.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class GeoIR(normalize: str | None = None, nan: float | None = None)

The GeoIR loads input data from IR-window channel observations interpolated in time to be closest to the nominal time of the precipitation estimates.

Parameters:

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the GeoIR input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO IR data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO IR observation from the desired time steps.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class GeoIRT(time_steps: List[int] | None = None, normalize: str | None = None, nan: float | None = None)

The GeoIR class represents IR-window channel observations from geostationary satellites in the retrieval input. The full IR input comprises 8 half-hourly observations before the median overpass time and 8 after the median overpass time. The GeoIR class allows selecting subsets of these time steps as well as only loading the nearest observations for every reference data pixel.

Parameters:

• time_steps – Optional list of time steps to load. The time steps are identified using zero-based indices with steps 0-7 to the eight time steps prior to the median overpass time and steps 8-15 to the eight time steps after the overpass time.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the GeoIR input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO IR data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO IR observation from the desired time steps.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class GeoT(channels: List[int] | None = None, time_steps: List[int] | None = None, normalize: str | None = None, nan: float | None = None)

The Geo class represents GOES-16 ABI observations in the retrieval input. The full IR input comprises 2 10-minute observations before the median overpass time and 2 after the median overpass time.

Parameters:

• channels – Optional list of zero-based indices identifying the GOES channels to load.

• time_steps – Optional zero-based indices of the time steps to load. Indices 0 and 1 correspond to 30 and 15 minutes before the median overpass time and indices 2 and 3 to 15 minutes after the median overpass time.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the Geo input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO observation from the desired time steps. The returned array will have the time and channel dimensions along the leading axes of the array.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class InputConfig

Base class for input data records used to define what input data to load.

abstract property name: str

String representation of the input.

classmethod parse(inpt: str | Dict[str, Any] | InputConfig) → InputConfig

Parse InputConfig object from an argument that can be either a string, a dictionary or an InputConfig object.

If ‘inpt’ is a string, this method will simply instantiate the InputConfig sub-class of the corresponding name, which will be instantiated with the default settings. If ‘inpt’ is a dictionary, it must have field ‘name’ specifying the name of the InputConfig sub-class to instantiate. All other keys in the dictionary will be passed to the constructor call of this class. Finally, if ‘inpt’ is allready an InputConfig sub-class object, it is returned as-is.

Parameters:

inpt – The inpt to parse as a InputConfig object.

Returns:

An object of an InputConfig sub-class.

abstract property stats: Dataset

xarray.Dataset containing summary statistics for the input.

to_dict() → Dict[str, Any]

.toml compatible dictionary representation of input config.

class PMW(channels: List[int] | None = None, include_angles: bool = True, normalize: str | None = None, nan: str | None = None)

InputData record class representing passive-microwave (PMW) observations.

Parameters:

• channels – An optional list of zero-based indices identifying channels to load. If ‘None’, all channels will be loaded.

• include_angles – Wether or not to include the eart-incidence angles of the observations in the input.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

property ang_stats: Dataset

xarray.Dataset containing summary statistics for the viewing angles.

load_data(pmw_data_file: Path, target_time: DataArray) → Dict[str, ndarray]

Load PMW observations from NetCDF file.

Parameters:

• pmw_data_file – A Path object pointing to the file from which to load the input data.

• target_time – Not used.

Returns:

A dictionary mapping the keys ‘obs_<sensor_name>’ the loaded PMW observations. If ‘include_angles’ is ‘True’ the dictionary will also containg the earth-incidence angles with the key ‘eia_<sensor_name>’.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class Seviri(channels: List[int] | None = None, normalize: str | None = None, nan: float | None = None, remap_obs: bool | None = False)

Special instance of the Geo class load observations from the SEVIRI sensor of the ‘austria’ domain.

Parameters:

• channels – Optional list of zero-based indices identifying the GOES channels to load.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

• remap_obs – Boolean flag indicatin whether or not to remap the observations to match the distribution of corresponding GOES channels.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the Geo input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO observation from the desired time steps. The returned array will have the time and channel dimensions along the leading axes of the array.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

class SeviriT(channels: List[int] | None = None, time_steps: List[int] | None = None, normalize: str | None = None, nan: float | None = None, remap_obs: bool | None = False)

Special instance of the Geo class load observations from the SEVIRI sensor of the ‘austria’ domain.

Parameters:

• channels – Optional list of zero-based indices identifying the GOES channels to load.

• normalize – An optional string specifying how to normalize the input data.

• nan – An optional float value that will be used to replace missing values in the input data.

• remap_obs – Boolean flag indicatin whether or not to remap the observations to match the distribution of corresponding GOES channels.

property features: Dict[str, int]

Dictionary mapping names of the input data variables loaded by the Geo input class to the corresponding number of features.

load_data(geo_data_file: Path, target_time: DataArray) → Dataset

Load GEO data from NetCDF file.

Parameters:

• geo_data_file – A Path object pointing to the file from which to load the input data.

• target_time – An xarray.DataArray containing the target times, which will be used to to interpolate the input observations to the nearest time step if ‘self.nearest’ is ‘True’.

Returns:

A dicitonary mapping the single key ‘obs_geo’ to an array containing the GEO observation from the desired time steps. The returned array will have the time and channel dimensions along the leading axes of the array.

property name: str

String representation of the input.

property stats: Dataset

xarray.Dataset containing summary statistics for the input.

calculate_input_features(inputs: List[str | Dict[str, Any] | InputConfig], stack: bool = True) → int | Dict[str, int]

Calculates the number of input features given a list of inputs.

Parameters:

• inputs – A list specifying the retrieval input.

• stack – If ‘True’, returns a single integer representing the total number of features of all inputs stacked along the channel/feature dimension. If ‘False’, returns a dict mapping input names to the corresponding number of features.

normalize(data: ndarray, stats: Dataset, how: str | None = None, nan: float | None = None) → ndarray

Normalize input data and replace missing values.

Parameters:

• data – An numpy.ndarray containing the data to normalize.

• stats – An xarray.Dataset containing the summary statistics of the data.

• how – A string specifying how to normalize the data. Should be one of [‘standardize’, ‘minmax’]>

• nan – If given, use this value to replace NAN values in the input.

Returns:

The give array ‘data’ normalized according to the given statistics and chosen normalization method and, if ‘nan’ is not None, with NAN values replaced with ‘nan’.

parse_retrieval_inputs(inputs: List[str | Dict[str, Any] | InputConfig]) → List[InputConfig]

Parse retrieval inputs.

Parameters:

inputs – A list specifying retrieval inputs. Each element in the list can be a string, a dictionary defining the retrieval input configuration or an InputConfig.

Returns:

A list containing the retrieval input configuration represented using InputConfig objects.