Baselines

This page summarizes baseline results for a range of existing satellite-based precipitation products evaluated using the SatRain benchmark dataset. The results are intended to provide a common point of reference for the development and evaluation of new precipitation retrieval algorithms.

Product	Primary Reference	Implementation	Evaluation Notebook
GPROF V07	https://doi.org/10.1007/978-3-030-24568-9_8		Notebook
GPROF V08	Upcoming	github.com/simonpf/gprof_nn	Notebook
ERA5	https://doi.org/10.1002/qj.3803		Notebook
PERSIANN CCS	https://doi.org/10.1175/JAM2173.1		Notebook
PERSIANN PDIR-Now	https://doi.org/10.1175/JHM-D-20-0177.1		Notebook
PU-Net	https://doi.org/10.1175/JHM-D-25-0162.1		Notebook
GPROF-IR	https://doi.org/10.48550/arXiv.2605.07167	github.com/simonpf/gprof_ir	Notebook

The results below are reported separately for the GMI- and ATMS-based subsets of SatRain.

Base Sensor: GMI

Quantitaive Precipitation Estimation

import xarray as xr
res_gprof_ir = xr.load_dataset("/home/simon/src/satrain/src/satrain/files/baselines/gprof_ir_gmi_austria.nc")
res_punet = xr.load_dataset("/home/simon/src/satrain/src/satrain/files/baselines/punet_gmi_austria.nc")
res_gprof = xr.load_dataset("/home/simon/src/satrain/src/satrain/files/baselines/gprof_v08_gmi_austria.nc")

from IPython.display import HTML, display
from satrain.baselines import get_ranked_results, plot_baselines_interactive, QPE_METRICS
results = get_ranked_results(base_sensor="gmi", metrics=list(QPE_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))

Precipitation Detection

The precipitation-detection skill of all retrievals is assessed based on their ability to identify reference precipitation rates exceeding 0.1 mm h\(^{-1}\). For algorithms that provide a precipitation mask or flag, this information is used to compute the probability of detection (POD), false alarm rate (FAR), and Heidke Skill Score (HSS). If an algorithm additionally provides probabilistic precipitation estimates, detection performance is further evaluated using the area under the Precision–Recall (PR) curve. For algorithms that do not provide dedicated precipitation-detection outputs, all pixels with retrieved precipitation rates exceeding 0.1 mm h\(^{-1}\) are classified as raining and evaluated using POD, FAR, and HSS. Because the area under the PR curve requires probabilistic detection output, this metric cannot be computed for products that do not provide precipitation probabilities.

from satrain.baselines import DETECTION_METRICS


results = get_ranked_results(base_sensor="gmi", metrics=list(DETECTION_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))

Heavy Precipitation Detection

The heavy-precipitation-detection skill of all retrievals is evaluated by testing their ability to identify reference precipitation rates exceeding 10 mm h\(^{-1}\). The metrics are the same as for the detection of precipitation.

from satrain.baselines import HEAVY_DETECTION_METRICS
results = get_ranked_results(base_sensor="gmi", metrics=list(HEAVY_DETECTION_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))

Base Sensor: ATMS

Quantitative Precipitation Estimation

from satrain.baselines import get_ranked_results, plot_baselines_interactive, QPE_METRICS

results = get_ranked_results(base_sensor="atms", metrics=list(QPE_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))

Precipitation Detection

from satrain.baselines import DETECTION_METRICS

results = get_ranked_results(base_sensor="atms", metrics=list(DETECTION_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))

Heavy Precipitation Detection

from satrain.baselines import HEAVY_DETECTION_METRICS

results = get_ranked_results(base_sensor="atms", metrics=list(HEAVY_DETECTION_METRICS))
fig = plot_baselines_interactive(results)
display(HTML(fig.to_html(full_html=False, include_plotlyjs="cdn")))