8 of 8 · reference
Appendix
Sources, terms, and a couple of the technical specifics — for the curious or for anyone who wants to dig deeper. Skip if you're skimming.
Glossary
| RMSE | Root mean squared error — the standard accuracy metric for forecast predictions vs. truth. Lower is better. |
| GFS | Global Forecast System — NOAA's operational global weather model. The industry-standard baseline. |
| ERA5 | ECMWF's gold-standard reanalysis dataset. Used as ground truth for validation. |
| CHIRPS | Climate Hazards InfraRed Precipitation with Station data. A satellite + station blended precipitation dataset. Second validation source. |
| EMOS-NGR | Ensemble Model Output Statistics — Non-homogeneous Gaussian Regression. The post-processing method that combines the three ensemble members and produces calibrated uncertainty. |
| FCN3 | NVIDIA's spherical Fourier neural operator weather model. One of the three ensemble members. |
| GraphCast | Google DeepMind's graph neural network weather model. Another ensemble member. |
| Calibrated uncertainty | A probability distribution where the stated probability matches reality — when the model says "80% chance of more than 5 mm," it should be right 80% of the time. |
| DMH / DINAC | Dirección de Meteorología e Hidrología, the Paraguayan national met service, sitting inside the civil aviation authority DINAC. |
| MAG | Ministerio de Agricultura y Ganadería — Paraguay's agriculture ministry. |
| Itaipú Binacional | The binational (Paraguay + Brazil) hydroelectric dam authority. Operates a substantial gauge network in the Paraná basin — exactly the region where this project's biggest validation gap is. |
Source materials
- Live technical demo — the canonical place to see results.
- Project repository — private; available on request.
- Validation methodology: 60 forecast initialization dates spanning October 2024 – March 2025, validated at lead times 24–240 hours in 6-hour steps. Bootstrap confidence intervals on all reported improvements.
Data inventory — what flows in, what comes out
The product behind the site is a scorecard: forecast-skill numbers stratified by model, variable, lead time, region, and severity. Three streams feed it — forecast inputs, ground-truth observations, and the metrics computed where they meet.
1. Forecast inputs
Multiple AI weather models run on Modal infrastructure. Three are current ensemble members; two more were evaluated and dropped or deferred.
| Model | Status | Notes |
|---|---|---|
| FCN3 | Active | NVIDIA spherical Fourier neural operator. A100 80GB. |
| GraphCast | Active | Google DeepMind graph neural network. Chained with PrecipitationAFNOv2 for precipitation output. |
| GFS | Active | NOAA physical model. Baseline and ensemble member. |
| Atlas | Evaluated, dropped | NVIDIA diffusion model. Audited on 8 dates; still 17–34% worse than GFS for Paraguay precipitation. Drying bias is structural. |
| AIFS | Deferred | ECMWF model. Blocked on an upstream library bug; cheap to revisit later. |
2. Ground-truth sources
No single truth source covers Paraguay well. The validation strategy is to score against the strongest available source for each variable and cross-check against the others.
| Source | Type | Status |
|---|---|---|
| ERA5 | Gridded reanalysis (ECMWF) | Primary scoring truth. ~5-day delay. |
| CHIRPS | Satellite + station blended precipitation | Secondary scoring truth. Sparse in the east belt. |
| CMORPH | Satellite precipitation | Cross-check source. |
| GHCN-D | Global gauge network | 24 Paraguayan stations exist on paper; ~13% fill rate on the 60-date validation window. |
| OGIMET SYNOPs | Cross-border surface observations | Added in the recent Stage A expansion to backfill borderline-region truth. |
| DMH EMA (live poller) | Paraguay national stations | 101 nominal, ~67 reporting live. Pulled every 5 min, archived daily. Running since 7 May. |
| DMH archive | Historical Paraguay stations | Not yet pulled. Access path is the paid records request (~$7.70/record under Decree 8701/2012) or an institutional waiver — the latter is what the relational push is trying to unlock. |
3. The output — the scorecard
Every forecast × truth pair produces metrics stratified five ways. The +25.7% headline on the project page is one slice of this. The per-severity table below is another. The full cube is what a commodity trader, reinsurer, or government buyer would actually license — the headline is the marketing surface, the cube is the asset.
| Dimension | Values |
|---|---|
| Metric | RMSE, MAE, bias, anomaly correlation (ACC), CRPS |
| Model | FCN3, GraphCast, GFS, EMOS-NGR ensemble |
| Variable | Precipitation (primary); 2 m temperature, mean sea-level pressure, total column water vapour, 500 hPa geopotential, winds (available) |
| Lead time | 24 h to 240 h in 6 h steps |
| Region | Full domain, east (soybean belt), Chaco |
| Severity bin | Dry (<1 mm), light (1–10), moderate (10–25), heavy (>25) |
The validation regions
The Paraguayan domain used throughout:
| Region | Lat range | Lon range | Use |
|---|---|---|---|
| Full domain | −27.5 to −19.0 | −63.0 to −54.0 | National-level reporting |
| East / soybean belt | −27.0 to −22.0 | −57.0 to −54.0 | Primary commercial relevance |
| Chaco (northwest) | −23.0 to −19.0 | −63.0 to −58.0 | Drier; secondary cattle/cotton region |
Stratified results — the honest table
Per-bin RMSE on the full × ERA5 view (60 dates × 35 × 37 cells = 77,700 observations). This is the table that lives behind the +25.7% headline.
| Truth bin | n obs | Mean truth (mm) | Ensemble RMSE | GFS RMSE | Skill vs. GFS |
|---|---|---|---|---|---|
| Dry (<1 mm) | 48,904 | 0.17 | 1.86 | 3.20 | +41.7% |
| Light (1–10 mm) | 20,705 | 3.80 | 4.98 | 9.18 | +45.8% |
| Moderate (10–25 mm) | 6,056 | 15.48 | 10.85 | 14.00 | +22.5% |
| Heavy (>25 mm) | 2,035 | 40.57 | 33.98 | 30.68 | −10.8% |
The pattern is consistent across all four validation views (full × ERA5, east × ERA5, full × CHIRPS, east × CHIRPS). Heavy events lose to GFS by 6–19% across the four. The bias is structural at 25 km cell resolution.
What's running passively right now
- EMA poller. A Modal cron job pulls Paraguay's public weather station feed every 5 minutes and archives a daily Parquet file. Gives us a forward-looking observation archive with no permission required.
- The deployed demo. Hosted on Netlify; regenerated on each repository push.
Why I'm doing this
Two reasons.
First, I think AI weather models are interesting and the application to underserved regions is more impactful than another marginal improvement on a North American or European benchmark. Paraguay is a real economy with real exposure to weather risk and worse forecasting infrastructure than its neighbors.
Second, I wanted to find out whether one person with a laptop can take a credible technical artifact from "research" to "actually used by a stakeholder" without an institution behind them. The honest answer to that is "I don't know yet — that's what the next 90 days are about."
That's the end of the plan site. Back to the overview if you want to reread anything, or how to help if anything specific came to mind.