Which model wins where
Average miss by forecast length
Mean absolute error: the typical gap between the forecast and what the weather station recorded. Lower is better. A dot marks the best model, plus any model that isn't clearly worse.
How fast each forecast degrades
Average miss at each lead time, with 95% confidence intervals. The dotted grey line is NOAA's 1991–2020 normal for each day; in bias-corrected mode it is shifted by each station's average departure this season, the same way as the forecasts. A forecast only adds value while it stays below that line.
Do forecasts close in on the answer?
Each faint line is one day: how far off a model's forecast for that day was when issued 7 days ahead, then 6, and so on down to 1. The shaded bands hold the middle half and the middle 80% of days, and the solid line is the median. If a model homes in on the answer, its band narrows as you read right.
How often each model changes its mind
Every day each model issues a new forecast for the same date. The chart shows the average size of each revision, often called jumpiness. The summary below also gives the flip-flop index (Griffiths et al., 2019), which counts only back-and-forth movement: a forecast that moves steadily in one direction scores zero. Neither score looks at what actually happened, so a steady model can still be steadily wrong.
The day-by-day views need a single city. Pick one from the City menu, or click a city on the map.
Day by day at one lead time
Thick ink is what the weather station recorded. Each coloured line is what a model said that many days ahead. Drag across the strip underneath to zoom in, or use the buttons. Click a day to open it below.
Seven forecasts for one day
Each line is one model's forecast for the selected day, from the run issued 7 days before (left) to the run issued the day before (right). The dashed line is what the station recorded. Click any day in the chart above or the error map below to open it here.
Where each model ran hot or cold
Who called the wet days
A wet day is 1 mm (0.04 in) or more at the rain gauge. The score is the critical success index (CSI, also called the threat score), a standard measure for yes/no forecasts: hits ÷ (hits + misses + false alarms). 1.0 is perfect, and dry days correctly left dry don't inflate it. Model rain is an average over a grid square about 25 km wide, so it falls on more days than at any single gauge; some over-forecasting of wet days is expected. Rain is never bias-corrected. Hover a cell for the counts.
How this works
- Forecasts come from the Open-Meteo Previous Runs API, which keeps what each model predicted 1 to 7 days before every hour, on a grid about 25 km wide, at the grid point nearest each city's official weather station. "1 day ahead" means the forecast valid for that day from the run about a day earlier, so the real lead time is roughly 24 to 48 hours.
- Highs and lows use only the forecast values at 00, 06, 12 and 18 UTC. The AI models (and ECMWF IFS beyond 6 days) are only produced every 6 hours, and the hours in between are interpolated, so taking every model at the same four times keeps the comparison like-for-like. Those four times can miss the real afternoon peak and overnight minimum, so raw highs generally come out too cold and raw lows too warm. GFS's 1-day lows, which include the HRRR blend, are an exception. Rain is the sum over the local day.
- Bias-corrected scores (the default) remove each model's average error at each station, for each lead time, estimated from every other day in the window, including later days a forecaster wouldn't have had; a version using only earlier days gives the same ranking for daily highs. NOAA normals get the same correction, so the baseline also absorbs this season's warm or cool departure. This is a simple version of the local calibration that weather services apply before forecasts reach the public. Switch to Raw to see the uncorrected output. Rain is never corrected.
- Confidence intervals come from a block bootstrap: the window is resampled 400 to 1,000 times in 7-day blocks, keeping every city's data for a date together. A model is marked best, or a city's winner, only when the 95% interval of its advantage excludes zero. These are separate tests, so across many cities and lead times one or two "clear" wins can be chance. The headline result, ECMWF AIFS best on bias-corrected daily highs at every lead, also passes a Bonferroni correction for 21 comparisons.
- NOAA GFS here is Open-Meteo's "seamless" feed, which blends in NOAA's 3 km HRRR model over the US for the first 18 to 48 hours. Its 1-day scores are really a GFS/HRRR blend.
- Observations are NOAA's official daily station record, GHCN-Daily: Central Park, O'Hare, Miami Intl, Bush Intercontinental, Denver Intl, Sky Harbor, downtown LA (USC) and Sea-Tac. NOAA's day runs midnight to midnight standard time, so during daylight saving it is offset by an hour from the forecast's day. Normals are NOAA's 1991–2020 daily normals for the same stations.
- Models: ECMWF IFS is the European physics model and NOAA GFS the US one. ECMWF AIFS and NOAA AIGFS are their AI counterparts. AIGFS is built on Google DeepMind's GraphCast and has been operational since late 2025; its archive is patchy before July 2026. Both AI models were trained mainly on ERA5, ECMWF's reconstruction of global weather since 1979.
- Scores: avg miss is mean absolute error, and bias is mean signed error. Jumpiness is the mean absolute change between consecutive runs. The flip-flop index (Griffiths et al., 2019) is the total movement across the seven runs minus their range, divided by five, so it is zero for a forecast that only ever moves one way. Both consistency scores use the raw forecasts as issued.
- Fair comparison: at each lead time, only days where all four models and the station have data are scored.
- Map outlines come from us-atlas (US Census Bureau boundaries).
- Snapshot: the data was downloaded on — and covers about 13 weeks ending at the latest station observations. This is a one-off study of one summer and early autumn.