Everything we change, and everything we get wrong

The front page now uses a faster early-warning model, after a live week

Before changing what residents see, we replayed every candidate model against all 18 floods in our record, twenty times over, each time with the real delays in when gauge readings reach this site, including the worst case where the sister gauge on Clanfield Brook is always too late to use. Then we ran five models live side by side, the one you saw among them, on every fifteen-minute update for a week: 672 updates, every answer kept, and eight checks written down before the week began. The model now on the front page warned of all 18 floods at least four hours ahead, against 15 for the previous one, with 48 false orange days instead of 97 and 3 false red days instead of 5. On Storm Bert it would have gone orange 25 hours ahead, against 19. The price is red: 5 hours ahead on Storm Bert, where the previous model managed 8. Seven of the eight checks passed. The one that failed was about how fresh the Clanfield readings are: since mid-September the Environment Agency has published only about sixty a day, where a gauge read every fifteen minutes gives ninety-six. We decided to switch anyway, because the replay had already assumed the worst, Clanfield never arriving in time, and the new model still warned of 17 of the 18 floods on rain alone; and on the live week Clanfield was usable on two updates in three. The previous model still runs behind the scenes and is still scored on every update, so if the new one does worse on real weather this winter, this record will show it and we can switch back the same day.

How the prediction works now →

Our own security work blanked two maps for about four hours

As part of the clean-up after the two attacks on 4 October, we changed how the maps load so that the private code the map provider gives us is never sent to visitors’ browsers, and then replaced that code so any old copy stops working. Between replacing it and the site picking up the new one there was a gap of about four hours on 6 October, and in that time the detailed sensor map and the members’ map showed a blank background. Our mistake was the order of the steps: the new code should have been in place before the old one was switched off. River levels, warnings and the forecast were not affected, and both maps have worked normally since. We checked afterwards that map tiles load at every zoom level and that the private code appears nowhere on the public site.

Attacking our own site found a quieter danger: a dead rain feed would have read as 0 mm

The second of the two attacks we ran on this site on 4 October (see the security entry for that day) did not only look for ways in. It went through everything the site does looking for anything that fails without saying so, and it found one that mattered to residents: if the weather service we read rainfall from stopped answering, this site would have shown no rain at all — on the front page, in the risk score and in the nine o’clock forecast — instead of saying the figure was missing. A broken feed would have looked exactly like a dry week. It was fixed and live the same evening: the front page now says “Rainfall data unavailable”, the risk score leaves rainfall out rather than counting it as none, and the stored forecast is marked as made without it. We then went back through every nightly forecast on record, 211 of them since February, and checked each night that had recorded no rain against an independent weather archive. Of 27 such nights, 25 were genuinely dry. On the other two, in May and July, the archive shows two to five millimetres over the week that we did not count; we cannot tell whether that was the feed failing or the two records measuring slightly different places. On both nights the brook stood at 0.9 metres, the forecast matched what it then did, and the colour shown was right. One limit remains: on a night the feed does fail, the forecast is still worked out without that day’s rain, so it will be marked as incomplete but will lean slightly calm.

We attacked our own site, twice, the way someone trying to break it would

On 4 October we ran two full attacks on this site. Each one went after every way in a hostile outsider might try: every page, every form, everything that saves or changes data, and the machinery that runs behind the scenes every fifteen minutes. The second was carried out without sight of the first, so it could not simply repeat it. Between them they raised 59 points. Six were serious, and all six were fixed and tested on the live site the same day. The worst would have let someone outside alter the site’s record of past floods; once it was closed we checked the record itself, and nothing in it has been changed since January. Neither attack found any way for an outsider to change the river level, the warning colour or the forecast you see. The second attack also turned up a fault that had nothing to do with intruders — a broken rain feed that would have looked like a dry week — and that is fixed too: see the entry for the same day. Most of the remaining points were hardening rather than holes. Update, 6 October: by the following morning every point rated serious or medium was closed and tested on the live site, along with most of the smaller ones, including moving the maps so they load through this site rather than from the map provider directly. Update, 7 October: that list is finished. Of the 45 points the second attack raised, 44 are fixed and tested on the live site, and one is kept as it is on purpose, because it is how members’ personal links are meant to work and anyone holding one still cannot sign in as that member. Since 2 October every change to the site is also checked automatically before it can go live.

At the largest text size, the menu slid off phone screens

Many people read this site with their phone’s text turned up. At the largest size, the menu at the bottom of the screen could be pushed partly out of view, and the text-size buttons could sit on top of what you were reading. Our own automatic checks caught it, not a resident. The text-size buttons now live in the top bar, where nothing sits underneath them, and every public page is now checked at normal and at large text, on a phone and on a computer, before any change goes live.

Beat the model: you can now try to do better than we did

The hardest question this site faces — more warning time, or fewer false alarms — is not one we should answer alone. Beat the model lets anyone move the real settings of the two models we run and see straight away how they would have done across the same six winters we tested on: floods caught, false alarms raised, and hours of warning before Storm Bert. It scores your settings with exactly the method we use ourselves, and it was checked to give exactly our own published figures before it went live. If you find a better balance, you can send it to us, and we will look at every one.

Try Beat the model →

We now test new models on the live site, and the first result stopped a change

A model can look good on past winters and still behave differently on the live site. Since 27 September every model we are considering runs alongside the one you see, on the live weather, every fifteen minutes, and its answer is kept. Within an hour it showed us something six years of replay could not: the Clanfield gauge, our one genuine early-warning signal, usually reaches us about half an hour after its reading is taken, and sometimes nearly three hours after. Our testing had assumed it arrived instantly. Tested again with the delays the site really sees, the model we had been preparing to switch to lost its edge in exactly the case that matters, when Clanfield is late. So we did not switch, and the front page still shows the same model it did before. Any future change will be made on what this record shows over a winter, not on what we expected to happen.

Clanfield is not upstream of Bampton, and we have stopped saying it is

We had been calling the Environment Agency gauge at Clanfield "upstream" across this site, and describing it as one of the two gauges on our own water. It is neither. Clanfield Brook is a separate, neighbouring stream about three kilometres away that drains to the Thames on its own; no water from it passes through Bampton. What is true is narrower and still useful: a storm that soaks our catchment usually soaks Clanfield’s too, a few hours earlier, so a rise there is a real early-warning signal — just not a measurement of water heading our way. We have corrected the wording on the Storm Bert chart, the sensor map, the low-data banner and the 13 September logbook entry to say "neighbouring stream, early-warning signal" rather than "upstream".

We now measure how many people use this site, properly

Until today the footer showed a visitor figure that was not a real count, and we have removed it rather than leave a number on the page we could not stand behind. Proper measurement started today. Separately, the site’s own database has kept a plain tally of page loads since 21 August: 315 over the month, 93 of them on the afternoon of the flood group talk. We will publish a monthly figure once a full month exists.

The sewage report is easier to read, and the maps now use Ordnance Survey

The downloadable sewage report has been redesigned so it can be handed to a councillor or a water company without needing anything explained. All three maps on the site now sit on the Ordnance Survey Outdoor map, with the Crown copyright acknowledgement it requires.

We took this site to the Bampton & District Flood Prevention Group

On the afternoon of 17 September we presented this site and the model behind it to the Bampton & District Flood Prevention Group at the Pavilion on Buckland Road, alongside parish representatives from the surrounding villages, the county and district councils, RAF Brize Norton and Thames Water. We showed the failures as well as the working parts: that the model under-called Storm Bert by seventy-four centimetres when run cold, that it now gets about twenty-one hours of warning on that storm, and that it would also have called red several times that autumn when nothing happened — which is exactly why nobody is being sent alerts yet. It was the busiest afternoon this site has ever had. Two things came out of the room that we are now building: a record of what people actually saw during a flood, parish by parish, and a map of where extra sensors should go.

A conversation about what was done to the brook fifty years ago

On 16 September we sat down with a long-standing Bampton resident who described the works carried out on the Shill Brook and the Great Brook decades ago, and the water courses they run through. Bampton flooded repeatedly in the 1960s and 1970s; those works are a large part of why it stopped. None of it is written down anywhere we can read, and most of what this site knows begins with a gauge installed long afterwards. We are recording it, because a channel that was dug or straightened fifty years ago still decides where the water goes today — and because the people who remember it will not always be here to ask. If you know something about how the brooks were changed, or where water used to stand, we would like to hear it.

Get in touch: the help and report page →

The false alarms can be cut. The price is warning time.

We re-ran the six-year replay four more ways, changing nothing in the live model. The live model catches every real event, with about a day’s notice on an average flood-risk day and twenty-one hours on Storm Bert — and across six winters it raised a false red 56 times and a false orange 97 times. Last winter alone it went red nineteen times; the brook never reached red once. A more cautious version of the model raised five false reds and seven false oranges over the same six winters and still caught every event, but only by waiting for the water: under two hours’ notice on an average orange, and half an hour on 24 September 2024. Stacking the two — the live model’s orange, red only when both agree — keeps the day’s notice at orange, cuts the false reds to five, and leaves the false oranges exactly where they were. Nothing on the site has changed. Whether the village would rather have more notice or fewer false alarms is not a question the model can answer, and we will put it to the flood group. One honesty note: last winter was used in building the model, so it is not a blind test of it.

Counting our gauges honestly: only one of them gives real advance warning

In July we wrote that four of our five upstream gauges were dead and that we had fixed them. Re-reading that in September, it flatters us. Those four were never going to warn Bampton in the first place, because they are not on our water. The Shill Brook is fed by its own small catchment; the Windrush, the Evenlode, the Leach and the Thames above us are separate rivers that never pass through the village. They tell us how wet the wider area is, which is worth knowing, but they cannot tell us water is on its way here. Counted honestly, this site has exactly two gauges that tell us anything useful: the one in the village, which only tells us water has already arrived, and Clanfield, which is the single genuine piece of advance warning we own. That is the whole case for the rain gauges and the level sensor we are asking for at Shilton and Black Bourton — not more rivers watched, but the right ones. (Corrected 27 September: this entry called Clanfield part of “our water”. It is not — see the entry for 27 September.)

Our usual line to the Environment Agency went quiet, and the level kept coming

The Environment Agency publishes the Bampton gauge two ways: a feed built for computers, which this site normally reads, and its own public website. On the afternoon of 13 September the computer feed stopped answering altogether — we tested it ourselves and got nothing after twenty-five seconds, twice. The gauge itself was fine, and the Agency’s website was still showing the same reading from it within a second. So the site switched to reading the Agency’s website instead, kept its fifteen-minute record intact, and said so on the front page. One thing we got wrong: the front page said “the EA data feed is not responding”, which reads as if the Agency had stopped measuring. It had not. We have reworded it to say what is actually happening.

We re-checked the Thames finding from sea level, and corrected our own earlier note

Gauges on different rivers are measured from different starting heights, so comparing their raw numbers is comparing apples with oranges. We went back and converted everything to height above sea level. At the moment the Shill Brook peaked on 25 November 2024 it stood at 68.17 metres. The Thames at Tadpole Bridge, just below where our water leaves the village, stood at 66.33 metres — 1.84 metres lower — and did not peak for another two days. Our brook fell 84 centimetres in the following twenty-two hours while the Thames was still rising. The Thames cannot have held our water back. An earlier working note of ours had quoted a record from a Thames gauge that turned out to sit above Radcot weir, upstream of where the brook joins the river; that figure was irrelevant and has been withdrawn.

The same rain, worth three times as much, depending which side of midnight you asked

Until last week this site worked out tomorrow’s peak once a night, at nine in the evening. We went back to Storm Bert and re-ran the same model four different ways, changing nothing inside it, to find out what had really made it under-call that flood by seventy-four centimetres. We expected the answer to be the weather forecast, which had said 23mm when 43mm fell. It was not. Handing the model a perfect forecast barely helped: that accounted for about one twentieth of the miss. Asking it only once a night accounted for nearly all of the rest. The reason is that the figure it uses for how wet the ground already is only moved forward once a day, at midnight. At nine in the evening on 23 November it still knew nothing about the twenty-one millimetres that had fallen that day. Three hours later the same rain was worth nearly a metre of rise instead of a third of a metre. The model is now asked every fifteen minutes instead. Not one number inside it was changed, and nothing was made cleverer. On Storm Bert that alone turns a seventy-four centimetre under-call into three centimetres, and turns no warning at all into about twenty-one hours of it. One part of that deserves saying plainly: the warning appears at midnight partly because of that daily rollover, not because the model watched the rain falling. On the morning of the 23rd this site would still have shown yellow, not red.

A slow feed silently cost us most of a day’s readings

On 8 September the Environment Agency’s feed began taking fifteen to twenty-five seconds to answer every question, when it normally takes well under one. Our collection run gave up waiting each time, and because everything else in that run — rainfall, ground wetness, the evening forecast — sat behind the river level, one slow answer cost all of it. The day’s record held fifty-seven readings instead of ninety-six. We found it that afternoon from our own run logs, not from a resident. The fix was in three parts: the rest of the run now carries on without the level; if the Agency’s feed fails, the site reads the same gauge from the Agency’s public website instead; and the front page names that source when it is used, rather than presenting it as live. The next morning we found the new fallback had been silently refusing nine pages in ten over a rounding difference, and fixed that too.

We tested our own theory that the Thames caused Storm Bert. It was wrong.

After the sealed exam we had a strong theory: on Storm Bert the Thames was in record flood, so the Shill Brook could not drain into it and backed up through the village — a resident saw a side stream flow the wrong way. We wrote the theory down, then pulled the fifteen-minute records for every gauge in the window and tested it properly. It failed: the flood water in Bampton sat well above the Thames the whole time, and water is not held up by a river beneath it. What the same records did show was simpler. The weather forecast the model was working from said 23mm of rain would fall. 43mm fell. The model did its sums correctly on half the rain, which is why it said 1.43 metres when the answer was 2.17. Rerunning that night with the upstream watcher included — the part of the site that reads rain and river gauges upstream rather than trusting the forecast — gave twenty-four hours of warning before the dangerous level, against none from the forecast model alone. On the strength of this the model was frozen: no further tuning, because a warning system cannot be built on numbers that keep moving underneath it. (Corrected in September. This entry named the weather forecast as the cause of the Storm Bert miss. Measured properly on 8 September, the forecast accounted for only about one twentieth of it, and asking the model just once a night for nearly all of the rest. The twenty-four hours of warning quoted here came from that earlier working; the measured figure is about twenty-one hours. See the entry for 8 September.)

The sealed exam: run cold against Storm Bert, the model under-called by 74cm

Throughout the six-year replay, the fortnight around Storm Bert was kept sealed — never opened, never used to tune anything — so it could be a single honest final test. On 2 August we opened it once. The model was given only what it would have known at each moment: the river level so far, the rain that had fallen, and the weather forecast as it was actually issued at the time. From about twenty-eight hours out to about five hours before the peak, its answer barely moved: a peak of 1.43 metres. The brook reached 2.17 metres. That is a miss of 0.74 metres, in the direction that matters, at the one event the whole site exists for. We publish it because it is true, and because it told us where to look next.

We replayed six years of history, showing the model only what it knew at the time

A model that has only ever been marked against a calm year has not really been marked. So we built a replay: the model runs against every day from January 2020 onward, and at each day it is given only the readings and the weather forecast that existed at that moment — never the answer. The model was never edited during this; every experimental tweak lived in a separate copy. The model itself is simple to state. Tomorrow’s peak is today’s level, plus a rise for rain (rain that has already fallen counted in full, forecast rain counted at well under half because forecasts overstate), plus an allowance for how wet the ground already is, scaled by season. The replay ran in dated blocks, finishing on 1 August: early 2020, spring 2020, early 2021, the winter of 2023–24 chosen for having the most high-water days, then the second half of 2021 and first half of 2022. Part way through we caught the test cheating: for most of the record it had been running without a weather forecast at all, because none had been kept before 2024. We recovered the forecasts that were actually issued at the time and reran everything — 2,275 days, every one with the real forecast. What it showed, in plain terms: on ordinary days the model is accurate, on average within four centimetres and usually within nine. On elevated days it runs about twenty centimetres too high — over-cautious, a false alarm rather than a missed one. On the thirty-one days the brook was actually flooding land it leans safely high on average but the spread is wide, up to half a metre either way, and no adjustment we tried made that better. One day in six years reached property-risk level. Storm Bert was kept sealed for a separate exam.

618 river readings from the outage week, recovered before they aged out of the EA feed

The Environment Agency keeps about a month of readings, then drops them. When our database was down from 24 July we lost the copy we would normally have kept. On 1 August, with a few days to spare, we went back to the Agency and recovered 618 readings for 24–30 July into the record, passing each through the same impossible-reading check as a live one. We did not invent the predictions the site would have made that week; that gap is shown as a gap, and labelled.

We went looking, and found our advance warning could not have worked

This site exists to give Bampton more notice than the gauge in the village can, by watching rivers upstream. On a dry morning in July we set out to prove those upstream gauges were live. Four of the five were not, and never had been — leaving only the Bampton gauge itself, which tells us water has already arrived, not that it is coming. (Re-read in September, this entry flattered us: see 13 September. Four of those five sit on other rivers and could never have warned Bampton anyway.) The six to eighteen hours of warning was not less accurate than we thought; it could not have happened at all. No flood exposed this and nobody reported it: we found it by checking on a quiet day. All five are now live.

Checking every gauge one by one found nine that had never reported

Having found four dead gauges, we did not assume the rest were fine. We tested all eighteen against the Environment Agency’s own records, one at a time. Nine of them were being asked for under the wrong reference number — each gauge has two, and they are not interchangeable — so they returned nothing at all instead of failing loudly. Nothing looks exactly like calm. Everything affected was on the Windrush, the Evenlode and the Leach. All eighteen now report, and an automatic check tells us the day any of them stops.

Eighteen gauges now feed this site, across four rivers

The site reads eighteen gauges: the one in the village on the Shill Brook, one on the Clanfield Brook, and sixteen more along the Thames, the Windrush, the Evenlode and the Leach. Five of those sixteen sit on the Thames above Bampton — water on its way here, rather than water already here. Six sit below the village, where a high Thames can hold our brook back. The remaining five are on tributaries that do not drain through Bampton at all; they show how wet the wider area is under the same weather.

Repairing the gauges exposed a fault that would have raised a false alarm

Within minutes of the gauges reporting again, the site announced the Thames rising a metre and a half in a single hour, on a dry day with no rain. It was taking two different measurements from the same gauge and comparing one against the other. In November that would have told the village something was coming that was not. Because we were watching the repair, we saw it the same morning, fixed it the same afternoon, and added a check that refuses any rise too fast to be real.

The site could have said "moderate" while the brook was at a record level

We asked our own risk score what it would report if the brook stood at Storm Bert levels. It answered "moderate". The score averaged the river level together with rainfall, ground and upstream conditions, so a dangerously high brook could be pulled back down by everything else being calm. The measured level now sets a floor the score cannot fall below. No resident ever saw this: the brook has not come close to those levels since the site launched, and we found it by asking the question rather than by living through the answer.

You will see the word "elevated" more often this winter

As part of the same change, a brook between 1.06m and 1.40m is now described as "elevated" rather than "moderate". Nothing about the river has changed and nothing has got worse — we have made the wording match what those levels have always meant. Expect to see it during ordinary wet spells.

We read our own front page as a resident would, and it contradicted itself

During the data outage the page correctly warned that the live river feed was unavailable, and directly underneath showed a green status light and the words "Trend: stable". Green reads as "safe". Nobody reported this. We found it by sitting down and reading our own front page the way someone in the village would, in the middle of the outage it happened in. It now shows grey, says the level is unknown, and points you to the Environment Agency.

We took the river reading away on purpose, and the site still said "low risk"

With no river reading at all, the site would still work out a risk level from everything else and call it low. Unknown is not the same as safe. We found this deliberately — by removing the reading and watching what the page did — rather than waiting to discover it during a flood. It now says plainly when it does not know.

The live river level kept working while the database did not

The database has been unavailable since 24 July. Since the 27th the level on the front page has come straight from the Environment Agency instead, and it has not stopped since. This is no longer a plan for an outage; it is something that has now held up during one. Stored history and predictions remain unavailable until the database returns — the number that matters most during a flood does not depend on it.

Our database ran out of its monthly allowance

The site could not show stored history or predictions. Live river levels were restored on 27 July by reading the Environment Agency directly. We are moving to a different arrangement so this does not recur.

Auditing our own accuracy figures found one impossible reading skewing them

We went back over the accuracy figures we publish, to check they were honest. A single physically impossible reading from the river gauge had been enough to distort them. We now reject impossible readings before they are stored, and we corrected the figures that had already been published rather than let them stand.

Routine security maintenance

We review and update the software this site is built on regularly. This site has no accounts and no forms — there is nothing of yours held here to expose. If that ever changes, or if anything happens that affects you, it will be recorded on this page. (Corrected 4 October: this is no longer true. Since September the site has had a members’ area for the flood observation log, which holds the name, email address and area of each person taking part and the places they record, and a form on Beat the model. Both were inside the scope of the two attacks recorded on 4 October.)

The site now tells you when a reading is old

Previously a reading that had stopped updating looked exactly like a fresh one. A banner now appears when the latest reading is more than about an hour and a half old.

Two separate systems fetch the river reading, so one failing does not stop it

The gauge is read every fifteen minutes. Two independent systems ask for it, running on different machines in different places, and either one on its own is enough — each run reaches back over the previous day and fills in anything missed. If one stops, the reading is still collected, and the record does not develop a hole nobody notices.

The site showed the same readings for ten days without updating

Between 22 May and 1 June several parts of the site quietly served old numbers instead of fetching new ones, and gave no sign anything was wrong. Anyone who looked during those ten days was reading stale figures without being told so. This one was live, not hypothetical, and we claim no credit for catching it early. We found the cause, fixed it, and added an automatic check that fails the build if it can happen again.

We switched off our own sewage report rather than leave it wrong

We checked what it was producing and found figures for the wrong stretch of river. We took it down the same day rather than leave a working-looking download handing people the wrong catchment, and it stayed down until it could be rebuilt properly.

Sewage discharges are recorded, including the ones in dry weather

The site records every storm overflow the water company reports within about fifteen kilometres of Bampton: when each discharge started, when it stopped, and how much rain fell at that spot at the time. A discharge with no rain behind it is flagged, because a storm overflow releasing in dry weather may fall outside the terms it is allowed to operate under. Any date range can be downloaded as a spreadsheet or a report.

Every prediction we make is published with the error it turned out to have

Each forecast is recorded before the event, then compared with what the brook actually did, and both numbers stay on the site with the difference between them in metres — the misses alongside the hits. We have not found another flood service that publishes its own errors this way, the official ones included. One limit we will not bury: the model has only ever been tested against levels the brook has actually reached since January 2026, which is a long way below Storm Bert. Where it matters most, it is untested.

The sewage page could show normal-looking data that was not real

When the real discharge data was unavailable, the page filled in with stand-in figures that looked like ordinary readings. We removed that, so it now says plainly when it does not know.