Did the Warm Homes Visits help households keep warm for less?
Question Did the Warm Homes Visits help households keep warm for less, well enough to justify funding the scheme for another year?
Answered against the evaluator's rubric, agreed on 7 October 2026 before any interview was read. Criterion A: the householder describes at least one change the visit brought about that was still in place or still in use at the interview (draught strips or reflector panels still fitted, still on the tariff they switched to, heating controls still used as advised, loft or cavity insulation installed through the referral, or a Warm Home Discount received after the adviser helped them apply); not met where no such change is described, where the change has since stopped, or where the householder puts it down to something other than the visit. Criterion B: the householder says the home has been warmer, or bills or top-ups lower, and links this to the visit or to something it brought about, in their own words or when asked; not met where neither is described, or where one is described but put down to something else, such as a mild winter, a fall in the price cap or other help, or not linked to the visit. Both criteria must be met in the same household. Good enough: more than half of the households interviewed meet both. Not good enough: half or fewer meet both. Cannot say: the interviews did not ask about changes since the visit.
Synthetic documents. All documents were written by Claude to test the method. They are not records of real people, and nothing in this report is evidence about anyone.
Yes: on the evaluator's rubric the scheme is good enough to fund for another year. Of the households interviewed, describe, in their own home, both a change the visit brought about that was still in place at the interview and a warmer home or lower bills that they link to the visit. The rubric asks for more than half, so the verdict would still hold if any one household were read the other way. Every interview asked what had changed since the visit, so the rubric's condition for giving no verdict does not arise. The weakest case is , whose saving comes from a cheaper tariff rather than a warmer home or less energy used.
4 interviews in 3 groups · 8 passages coded · 10 October 2026
Verification in this report:
3 numbers, every one computed directly from the coded passages rather than written by the AI. Click any number to see who it counts.
8 quotations, every one found word for word in its interview. Click an interview's ID beside a quotation to read it in place.
22 findings and 3 high-level findings, every one checked against the interviews after they were coded a second time.
Changes that lasted
Every household interviewed, , describes a change the visit brought about that was still in place or in use about six months later. No interview describes a change that has since stopped. The changes differ from home to home:
Loft insulation, fitted through the adviser's referral for a grant: "The loft. No question. The lass who came, she looked at the house and said I was a good candidate for the grant-funded insulation, so I didn't pay for it." ()
Draught strips on both doors, checked that morning: "She put those strips along the bottom of the front door and the back one, and they're still on." () The adviser also made this household's Warm Home Discount application, and the money went on the meter in January.
A tariff switch the adviser made during the visit: "Got me onto a fixed deal with a different supplier. He did the whole lot, I just had to say yes and give him the meter readings." ()
Reflector panels, with the heating timer still used as the adviser showed: "They're still there, nobody's touched them." ()
The tariff switch comes from a householder who is otherwise dismissive of the visit, calling it "a waste of an afternoon" (). Asked whether they would have switched without the adviser, they answered "No. Probably not." (), and they are still on the deal.
Warmer homes and lower bills
All households describe a warmer home or lower energy costs and put it down to the visit:
"It's the same boiler, same me, same house, and the only thing that's different is the loft." ()
"That's why I think it's down to her visit, because nothing else was different." ()
"If somebody asked me, I'd say it's a bit of a faff. But I'm not paying the twenty quid, am I?" ()
"But I haven't changed anything else, and the housing association hasn't done any work here. So I put it down to what she did." ()
These accounts qualify the finding:
describes no warmer home: "It's the same flat. Same radiators, same window that doesn't shut properly." (). The lower cost is a fixed deal about twenty pounds a month cheaper, a saving on price rather than on energy used.
In the link between the loft and the lower direct debit is the householder's own inference: "They didn't say why, but I put two and two together." ()
links the longer-lasting top-ups to the visit but adds "I can't prove it." ()
Limitations
The verdict rests on what the householders interviewed said about their own homes. It does not measure energy use, and it covers only the households interviewed, not every household visited. No householder's link between the visit and a warmer home or lower bills was checked against bills or meter readings. The interviews say nothing about what the scheme costs, so they cannot show whether the savings are worth that cost.
Annexes
How this report was written
What was read. Rubicon read all 4 documents in full. The counts below are of households, one document each.
The workflow x-ray beside Step by step, below, draws the same procedure, a box for each stage and what it made.
Step by step
The analysis was written down as a workflow before it was counted: what to look for, how to code it, what to count. It was then counted directly from the coded passages rather than by the AI, so anyone can check a number, and another coder can follow the same definitions and code the interviews afresh.
What was read
All 4 documents, read in full: 1 older person alone, 2 family with children, 1 working-age adult.
Coding: Lasting change
one row for each document, read whole; 4 rows.
The instruction a second coder would follow
This is an interview with a householder about six months after a Warm Homes Visit, in which an adviser checked their energy tariff and helped them switch or apply for the Warm Home Discount, fitted draught strips and radiator reflector panels, advised on heating controls, and referred them for grant-funded loft or cavity insulation. Read the whole interview and give one row for the household. Quote the passage that best shows whether a change the visit brought about was still in place or in use at the interview; where there is none, quote the passage that shows that. Code each column exactly as defined.
Asked about changes
Whether the interviewer asked the householder, in any words, what has changed since the visit or whether anything came of it. yes: such a question is asked anywhere in the interview. no: the interview never asks about changes since the visit, even if the householder mentions some unprompted.
Lasting change
Whether the householder describes at least one change the visit brought about that was still in place or still in use at the interview. yes: at least one of these, described as brought about by the visit (done by the adviser, or arranged, applied for or learned at the visit) and still in place or in use when interviewed: draught strips or radiator reflector panels still fitted; still on a tariff they switched to through the visit; heating controls (timer, thermostat, radiator valves) still used as the adviser showed; loft or cavity insulation installed through the visit's referral; or a Warm Home Discount received after the adviser helped them apply (this counts although it is paid once). no: no change brought about by the visit is described; or every such change has since stopped (strips or panels removed or fallen off and not replaced, back on the old tariff, controls set back as before, a referral not followed up); or the householder puts the change down to something other than the visit, such as the landlord or housing association, or says they did it before or without the visit. Advice the householder says they already followed before the visit is not a change. Tone does not decide it: a householder who is critical of the visit but describes such a change still in place is yes; one who praises the visit but names nothing still in place is no.
The coded passages 4 rows
The loft. No question. The lass who came, she looked at the house and said I was a good candidate for the grant-funded insulation, so I didn't pay for it.
She put those strips along the bottom of the front door and the back one, and they're still on.
Got me onto a fixed deal with a different supplier. He did the whole lot, I just had to say yes and give him the meter readings.
She put those silver reflector panels behind two radiators, one in their bedroom and one in the front room. They're still there, nobody's touched them.
Coding: Warmer linked
one row for each document, read whole; 4 rows.
The instruction a second coder would follow
This is an interview with a householder about six months after a Warm Homes Visit, in which an adviser checked their energy tariff and helped them switch or apply for the Warm Home Discount, fitted draught strips and radiator reflector panels, advised on heating controls, and referred them for grant-funded loft or cavity insulation. Read the whole interview and give one row for the household. Quote the passage that best shows whether the householder links a warmer home or lower bills to the visit; where they do not, quote the passage that shows that. Code the column exactly as defined.
Warmer linked
Whether the householder says their home has been warmer, or their bills, direct debit or prepayment top-ups lower or lasting longer, and links this to the visit or to something it brought about. yes: they describe a warmer home or lower energy costs in their own case, and put it down, at least in part, to the visit or to a change it brought about (a measure fitted, a tariff switch, a Warm Home Discount, insulation arranged, controls used as shown), in their own words or by agreeing when asked. no: they describe neither a warmer home nor lower costs (feeling reassured, supported or less alone is not a warmer home); or they describe one but put it down only to something else, such as a new boiler, the landlord or housing association, a mild winter, a fall in the price cap or other help; or they do not link it to the visit when the interview gives them the chance. Tone does not decide it: grudging words that still credit the visit with lower bills are yes; warm praise for the adviser with the house still cold and bills unchanged is no.
The coded passages 4 rows
It's the same boiler, same me, same house, and the only thing that's different is the loft.
That's why I think it's down to her visit, because nothing else was different.
If somebody asked me, I'd say it's a bit of a faff. But I'm not paying the twenty quid, am I?
But I haven't changed anything else, and the housing association hasn't done any work here. So I put it down to what she did.
Counting: Change
Households by lasting change
The table 2 cells
Lasting change
Households
Yes
of 4
No
of 4
Counting: Warmth
Households by warmer linked
The table 2 cells
Warmer linked
Households
Yes
of 4
No
of 4
Counting: Both
Households by asked about changes and lasting change and warmer linked
The table 4 cells
Asked about changes
Lasting change
Warmer linked
Households
Yes
Yes
Yes
of 4
No
No
Yes
of 4
No
Yes
Yes
of 4
Yes
No
Yes
of 4
Judging: Verdict
The evaluator's rubric, agreed on 7 October 2026 before any interview was read.
How many of the households interviewed describe at least one lasting change made because of the visit and, in the same household, a warmer home or lower bills they link to the visit? · level met: Good enough
Verdict: Good enough, since the rule for Good enough holds: both_in_same_household is Good enough.
The verdicts the rule could reach
Good enough
More than half of the households interviewed describe at least one lasting change made because of the visit and, in the same household, a warmer home or lower bills they link to the visit.
Not good enough
Half or fewer of the households interviewed describe both.
Writing: Answer
The instruction the answer was written to
The checks that were run
8 of 8 quotations found word for word in their document.
3 numbers in the answer, every one computed directly from the coded passages rather than written by the AI; 0 written by the AI without a count behind them.
0 citations to passages that do not exist; 0 quotations not in the passage they cite.
Second coding
The same documents were coded again to the same definitions; the run does not record whether a separate coder did it, so it is not called independent here. The second coder's rows were not kept in this run's zip.
The check
A reader ruled on every place the two codings differed, and checked every sentence and quotation of the answer against the documents, correcting the coding and the answer where they were wrong. The answer above is the corrected one, recounted after the check.
The check's record check.md
Counts below are of disagreements in recode/disagreements.md, rows in coded/*.json, and sentences in answer.md.
Disagreements ruled for the rows: 0
Disagreements ruled for the second coder: 0
Disagreements left unsettled: 0
Rows changed: 1 (quotation only; no value changed)
Definitions changed: 0
Sentences changed: 14 (9 rewritten, 5 added), plus the closing "The verdict against the rubric" section cut
Rulings on the coding
recode/disagreements.md: "The two coders placed every document and value alike." There was nothing to rule.
recode/unclear.md, warmer_linked, SYNTHETIC-A-03 ("Same radiators, same window that doesn't shut properly"; a fixed deal "about twenty quid a month less"): the definition settles it. It counts "bills, direct debit or prepayment top-ups lower" and names "a tariff switch" among the changes a lower cost can be put down to, so yes stands. The answer now says the saving is on price, not warmth or energy used.
recode/unclear.md, warmer_linked, SYNTHETIC-A-01 ("They didn't say why, but I put two and two together"): the definition settles it, since it accepts the householder's link "in their own words". Yes stands; the answer now names the inference.
Row changes
warmer_linked, SYNTHETIC-A-03: quotation changed from "It's about twenty quid a month less than I was paying." to "If somebody asked me, I'd say it's a bit of a faff. But I'm not paying the twenty quid, am I?" The old passage states the saving but not its link to the visit, which the prompt asks the quotation to show; the new one is the householder's answer to "Would you recommend the visit?" and credits the visit with the saving. Value stays yes. Row id is now warmer_linked.yjzqw.
Answer changes
Opening: states the answer plainly ("Yes ... good enough to fund for another year"); adds that the verdict survives any one household being read the other way (the rule's threshold is more than half); moves in the "every interview asked" point from the cut verdict section; names SYNTHETIC-A-03 as the weakest case. Cut "No household interviewed falls short on either criterion", which repeated the count.
Opening: the old verdict section cited {t_both.c1.within.asked_about_changes} as the number of interviews that asked about changes; that cell counts households meeting all three conditions within those asked and warmer, not interviews asked. Replaced with the plain statement, which each document's first question bears out ("What's changed since then?" or similar).
Changes that lasted: added "No interview describes a change that has since stopped" (no document describes one).
SYNTHETIC-A-02 bullet: "after the adviser made the application with her" changed to "made this household's Warm Home Discount application, and the money went on the meter in January"; the householder's sex is not stated, and the passage is "She sat with me and did the application on her phone ... The money went on the meter in January."
SYNTHETIC-A-03 paragraph: "he says he would not have switched without the adviser" was overstated; the householder said "No. Probably not." Now quoted exactly with the id. "he" removed (sex not stated). "a waste of an afternoon" given its id. Cut "His account counts on what it describes, not on its tone" (method narration).
Warmer homes: cut "Each rules out other causes in their own words", which the SYNTHETIC-A-03 quotation did not do. Replaced that bullet's quotation with the new row's.
Warmer homes: added the accounts that go against or weaken the finding: SYNTHETIC-A-03 ("It's the same flat. Same radiators, same window that doesn't shut properly."), SYNTHETIC-A-01 ("They didn't say why, but I put two and two together."), SYNTHETIC-A-04 ("I can't prove it.").
Cut the section "The verdict against the rubric": it repeated the opening.
Limits: added that the interviews say nothing about the scheme's cost, which bears on "justify funding".
Checked and unchanged: the four lasting-change quotations and their rows, the SYNTHETIC-A-01, -02 and -04 warmth quotations, "Draught strips on both doors, checked that morning" ("I checked this morning"), "Reflector panels, with the heating timer still used" ("I still use it"), and the existing limits.
No group comparison and no count of others' cases appears in the report, so those rules needed no change. recount.py reports no faults.
The record
This report was drawn directly from the run's zip, did-the-warm-homes-visits-help-households-keep-warm-2026-10-10-revision-1-d2b3ed6a.zip. The zip's fingerprint is d2b3ed6abc1905ac0fd40178aa1bc9f685288e606999209af48e25e2686ba0ab. It is the SHA-256 of the zip's SHA256SUMS file, which lists the SHA-256 of every other file in the zip, so changing any file breaks the match. To check that the zip is the one this report was drawn from, unzip it, run sha256sum -c SHA256SUMS, then sha256sum SHA256SUMS, and compare the result with the fingerprint above. Rubicon's own code, which is public, checks the rest: python verify.py <report> <zip> recounts every number from the coded passages in the zip, draws the report again from the zip and says whether it matches this one exactly.
Revisions
Changes made after the first report, each recounted and checked before this version was drawn. Earlier versions keep their own files.
10 October 2026: Each table now says what it counts: every count names its unit, households, beside the number it is out of. Workflow: every tabulate step names its unit. Coding and answer unchanged. Recounted the whole workflow; no count moved. The recount also states how near each verdict is to moving (its margin), which the engine that made the earlier recount did not.
Every interview at a glance
Each row is one interview, each column a reason as the coding defines it. Click a mark to read the passages.
🔥 Carbon cost of this report: Between 0.00062 and 0.098 km driven in an average UK family car
Making this report used between 0.8 and 132 watt-hours of electricity, which is the carbon of driving an average UK family car for that distance. A ballpark only: Anthropic publishes no energy figures, and the run, made in the evaluator's own Claude, records no token counts, so the estimate is made from the run's size. The 4 documents were read in full once by each of the 2 coding steps, by the second coder and by the check, and the writer read the tables; everything the run wrote counts as output. That comes to about 15,544 tokens read and 2,678 written, at 4 characters a token. It covers the first run and its revision. The zip holds only the latest state, so each revision is counted at the low end as rereading the tables and writing the answer again, and at the high end as a whole run again.
High: 0.39 Wh per thousand tokens read and 1.95 per thousand written, with what was read counted 10 times over for the agent's own overhead. Simon P. Couch, 'Electricity use of AI coding agents', 20 January 2026, https://simonpcouch.com/blog/2026-01-20-cc-impact/: about 390 Wh per million input tokens and 1,950 per million output tokens for Claude Opus, back-solved from Epoch AI's estimate for a 100,000-token prompt with output at five times the input rate, as Anthropic's prices are. The author calls them napkin maths. Our assumption, not a measurement: an agent re-sends its instructions, earlier turns and tool output with every request, so it reads far more than the documents themselves. The low case counts only what was read and written; the high case counts what was read ten times over.
Low: the same rates divided by 20, what was read counted once. Oviedo et al., 'Energy use of AI inference, efficiency pathways, and test-time scaling', arXiv:2509.20241: a median 0.31 Wh per query for frontier models, and widely cited estimates overstated by 4 to 20 times. The low case takes the high case's rates divided by the top of that range.
0.124 kg CO2e per kWh. DESNZ greenhouse gas reporting conversion factors, UK grid electricity, 2024.
0.16601 kg CO2e per km. DESNZ greenhouse gas reporting conversion factors, average car of unknown fuel, 2024.