Checkable AI for evaluators

Example report: people inside focus groups

An evaluator used the Rubicon plugin for Claude to answer one evaluation question from a set of transcripts. Click any number, source ID or quotation in the report below to see the passages behind it. Each transcript records a focus group of several researchers, so this report counts the researchers, not the transcripts, and one table counts the groups.

The question
How do the researchers in these four focus groups see luck in their own work?
The transcripts
Four anonymised focus groups with 17 researchers at Dutch universities, from a published dataset by Martin Sand and Karin Jongsma (Focus Group Interviews "Luck in Science", 4TU.ResearchData 2020, CC BY 4.0).

The original report file, byte for byte (this page is the same report, with its scripts moved to a file for the site) · Word copy · About the Rubicon plugin for Claude

Rubicon report

How researchers see luck in their own work

Question How do the researchers in these four focus groups see luck in their own work?

Framework analysis of the four transcripts, each participant (by speaker label, moderators and project staff left out) being one case, 17 in all. (1) What participants say about luck in their own research or career, grouped into kinds found in the material, and whether they set luck beside their own work, preparation or choices. (2) How many participants credit luck with a part in an outcome of their own work, keeping research outcomes (whether experiments, methods, projects and papers produced results) apart from career outcomes (posts, places, degrees, awards, grants, the people they came to work with); general remarks about science and stories about other people do not count. (3) In how many of the four groups any participant argues that prizes, authorship, recognition, degrees, hiring or funding should be given differently because luck plays a part, with the opposite view (that credit should go to whoever got the result, luck or not) reported beside it. Bare assent to someone else's suggestion is weak. No rubric: the question asks for no verdict of worth.

Most of these researchers credit luck with a part in outcomes of their own. Of the participants, said whether luck played a part in some outcome of their own, and said it did, for good or ill. They credit it about as often for their careers () as for their research results (). Few deny it plainly for an outcome of their own (), and who credit luck say in the same passage that their own work, drive or choices also counted. In every group () one participant argues that because results depend partly on luck, some form of credit (a prize, funding, a degree or a grant) should be given differently, and in every group () someone argues that credit should go to whoever got the result, luck or not; in every group but physics, where the two views are held by one participant each, more participants take the second view.

4 interviews in 4 groups · 80 passages coded · 12 October 2026 · Run 2026-10-12-a493

  • Verification in this report:
  • 38 numbers, every one computed directly from the coded passages rather than written by the AI. Click any number to see who it counts.
  • 80 quotations, every one found word for word in its interview. Click an interview's ID beside a quotation to read it in place.
  • 47 findings and 5 high-level findings, every one checked against the interviews after they were coded a second time.

Where they see luck in their own work

The luck these researchers see in their own work takes four main forms, and a fifth appears once. Each count below is of participants who credit luck in that form, out of the who spoke about luck in their own work.

  • People and timing: meeting the right people at the right time. participants. The right PhD supervisor (), a visiting speaker's talk and a master's student who happened to choose the project (), a professor who supported a run of grant proposals (), a career path "very accidental. Luckily." ().
  • Doing the work: whether an experiment, method or material works. participants. A master's student's experiment that worked "in the last month" (), a thesis method that "worked out well" and opened the way to a PhD (), and, in the neuroscience group, a run of bad luck: a mouse line that "stopped breeding" (), "a genetic event that I had that I had no control over" (), equipment that arrived too late ().
  • Being chosen: applications, awards, exams and grants. participants. An award introduced in the speaker's year (), a "charmed life" of a scholarship, a PhD mentor and a post (), a grant reviewers rated highly that "was still killed by the committee" ().
  • The unexpected: a finding or idea nobody planned. participants. A protein that "does something completely different" (), a solution that came "during a shower" (), and a result opposite to the literature that its finder counted as "bad luck to us" ().
  • Background. participants: a neuroscientist whose parents helped out while they worked for free and who said "you're lucky to be born to people who made good decisions" ().

Luck here is not only good fortune. In the neuroscience group much of it is bad luck: half of a paper lost when a mouse line died out (), a few months lost to a faulty mouse line (), equipment that came too late for a thesis committee's deadline, which the speaker gave as one reason, though probably not the main one, for having to switch labs (), and a grant "killed by the committee" ().

Who denies it, and the place of effort

participants said plainly that an outcome of their own was not luck, and only one of them denied luck throughout: the aerospace engineer who said of their PhD topic "I did hard work. Its not luck." (), though the same speaker, telling of a simulation that gave a more relevant result than expected, asked whether that was luck and answered only "It just happened." (). The others deny it for one part of their work and credit it for another: one says "During work, I’m never lucky" but calls their career "very accidental" (); a biophysicist says their experiments need "relatively little luck" yet says "I felt lucky" when a protein surprised them (). In the neuroscience group, 2 participants hedge in the same way without denying luck outright: one says funding was "a little less more of luck" because they kept writing proposals, while calling the rest of their path lucky (); another, having said of their career "every step I was very lucky", adds that they could have taken any of the PhD places they applied for ().

participants who credit luck say in the same passage that their own work, drive or choices also counted, as in "it's not only luck, I had also been working hard" (), "if you drive something it doesn’t mean you get the outcome" () or "at the end I didn't give up and pushed it" (). The others describe luck in their own outcomes without saying, in that passage, what their own part was.

Differences between the groups

Every participant in the neuroscience and physics groups credits luck with a part in some outcome of their own; in the bionanoscience group all but one do so plainly, and that one only in passing; in the aerospace engineering group, whose transcript is only rough notes, fewer do: three of its seven participants, with nothing recorded about luck in their own work from three others. Whether the groups really differ is unclear with four groups of a few people each: the bionanoscience group would match the neuroscience and physics groups if its one passing remark were read as plain, and the aerospace notes may simply have left accounts out.

Credits luck for this reason Raised, not tied to that
Aerospace engineering (aerodynamics)3 +12
Neuroscience40 +2
Theoretical physics and biophysics31
Bionanoscience2 +10
Participants in each group. Paler, and + beside a count: those that say it only implicitly, which the count leaves out. Click a bar for the passages behind it.

In the neuroscience group all participants credit luck, in the physics and biophysics group , in the bionanoscience group (with one more only in passing ()), and in the aerospace engineering group who spoke of it; the other participants in that group said nothing about luck in their own work in the notes. Crediting luck does not exclude denying it elsewhere: in the physics group one participant who credits luck also denies it for their experiments (), and in the neuroscience group two hedge it for one outcome (). The aerospace group holds the only participant who denies luck throughout ().

Should luck change how credit is given?

In all groups a participant argues that because results depend partly on luck, credit should be given differently, and in each group one participant argues it firmly:

  • Aerospace engineering: "Luck doesn't strike twice" (), so a lucky finding may deserve a prize but not funding, since "If you don't know how it works, the chances of reproducing it are low" and "some sort of funding for similar things, that is not the right approach" ().
  • Bionanoscience: a master's degree or PhD should not rest "only basing on a great outcome", which "would be super-unfair because we said this might be lucky or not" (). The same participant held that a prize for the discoverer is still right (). Another calls glorifying individuals "unfair because two equally hardworking people and one is lucky and one is not", without saying prizes should be given differently ().
  • Physics and biophysics: "let's abolish prizes", because "the funding is not allocated well" (), and, set against a race where the fastest wins, "The problem is because you don’t know what’s there to discover" ().
  • Neuroscience: "because there are so many good proposals", "skip the whole reviewing and just completely based on luck" (), an idea another participant liked ().

Against this, in all groups someone argues that the one who got the result should get the credit: "the one who finds it should get the prize" (); "It doesn’t matter if it's fair" (); "the person should get the prize, the one who did the discovery" (); "Most prizes are awarded based on the impact of the discovery rather than the effort" (). In the aerospace engineering, neuroscience and bionanoscience groups more participants hold this view than argue for a change: against in aerospace engineering, against in neuroscience, and against in bionanoscience, where the one who argues for a change over degrees is also one of those who keep prizes for the discoverer; in neuroscience, too, two of those who keep it lean towards a change, one only by assent () and one by proposing grants given to people rather than projects, without tying this to luck (). In the physics group the two views are held by one participant each ( and ).

Limitations

  • The aerospace engineering transcript is rough notes, not a full record, and several of its participants say nothing about luck in their own work in it; what they said may have been left out, so that group's counts are likely low.
  • Every group was asked directly about its members' own experience of luck, and most views on credit answer the moderator's questions about prizes and the scenario of two equally talented scientists; these are mostly answers to prompts, though the neuroscience group's proposal to fund grants by lot was raised before any question about credit.
  • The groups are small and come from four disciplines, one group each, so a difference between groups is also a difference between a handful of people and between moderators' questions; it says nothing about the disciplines at large.
  • The finding that a participant in every group argues for a change rests in two groups on one reading each. In bionanoscience, the degree passage () is read as arguing that a degree should not rest on an outcome that may be luck; read instead as calling a way of giving credit unfair without saying it should change, it would be weak, and the bionanoscience group would show the view only in passing (). In physics, the call to abolish prizes is read as tied to luck through "you don’t know what’s there to discover" (); read as resting only on misallocated money, it would be weak too (). Either reading the other way would leave the change view argued firmly in fewer than all the groups (); both together, in only half of them.
  • That every participant in the bionanoscience group but one credits luck, rather than all, rests on reading the remark that "when it actually works so smoothly" it is "very lucky" () as a remark about that kind of work in general rather than about an outcome of the speaker's own (); read the other way, the bionanoscience group would match the neuroscience and physics groups.
  • The share who set their own effort beside luck () includes two placements a reader could take either way: the aerospace engineer whose thesis method worked, read as setting their own work beside luck ("If you put more work in the chances are higher") (), and the neuroscientist whose funding came through after they "didn't give up" (); read the other way, those who set effort beside luck would be a minority rather than about half of those who credit luck.
  • That one aerospace engineer denies luck throughout rests on reading their account of a simulation's surprising result, which they asked whether to call luck and answered with "It just happened." (), as a question left open rather than a plain credit of luck (1); read the other way, no participant would deny luck throughout.

Annexes

How this report was written

What was read. Rubicon read all 4 documents in full.

Coding. It then marked every passage that bears on the question and recorded yes for every row: the row records that this speaker took part as a participant and the participant's speaker label exactly as the transcript gives it, such as Sp3 or SP3. It also marked every passage that bears on the question and recorded what kind of outcome of the speaker's own the passage is about, where the speaker locates the luck in the outcome and the speaker's label exactly as the transcript gives it, such as Sp3 or SP3, with two yes-or-no codes besides, each defined in the codebook. It also marked every passage that bears on the question and recorded whether the speaker argues that luck should change how credit is given and the speaker's label exactly as the transcript gives it, such as Sp3 or SP3. A passage that only implies a code, in the way the code's definition names, was marked implicit and counted apart from the passages that say it explicitly.

Second coding and check. The same documents were coded again to the same definitions; the run does not record whether a separate coder did it, so it is not called independent here. The two codings differed in 6 places. A reader ruled on each against the text. It searched the documents each count leaves out for passages both coders had missed, and added them. It also checked every sentence and quotation of the answer against the documents, so the coding and the answer in this report are the corrected ones, counted again after the check.

Counting. The coded passages were then counted in 9 tables, compared by these codings. The annex below gives each.

Writing and checks. Rubicon wrote the answer from these counts, citing for each finding the passages it rests on. Every number in the answer, 38 in all, was computed directly from the coded passages rather than written by the AI, and all 80 quotations were found word for word in their documents.

Revisions. The report was revised once after it was first drawn, and counted and checked again; the changes are listed at the end of this annex.

The workflow x-ray beside Step by step, below, draws the same procedure, a box for each stage and what it made.

Step by step

The analysis was written down as a workflow before it was counted: what to look for, how to code it, what to count. It was then counted directly from the coded passages rather than by the AI, so anyone can check a number, and another coder can follow the same definitions and code the interviews afresh.

What was read

All 4 documents, read in full: 1 2019-04-12, 1 2019-07-04, 1 2019-08-09, 1 2019-10-31.

Coding: Participants

one row for each passage that fits; 17 rows.

The instruction a second coder would follow
List every participant in the focus group: one row per participant, quoting the line in which they introduce themselves (their name, discipline or research). Leave out the moderator and any member of the research project team.
Case

The participant's speaker label exactly as the transcript gives it, such as Sp3 or SP3.

Participant

yes for every row: the row records that this speaker took part as a participant.

Implicit: Never weak.

The coded passages 17 rows
  • Sp1: [name], Phd. Computational aerodynamics.
  • Sp2: [name], BA/MA, PhD aerospace engineering
  • Sp3: [name], Germany, experimental side of things
  • Sp4: [name], Belgium, aerospace engineering
  • Sp5: [name], China, aerdodynamics
  • Sp6: [name] China, simulation in aerodynamics
  • Sp7: [name], India, experimental aerodynamics
  • My name is [name] I just graduated from a PhD two months ago
  • My name is [name]. I am from Denmark.
  • I am [name], a second-year PhD student.
  • I am a theoretical physicist currently working in
  • I am a final year PhD student in the [name department].
  • I am a last year postdoc.
  • My discipline is neurobiology
  • I am a neuroscientist by training
  • I am interested how the retina works
  • I am [name], a post-doc in neuroscience

Coding: Own luck

one row for each passage that fits; 39 rows.

The instruction a second coder would follow
Find every passage in which a participant says whether luck (good or bad, chance, fortune, accident, being in the right place at the right time) played a part in an outcome of their own research or career. Code only what the speaker says about their own work or career, including their own lab or team where they speak of 'we'. Leave out general remarks about science, hypothetical people, and stories about colleagues or famous scientists. Leave out the moderator. Give one row for each outcome a participant speaks of; several rows from one participant are expected.
Case

The speaker's label exactly as the transcript gives it, such as Sp3 or SP3. Where the transcript's label is plainly the moderator speaking, leave the passage out.

Credits luck

yes where the speaker says luck (good or bad) played a part in an outcome of their own; no where the speaker says an outcome of their own was not luck, or that they are never lucky in their work.

Implicit: Weak where the passage only hints at its value: a hedged 'maybe you can call that luck' or 'I don't know if that's lucky', a hypothetical about themselves, or a remark about the kind of work they do in general rather than an outcome they had.

Area

What kind of outcome of the speaker's own the passage is about.

Implicit: Weak where the outcome could be read as either research or career.

CodeMeansCountsDoes not count
ResearchWhether their experiments, methods, calculations, projects or papers produced results.an experiment or method working or failing; an animal line dying and losing data; an unexpected finding; a paper accepted or rejectedthe grant that funds the work, or the post or supervisor they got (career)
CareerTheir path through science: places on courses, PhD and other posts, degrees, awards, exams, grants and funding, and the supervisors and colleagues they came to work with.getting a PhD place; an award; a grant rejected; meeting the right supervisor; being in science at allwhether the work itself produced results (research)
Kind

Where the speaker locates the luck (or its absence) in the outcome.

Implicit: Weak where the passage could fit two kinds equally.

CodeMeansCountsDoes not count
People and timingMeeting the right or wrong people, or being in the right place at the right time.a supervisor met by chance; a talk that sparked an experiment; a student who chose their project; being where a position happened to bea formal decision by a panel or committee (selection)
SelectionDecisions others make about them: applications, awards, scholarships, exams, grant reviews.an award introduced in their year; one application out of many succeeding; a grant killed by a committee; an exam failinga paper's acceptance, which is the work's result (work_outcome)
Work outcomeWhether an experiment, method, material, piece of equipment, project or paper works or fails for reasons outside their control.a method that happened to work; a mouse line dying out; equipment arriving late; a paper accepted this timea result that surprised them and pointed somewhere new (unexpected)
UnexpectedA finding, result or idea they did not expect, which changed or opened a direction.a protein doing something else; a result opposite to the literature; an idea in the showera planned experiment simply working or failing (work_outcome)
BackgroundCircumstances they were born or brought up into, such as family money or education, that opened or closed their path.parents who could support them through unpaid work; an educational background that gave them more opportunitiesa person met along the way (people_and_timing)
Effort too

yes where the same passage also says their own work, preparation, drive or choices contributed to the outcome, or decided it; no where it does not.

Implicit: Weak where the passage only implies effort, without saying it.

The coded passages 39 rows
Research 21 passages
  • So, I chose B, it was nice and it worked. And I could say: how lucky because I chose B. No, this topic was offered to me, I did hard work. Its not luck.
  • If you put more work in the chances are higher. In case of my ma thesis, testing a new measurement method. I was lucky it worked out well, and therefore I could continue to a PhD.
  • Then I think I am so lucky I figured this out during a shower. I sometimes think I am lucky.
  • So, I checked everything and checked for errors in the code. During work, I’m never lucky.
  • After a few weeks we found a result that was way more relevant than what I was working on. Is it luck? Well, you go different ways, you choose this path or another. It just happened.
  • When I think back of my master’s thesis and stumbled into this project, I had no idea of it and I completely relied on my professor at that time, so in that sense it was lucky that in the end something worked out.
  • The non-lucky part comes in, for example, in my master’s thesis after half a year, I decided this isn’t going to work and I now shift the project and then the rest of the half year something worked out. But also ways these decisions going into some direction it's quite luck based, at least for me.
  • when it actually works so smoothly the way you designed it and the way you thought about it – that’s luck. That’s actually very lucky.
  • then you're like: Okay, well, I guess I got lucky it got accepted this time in this journal.
  • But this depends most of the time on execution and being sharp in the end. So, I would say there's relatively little luck.
  • For me, I felt lucky when I realised one of the proteins I worked with does something completely different in addition to what it does for me.
  • For my case, I would say the luck happened not actually on the scientific side but by connection with the people in this specific case.
  • Of course, unexpected things pop up and, sure, luck in this respect, yes, otherwise we wouldn’t be here.
  • I guess it's lucky I went to that talk and then thought about I can do that experiment. Then it was lucky I talked to a master’s student who was looking for a project
  • Then, of course, you're also lucky that the master student works for nine months and then in the last month suddenly the experiment works.
  • For me, there’s a little bit of drive behind it. The thing is, if you drive something it doesn’t mean you get the outcome.
  • I already told you about this bad luck for my PhD, where I lost something that probably would have turned into a
  • there I was definitely fortunate I never had any really serious issues. Some of them you can mitigate by doing a lot of preparation getting plans in surgery but some of that is like rolling a dice.
  • I think everybody at the table will agree this is bad luck because this is purely a genetic event that I had that I had no control over.
  • But it was bad luck to us, because we couldn’t continue what we wanted to do because this little problem was not something that we expected.
  • Maybe you can call that luck. It was very exciting, at least. It was completely unexpected on the one hand but on the other hand fully predicted.
Career 18 passages
  • sometimes it is also about being at the right time at the right place. For example, I wouldn't be here if I wasn't searching for a master thesis topic.
  • I personally have been very, I think most of my career path very accidental. Luckily.
  • In India the selection as from 80.000 students, I was lucky and the only one applying.
  • There was somebody better than me there, so I was not the best candidate. I was not unlucky, there was someone better than me.
  • I was luckily one of the ones who actually had done a lot different things, had a fairly good publication record and stuff so I get this award. But that’s also luck. But it's not only luck, I had also been working hard to do some stuff and write this.
  • There's luck in the sense meeting the right people. The right people teach you the right things, motivate you at the right time to do the right way. I experienced this with respect to a PhD supervisor.
  • Somehow, I'm lucky to still be around, right? In life, even that. To be in science, to get along with the people around me
  • Because you're lucky to be born to people who made good decisions and like them to have a high income.
  • But at a earlier stage of your career, at least for me, I didn't really have that sense was not very well developed, certainly not as well as it is now. I think then maybe luck may have played a big role.
  • I feel if I look back at my life it's been a charmed life because I've been lucky in every hospital, you know, branch from my gain a scholarship to my undergrad degree and then finding a very good PhD mentor to finding this position here.
  • I think in terms of funding, that has been a little less more of luck because I wrote about 10 or 11 grant proposals in the first couple of years of my postdoc
  • So, in that case, it's more that you go on until it works. But I was also fortunate that my professor was supporting me writing all these proposals.
  • I think I can say the same for my career as you said, so every step I was very lucky.
  • On the other hand, when I did my masters, I applied for three positions for PhD and there are five open positions and I could get all of them
  • It's for sure bad luck but on the other hand I could obtain some other scholarship to study in Europe.
  • but at the end it was still killed by the committee. So, that was bad luck.
  • So, that was bad luck. I had to struggle with funding for many years so that was pretty difficult but at the end I didn't give up and pushed it, a lot of suffering but in the end, it worked.
  • But that was very, very bad luck. We just couldn’t get the equipment I needed for my project in time to fulfil the mandate of my thesis committee.

Coding: Credit view

one row for each passage that fits; 24 rows.

The instruction a second coder would follow
Find every passage in which a participant says how prizes, authorship, recognition, degrees, hiring or funding should be given, in a discussion where luck bears on it (for example the moderator's question about two equally hard-working scientists, only one of whom makes a discovery). Leave out the moderator. Leave out passages about whether scientists may feel proud, which is not about how credit is given.
Case

The speaker's label exactly as the transcript gives it, such as Sp3 or SP3.

View

Whether the speaker argues that luck should change how credit is given.

Implicit: Weak where the speaker only assents to someone else's proposal ('I like that idea'), or calls the present way unfair because of luck without saying credit should be given differently, or argues for a change in a discussion of luck without tying the change to luck themselves.

CodeMeansCountsDoes not count
ChangeArgues that because results depend partly on luck, some form of credit should be given differently: on something other than the result, by lot, not at all, or not as funding.abolish prizes and spread the money; allocate grants by lottery since good proposals outnumber money; a prize is fine but funding for a lucky finding is not; a degree should not rest on an outcome that may be luckobjections to prizes on other grounds, such as science being a group effort or prizes depending on friendships, unless luck is also given as a reason
KeepArgues that credit should go to whoever got the result even though luck played a part.the one who finds it should get the prize; prizes reward impact, not effort; it is not fair but that is how it worksa bare description of what happens with no view on whether it should
The coded passages 24 rows
Change 13 passages
  • Luck doesn't strike twice.
  • There is a difference between prizes and grants. A grant means your continuing your work. If you don't know how it works, the chances of reproducing it are low.
  • I think a prize would be ok, but some sort of funding for similar things, that is not the right approach.
  • We have many prizes around and we all often glorify individuals which in that sense is unfair because two equally hardworking people and one is lucky and one is not.
  • for example, getting a master degree or PhD only basing on a great outcome, and this would be super-unfair because we said this might be lucky or not.
  • If I could I’d say let's abolish prizes.
  • The merit comes anyway and the funding is not allocated well so it's just a misallocation of money, in my view.
  • The problem is because you don’t know what’s there to discover.
  • But it's also random. It's very bad: I discovered something and you did not.
  • I once proposed because there are so many good proposals that we just skip the whole reviewing and just completely based on luck.
  • I like that idea.
  • So, I think it would be extremely important to grade scientists on a kind of risk-taking.
  • One way that HH and I tried to address this is by having grants allotted to people based on their believe that individuals are good enough so they have more leeway to experiment and more time before they are evaluated on their output
Keep 11 passages
  • I think if he wins prizes he has deserved it. He has discovered it by luck, but worked hard for finding it out and published about it.
  • It is not fair, but it happens. It’s life. We don't value effort, we value result. It is how it works. Its not fair but yes.
  • What should the other guy get a prize for? for being smart?
  • If there is no result, there is no result. If he doesn’t bring anything to the table. Then no.
  • Then still, yes the one who finds it should get the prize.
  • Yes, I think it would. It doesn’t matter if it's fair.
  • Yes, in a way you also acknowledge the person for doing the work but also what he did, so in that sense I think that makes sense.
  • I would say, yes, the person should get the prize, the one who did the discovery.
  • Most prizes are awarded based on the impact of the discovery rather than the effort that went into it.
  • But I agree with [name SP2], these prizes are judged by the impact and the impact sometimes is not completely proportional to the amount of effort you put in.
  • But imagine you put in an equal amount of effort, you didn't get results and your colleague does but then you still get a prize, what would that prize be for?

Counting: Base

Participants by participant

The table 2 cells
ParticipantParticipants
Yes of 17
No of 17

Counting: Credit

Participants by credits luck

The table 2 cells
Credits luckParticipants
Yes of 14 +2 implicit
No of 14 +2 implicit

Counting: Area

Participants by credits luck and area

The table 4 cells
Credits luckAreaParticipants
YesCareer of 14
YesResearch of 14 +4 implicit
NoResearch of 14
NoCareer of 14 +3 implicit

Counting: Kind

Participants by credits luck and kind

The table 10 cells
Credits luckKindParticipants
YesPeople and timing of 14 +1 implicit
YesSelection of 14
YesWork outcome of 14 +1 implicit
YesUnexpected of 14 +3 implicit
NoWork outcome of 14
NoUnexpected of 14
YesBackground of 14
NoBackground of 14
NoPeople and timing of 14
NoSelection of 14 +3 implicit

Counting: Effort

Participants by credits luck and effort too

The table 4 cells
Credits luckEffort tooParticipants
YesNo of 14 +1 implicit
YesYes of 14 +1 implicit
NoYes of 14 +1 implicit
NoNo of 14 +2 implicit

Counting: Credit group

Participants by discipline and credits luck

The table 8 cells
DisciplineCredits luckParticipants
NeuroscienceYes of 4
Aerospace engineering (aerodynamics)Yes of 4 +1 implicit
Theoretical physics and biophysicsYes of 3
Aerospace engineering (aerodynamics)No of 4
BionanoscienceYes of 3 +1 implicit
Theoretical physics and biophysicsNo of 3
BionanoscienceNo of 3
NeuroscienceNo of 4 +2 implicit

Counting: Base group

Participants by discipline

The table 4 cells
DisciplineParticipants
Aerospace engineering (aerodynamics) of 7
Neuroscience of 4
Bionanoscience of 3
Theoretical physics and biophysics of 3

Counting: View groups

Documents by view

The table 2 cells
ViewDocuments
Change of 4
Keep of 4

Counting: View people

Participants by discipline and view

The table 8 cells
DisciplineViewParticipants
Aerospace engineering (aerodynamics)Keep of 5
NeuroscienceKeep of 4
BionanoscienceKeep of 3
Aerospace engineering (aerodynamics)Change of 5
BionanoscienceChange of 3 +1 implicit
NeuroscienceChange of 4 +2 implicit
Theoretical physics and biophysicsChange of 3 +1 implicit
Theoretical physics and biophysicsKeep of 3

Writing: Answer

The instruction the answer was written to

The checks that were run

  • 80 of 80 quotations found word for word in their document.
  • 38 numbers in the answer, every one computed directly from the coded passages rather than written by the AI; 0 written by the AI without a count behind them.
  • 0 citations to passages that do not exist; 0 quotations not in the passage they cite.

Second coding

The same documents were coded again to the same definitions; the run does not record whether a separate coder did it, so it is not called independent here.

The second coder's rows
  • Credit view: 21 rows
  • Own luck: 56 rows
  • Participants: 17 rows

In the run's zip, under check/coded/.

The check

A reader ruled on every place the two codings differed, and checked every sentence and quotation of the answer against the documents, correcting the coding and the answer where they were wrong. The answer above is the corrected one, recounted after the check.

The check's record check.md
Counts
  • Disagreements ruled: 6 in all; 4 for the rows (one of them with a row also made weak), 2 for the second coder, 0 unsettled.
  • Rows changed: 11 (own_luck: 2 made weak, 5 added; credit_view: 1 made firm, 3 added). Workflow: one value ("background") added to own_luck.kind; only FG4 holds a passage it fits, and the other three transcripts were searched for it.
  • Sentences changed in answer.md: 31 (listed below), including 4 Limitations lines added or replaced for narrow judgements.
Rulings on the disagreements

1. own_luck.credits_luck = no, FG2 (second coder only). FG2 SP4: "The non-lucky part comes in ... I decided this isn’t going to work and I now shift the project", followed at once by "But also ways these decisions going into some direction it's quite luck based, at least for me." The speaker calls the decision the non-lucky part but credits luck for how the shift worked out; no outcome is said to be not luck. For the rows: no "no" row added. 2. own_luck.credits_luck = no, FG4 (firm for the first coder, weak for the second). FG4 SP3 says funding "has been a little less more of luck" because of writing many proposals, then "But I was also fortunate that my professor was supporting me". Less luck, hedged, is not a plain "not luck": the column's weakness (a hedged passage) applies. For the second coder: row own_luck.qbp26 made weak. The second coder's other passage (FG4 SP2, "red flags during the interview") opens "Yes, but" in answer to SP5 calling the clash bad luck and does not say the clash was not luck, so it was not added. 3. own_luck.kind = unexpected, FG3 (firm/weak). FG3 SP4, asked about their own work: "I felt lucky when I realised one of the proteins I worked with does something completely different": a plain own unexpected finding, firm. FG3 SP2, "unexpected things pop up and, sure, luck in this respect, yes, otherwise we wouldn’t be here", answers the question about surprises with a remark about the kind of work in general and theoretical physicists as a whole ("we wouldn’t be here"), which the credits_luck definition makes weak. For the rows (FG3 stays firm), with own_luck.cgesf made weak. 4. own_luck.effort_too = yes, FG1 (firm/weak). FG1 Sp1: "No, this topic was offered to me, I did hard work. Its not luck." says plainly that their own work decided the outcome. For the rows. (Sp3's thesis row also stays yes: "If you put more work in the chances are higher" frames their own case; see Narrow judgements.) 5. credit_view.view = change, FG2 (weak for the first coder, firm for the second). FG2 SP3, after the prize question: "getting a master degree or PhD only basing on a great outcome, and this would be super-unfair because we said this might be lucky or not." This is an argument that a degree should rest on something other than an outcome that may be luck, which the definition lists under "counts" ("a degree should not rest on an outcome that may be luck"); it is the speaker's own reason, not assent, and not a complaint about present practice. For the second coder: credit_view.hphan made firm. FG2 SP4's "unfair because two equally hardworking people and one is lucky" stays weak (calls it unfair without saying credit should change, and goes on "we would need to clarify first what is the prize for"). 6. credit_view.view = change, FG3 (firm for the first coder, weak for the second). FG3 SP4, answering the moderator's two-scientists question (the line labelled "SP2:" is the moderator's): "let's abolish prizes ... the funding is not allocated well". The luck tie comes a few turns later in SP4's own words, set against SP3's 100m race: "The problem is because you don’t know what’s there to discover", then "Exactly" to SP3's "It's even more random", then "I have another argument why prizes in science are really bad", which marks the earlier turn as an argument against prizes. For the rows, adding credit_view.72nwx so the tie is quoted. This is a reading a careful reader could take the other way (see Narrow judgements).

Omissions found
  • FG1 Sp1 (own_luck.muuvc, weak, research, unexpected, effort_too yes): "After a few weeks we found a result that was way more relevant than what I was working on. Is it luck? Well, you go different ways, you choose this path or another. It just happened." Own outcome, left as an open question, so weak. Qualifies the claim that Sp1 denies luck throughout.
  • FG4 SP3 (own_luck.q8hvk, career, people_and_timing, effort_too yes): "But I was also fortunate that my professor was supporting me writing all these proposals." Adds SP3 to people and timing (t_kind.c1).
  • FG4 SP4 (own_luck.qhn8m, no, weak): "when I did my masters, I applied for three positions for PhD ... I could get all of them", followed by "it's difficult to say whether it's luck". A hedged "not luck" for an outcome SP4 had called lucky.
  • FG4 SP5 (own_luck.p2dma, career, selection, effort_too yes): "So, that was bad luck. I had to struggle with funding for many years ... but at the end I didn't give up and pushed it". Sets own effort beside luck; contradicts "most ... without saying their own effort also decided" and "only" in the effort sentence (t_effort.c2 from 5 to 6 of 12).
  • FG4 SP2 (own_luck.p7e4t, career, background): "Because you're lucky to be born to people who made good decisions", answering how their own unpaid periods supported by their parents relate to luck. No kind fitted; "background" added to the workflow (the other transcripts hold no own-case passage it fits).
  • FG1 Sp2 (credit_view.jaz8m, change): "Luck doesn't strike twice.", the speaker's own tie to luck for the prize-but-not-funding view, which the report's quotes lacked.
  • FG4 SP3 (credit_view.eda66, change, weak): grants "allotted to people ... as opposed to grants getting allotted on the project basis", argued in the discussion of risk and failure without tying it to luck.
  • Searched and not added: FG2 SP3 "shifting toward something came more or less luck again" (may be SP4 mislabelled, and general); FG3 SP2 "they did not have the luck you had" (general "you"; SP2 already counted); FG3 SP4 "I was not lucky it was accepted" (within a general account of a submission strategy; SP4 already firm "no"); FG4 SP2 "I hope we get lucky" experiments and SP4 "is it luck? Is it risk-taking?" (both speakers already counted; would not change a cell); FG4 SP4 and SP2 against prizes as "a group effort" (other grounds, does_not_count).
Changes to answer.md

Opening 1. "Most ... do credit luck with a part in their own results": now "outcomes of their own" (career outcomes are not results). 2. "{t_credit.c1} of those said it did ... one more said so only in passing": weak mention dropped (does not change "most"); now two weak rows. 3. "Few deny it outright, and most who credit luck do so without saying their own effort also decided": wrong after FG4 SP5; now "{t_effort.c2.within.credits_luck} who credit luck say in the same passage". 4. "Arguing that luck should change ... is rarer, and always a minority view ... firmly in {c2} groups ... bionanoscience only in passing": after rulings 5 and 6 every group has a firm change view, and in physics it is not a minority (one each way); rewritten.

Where they see luck 5. "takes four forms": a fifth (background) added. 6. People and timing bullet: added FG4 SP3's professor [own_luck.q8hvk]. 7. "a 'charmed life' of scholarships and posts": passage names one scholarship, a PhD mentor and a post. 8. Background bullet added [own_luck.p7e4t]. 9. "lost data, lost months and a lost PhD project [7szpz] [4te7b] [rzyqp]": the equipment was "maybe not the main reason" for switching labs, and rzyqp is a grant, not a PhD project; corrected with each loss as told.

Who denies it 10. "{t_credit.c2} ... said an outcome was not luck, only one denied throughout": now "plainly", with Sp1's open question about a simulation [own_luck.muuvc] named as the account against it. 11. "calls his career": gender is not given in the transcripts; "their". 12. "a physicist says his experiments": FG3 SP4 works in biophysics; "a biophysicist ... their". 13. "a neuroscientist says grant funding came from writing ten or more proposals": the passage hedges ("a little less more of luck"), now weak; rewritten as a hedge. 14. Added FG4 SP4's hedge on PhD places [own_luck.qhn8m]. 15. "Only {t_effort...} ... in the same breath": "only" dropped; now 6 of 12; FG4 SP5 quote added. 16. "The rest describe luck as something that happened to them": overstated (they say nothing either way in that passage); corrected.

Differences between the groups 17. "The neuroscience and physics groups credited luck the most ... aerospace least": one weak row in bionanoscience and rough notes in aerospace decide it; now stated as unclear. 18. Added the sentence saying why it is unclear. 19. "and and" and "the rest of that group": typo fixed; "in the notes". 20. Added: crediting luck does not exclude denying it (FG3 SP4, FG4 SP3, SP4), the accounts against "every participant credits luck".

Should luck change how credit is given 21. "In {c2} groups ... each time one person": now all groups. 22. Aerospace bullet: speaker's own luck tie added [credit_view.jaz8m]. 23. Bionanoscience: "nobody argues for a change outright ... neither calls for credit to be given differently" was wrong for SP3 (ruling 5); rewritten as a bullet, noting SP3 also keeps prizes for the discoverer. 24. Physics bullet: luck tie added [credit_view.72nwx]. 25. Neuroscience: "since good proposals outnumber the money" was SP5's point; SP4's words "because there are so many good proposals" used. 26. "In three groups more participants hold this view ({c1}, {c2}, {c3})": cell ids reordered after recount; now each set against the change count, with SP3 in bionanoscience in both, and neuroscience's two weak leaners named since they would make it level. 27. Added: physics one each way.

Limitations 28. "three of its seven participants": no bare count; "several". 29. "these are answers to prompts, not views raised unprompted": FG4's lottery proposal came before any question about credit; FG1 was asked "Own experiences with luck?", not "come across luck"; corrected. 30. Old last line on the two placements: its claim that the unfairness reading decides whether bionanoscience is a fourth group no longer holds; replaced by the narrow-judgement lines (FG2/FG3 change, bionanoscience weak row, effort share). 31. Added the line on Sp1's open question.

Narrow judgements
  • A participant in every group argues for a change (t_view_groups.c1, 4 of 4). Rests on two readings: FG2 SP3's degree passage [credit_view.hphan] held firm, and FG3 SP4's abolish-prizes argument held tied to luck through [credit_view.72nwx]. Either could fairly go the other way (the first as a complaint of unfairness, the second as resting on misallocated money); one flipped gives three groups, both gives two. Line added.
  • Someone in every group argues credit should go to whoever got the result (t_view_groups.c2, 4 of 4). Each group has at least one plain statement (FG1 Sp4, FG2 SP3, FG3 SP2, FG4 SP2). Would stand; no line.
  • More participants keep than change in every group but physics (t_view_people.c1 to c8). In neuroscience it holds on firm rows only; the two weak leaners are named in the text. Holds on firm rows; no further line.
  • Most credit luck (t_credit.c1, 12 of 14). Many firm rows; one participant flipped leaves it standing. No line.
  • About as often for careers as research (t_area.c1, 9; t_area.c2, 8 plus 4 weak). Would stand. No line.
  • Effort set beside luck by half of those who credit it (t_effort.c2, 6 of 12). Rests on FG1 Sp3's thesis row (effort_too yes, from "If you put more work in the chances are higher") and the new FG4 SP5 row; both read the other way make it a minority. Line added.
  • Only one participant denies luck throughout (FG1 Sp1). Rests on [own_luck.muuvc] being weak; read as a plain credit of luck, nobody denies throughout. Line added.
  • Group differences (t_credit_group). Already called unclear in the text; the bionanoscience weak row [own_luck.ufmv9] would make that group match neuroscience and physics. Line kept, reworded.

The record

This is run 2026-10-12-a493. Its ID stays the same through every revision, and the zip's fingerprint, below, names this version of it. This report was drawn directly from the run's zip, how-do-the-researchers-in-these-four-focus-groups-2026-10-12-revision-1-c5787ea9.zip. The zip's fingerprint is c5787ea9d088771cb514431b864e7bea7d4e97dd5db078f02832edb043f0207c. It is the SHA-256 of the zip's SHA256SUMS file, which lists the SHA-256 of every other file in the zip, so changing any file breaks the match. To check that the zip is the one this report was drawn from, unzip it, run sha256sum -c SHA256SUMS, then sha256sum SHA256SUMS, and compare the result with the fingerprint above. Rubicon's own code, which is public, checks the rest: python verify.py <report> <zip> recounts every number from the coded passages in the zip, draws the report again from the zip and says whether it matches this one exactly.

Revisions

Changes made after the first report, each recounted and checked before this version was drawn. Earlier versions keep their own files.

  • 12 October 2026: The summary of the differences between the groups said that fewer participants credit luck in the aerospace engineering group, which read as contradicting the later count of three of the four who spoke of it, a higher share than in the bionanoscience group. The sentence now says fewer out of what: three of the group's seven participants, with nothing recorded about luck in their own work from three others. Answer only; the workflow, the coding and every count are as before. Recounted with `recount.py --answer`.

Every interview at a glance

Each row is one interview, each column a reason as the coding defines it. Click a mark to read the passages.

Only implicitly, so left out of the counts
ChangeKeep
2019-04-12
2019-07-04
2019-08-09
2019-10-31
🔥 Carbon cost of this report: Between 0.0058 and 1.3 km driven in an average UK family car

Making this report used between 7.8 and 1718 watt-hours of electricity, which is the carbon of driving an average UK family car for that distance. A ballpark only: Anthropic publishes no energy figures, and the run, made in the evaluator's own Claude, records no token counts, so the estimate is made from the run's size. The 4 documents were read in full once by each of the 3 coding steps, by the second coder and by the check, and the writer read the tables; everything the run wrote counts as output. That comes to about 205,213 tokens read and 30,207 written, at 4 characters a token. It covers the first run and its revision. The zip holds only the latest state, so each revision is counted at the low end as rereading the tables and writing the answer again, and at the high end as a whole run again.

  • High: 0.39 Wh per thousand tokens read and 1.95 per thousand written, with what was read counted 10 times over for the agent's own overhead. Simon P. Couch, 'Electricity use of AI coding agents', 20 January 2026, https://simonpcouch.com/blog/2026-01-20-cc-impact/: about 390 Wh per million input tokens and 1,950 per million output tokens for Claude Opus, back-solved from Epoch AI's estimate for a 100,000-token prompt with output at five times the input rate, as Anthropic's prices are. The author calls them napkin maths. Our assumption, not a measurement: an agent re-sends its instructions, earlier turns and tool output with every request, so it reads far more than the documents themselves. The low case counts only what was read and written; the high case counts what was read ten times over.
  • Low: the same rates divided by 20, what was read counted once. Oviedo et al., 'Energy use of AI inference, efficiency pathways, and test-time scaling', arXiv:2509.20241: a median 0.31 Wh per query for frontier models, and widely cited estimates overstated by 4 to 20 times. The low case takes the high case's rates divided by the top of that range.
  • 0.124 kg CO2e per kWh. DESNZ greenhouse gas reporting conversion factors, UK grid electricity, 2024.
  • 0.16601 kg CO2e per km. DESNZ greenhouse gas reporting conversion factors, average car of unknown fuel, 2024.

Made with Rubicon, free for Claude: rubicon.causalmap.app