Do not take our word for it.
Real reliability questions. Controlled experiments. Evidence you can inspect.
Every claim in this series has a run behind it.
Each episode takes one thing people say about software reliability, builds a fixture that can prove it wrong, and publishes what the run produced. The scope is stated, the limitations are stated, and an experiment that fails is published as a failure.
No experiment has published yet
Read how the experiments workHow an experiment works
Every episode follows the same four steps, in the same order, whatever the result turns out to be. The criteria are fixed before the run, so the experiment cannot be quietly rewritten once the numbers arrive.
- 01
The claim
One proposition about reliability, stated so it can be wrong.
- 02
The experiment
A controlled fixture, with the pass and fail criteria fixed before the run.
- 03
The evidence
The artifacts the run produced, published so you can check them.
- 04
The verdict
Supported, contradicted, or inconclusive, inside a stated scope.
The three results an experiment can have
- Supported within the tested scope
- The experiment behaved as the claim predicted, for the systems, versions and conditions stated in the scope. It is not a general guarantee.
- Contradicted by the experiment
- The run produced the outcome the claim said it would not. The claim does not hold under these conditions.
- Inconclusive
- The run did not establish the conditions the claim needs, so it neither supports nor contradicts it. The experiment is reported anyway.
An experiment that has not run yet has no result at all, so it is not listed here as one. Nothing on this site describes a planned experiment as a completed demonstration.
The experiments
No experiment has published yet. Episodes appear here once the run is complete, the evidence has been reviewed, and a verdict has been selected. Nothing is listed before that, including work that is already in production.
The argument behind the experiments
The Reliability Thesis sets out why verification, authority and evidence have to change as software becomes autonomous. PROVE IT takes one claim from that argument at a time and tries to break it in a lab. If the two ever disagree, the experiment wins.
Read The Reliability ThesisRequest this experiment against your workflow
We will run the same controlled demonstration on a system you choose, and publish the evidence to you rather than to the internet.
