ARIANE 501 / AN ASSUMPTION-CHALLENGING LAB
The 37-Second
Blind Spot.
A Million Green Tests,
One Missing Assumption.
A testable disaster. A misleading green dashboard. A lesson for the systems we build with AI.
Retrospective documentary analysis + synthetic mechanism experiment.
No flight telemetry. No claim of independent AI discovery.
One unchecked conversion.
Two identical failure paths.
The weak oracle sees a frame.
It does not see the loss of navigation.
01 / HYPOTHESIS & VERDICT
More tests cannot discover
a failure your test world excludes.
The disaster was testable and preventable.
Monte Carlo would not automatically have found it.
The hypothesis is that representative simulation and AI-assisted test design could expose an Ariane 501-like failure before deployment. The inquiry establishes feasible detectability; our toy experiment tests the conditions needed, not an unconditional AI-discovery guarantee.
Read the hypothesis assessment02 / FIRST, GET THE CLOCK RIGHT
37 seconds after ignition.
About 30 after lift-off.
- H0
Ignition command
The inquiry's reference clock. Not lift-off.
- H0 + 36.7 s
Backup SRI fails
The alignment conversion exceeds its range.
- About 0.05 s later
Active SRI follows
Identical software, the same systematic error.
- About H0 + 39 s
Launcher disintegrates
Erroneous steering causes destructive loads.
Timing from the 1996 Inquiry Board report. Its 72 ms data cycle and approximately 0.05 s failure separation are different statements. Read the chronology and source.
03 / SAME INPUTS. DIFFERENT TEST WORLDS.
Three ways to get a green result.
Only one changes the design.
The expanded-input experiments share identical random inputs. Replacing the implementation, weakening the oracle, and removing alignment can all produce zero detections. They do not mean the same thing.
| Test world | Opportunities | Navigation losses | Detected |
|---|---|---|---|
| Legacy-like supportThe input distribution excludes the failure region. | 1,000,000 | 0 | 0 |
| Expanded supportThe mechanism executes; the safety oracle sees the loss. | 100,000 | 25,005 | 25,005 |
| Rare-tail supportThe trigger exists, but the test budget matters. | 200,000 | 96 | 96 |
| Idealized SRI stubA nominal-output harness removes the failing path. | 100,000 | 0 | 0 |
| Packet-presence oracleDiagnostics count as packets. A green result hides navigation loss. | 100,000 | 25,005 | 0 |
| Post-launch alignment removedAn explicit intervention blocks this modeled chain. | 100,000 | 0 | 0 |
Modeled navigation losses hidden by the packet-presence oracle.
A million green tests are not a safety argument without a meaningful contract.
A trial is one independent conversion opportunity, not a launch. Rates are not Ariane failure probabilities. Distributions, seeds, confidence intervals, and limitations / Raw results (JSON)
04 / FROM POSSIBLE FAILURE TO ACTIONABLE FINDING
What would have to be true
for this failure to be uncovered?
An input must reach real failing behavior, the evaluation must recognize it, and the evidence must survive into a decision. These are separate, testable obligations.
Expose the dangerous input
Show that the triggering region exists in the test support. Start with a deterministic boundary witness.
Reach the vulnerable lifecycle
Run the input while the function is active. Include state, ordering, retries, and transitions.
Preserve executable semantics
Exercise the conversion and exception behavior. A nominal-output mock can erase the defect.
Observe the right contract
Distinguish navigation from diagnostics, or real task completion from a success-shaped response.
Defend the rare-event budget
Choose a probability target and justify the sampling model. More tests cannot fix zero detection probability.
Retain evidence and enforce a response
Save a replayable falsifier, name a decision owner, and test that the release gate actually fails.
Remove one prerequisite. Keep the others.
CONTROLLED SYNTHETIC EXPERIMENTThe complete experiment crosses 128 configurations of seven process switches; retention and response are separate. These nine controls isolate missing prerequisites and include a guarded negative control. No additional Monte Carlo draws.
| Control | Actual navigation loss | Injected detection | Random budget | Campaign detection chance | 95% target met | Replay retained | Modeled block |
|---|---|---|---|---|---|---|---|
| All prerequisites present | Yes | Yes | 5,990 | 95.000084% | Yes | Yes | Yes |
| No overflow exposure | No | No | 5,990 | 0.000000% | No | No | No |
| Wrong lifecycle only | No | No | 5,990 | 0.000000% | No | No | No |
| Idealized SRI stub | Yes | No | 5,990 | 0.000000% | No | No | No |
| Packet-only oracle | Yes | No | 5,990 | 0.000000% | No | No | No |
| Too little random budget | Yes | Yes | 10 | 0.498876% | No | Yes | Yes |
| No replay evidence | Yes | Yes | 5,990 | 95.000084% | Yes | No | No |
| No enforced response | Yes | Yes | 5,990 | 95.000084% | Yes | Yes | No |
| Guarded negative control | No | No | 5,990 | 0.000000% | No | No | No |
At the invented rare-tail p = 0.0005, 5,990 tests are the minimal 95% budget. A short budget can still detect the injected fault; it just does not meet that random-discovery target. Retention and response do not alter detection. The Boolean response is not a measurement of human behavior.
05 / GO BEYOND THE DASHBOARD
The research is the product.
The demo is an invitation to question it.
Read the complete study
The inquiry, the hypothesis, causal chain, detection prerequisites, and AI-evaluation analogy.
Full research → 02Inspect the method
Conversion semantics, independent samples, temporal trace, uncertainty, and scope.
Methods & limitations → 03Challenge an assumption
Nine BLIND-SPOT steps, executable contracts, release gates, and a teaching exercise.
Evaluation framework → 04Follow the evidence
Every source, access limitation, invented variable, and open historical quantity.
Provenance register →OPEN ABOUT THE WORK. OPEN ABOUT THE LIMITS.
AI helped build this lab.
That is not evidence AI
would have caught the fault.
We knew the answer before designing the experiment. No blinded AI evaluation, trained model, flight telemetry, or independent human scientific validation was performed in-session.
Alternative web search located sources; direct source-text verification established the claims. The source catalog preserves access limitations, academic mirrors, and metadata-only checks.
Read the AI usage disclosure →Usage totals, verified model identifiers, time, and cost are unknown, not zero. Model artifact version 1.1.