Skip to main content
James Scott Institute for Adversarial Frontier Physics

Review programmes

Six programmes, one method

Each programme applies the same reasoning sequence to a different object: a claim, a measurement, a model, a contradiction, a replication, or a decision. Every programme produces a written record with an explicit evidentiary state.

Engagement
Scoped in writing before work begins
Access
Read-only, partner-defined
Output
Written record with decision state
Confidentiality
Default; publication only by consent

Programme 01

Claim and evidence review

The claim is decomposed into separable propositions, and each proposition is assessed against what actually supports it.

Suited to
A consequential result approaching publication, disclosure, or an investment decision.
Inputs required
The claim as stated, the analysis behind it, and the records that were not used along with the reason they were excluded.
Output
A proposition-by-proposition record: support, contradiction, load-bearing assumptions, falsification conditions, and the justified decision state.
Not this programme
Editorial improvement of a manuscript, or endorsement of a result for external audiences.

Sequence

  1. 01Decompose the claim into propositions that can fail independently.
  2. 02Attach each item of evidence to the proposition it actually bears on.
  3. 03Enumerate the surviving alternative explanations.
  4. 04Identify the assumptions that carry the conclusion.
  5. 05State what observation would falsify each proposition.
  6. 06Record the decision state, with dissent retained.
Figure S1

Assumption dependency

Illustrative

Which conclusions fail if a given assumption fails. Load-bearing assumptions are the ones worth testing first.

Dependency graph from assumptions through intermediate results to conclusions, with load-bearing paths emphasised.

linearityisotropysteady stateclosed formestimatorreported claimDashed edge: load-bearing assumption with no independent verification. If it fails, the claim fails.
SyntheticDecision-relevant

Programme 02

Measurement and uncertainty audit

The calibration chain is reconstructed and the uncertainty budget rebuilt, with systematic terms separated from repeatability.

Suited to
A quantitative result whose interpretation depends on the size of the stated interval.
Inputs required
Calibration records, traceability references, environmental conditions, and the analysis transformations applied to raw values.
Output
A reconstructed budget, the dominant contributor named, the weakest link in the chain identified, and a statement of whether the reported interval is defensible.
Not this programme
Instrument calibration itself, or issuing calibration certificates.

Sequence

  1. 01Reconstruct the chain from reference to reported value.
  2. 02Separate random, systematic, and correction terms.
  3. 03Test independence before combining terms in quadrature.
  4. 04Identify the dominant contributor and the weakest traceability link.
  5. 05Compare the reconstructed interval with the reported one.
Figure S2

Measurement chain and error propagation

Illustrative

The chain from physical quantity to reported value, with the stage that limits the result marked, and the propagation of terms through it.

Measurement chain diagram with the limiting stage marked, alongside a tree showing how error terms propagate to the reported value.

InstrumentCalibrationEnvironmentdominant termTransformDerivedInterpretationTraceability chain: each stage contributes systematic and random terms.σ_instrumentσ_calibrationσ_environmentquadrature sumσ_combined → interpretive stateDashed edge: correlated term, quadrature addition not valid without a covariance estimate.
SyntheticDecision-relevant

Programme 03

Model admissibility assessment

Consistency, well-posedness, identifiability, and convergence are examined before accuracy is discussed at all.

Suited to
Results that depend on a numerical or analytic model extended beyond its validated range.
Inputs required
Model formulation, stated validity range, parameter provenance, discretisation choices, and convergence evidence.
Output
A statement of admissibility, the range in which the model may be used, and the specific numerical evidence that is missing.
Not this programme
Producing a better-fitting model, or tuning parameters on the partner's behalf.

Sequence

  1. 01Check dimensional and structural consistency of the formulation.
  2. 02Establish whether the problem as posed is well-posed.
  3. 03Test whether parameters are identifiable from the available data.
  4. 04Assess convergence and residual behaviour under refinement.
  5. 05State the admissible range, separately from any question of fit.
Figure S3

Convergence and residual structure

Illustrative

Convergence under refinement and the structure of the remaining residuals. Structured residuals indicate a missing term, not a need for more resolution.

Convergence curve under grid refinement alongside a residual panel showing structured rather than random residuals.

iterationlog residual normtoleranceconverged configurationstalled configuration — result not admissible
SyntheticDecision-relevant

Programme 04

Contradiction and anomaly analysis

Inconsistent records are registered, traced to their origin, and kept acting on the release state until resolved on stated grounds.

Suited to
A programme carrying an unexplained inconsistency, or a result that some attempts recover and others do not.
Inputs required
All attempts including the unsuccessful ones, the conditions that differed, and the grounds on which any record was set aside.
Output
A contradiction register with provenance for each entry, the discriminating test, and a statement of what remains unresolved.
Not this programme
Explaining away an inconsistency to permit publication.

Sequence

  1. 01Register each inconsistent record with its provenance.
  2. 02Separate contradictions of measurement from contradictions of interpretation.
  3. 03Identify which contradictions are decision-relevant.
  4. 04Specify the test that would discriminate.
  5. 05Hold the release state until the register is resolved or explicitly scoped.
Figure S4

Signal, noise, and residual

Illustrative

Separating a candidate feature from the residual structure of the measurement. Where the residual is structured, the feature cannot be attributed to nature.

Signal and noise traces with the residual plotted beneath, showing structured residual behaviour under the candidate feature.

channelresponse / residualfitted modelobservation (synthetic)residual drift after channel 0.6
SyntheticDecision-relevant

Programme 05

Replication design review

A replication is designed so that its outcome is informative in both directions, with sensitivity established before it runs.

Suited to
Before committing resources to a replication attempt, or when an attempt has produced an uninformative null.
Inputs required
The original protocol, the proposed attempt, the shared apparatus or pipeline, and the intended sensitivity.
Output
An assessment of independence, the required sensitivity, the conditions that must be held and reported, and the interpretation rules agreed in advance.
Not this programme
Conducting the replication, or arbitrating a priority dispute.

Sequence

  1. 01Map what the attempt shares with the original.
  2. 02Establish the sensitivity required for a null to be informative.
  3. 03Fix the analysis path before data are examined.
  4. 04Define in advance what each outcome would license.
  5. 05Record the attempt in the replication topology, whatever it returns.
Figure S5

Parameter constraint region

Illustrative

The region of parameter space that a proposed attempt would actually constrain, against the region relevant to the claim.

Parameter space with the constrained region and the claim-relevant region shown, with their overlap marked.

parameter aparameter badmissible regionshaded: excluded by an independent constraintA physical bottleneck often appears as an admissible region that no achievable configuration reaches.
SyntheticDecision-relevant

Programme 06

Decision-state adjudication

A stated position on what the evidence permits now: hold, scope, proceed with conditions, or stop.

Suited to
A decision that is due — funding, scale-up, disclosure, or termination — where the evidence is genuinely incomplete.
Inputs required
The decision actually faced, its timing, the variables that gate it, and the cost of being wrong in each direction.
Output
The justified release state, the conditions attached to it, the primary bottleneck, and the cheapest decisive next action.
Not this programme
Supplying a favourable position, or making the decision on the partner's behalf.

Sequence

  1. 01State the decision, its deadline, and the cost of error in each direction.
  2. 02Identify which evidence is decision-relevant and which is not.
  3. 03Locate the primary bottleneck.
  4. 04Name the cheapest action that would change the state.
  5. 05Issue the release state with its conditions and dissent.
Figure S6

Evidence threshold and release state

Illustrative

Where the accumulated evidence sits against the threshold required for the decision at hand. Thresholds are set by consequence, not by convention.

Evidence accumulation against a decision threshold, with the current position and the distance to the threshold marked.

HoldScopeProceed with conditionsProceedcurrent evidence stateA decision state is a claim about sufficiency of evidence, not a prediction of outcome.
SyntheticDecision-relevant

Engagements begin with eligibility review

Scope, access, and confidentiality are established in writing before any review begins. Nothing confidential, classified, export-controlled, or embargoed should be sent through this website.

Apply for a pilot projectLab attachment