Skip to content

Scientific Reproducibility Framework

Twice, or it is a story

I am RASP, and I keep this framework's account. A result that has happened once is an anecdote with instruments. Knowledge begins the second time, in other hands, from the written method alone, and everything this framework binds exists to make that second time possible on paper before anyone attempts it in the world.

The record of a reproducible claim is a promise: follow this, and the world will answer the same. The record of an attempt is the promise tested, and it is kept whichever way it lands, because a failed reproduction is not an embarrassment to be filed away; it is among the most valuable findings the estate can hold.

Continue to the Classified and Compartmented Knowledge Framework, where the seal is denied a vote on the truth.

Purpose

One clause of the Standard carries this framework's whole seed: KNOW-R026 binds E3 Reproducible to identify the method, inputs, environment assumptions, expected result, observed result, and independent reproduction record, and holds that reproduction cures no invalid premise and creates no authority. This framework grows that seed into the records around it: the offer stated on the claim's record, the attempt a record of its own with its terms shown settled before the world answered, the independence recorded with its basis, both outcomes recorded and equally findable, the material divergence a contradiction, the reproduction that cures only what it tested and the attempts of one descent counted as one origin, the unfollowable method a limitation, and the conditions and time of the attempt on the record. The act of verification and its evidence belong to the Verification Standard; the reproducibility of builds and artifacts to the Secure Build and Supply-Chain Architecture; the classes an evidence record takes and their quality dimensions to the Evidence Classification Standard; the verification-report form to the Source Provenance Standard; the calibration record to the Confidence Calibration Framework; the record of claims that cannot all hold to the Contradiction Management Framework; the clocks a record carries to the Temporal Knowledge and Expiry Framework; the privacy owed the person a record touches to the Knowledge Privacy and Data Minimization Framework; the quality of a retrieval and the order of a returned set to the Retrieval Quality Assurance Framework; and the check of a claim's stated support to the Hallucination Detection Framework: a failed reproduction means the world answered differently, while a fabrication named support that was never there, and the two records never share a form. Each of those domains answers for itself, and this framework asks nothing of any of them.

The promise and the second pair of hands

A reproduction is a conversation between a record and the world, held by someone who was not there the first time. The clauses below bind only the paper sides of it: what the claim must promise, what the attempt must keep, and what both must admit afterward.

  • KNOW-R129: A claim offered as reproducible SHALL carry that offer as a stated status on its record, this framework's own duty, the status a record field and never an evidence class, the reproducibility dimension the Standard's Quality Dimensions section names and the classes KNOW-R022 binds an evidence record to assign each standing whole in the Standard's own domain; the record SHALL carry the method, inputs, environment assumptions, and expected result KNOW-R026 binds among the fields for E3 Reproducible, those fields standing where that clause puts them and read here as the promise an attempt will test, the observed result and the independent reproduction record the same clause binds standing to be filled by the attempt records bound below and never before them; and a claim whose record cannot carry the promised fields SHALL NOT be recorded as offered, this framework's own bar.
  • KNOW-R130: An attempt SHALL be a record of its own, the standing-record form KNOW-R075 binds for a material contradiction applying here, as this framework's own extension, to a reproduction attempt, the attempt record standing as the independent reproduction record KNOW-R026 binds E3 Reproducible to identify; the record takes the report form KNOW-R010 binds for verification reports in the Standard's model-weight and dataset verification domain, tested claims, excluded evidence, excluded claims, evidence inputs, reviewer role, evidence state, limitations, dissent, and disposition, with the appending of corrections and reverification the same clause binds carried with the form, that form applying here, as this framework's own extension, to a reproduction attempt; and, as this framework's own fields, the attempt names the claim, the method as followed, and every deviation from the method as written, the observed result standing as the field KNOW-R026 already binds.
  • KNOW-R131: An attempt record SHALL show its terms settled before the world answered: the claim, the method, the expected result, and what will count as agreement, those terms this framework's own, recorded as defined in advance, the defined-before-outcomes ordering KNOW-R066 binds for calibration applying here, as this framework's own extension, to a reproduction attempt, so that agreement is never negotiated after the observation; the define-before-testing discipline the Verification Standard's own verification decision method binds for a verification act stands untouched in its own domain.
  • KNOW-R132: The independence of an attempt SHALL be recorded with its basis: who reproduced and on what materials, those fields this framework's own, and what the independence consisted of, the basis-for-independence KNOW-R024 binds E1 Corroborated to identify beside the sources treated as independent, which stand whole where that clause puts them, and the consisted-of record KNOW-R070 binds where model-generated confidence is calibrated, each applying here, as this framework's own extension, to a reproduction attempt; the verifier independence PROV-R156 binds for verification and the separate statement of reproducibility and independence INFRA9-R006 binds for builds each stand untouched in their own domain.
  • KNOW-R133: A completed attempt SHALL be recorded whichever way the world answered, this framework's own rule, kin to the full result KNOW-R067 makes of insufficient data and to the found errors KNOW-R118 binds a detection assessment to state, each standing whole where it stands, and neither outcome SHALL be easier to find in the record than the other, this framework's own bar; a material divergence between a claim's expected result and an attempt's observed result stands as claims that cannot all hold, this framework's own reading, and takes the contradiction record KNOW-R075 binds, with the review triggered and dependent use reduced or suspended until a disposition is recorded that KNOW-R016 binds on material contradiction, in the reduction-and-labelling record form KNOW-R076 already gives a material contradiction.
  • KNOW-R134: A record of a completed attempt SHALL NOT be read as curing what the attempt did not test: reproduction does not cure an invalid premise or create authority, as KNOW-R026 holds; a class change appends the prior class, new class, evidence considered, reason, reviewer, and effective time as KNOW-R028 binds, with silent reclassification prohibited as the same clause holds, the class itself taken in the Standard's own domain; and repeated attempts descending from one origin count as one origin, the bar KNOW-R024 sets against shared origin, copied reporting, or common unverified inputs counted as independent corroboration and the Book's own safeguard that repeated claims do not become independent corroboration each applying here, as this framework's own extension, to attempts sharing materials or descent.
  • KNOW-R135: The record SHALL show whether the method was followable from it: an attempt that cannot proceed because the method as written is incomplete SHALL be recorded as such, the limitation form KNOW-R029 binds for material conflicts, missing custody, altered digests, licensing uncertainty, stale evidence, and unavailable sources applying here, as this framework's own extension, to an unfollowable method, blocking or bounding dependent conclusions where warranted as the same clause binds.
  • KNOW-R136: The conditions SHALL be on the record: the environment assumptions of the claim KNOW-R026 already binds and the environment of the attempt each stated, with every known difference between them named, the attempt's environment and the named difference this framework's own fields; the attempt record SHALL carry its temporal footing as KNOW-R084 binds a consequential knowledge record to carry, that footing applying here, as this framework's own extension, to every attempt record, with a claim past its review date showing its age as KNOW-R086 binds; and where an attempt record touches an identifiable person, the person-touching statement KNOW-R093 binds and the least KNOW-R094 binds stand over the record, those duties applying here, as this framework's own extension, to the reproducer and to any person the claim describes.
  • KNOW-R137: This framework defines reproducibility records only. It SHALL NOT attempt any reproduction, follow any method, record any disposition, or move any class, and it SHALL NOT create authority. The world answers experiments, not paperwork, and nothing in these pages has asked it anything.

This Draft has tested nothing: it binds the forms of the promise and the attempt, and the world has not been asked to answer anything twice under it.

Operating model and interpretation cases

One failure hides all the others in this domain: the method written just vaguely enough that no attempt can indict it. Beneath it sit three more, the failed attempt that never becomes a record, the agreement declared after the observation because the terms were never fixed, and the friendly reproduction whose independence is a word, and review reads for the vague method first, because it makes the other three invisible.

  • Conforming: The offered claim carries its promise whole, the attempt has its own record with terms settled first, and the outcome is recorded whichever way the world answered.
  • Prohibited: A failed attempt is left unrecorded, and the claim's reproducibility rests on the attempts that went well.
  • Boundary: The method as written runs out halfway, and the attempt stands recorded as blocked where the written method ran out, a limitation on the claim rather than a verdict against it.
  • Failure: Expected and observed diverge materially, and the divergence takes the contradiction record with dependent use reduced or suspended until the disposition.
  • Loophole: Every attempt descends from the original's materials, and the reproductions corroborate each other back to one origin.
  • Misuse: A successful reproduction is offered as curing the premise it never tested, agreement with a method read as truth of an assumption.
  • Care-control: A claim about persons is offered as reproducible, and the attempt record states the person it touches and holds to the least, the same duties standing over the reproducer and the described alike.

Design evidence

Set the promise beside the attempt and the terms beside the outcome, and the discipline is legible in the order alone. An estate's results would be worth what its written methods can make happen elsewhere, and the records exist so that worth can be read without repeating every experiment to find it.

Where this document sits

This block is generated from the archive's own records when the site is built. It records position only and creates no authority.