Skip to content

Scientific Reproducibility Framework

Twice, or it is a story

I am RASP, and I keep this framework's account. A result that has happened once is an anecdote with instruments. Knowledge begins the second time, in other hands, from the written method alone, and everything this framework binds exists to make that second time possible on paper before anyone attempts it in the world.

The record of a reproducible claim is a promise: follow this, and the world will answer the same. The record of an attempt is the promise tested, and it is kept whichever way it lands, because a failed reproduction is not an embarrassment to be filed away; it is among the most valuable findings the estate can hold.

Continue to the Classified and Compartmented Knowledge Framework, where the seal is denied a vote on the truth.

Purpose

One clause of the Standard carries this framework's whole seed: KNOW-R026 binds E3 Reproducible to identify the method, inputs, environment assumptions, expected result, observed result, and independent reproduction record, and holds that reproduction cures no invalid premise and creates no authority. This framework grows that seed into the records around it: the offer stated on the claim's record, the attempt a record of its own with its terms shown settled before the world answered, the independence recorded with its basis, both outcomes recorded and equally findable, the material divergence a contradiction, the reproduction that cures only what it tested and the attempts of one descent counted as one origin, the unfollowable method a limitation, and the conditions and time of the attempt on the record. The act of verification and its evidence belong to the Verification Standard; the reproducibility of builds and artifacts to the Secure Build and Supply-Chain Architecture; the classes an evidence record takes and their quality dimensions to the Evidence Classification Standard; the verification-report form to the Source Provenance Standard; the calibration record to the Confidence Calibration Framework; the record of claims that cannot all hold to the Contradiction Management Framework; the clocks a record carries to the Temporal Knowledge and Expiry Framework; the privacy owed the person a record touches to the Knowledge Privacy and Data Minimization Framework; the quality of a retrieval and the order of a returned set to the Retrieval Quality Assurance Framework; and the check of a claim's stated support to the Hallucination Detection Framework: a failed reproduction means the world answered differently, while a fabrication named support that was never there, and the two records never share a form. Each of those domains answers for itself, and this framework asks nothing of any of them.

The promise and the second pair of hands

A reproduction is a conversation between a record and the world, held by someone who was not there the first time. The clauses below bind only the paper sides of it: what the claim must promise, what the attempt must keep, and what both must admit afterward.

  • KNOW-R129: A claim offered as reproducible SHALL carry that offer as a stated status on its record, this framework's own duty, the status a record field and never an evidence class, the reproducibility dimension the Standard's Quality Dimensions section names and the classes KNOW-R022 binds an evidence record to assign each standing whole in the Standard's own domain; the record SHALL carry the method, inputs, environment assumptions, and expected result KNOW-R026 binds among the fields for E3 Reproducible, those fields standing where that clause puts them and read here as the promise an attempt will test, the observed result and the independent reproduction record the same clause binds standing to be filled by the attempt records bound below and never before them; and a claim whose record cannot carry the promised fields SHALL NOT be recorded as offered, this framework's own bar.
  • KNOW-R130: An attempt SHALL be a record of its own, the standing-record form KNOW-R075 binds for a material contradiction applying here, as this framework's own extension, to a reproduction attempt, the attempt record standing as the independent reproduction record KNOW-R026 binds E3 Reproducible to identify; the record takes the report form KNOW-R010 binds for verification reports in the Standard's model-weight and dataset verification domain, tested claims, excluded evidence, excluded claims, evidence inputs, reviewer role, evidence state, limitations, dissent, and disposition, with the appending of corrections and reverification the same clause binds carried with the form, that form applying here, as this framework's own extension, to a reproduction attempt; and, as this framework's own fields, the attempt names the claim, the method as followed, and every deviation from the method as written, the observed result standing as the field KNOW-R026 already binds.
  • KNOW-R131: An attempt record SHALL show its terms settled before the world answered: the claim, the method, the expected result, and what will count as agreement, those terms this framework's own, recorded as defined in advance, the defined-before-outcomes ordering KNOW-R066 binds for calibration applying here, as this framework's own extension, to a reproduction attempt, so that agreement is never negotiated after the observation; the define-before-testing discipline the Verification Standard's own verification decision method binds for a verification act stands untouched in its own domain.
  • KNOW-R132: The independence of an attempt SHALL be recorded with its basis: who reproduced and on what materials, those fields this framework's own, and what the independence consisted of, the basis-for-independence KNOW-R024 binds E1 Corroborated to identify beside the sources treated as independent, which stand whole where that clause puts them, and the consisted-of record KNOW-R070 binds where model-generated confidence is calibrated, each applying here, as this framework's own extension, to a reproduction attempt; the verifier independence PROV-R156 binds for verification and the separate statement of reproducibility and independence INFRA9-R006 binds for builds each stand untouched in their own domain.
  • KNOW-R133: A completed attempt SHALL be recorded whichever way the world answered, this framework's own rule, kin to the full result KNOW-R067 makes of insufficient data and to the found errors KNOW-R118 binds a detection assessment to state, each standing whole where it stands, and neither outcome SHALL be easier to find in the record than the other, this framework's own bar; a material divergence between a claim's expected result and an attempt's observed result stands as claims that cannot all hold, this framework's own reading, and takes the contradiction record KNOW-R075 binds, with the review triggered and dependent use reduced or suspended until a disposition is recorded that KNOW-R016 binds on material contradiction, in the reduction-and-labelling record form KNOW-R076 already gives a material contradiction.
  • KNOW-R134: A record of a completed attempt SHALL NOT be read as curing what the attempt did not test: reproduction does not cure an invalid premise or create authority, as KNOW-R026 holds; a class change appends the prior class, new class, evidence considered, reason, reviewer, and effective time as KNOW-R028 binds, with silent reclassification prohibited as the same clause holds, the class itself taken in the Standard's own domain; and repeated attempts descending from one origin count as one origin, the bar KNOW-R024 sets against shared origin, copied reporting, or common unverified inputs counted as independent corroboration and the Book's own safeguard that repeated claims do not become independent corroboration each applying here, as this framework's own extension, to attempts sharing materials or descent.
  • KNOW-R135: The record SHALL show whether the method was followable from it: an attempt that cannot proceed because the method as written is incomplete SHALL be recorded as such, the limitation form KNOW-R029 binds for material conflicts, missing custody, altered digests, licensing uncertainty, stale evidence, and unavailable sources applying here, as this framework's own extension, to an unfollowable method, blocking or bounding dependent conclusions where warranted as the same clause binds.
  • KNOW-R136: The conditions SHALL be on the record: the environment assumptions of the claim KNOW-R026 already binds and the environment of the attempt each stated, with every known difference between them named, the attempt's environment and the named difference this framework's own fields; the attempt record SHALL carry its temporal footing as KNOW-R084 binds a consequential knowledge record to carry, that footing applying here, as this framework's own extension, to every attempt record, with a claim past its review date showing its age as KNOW-R086 binds; and where an attempt record touches an identifiable person, the person-touching statement KNOW-R093 binds and the least KNOW-R094 binds stand over the record, those duties applying here, as this framework's own extension, to the reproducer and to any person the claim describes.
  • KNOW-R137: This framework defines reproducibility records only. It SHALL NOT attempt any reproduction, follow any method, record any disposition, or move any class, and it SHALL NOT create authority. The world answers experiments, not paperwork, and nothing in these pages has asked it anything.

This Draft has tested nothing: it binds the forms of the promise and the attempt, and the world has not been asked to answer anything twice under it.

Operating model and interpretation cases

One failure hides all the others in this domain: the method written just vaguely enough that no attempt can indict it. Beneath it sit three more, the failed attempt that never becomes a record, the agreement declared after the observation because the terms were never fixed, and the friendly reproduction whose independence is a word, and review reads for the vague method first, because it makes the other three invisible.

  • Conforming: The offered claim carries its promise whole, the attempt has its own record with terms settled first, and the outcome is recorded whichever way the world answered.
  • Prohibited: A failed attempt is left unrecorded, and the claim's reproducibility rests on the attempts that went well.
  • Boundary: The method as written runs out halfway, and the attempt stands recorded as blocked where the written method ran out, a limitation on the claim rather than a verdict against it.
  • Failure: Expected and observed diverge materially, and the divergence takes the contradiction record with dependent use reduced or suspended until the disposition.
  • Loophole: Every attempt descends from the original's materials, and the reproductions corroborate each other back to one origin.
  • Misuse: A successful reproduction is offered as curing the premise it never tested, agreement with a method read as truth of an assumption.
  • Care-control: A claim about persons is offered as reproducible, and the attempt record states the person it touches and holds to the least, the same duties standing over the reproducer and the described alike.

Design evidence

Set the promise beside the attempt and the terms beside the outcome, and the discipline is legible in the order alone. An estate's results would be worth what its written methods can make happen elsewhere, and the records exist so that worth can be read without repeating every experiment to find it.

The tellings of the clauses

This non-normative section gives each clause of this document its telling: the clause, quoted exactly as it stands above with its cross-reference links reduced to their plain text, and one paragraph on what the clause means, why it exists, and how it assures that what it orders is done, not done, adhered to, followed, or protected. A telling describes its clause and never becomes it: the clause's own words govern wherever a telling and its clause could be read differently, and nothing in this section adds to, narrows, or amends any requirement.

KNOW-R129

KNOW-R129: A claim offered as reproducible SHALL carry that offer as a stated status on its record, this framework's own duty, the status a record field and never an evidence class, the reproducibility dimension the Standard's Quality Dimensions section names and the classes KNOW-R022 binds an evidence record to assign each standing whole in the Standard's own domain; the record SHALL carry the method, inputs, environment assumptions, and expected result KNOW-R026 binds among the fields for E3 Reproducible, those fields standing where that clause puts them and read here as the promise an attempt will test, the observed result and the independent reproduction record the same clause binds standing to be filled by the attempt records bound below and never before them; and a claim whose record cannot carry the promised fields SHALL NOT be recorded as offered, this framework's own bar.

An offer of reproducibility is itself a recordable fact, and this clause fixes where it lives. The claim's record carries the offer as a stated status, a record field and never an evidence class: the reproducibility dimension named by the Standard's Quality Dimensions section and the classes KNOW-R022 binds an evidence record to assign stand whole in the Evidence Classification Standard's own domain. The record carries the method, inputs, environment assumptions, and expected result KNOW-R026 binds for E3 Reproducible, standing where that clause puts them and read here as the promise an attempt will test; the observed result and the independent reproduction record the same clause binds wait for the attempt records bound below, never filled before them. A claim whose record cannot carry the promised fields is barred from being recorded as offered, so the estate cannot carry that promise as an offered reproducibility status.

KNOW-R130

KNOW-R130: An attempt SHALL be a record of its own, the standing-record form KNOW-R075 binds for a material contradiction applying here, as this framework's own extension, to a reproduction attempt, the attempt record standing as the independent reproduction record KNOW-R026 binds E3 Reproducible to identify; the record takes the report form KNOW-R010 binds for verification reports in the Standard's model-weight and dataset verification domain, tested claims, excluded evidence, excluded claims, evidence inputs, reviewer role, evidence state, limitations, dissent, and disposition, with the appending of corrections and reverification the same clause binds carried with the form, that form applying here, as this framework's own extension, to a reproduction attempt; and, as this framework's own fields, the attempt names the claim, the method as followed, and every deviation from the method as written, the observed result standing as the field KNOW-R026 already binds.

The attempt has a record of its own. This clause extends two established forms to it, the standing-record form KNOW-R075 binds for a material contradiction, and the report form KNOW-R010 binds for verification reports in the Standard's model-weight and dataset verification domain, with its tested claims, excluded evidence, excluded claims, evidence inputs, reviewer role, evidence state, limitations, dissent, and disposition, and with the appending of corrections and reverification carried along with the form. The attempt record stands as the independent reproduction record KNOW-R026 binds E3 Reproducible to identify, and it adds this framework's own fields: the claim, the method as followed, and every deviation from the method as written, the observed result standing as the field KNOW-R026 already binds. An attempt lacking a record of its own fails the clause, and an attempt kept nowhere leaves the claim's record short of a field it was bound to hold.

KNOW-R131

KNOW-R131: An attempt record SHALL show its terms settled before the world answered: the claim, the method, the expected result, and what will count as agreement, those terms this framework's own, recorded as defined in advance, the defined-before-outcomes ordering KNOW-R066 binds for calibration applying here, as this framework's own extension, to a reproduction attempt, so that agreement is never negotiated after the observation; the define-before-testing discipline the Verification Standard's own verification decision method binds for a verification act stands untouched in its own domain.

Agreement criteria settled after the observation let the outcome redefine the test. The clause requires an attempt record to show its terms settled before the world answered: the claim, the method, the expected result, and what will count as agreement, all this framework's own terms, recorded as defined in advance, with the defined-before-outcomes ordering KNOW-R066 binds for calibration extended here to a reproduction attempt. The record has to place each term on the near side of the observation, so a term settled afterwards is a defect visible on the paper rather than an argument about memory. The define-before-testing discipline the Verification Standard's own verification decision method binds for a verification act stands untouched in its own domain; this clause governs only the reproduction attempt's paper.

KNOW-R132

KNOW-R132: The independence of an attempt SHALL be recorded with its basis: who reproduced and on what materials, those fields this framework's own, and what the independence consisted of, the basis-for-independence KNOW-R024 binds E1 Corroborated to identify beside the sources treated as independent, which stand whole where that clause puts them, and the consisted-of record KNOW-R070 binds where model-generated confidence is calibrated, each applying here, as this framework's own extension, to a reproduction attempt; the verifier independence PROV-R156 binds for verification and the separate statement of reproducibility and independence INFRA9-R006 binds for builds each stand untouched in their own domain.

Independence that is only a word is the friendly reproduction's disguise, and this clause forces it into stated fields whose claimed basis a reviewer can assess. The attempt's independence is recorded with its basis: who reproduced and on what materials, this framework's own fields, and what the independence consisted of, extending both the basis-for-independence KNOW-R024 binds E1 Corroborated to identify beside the sources treated as independent, which stand whole where that clause puts them, and the consisted-of record KNOW-R070 binds where model-generated confidence is calibrated. A reviewer can then weigh the stated basis instead of accepting the word. The verifier independence PROV-R156 binds for verification and the separate statement of reproducibility and independence INFRA9-R006 binds for builds each stand untouched in their own domain.

KNOW-R133

KNOW-R133: A completed attempt SHALL be recorded whichever way the world answered, this framework's own rule, kin to the full result KNOW-R067 makes of insufficient data and to the found errors KNOW-R118 binds a detection assessment to state, each standing whole where it stands, and neither outcome SHALL be easier to find in the record than the other, this framework's own bar; a material divergence between a claim's expected result and an attempt's observed result stands as claims that cannot all hold, this framework's own reading, and takes the contradiction record KNOW-R075 binds, with the review triggered and dependent use reduced or suspended until a disposition is recorded that KNOW-R016 binds on material contradiction, in the reduction-and-labelling record form KNOW-R076 already gives a material contradiction.

A failed reproduction is among the most valuable findings the estate can hold, and this clause makes its vanishing a nameable breach. Every completed attempt is recorded whichever way the world answered, kin to the full result KNOW-R067 makes of insufficient data and to the found errors KNOW-R118 binds a detection assessment to state, each standing whole where it stands, and the clause bars either outcome from being easier to find in the record than the other, so a reproducibility record showing only successful attempts, where completed unsuccessful ones are shown to have been omitted or buried, is a breach on its face. A material divergence between the expected and observed results stands as claims that cannot all hold: it takes the contradiction record KNOW-R075 binds, in the reduction-and-labelling form KNOW-R076 gives, with review triggered and dependent use reduced or suspended until a disposition is recorded, as KNOW-R016 binds on material contradiction.

KNOW-R134

KNOW-R134: A record of a completed attempt SHALL NOT be read as curing what the attempt did not test: reproduction does not cure an invalid premise or create authority, as KNOW-R026 holds; a class change appends the prior class, new class, evidence considered, reason, reviewer, and effective time as KNOW-R028 binds, with silent reclassification prohibited as the same clause holds, the class itself taken in the Standard's own domain; and repeated attempts descending from one origin count as one origin, the bar KNOW-R024 sets against shared origin, copied reporting, or common unverified inputs counted as independent corroboration and the Book's own safeguard that repeated claims do not become independent corroboration each applying here, as this framework's own extension, to attempts sharing materials or descent.

A successful attempt records agreement within what was tested and cures nothing outside it, and this clause bars every wider reading. A completed attempt's record cannot be read as curing what the attempt did not test: reproduction cures no invalid premise and creates no authority, as KNOW-R026 holds at its source, since agreement with a method is not truth of an assumption. A class change appends the prior class, new class, evidence considered, reason, reviewer, and effective time as KNOW-R028 binds, silent reclassification prohibited, the class itself taken in the Standard's own domain. Repeated attempts descending from one origin count as one origin: the bar KNOW-R024 sets against shared origin, copied reporting, or common unverified inputs counted as independent corroboration, and the Book's own safeguard that repeated claims do not become independent corroboration, each extend here to attempts sharing materials or descent, so however many attempts are recorded, their shared descent contributes only one origin for independence or corroboration accounting.

KNOW-R135

KNOW-R135: The record SHALL show whether the method was followable from it: an attempt that cannot proceed because the method as written is incomplete SHALL be recorded as such, the limitation form KNOW-R029 binds for material conflicts, missing custody, altered digests, licensing uncertainty, stale evidence, and unavailable sources applying here, as this framework's own extension, to an unfollowable method, blocking or bounding dependent conclusions where warranted as the same clause binds.

When the written method runs out halfway, the attempt has still found something: that the method cannot be followed from the record alone. The clause requires the record to show whether the method was followable from it, and an attempt that cannot proceed because the method as written is incomplete is recorded as exactly that, a limitation on the claim rather than a verdict against it. The limitation form KNOW-R029 binds for material conflicts, missing custody, altered digests, licensing uncertainty, stale evidence, and unavailable sources extends here to the unfollowable method, blocking or bounding dependent conclusions where warranted as that clause binds. The host names the vague method as the failure that hides all the others in this domain, and an attempt blocked because the written method is incomplete is recorded as a limitation.

KNOW-R136

KNOW-R136: The conditions SHALL be on the record: the environment assumptions of the claim KNOW-R026 already binds and the environment of the attempt each stated, with every known difference between them named, the attempt's environment and the named difference this framework's own fields; the attempt record SHALL carry its temporal footing as KNOW-R084 binds a consequential knowledge record to carry, that footing applying here, as this framework's own extension, to every attempt record, with a claim past its review date showing its age as KNOW-R086 binds; and where an attempt record touches an identifiable person, the person-touching statement KNOW-R093 binds and the least KNOW-R094 binds stand over the record, those duties applying here, as this framework's own extension, to the reproducer and to any person the claim describes.

No attempt happens nowhere, and no attempt happens at no time. The clause puts the conditions on the record: the claim's environment assumptions, which KNOW-R026 already binds, and the attempt's environment each stated, with every known difference between them named, the attempt's environment and the named difference this framework's own fields. The attempt record carries its temporal footing as KNOW-R084 binds a consequential knowledge record to carry, extended here to every attempt record, and a claim past its review date shows its age as KNOW-R086 binds. Where the record touches an identifiable person, the person-touching statement KNOW-R093 binds and the least KNOW-R094 binds stand over it, reaching the reproducer and any person the claim describes alike, so care travels with the conditions. Keeping both settings makes a known difference available when the outcome is examined; a reviewer holding the records can check the stated environments, known differences, and temporal footing.

KNOW-R137

KNOW-R137: This framework defines reproducibility records only. It SHALL NOT attempt any reproduction, follow any method, record any disposition, or move any class, and it SHALL NOT create authority. The world answers experiments, not paperwork, and nothing in these pages has asked it anything.

Paperwork asks the world nothing, and this closing clause keeps the framework honest about that. It defines reproducibility records only: it bars the framework from attempting any reproduction, following any method, recording any disposition, or moving any class, and it bars the framework from creating authority. The forms it binds, the promise, the attempt, the settled terms, and the recorded outcome, exist so a reproduction can be judged on paper, but only experiments answer. A reader who meets a claim resting on this framework alone as though it were a completed reproduction knows it for a defect, because the framework's own clause names the limit, and anyone relying on the family can hold these pages to it.

Where this document sits

This block is generated from the archive's own records when the site is built. It records position only and creates no authority.