Draft revision of the Institute’s certainty assessment framework, version 2
Consultation on the second version of the framework the Institute uses to rate certainty in a body of evidence. The revision retains the four-level GRADE scale and alters the way the two lowest ratings are explained, after the assessment committee recorded…
§1Consultation and window
§1.1Window
- Opened
- 2 May 2024
- Closed
- 27 June 2024
- Duration
- 56 days
- Status
- Closed — the disposition is published
- Document under consultation
- Methodology
- Submissions received
- 15
§1.2Consultation questions
The secretariat put the following questions. A submission was not required to address them and several did not; the questions record what the Institute was uncertain about rather than restricting what a respondent could raise.
- Does the revised framework distinguish clearly enough between an absence of evidence and a body of conflicting evidence?
- Is the revised treatment of imprecision applicable to safety outcomes, where the event of interest may not have occurred in any contributing trial?
- Does the framework remain interoperable with published GRADE guidance, such that an Institute rating can be read against an external assessment using the same scale?
§1.3How a submission is handled
- Every submission is signed. The Institute publishes no anonymous submission.
- Every submission carries a declared interest. A declaration of no interest is itself a declaration and is recorded as one. A declaration is not a disqualification and the Institute has never given a submission less weight on that ground.
- A submission from an employee of a marketing-authorisation holder is identified as an industry submission on its face, following a unanimous consultation on the conflicts policy.
- Every submission receives a secretariat response and a disposition. Submissions that are not accepted remain published in full; the Institute does not remove a submission because it disagrees with it.
- There is no voting, no reputation score, no reply thread and no editing after receipt.
§1.4Outcome
The framework was amended before ratification. The rating scale itself was left unchanged, so that Institute ratings remain readable against external assessments, and the distinction between absent and conflicting evidence is now carried in a standing formulation applied wherever a low or very low rating appears.
The disposition table, listing each submission with its disposition and the resulting amendment, is at §3.