Draft standard CEI-MS-19: Purity determination for peptides… — submissions
The 16 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
16 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A superseded version should remain reachable from the version that replaced it
The respondent states that the draft supersedes an earlier document and that a reader who cited the earlier version has no way to reach it from the new one, which makes it impossible to see what changed.
The respondent asks that every version carry a link both to what it supersedes and to what supersedes it.
The secretariat notes this submission. The corrections and versioning policy already requires bidirectional version links and every superseded document is retained at its own address.
No amendment arises. The requirement is stated in the corrections and versioning policy and the amendment log of this document links to the version it replaced. The respondent is correct that the link was absent from the draft page furnished for consultation, which was a defect of the consultation copy and not of the policy.
The standard would be applied more consistently with a worked example
The respondent states that the requirement most often misapplied in practice is the one the draft states most briefly, and that a worked calculation would settle it.
The respondent proposes a worked example showing the same data treated correctly and incorrectly, with both results stated.
The secretariat accepts this submission. An example showing only the correct treatment does not show the reader what they are doing wrong.
A worked example is added showing the same primary data treated two ways, with both resulting figures stated and the difference between them quantified, so that the consequence of the misapplication is visible rather than described.
The document set should be published in translation
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
References should carry a persistent identifier for every cited source
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent, who works in a library setting, states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
The standard does not say what to do with a result outside the expected range
The respondent states that the draft describes how to obtain a result and stops, and that the handling of an unexpected result is where the integrity of a determination is decided.
The respondent proposes that repetition be permitted only under stated conditions and that every determination performed be reported, including those superseded.
The secretariat accepts this submission. A standard silent on repetition permits the selective reporting it exists to prevent.
The standard now requires that every determination performed on a sample be reported, that a repeat be permitted only where an assignable cause is documented, and that the assignable cause be stated on the face of the report.
The draft prescribes a procedure where it should state performance
Several clauses specify a gradient, a column chemistry and a temperature. The respondent states that fixing these prevents a laboratory from using a validated equivalent, and that the Institute settled on stating performance when it scoped the series.
The respondent proposes that the prescriptive clauses be converted into performance requirements with the prescribed conditions retained as a worked example.
The secretariat accepts this submission. The draft departed from the position settled when the series was scoped, and the departure was not deliberate.
The prescriptive clauses are converted to performance requirements, the specific conditions are retained as a worked example clearly labelled as one, and a laboratory using a different validated procedure that meets the stated performance reports a conformant determination.
Absence of evidence is presented in a form a reader will take as negative evidence
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
The matrix interferes with the determination and no control is required
The respondent states that the matrices in scope can suppress or enhance the response, and that without an inhibition and enhancement control a result cannot be attributed to the analyte.
Data showing recovery outside acceptable limits in an uncontrolled matrix accompanied the submission.
The secretariat accepts this submission. A determination on an interfering matrix without a validated control establishes nothing, and the draft permitted one.
An inhibition and enhancement control is now required for every determination under this standard, with stated recovery limits, and a determination reported without it is recorded as not conformant rather than as a low result.
The document is unreadable without specialist training
The respondent, a trustee of a patient organisation, states that the draft is written for a reader who already understands certainty grading, and that the people most affected by the subject matter will not reach the assessment at all.
The respondent proposes a plain-language summary at the head of every document, written to the same standard of accuracy as the document itself and not as a promotional abstract.
The secretariat accepts this submission in part. A plain-language summary is added. The proposal that it replace the technical abstract is declined, because the abstract is the part of the document other assessors read and cite.
Every document now opens with a plain-language summary of not more than 150 words, placed above the technical abstract and carrying the same certainty language, so that the two cannot diverge.
The standard should state a pass limit
The respondent states that a standard which reports a figure without a limit leaves the reader without a criterion, and that suppliers will set their own.
The respondent proposes a universal acceptance limit for the determination.
The secretariat does not accept this submission. An appropriate limit depends on the compound, the presentation and the intended route, none of which the standard fixes.
The standard states performance and reports the figure as determined. Setting a universal limit would attach a pass or fail to materials whose acceptable ranges genuinely differ, which would be a stronger claim than the Institute can support. The submission remains published in full and the point is recorded in the limitations.
Rounding conventions are not stated and change the reported result
The respondent states that the draft reports figures to varying precision and does not specify when rounding occurs in the calculation chain, which can move a result across an acceptance limit.
The respondent proposes that rounding occur only at the reporting step and that the convention be stated.
The secretariat accepts this submission. Intermediate rounding is a silent source of disagreement between laboratories analysing the same material.
The standard now requires that no rounding be applied before the reporting step, states the reporting precision for each quantity, and states the rounding convention used at that step.
A determination does not record the version of the standard it was made under
The respondent states that the standard will be revised and that results reported under different versions will circulate together, indistinguishable from one another.
The respondent proposes that the standard identifier and version be carried on the face of every determination.
The secretariat accepts this submission. A conformance claim against an unspecified version is not a conformance claim.
Every determination reported under this standard now carries the standard identifier and version on its face, and a claim of conformance that does not identify the version is recorded by the Institute as not conformant.
The search date is not on the face of the document
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
Acceptance criteria are expressed in terms specific to one instrument family
The respondent states that two criteria are expressed in units reported by one manufacturer's software and not by others, which makes conformance a function of the instrument rather than of the determination.
The respondent proposes that criteria be expressed in instrument-independent quantities.
The secretariat accepts this submission in part. Both criteria are restated in instrument-independent terms. The conversion from the manufacturer-specific quantity is given rather than removed, because laboratories will continue to read it from their software.
The acceptance criteria are now expressed in instrument-independent quantities, with the conversion from the common manufacturer-specific reporting units given in a note so that a laboratory can check conformance without recalculating by hand.
The standard should require documented operator competence
The respondent states that the determination described requires judgement, particularly at the integration step, and that a competence requirement would improve consistency more than any change to the criteria.
The respondent proposes that conformance be conditional on documented operator training.
The secretariat does not accept this submission. Operator competence is a quality-system matter governed by accreditation requirements the Institute does not administer and cannot verify.
The standard states performance and remains silent on competence. Where judgement affects the result, as at the integration step, the standard requires the judgement to be recorded rather than the operator to be qualified, which is the part the Institute can check from a report. The submission remains published in full.
The same concept is given three different names in one document
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.