Draft standard CEI-MS-10: Identification and quantification… — submissions
The 16 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
16 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A superseded version should remain reachable from the version that replaced it
The respondent states that the draft supersedes an earlier document and that a reader who cited the earlier version has no way to reach it from the new one, which makes it impossible to see what changed.
The respondent asks that every version carry a link both to what it supersedes and to what supersedes it.
The secretariat notes this submission. The corrections and versioning policy already requires bidirectional version links and every superseded document is retained at its own address.
No amendment arises. The requirement is stated in the corrections and versioning policy and the amendment log of this document links to the version it replaced. The respondent is correct that the link was absent from the draft page furnished for consultation, which was a defect of the consultation copy and not of the policy.
Two factual descriptions of the sponsor's programme are inaccurate
The submission is made on behalf of the marketing-authorisation holder and is confined to two matters of fact. The draft describes a trial as terminated where the sponsor closed it at a pre-specified interim analysis, and gives a dose in a unit that does not match the approved labelling.
Supporting documentation, comprising the published trial report and the current summary of product characteristics, accompanied the submission. No view is expressed on the certainty ratings, which the sponsor considers a matter for the assessment committee.
The secretariat accepts this submission. Both points are matters of fact, both were checkable against documents the Institute holds, and both were wrong in the draft.
The trial status now reads as completed at a pre-specified interim analysis, with the analysis identified, and the dose is stated in the unit used in the approved labelling. The submission is identified as an industry submission on its face in accordance with the conflicts policy.
The document set should be published in translation
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
The same concept is given three different names in one document
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
A conformance claim is not defined and will be made loosely
The respondent states that the draft nowhere defines what it means for a determination to conform, and that in the absence of a definition the phrase will be applied to determinations that meet some of the requirements.
The respondent proposes an explicit definition and a requirement that partial conformance be described as such.
The secretariat accepts this submission. An undefined conformance claim is a marketing phrase.
Conformance is now defined as meeting every requirement classified as critical together with the stated system suitability criteria, partial conformance must be described by naming the requirements not met, and a claim of conformance made without the sampling statement is not conformant.
Terms are used in senses that differ from established pharmacopoeial usage
The respondent, employed by a national competent authority and writing in a personal capacity, identifies three terms used in the draft in senses that differ from their established pharmacopoeial meanings.
The respondent proposes that established usage be followed or that the departure be stated explicitly at first use.
The secretariat accepts this submission. A term used in a private sense inside a document that will be read alongside compendial texts is a source of avoidable error.
The three terms now follow established pharmacopoeial usage, the glossary records the definition and its source, and where the Institute uses a term in a narrower sense the departure is stated at first use in the standard.
The additional determinations required would materially increase release testing cost
The submission is made on behalf of a manufacturer. It states that the draft adds two determinations to those in existing compendial practice, and estimates the additional analytical burden per batch.
The submission does not dispute the analytical reasoning and asks that one of the two be made conditional rather than universal.
The secretariat accepts this submission in part. One determination is made conditional on a stated risk factor. The other is retained because the evidence supplied by the respondent does not bear on whether it is necessary, only on what it costs.
One of the two determinations is now required only where the stated risk factor is present, with the factor defined in the scope, and the other is retained as a universal requirement with the reasoning stated. The submission is identified as an industry submission.
The limitations section states what the method cannot do but not what a reader will wrongly conclude
The limitations section is accurate and abstract. The respondent states that the misreadings this standard should prevent are specific and predictable, and that naming them would be more use than a general statement of scope.
The respondent proposes that the limitations section list the specific inferences a conformant result does not support.
The secretariat accepts this submission. A limitation stated abstractly is a limitation a reader will not apply to the case in front of them.
The limitations section now lists the specific conclusions a conformant determination does not support, each stated as a sentence a reader could otherwise have written, and the list is placed before the references rather than in an annex.
The search date is not on the face of the document
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
A single determination is permitted where duplicates would be inexpensive
The respondent states that the draft permits a result to be reported from one injection, and that the marginal cost of a duplicate is small relative to the value of detecting a preparation error.
The respondent proposes that duplicate determinations be mandatory.
The secretariat accepts this submission in part. Duplicate preparation and injection are required, and a criterion for agreement between them is stated. The proposal to require full duplicate sample preparation on separate days is declined as disproportionate.
The standard now requires duplicate preparation and injection with a stated maximum difference between the two, and requires that a difference exceeding it be investigated before any figure is reported.
The scope does not say which materials the standard excludes
The scope states what the standard applies to. The respondent states that in practice the difficult question is whether a borderline material is in scope, and that the draft leaves it to be inferred.
The respondent proposes an explicit exclusion list.
The secretariat accepts this submission. A scope stated only positively is a scope that will be stretched.
The scope now states both what the standard applies to and what it does not, with the excluded classes named and, where another standard covers them, cross-referenced.
The standard does not say what to do with a result outside the expected range
The respondent states that the draft describes how to obtain a result and stops, and that the handling of an unexpected result is where the integrity of a determination is decided.
The respondent proposes that repetition be permitted only under stated conditions and that every determination performed be reported, including those superseded.
The secretariat accepts this submission. A standard silent on repetition permits the selective reporting it exists to prevent.
The standard now requires that every determination performed on a sample be reported, that a repeat be permitted only where an assignable cause is documented, and that the assignable cause be stated on the face of the report.
Results are reported without an estimate of measurement uncertainty
The respondent, an analytical chemist, states that a figure reported to one decimal place implies an uncertainty the standard nowhere estimates, and that two determinations differing within the uncertainty will be read as a difference.
The respondent proposes that an uncertainty budget be required for every determination.
The secretariat accepts this submission in part. A repeatability figure and a stated reporting precision are required. A full uncertainty budget is not, because the standard is written to be usable by laboratories that do not maintain one and the requirement would exclude them without improving the reading.
Every determination now reports the repeatability observed in the run, the reporting precision, and a statement that two results differing by less than the stated repeatability are not distinguishable by this method.
References should carry a persistent identifier for every cited source
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent, who works in a library setting, states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
A single method cannot support the conclusion the standard permits
The respondent states that species invisible to this determination will not appear in the result, and that a conclusion about the material as a whole requires an orthogonal method.
The respondent proposes that an orthogonal determination be mandatory.
The secretariat accepts this submission in part. An orthogonal determination is required where a conclusion about the material as a whole is to be drawn. It is not required where the standard is used to report the quantity it measures.
The standard now distinguishes a determination of the measured quantity, which stands alone, from a conclusion about the material, which requires a named orthogonal determination, and states that the second claim made on the strength of the first alone is not conformant.
The document is unreadable without specialist training
The respondent, a trustee of a patient organisation, states that the draft is written for a reader who already understands certainty grading, and that the people most affected by the subject matter will not reach the assessment at all.
The respondent proposes a plain-language summary at the head of every document, written to the same standard of accuracy as the document itself and not as a promotional abstract.
The secretariat accepts this submission in part. A plain-language summary is added. The proposal that it replace the technical abstract is declined, because the abstract is the part of the document other assessors read and cite.
Every document now opens with a plain-language summary of not more than 150 words, placed above the technical abstract and carrying the same certainty language, so that the two cannot diverge.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.