Draft standard CEI-MS-11: Determination of aggregate content… — submissions
The 12 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
12 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
Nothing in the standard addresses transfer of the method between laboratories
The respondent states that determinations under this standard will be performed by different laboratories on the same material and that without transfer criteria their results are not comparable.
The respondent proposes a transfer protocol with acceptance criteria for inter-laboratory agreement.
The secretariat accepts this submission in part. Acceptance criteria for agreement between laboratories are added. A full transfer protocol is not, because transfer is a quality-system activity rather than a determination and the series does not address quality systems.
The standard now states the maximum difference between laboratories analysing the same sample at which results are treated as agreeing, and requires that a comparison across laboratories state whether that criterion was met.
The document set should be published in translation
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
A conformance claim is not defined and will be made loosely
The respondent states that the draft nowhere defines what it means for a determination to conform, and that in the absence of a definition the phrase will be applied to determinations that meet some of the requirements.
The respondent proposes an explicit definition and a requirement that partial conformance be described as such.
The secretariat accepts this submission. An undefined conformance claim is a marketing phrase.
Conformance is now defined as meeting every requirement classified as critical together with the stated system suitability criteria, partial conformance must be described by naming the requirements not met, and a claim of conformance made without the sampling statement is not conformant.
A single determination is permitted where duplicates would be inexpensive
The respondent states that the draft permits a result to be reported from one injection, and that the marginal cost of a duplicate is small relative to the value of detecting a preparation error.
The respondent proposes that duplicate determinations be mandatory.
The secretariat accepts this submission in part. Duplicate preparation and injection are required, and a criterion for agreement between them is stated. The proposal to require full duplicate sample preparation on separate days is declined as disproportionate.
The standard now requires duplicate preparation and injection with a stated maximum difference between the two, and requires that a difference exceeding it be investigated before any figure is reported.
Terms are used in senses that differ from established pharmacopoeial usage
The respondent, employed by a national competent authority and writing in a personal capacity, identifies three terms used in the draft in senses that differ from their established pharmacopoeial meanings.
The respondent proposes that established usage be followed or that the departure be stated explicitly at first use.
The secretariat accepts this submission. A term used in a private sense inside a document that will be read alongside compendial texts is a source of avoidable error.
The three terms now follow established pharmacopoeial usage, the glossary records the definition and its source, and where the Institute uses a term in a narrower sense the departure is stated at first use in the standard.
Quantitative claims are reproduced without the method that produced them
Several figures in the draft are quoted from sources that determined them by different methods. A figure obtained by one determination and a figure obtained by another are not comparable, and the draft places them in the same sentence without distinguishing them.
The respondent, an analytical chemist, proposes that every quantitative claim carry the method that produced it at the point of use rather than in the reference.
The secretariat accepts this submission. Placing two figures side by side is an implicit claim that they are the same kind of quantity, and in the cases identified they were not.
Every quantitative claim now carries the determination that produced it at the point of use, and figures obtained by non-comparable methods are no longer presented in the same row or sentence.
A determination does not record the version of the standard it was made under
The respondent states that the standard will be revised and that results reported under different versions will circulate together, indistinguishable from one another.
The respondent proposes that the standard identifier and version be carried on the face of every determination.
The secretariat accepts this submission. A conformance claim against an unspecified version is not a conformance claim.
Every determination reported under this standard now carries the standard identifier and version on its face, and a claim of conformance that does not identify the version is recorded by the Institute as not conformant.
References should carry a persistent identifier for every cited source
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent, who works in a library setting, states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
Point estimates are given without an interval
Several estimates in the draft appear as single figures. The respondent states that a point estimate without an interval invites a precision the underlying data do not support, and that the effect is worst where the estimate is drawn from a small contributing set.
The respondent proposes that no point estimate appear anywhere in the document set without its interval, including in summary tables and in the abstract.
The secretariat accepts this submission in part. Intervals are added wherever the source reports one. The proposal is declined for figures the source published without an interval, because the Institute will not compute an interval a source did not report.
Every estimate now carries its interval where the source reported one, and where it did not, the estimate is annotated as reported without an interval rather than left to appear as a precise figure.
Acceptance criteria are expressed in terms specific to one instrument family
The respondent states that two criteria are expressed in units reported by one manufacturer's software and not by others, which makes conformance a function of the instrument rather than of the determination.
The respondent proposes that criteria be expressed in instrument-independent quantities.
The secretariat accepts this submission in part. Both criteria are restated in instrument-independent terms. The conversion from the manufacturer-specific quantity is given rather than removed, because laboratories will continue to read it from their software.
The acceptance criteria are now expressed in instrument-independent quantities, with the conversion from the common manufacturer-specific reporting units given in a note so that a laboratory can check conformance without recalculating by hand.
The standard does not say what to do with a result outside the expected range
The respondent states that the draft describes how to obtain a result and stops, and that the handling of an unexpected result is where the integrity of a determination is decided.
The respondent proposes that repetition be permitted only under stated conditions and that every determination performed be reported, including those superseded.
The secretariat accepts this submission. A standard silent on repetition permits the selective reporting it exists to prevent.
The standard now requires that every determination performed on a sample be reported, that a repeat be permitted only where an assignable cause is documented, and that the assignable cause be stated on the face of the report.
Declared interests should appear on the document rather than on a separate page
The draft links to a central conflicts register. The respondent argues that a reader assessing whether to rely on a particular document should not have to leave it to find out who assessed it and what they declared.
The respondent proposes that the interests of every named contributor to a document be printed on that document.
The secretariat notes this submission and records that the draft already provides for it, which the respondent could reasonably have missed because the provision sits in an appendix.
Every document carries the declared interests of its named contributors in its front matter, and the central register exists so that a reader can see a person across all documents rather than one at a time. No amendment arises; the provision has been moved from the appendix into the body of the methodology document so that it is findable.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.