Draft standard CEI-MS-15: Protocol for independent… — submissions
The 12 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
12 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
Nothing requires the report to state who performed the determination
The respondent states that certificates in circulation frequently reproduce results generated by a third party without identifying it, and that a reader cannot tell whether the releasing party performed the work.
The respondent proposes that the performing laboratory be a required field.
The secretariat accepts this submission. Attribution of a determination is a precondition of assessing it.
The report now requires the identity of the laboratory that performed the determination, its accreditation status where claimed, and a statement of whether the releasing party is the performing party, with the fields classified as critical.
The additional determinations required would materially increase release testing cost
The submission is made on behalf of a manufacturer. It states that the draft adds two determinations to those in existing compendial practice, and estimates the additional analytical burden per batch.
The submission does not dispute the analytical reasoning and asks that one of the two be made conditional rather than universal.
The secretariat accepts this submission in part. One determination is made conditional on a stated risk factor. The other is retained because the evidence supplied by the respondent does not bear on whether it is necessary, only on what it costs.
One of the two determinations is now required only where the stated risk factor is present, with the factor defined in the scope, and the other is retained as a universal requirement with the reasoning stated. The submission is identified as an industry submission.
A determination does not record the version of the standard it was made under
The respondent states that the standard will be revised and that results reported under different versions will circulate together, indistinguishable from one another.
The respondent proposes that the standard identifier and version be carried on the face of every determination.
The secretariat accepts this submission. A conformance claim against an unspecified version is not a conformance claim.
Every determination reported under this standard now carries the standard identifier and version on its face, and a claim of conformance that does not identify the version is recorded by the Institute as not conformant.
The search date is not on the face of the document
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
A single determination is permitted where duplicates would be inexpensive
The respondent states that the draft permits a result to be reported from one injection, and that the marginal cost of a duplicate is small relative to the value of detecting a preparation error.
The respondent proposes that duplicate determinations be mandatory.
The secretariat accepts this submission in part. Duplicate preparation and injection are required, and a criterion for agreement between them is stated. The proposal to require full duplicate sample preparation on separate days is declined as disproportionate.
The standard now requires duplicate preparation and injection with a stated maximum difference between the two, and requires that a difference exceeding it be investigated before any figure is reported.
The standard assumes a reference standard of known assigned content
The respondent states that for many compounds in scope no reference standard with an assigned content exists, and that the standard as drafted has nothing to say to a laboratory in that position.
The respondent proposes that the standard state what can be determined without one and how such a determination should be labelled.
The secretariat accepts this submission. A standard applicable only where a reference standard exists is silent in the cases most in need of it.
The standard now states which determinations remain available without a reference standard of assigned content, requires that such determinations be labelled as relative rather than absolute, and requires the assumption used in place of an assigned content to be stated.
The same concept is given three different names in one document
The draft refers to the same quantity as a response rate, a responder proportion and a categorical outcome in different sections. The respondent, who works in health-technology assessment, states that a reader cannot tell whether the three refer to one thing or to three.
The respondent proposes that the glossary term be used at every occurrence and that the glossary entry be linked at first use in each section rather than only at first use in the document.
The secretariat accepts this submission. The variation was stylistic and its cost to the reader exceeds any benefit.
A single term is now used throughout for each defined concept, and the glossary entry is linked at first use within each section rather than once per document.
Terms are used in senses that differ from established pharmacopoeial usage
The respondent, employed by a national competent authority and writing in a personal capacity, identifies three terms used in the draft in senses that differ from their established pharmacopoeial meanings.
The respondent proposes that established usage be followed or that the departure be stated explicitly at first use.
The secretariat accepts this submission. A term used in a private sense inside a document that will be read alongside compendial texts is a source of avoidable error.
The three terms now follow established pharmacopoeial usage, the glossary records the definition and its source, and where the Institute uses a term in a narrower sense the departure is stated at first use in the standard.
The standard should require documented operator competence
The respondent states that the determination described requires judgement, particularly at the integration step, and that a competence requirement would improve consistency more than any change to the criteria.
The respondent proposes that conformance be conditional on documented operator training.
The secretariat does not accept this submission. Operator competence is a quality-system matter governed by accreditation requirements the Institute does not administer and cannot verify.
The standard states performance and remains silent on competence. Where judgement affects the result, as at the integration step, the standard requires the judgement to be recorded rather than the operator to be qualified, which is the part the Institute can check from a report. The submission remains published in full.
A conformance claim is not defined and will be made loosely
The respondent states that the draft nowhere defines what it means for a determination to conform, and that in the absence of a definition the phrase will be applied to determinations that meet some of the requirements.
The respondent proposes an explicit definition and a requirement that partial conformance be described as such.
The secretariat accepts this submission. An undefined conformance claim is a marketing phrase.
Conformance is now defined as meeting every requirement classified as critical together with the stated system suitability criteria, partial conformance must be described by naming the requirements not met, and a claim of conformance made without the sampling statement is not conformant.
A sortable table implies a comparison the underlying data do not support
The draft presents a sortable table whose columns are drawn from sources of differing quality. The respondent states that sorting on such a column produces an ordering that looks like a ranking and is not one.
The respondent proposes that sorting be disabled on any column whose values are not commensurable.
The secretariat notes this submission and records that the point is correct in principle.
No amendment arises here because every sortable table in the document set already carries a standing statement above it that the ordering is not a ranking and that the values in each column are commensurable only where the column header says so. The proposal to disable sorting was considered and not adopted, because a reader who cannot sort a table generally sorts it elsewhere and without the statement.
The document set should be published in translation
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.