Draft certificate-of-analysis minimum data standard — submissions
The 13 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
13 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The document set should be published in translation
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
Nothing requires a laboratory to demonstrate ongoing performance
The respondent states that suitability criteria assessed within a run demonstrate that the run worked and not that the laboratory performs consistently over time.
The respondent proposes participation in a proficiency scheme as a condition of conformance.
The secretariat accepts this submission in part. A trend requirement on system suitability data is added. Proficiency scheme participation is not made a condition of conformance, for the same reason competence requirements are not.
The standard now requires that system suitability results be trended across runs and that a determination report whether the trend was within the laboratory's stated limits, which places the demonstration inside the document the Institute can read.
The search date is not on the face of the document
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
The standard would be applied more consistently with a worked example
The respondent states that the requirement most often misapplied in practice is the one the draft states most briefly, and that a worked calculation would settle it.
The respondent proposes a worked example showing the same data treated correctly and incorrectly, with both results stated.
The secretariat accepts this submission. An example showing only the correct treatment does not show the reader what they are doing wrong.
A worked example is added showing the same primary data treated two ways, with both resulting figures stated and the difference between them quantified, so that the consequence of the misapplication is visible rather than described.
Nothing in the standard addresses transfer of the method between laboratories
The respondent states that determinations under this standard will be performed by different laboratories on the same material and that without transfer criteria their results are not comparable.
The respondent proposes a transfer protocol with acceptance criteria for inter-laboratory agreement.
The secretariat accepts this submission in part. Acceptance criteria for agreement between laboratories are added. A full transfer protocol is not, because transfer is a quality-system activity rather than a determination and the series does not address quality systems.
The standard now states the maximum difference between laboratories analysing the same sample at which results are treated as agreeing, and requires that a comparison across laboratories state whether that criterion was met.
Results are reported without an estimate of measurement uncertainty
The respondent, an analytical chemist, states that a figure reported to one decimal place implies an uncertainty the standard nowhere estimates, and that two determinations differing within the uncertainty will be read as a difference.
The respondent proposes that an uncertainty budget be required for every determination.
The secretariat accepts this submission in part. A repeatability figure and a stated reporting precision are required. A full uncertainty budget is not, because the standard is written to be usable by laboratories that do not maintain one and the requirement would exclude them without improving the reading.
Every determination now reports the repeatability observed in the run, the reporting precision, and a statement that two results differing by less than the stated repeatability are not distinguishable by this method.
Quantitative claims are reproduced without the method that produced them
Several figures in the draft are quoted from sources that determined them by different methods. A figure obtained by one determination and a figure obtained by another are not comparable, and the draft places them in the same sentence without distinguishing them.
The respondent, an analytical chemist, proposes that every quantitative claim carry the method that produced it at the point of use rather than in the reference.
The secretariat accepts this submission. Placing two figures side by side is an implicit claim that they are the same kind of quantity, and in the cases identified they were not.
Every quantitative claim now carries the determination that produced it at the point of use, and figures obtained by non-comparable methods are no longer presented in the same row or sentence.
References should carry a persistent identifier for every cited source
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent, who works in a library setting, states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
The standard does not say what to do with a result outside the expected range
The respondent states that the draft describes how to obtain a result and stops, and that the handling of an unexpected result is where the integrity of a determination is decided.
The respondent proposes that repetition be permitted only under stated conditions and that every determination performed be reported, including those superseded.
The secretariat accepts this submission. A standard silent on repetition permits the selective reporting it exists to prevent.
The standard now requires that every determination performed on a sample be reported, that a repeat be permitted only where an assignable cause is documented, and that the assignable cause be stated on the face of the report.
Acceptance criteria are expressed in terms specific to one instrument family
The respondent states that two criteria are expressed in units reported by one manufacturer's software and not by others, which makes conformance a function of the instrument rather than of the determination.
The respondent proposes that criteria be expressed in instrument-independent quantities.
The secretariat accepts this submission in part. Both criteria are restated in instrument-independent terms. The conversion from the manufacturer-specific quantity is given rather than removed, because laboratories will continue to read it from their software.
The acceptance criteria are now expressed in instrument-independent quantities, with the conversion from the common manufacturer-specific reporting units given in a note so that a laboratory can check conformance without recalculating by hand.
Absence of evidence is presented in a form a reader will take as negative evidence
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
Terms are used in senses that differ from established pharmacopoeial usage
The respondent, employed by a national competent authority and writing in a personal capacity, identifies three terms used in the draft in senses that differ from their established pharmacopoeial meanings.
The respondent proposes that established usage be followed or that the departure be stated explicitly at first use.
The secretariat accepts this submission. A term used in a private sense inside a document that will be read alongside compendial texts is a source of avoidable error.
The three terms now follow established pharmacopoeial usage, the glossary records the definition and its source, and where the Institute uses a term in a narrower sense the departure is stated at first use in the standard.
The limitations section states what the method cannot do but not what a reader will wrongly conclude
The limitations section is accurate and abstract. The respondent states that the misreadings this standard should prevent are specific and predictable, and that naming them would be more use than a general statement of scope.
The respondent proposes that the limitations section list the specific inferences a conformant result does not support.
The secretariat accepts this submission. A limitation stated abstractly is a limitation a reader will not apply to the case in front of them.
The limitations section now lists the specific conclusions a conformant determination does not support, each stated as a sentence a reader could otherwise have written, and the list is placed before the references rather than in an annex.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.