NOTE The proper distribution of proficiency test items can present severe problems for some types of material, e g. those which require uninterrupted storage in cold conditions or which should not be exposed to X-rays, shock or vibration. Most types of chemical materials would benefit from air-tight packaging to avoid contamination by atmospheric contaminants, e.g. fuel vapours or engine exhaust gases which can be encountered during transport.

  1. The proficiency testing provider shall specify relevant environmental conditions for the transport of proficiency test items. Where relevant, the proficiency testing provider shall monitor the pertinent environmental conditions of the proficiency test item during transport and assess the impact of environmental influences on the proficiency test item.

  2. In proficiency testing schemes where participants are required to transport the proficiency test items to other participants, documented instructions for this transport shall be supplied.

  3. The proficiency testing provider shall ensure that labels are securely attached to the packaging of

individual proficiency test items and are designed to remain legible and intact throughout the proficiency testing round.

  1. The proficiency testing provider shall follow a procedure to enable the confirmation of delivery of the proficiency test items.

NOTE This could be achieved in accordance with 4.6.1.1 by asking participants to inform the proficiency testing provider if proficiency test items have not been received in line with the schedule of dates provided.

  1. Data analysis and evaluation of proficiency testing scheme results

    1. Data analysis and records

      1. All data processing equipment and software shall be validated in accordance with procedures before being brought into use. Computer system maintenance shall include a back-up process and system recovery plan. The results of such maintenance and operational checks shall be recorded.

      2. Results received from participants shall be recorded and analysed by appropriate methods. Procedures shall be established and implemented to check the validity of data entry, data transfer, statistical analysis, and reporting.

      3. Data analysis shall generate summary statistics and performance statistics, and associated information consistent with the statistical design of the proficiency testing scheme.

      4. The influence of outliers on summary statistics shall be minimized by the use of robust statistical methods or appropriate tests to detect statistical outliers.

      5. The proficiency testing provider shall have documented criteria and procedures for dealing with

test results that may be inappropriate for statistical evaluation, e.g. miscalculations, transpositions and other gross errors.

    1. The proficiency testing provider shall have documented criteria and procedures to identify and manage proficiency test items that have been distributed and are subsequently found to be unsuitable for performance evaluation, e.g. because of inhomogeneity, instability, damage or contamination.

  1. Evaluation of performance

    1. The proficiency testing provider shall use valid methods of evaluation which meet the purpose of the proficiency testing scheme. The methods shall be documented and include a description of the basis for the evaluation. The evaluation of performance shall not be subcontracted (see 5.5.2).

    2. Where appropriate for the purpose of the proficiency testing scheme, the proficiency testing

provider shall provide expert commentary on the performance of participants with regard to the following:

  1. overall performance against prior expectations, taking measurement uncertainties into account;

  2. variation within and between participants, and comparisons with any previous proficiency testing rounds, similar proficiency testing schemes, or published precision data;

  3. variation between methods or procedures;

  4. possible sources of error (with reference to outliers) and suggestions for improving performance;

  5. advice and educational feedback to participants as part of the continual improvement procedures of participants;

  6. situations where unusual factors make evaluation of results and commentary on performance impossible;

  7. any other suggestions, recommendations or general comments; and

  8. conclusions.

NOTE It can be useful to provide individual summary sheets for participants periodically during or after completion of a particular proficiency testing scheme. These can include updated summaries of performance for individual participants over successive proficiency testing rounds of a continuous proficiency testing scheme. Such summaries can be further analysed and trends highlighted, if required,

  1. Reports

    1. Proficiency test reports shall be clear and comprehensive and include data covering the results of all participants, together with an indication of the performance of individual participants. The authorization of the final report shall not be subcontracted (see 5.5.2).

NOTE Where all original data cannot be reported to participants, a summary of the results, e g. in tabulated or graphical form, can be supplied.

  1. Reports shall include the following, unless it is not applicable or the proficiency testing provider has valid reasons for not doing so:

  1. the name and contact details for the proficiency testing provider;

  2. the name and contact details for the coordinator;

  3. the name(s), function(s), and signature(s) or equivalent identification of person(s) authorizing the report;

  4. an indication of which activities are subcontracted by the proficiency testing provider;

  5. the date of issue and status (e.g. preliminary, interim, or final) of the report;

  6. page numbers and a clear indication of the end of the report;

  7. a statement of the extent to which results are confidential;

  8. the report number and clear identification of the proficiency testing scheme;

  9. a clear description of the proficiency test items used, including necessary details of the proficiency test item's preparation and homogeneity and stability assessment;

  10. the participants' results;

  11. statistical data and summaries, including assigned values and range of acceptable results and graphical displays;

  12. procedures used to establish any assigned value;

  13. details of the metrological traceability and measurement uncertainty of any assigned value;

n) procedures used to establish the standard deviation for proficiency assessment, or other criteria for evaluation;

  1. assigned values and summary statistics for test methods/procedures used by each group of participants (if different methods are used by different groups of participants);

  2. comments on participants' performance by the proficiency testing provider and technical advisers;

  3. information about the design and implementation of the proficiency testing scheme;

  4. procedures used to statistically analyse the data;

  5. advice on the interpretation of the statistical analysis; and

  6. comments or recommendations, based on the outcomes of the proficiency testing round.

NOTE For continuous proficiency testing schemes, it can be sufficient to have simpler reports, such that many of the elements in this clause could be excluded from routine reports, but included in proficiency testing scheme protocols or in periodic summary reports that are available to participants.

  1. Reports shall be made available to participants within planned timescales. In sequential proficiency testing schemes, e.g. where the turn-around time may be very long, and in schemes involving perishable materials, preliminary or anticipated results may be provided before final results are disclosed.

NOTE This allows for early investigation of possible error.

  1. The proficiency testing provider shall have a policy for the use of reports by individuals and organizations.

  2. When it is necessary to issue a new or amended report for a proficiency testing scheme, this shall include the following:

  1. a unique identification;

  2. a reference to the original report that it replaces or amends; and

  3. a statement concerning the reason for the amendment or re-issue.

  1. Communication with participants

    1. The proficiency testing provider shall make detailed information available about the proficiency testing scheme. This shall include:

  1. relevant details of the scope of the proficiency testing scheme;

  2. any fees for participation;

  3. documented eligibility criteria for participation;

  4. confidentiality arrangements; and

  5. details of how to apply.

    1. Participants shall be advised promptly by the proficiency testing provider of any changes in proficiency testing scheme design or operation.

    2. There shall be documented procedures for enabling participants to appeal against the evaluation of their performance in a proficiency testing scheme. The availability of this process shall be communicated to proficiency testing scheme participants.

    3. Relevant records of communications with participants shall be maintained and retained, as appropriate.

    4. If the proficiency testing provider issues statements of participation or performance, they shall contain sufficient information to not be misleading.

  1. Confidentiality

    1. The identity of participants in a proficiency testing scheme shall be confidential and known only to persons involved in the operation of the proficiency testing scheme, unless the participant waives confidentiality.

    2. All information supplied by a participant to the proficiency testing provider shall be treated as confidential.

NOTE Participants can elect to waive confidentiality within the proficiency testing scheme for the purposes of discussion and mutual assistance, e.g. to improve performance. Confidentiality can also be waived by the participant for regulatory or recognition purposes. In most instances, the proficiency testing results can be provided to the relevant authority by the participants themselves.

  1. When an interested party requires the proficiency testing results to be directly provided by the proficiency testing provider, the participants shall be made aware of the arrangement in advance of participation.

  2. In exceptional circumstances, when a regulatory authority requires proficiency testing results to be directly provided to the authority by the proficiency testing provider, the affected participants shall be notified of this action in writing.

5 Management requirements

  1. Organization

    1. The proficiency testing provider, or the organization of which it is part, shall be an entity that is legally identifiable and accountable.

    2. It is the responsibility of the proficiency testing provider to carry out its proficiency testing operations in such a way as to meet the requirements of this International Standard and to satisfy the needs of the participants, regulatory authorities and organizations providing recognition.

    3. The management system shall cover work carried out in the proficiency testing provider's permanent facilities, at sites away from its permanent facilities, and in associated temporary facilities.

    4. If the proficiency testing provider is part of an organization performing other activities, then the proficiency testing provider shall identify the responsibilities of key personnel in the organization that have an involvement in or could have influence on the proficiency test activities, in order to identify potential conflicts of interest. Where potential conflicts of interest are identified, procedures shall be put in place to ensure that all activities of the proficiency testing provider are conducted with impartiality.

    5. The proficiency testing provider shall:

  1. have managerial and technical personnel who, irrespective of other responsibilities, have the authority and resources needed to carry out their duties, including the implementation, maintenance and improvement of the management system, and to identify the occurrence of departures from the management system or from the procedures for providing proficiency testing schemes, and to initiate actions to prevent or minimize such departures;

  2. have arrangements to ensure that its management and personnel are free from any undue internal or external commercial, financial and other pressures that may adversely affect the quality of their work;

  3. have policies and procedures to ensure the protection of its participants' confidential information and proprietary rights, including procedures for their protection during electronic storage and transmission;

  4. have policies and procedures to avoid involvement in any activities that might diminish confidence in its competence, impartiality, judgement or operational integrity;

  5. define the organization and management structure, its place in any parent organization, and the relationships between quality management, technical operations and support services;

  6. specify the responsibility, authority, interrelationships and required competence of all personnel who manage, perform or verify work affecting the quality of the operation of proficiency testing schemes;

  7. ensure that the personnel are aware of the relevance and importance of their activities and how they contribute to the achievement of the objectives of the management system;

  8. provide adequate supervision of technical staff, including trainees, by persons familiar with procedures for each activity;

  9. have technical management which has overall responsibility for the technical operations and the provision of the resources needed to ensure the required quality of proficiency testing schemes, including access to the necessary technical expertise and experience in the relevant field of testing, calibration or inspection, as well as statistics, as indicated in 4.4.1.4;

  10. appoint a member of staff as quality manager (named as appropriate) who, irrespective of other duties and responsibilities, shall have defined responsibility and authority for ensuring that the management system is implemented and followed at all times; the quality manager shall have direct access to the highest level of management at which decisions are taken on the proficiency testing provider's policies or resources; and

  11. appoint deputies for key managerial personnel.

NOTE Where proficiency testing providers have a small number of personnel, individuals can have more than one function and it can be impractical to appoint deputies for all major functions.

    1. Top management shall ensure that appropriate communication processes are established within the organization and that communication takes place regarding the effectiveness of the management system.

  1. Management system

    1. The proficiency testing provider shall establish, implement and maintain a management system appropriate to its scope of activities, including the type, range and volume of proficiency testing that it provides.

    2. The proficiency testing provider shall define and document its policies, programmes, procedures and instructions to the extent necessary to assure the quality of all aspects of proficiency testing. The system's documentation shall be communicated to, understood by, available to, and implemented by the appropriate personnel.

NOTE These aspects include, but are not limited to, proficiency test item quality (e.g. homogeneity and stability), characterization (e.g. equipment calibration and method validation), assignment of property values (e.g. use of appropriate statistical procedures), evaluation of participant performance, distribution of proficiency test items, storage and transport procedures, statistical treatment of test results, and reporting.

  1. The proficiency testing provider's management system policies related to quality, including a quality policy statement, shall be defined in a quality manual (named as appropriate). The overall objectives shall be established and reviewed during management review. The quality policy statement shall be issued under the authority of top management. It shall include at least the following:

  1. the management's commitment to the quality of its proficiency testing services to participants and other customers;

  2. the management's statement of the standard of service;

  3. the purpose of the management system related to quality;

  4. a requirement that all personnel concerned with the proficiency testing activities familiarize themselves with the quality documentation and implement the policies and procedures in their work; and

  5. the management's commitment to comply with this International Standard and to continually improve the effectiveness of the management system.

    1. Top management shall provide evidence of commitment to the development and implementation of the management system and to continually improving its effectiveness.

    2. Top management shall communicate to the organization the importance of meeting customer requirements, as well as statutory and regulatory requirements.

    3. The quality manual shall include or make reference to the supporting procedures, including technical procedures. It shall outline the structure of the documentation used in the management system.

    4. The roles and responsibilities of technical management and the quality manager, including their responsibility for ensuring compliance with this International Standard, shall be defined in the quality manual.

    5. Top management shall ensure that the integrity of the management system is maintained when changes to the management system are planned and implemented.

  1. Document control

    1. General

The proficiency testing provider shall establish and maintain procedures to control all documents that form part of its management system (internally generated, or from external sources), such as regulations, standards, other normative documents, proficiency testing scheme protocols, test or calibration methods, or both test and calibration methods, as well as drawings, software specifications, instructions and manuals.

  1. Document approval and issue

    1. All documents issued as part of the management system shall be reviewed and approved for use by authorized personnel prior to issue. A master list or equivalent document control procedure identifying the current revision status and distribution of documents in the management system shall be established and be readily available, in order to prevent the use of invalid or obsolete documents, or both.

T

5.3.2.2

he procedures adopted shall also ensure that:
  1. authorized editions of appropriate documents are available at all locations where activities essential to the effective operation of proficiency testing schemes are performed;

  2. documents are periodically reviewed and updated, as necessary, to ensure continuing suitability and compliance with applicable requirements;

  3. invalid or obsolete documents are promptly removed from all points of issue or use, or otherwise assured against unintended use; and

  4. obsolete documents retained for either legal or knowledge preservation purposes are suitably marked.

5.3.2.3 Management system documents generated by the proficiency testing provider shall be uniquely identified. Such identification shall include the date of issue or revision identification, or both, page numbering, the total number of pages or a mark to signify the end of a document, and the issuing authority/authorities.

5.3.3 Document changes

  1. Changes to documents shall be reviewed and approved by the same function that performed the

original review and approval, unless specifically designated otherwise. The designated personnel shall have access to pertinent background information upon which to base their review and approval.

  1. Where practicable, the altered or new text shall be identified in the document or the appropriate

attachments.

  1. If the proficiency testing provider's document control system allows for the amendment of documents by hand, pending re-issue of the documents, the procedures and authorities for such amendments shall be defined. Amendments shall be clearly marked, initialled and dated. A revised document shall be issued as soon as practicable.

  2. Procedures shall be established to describe how changes in documents maintained in computerized systems are made and controlled.

  1. Review of requests, tenders and contracts

    1. The proficiency testing provider shall establish and maintain policies and procedures for the review of requests, tenders and contracts. These reviews shall ensure that:

  1. the requirements, including those for test and calibration methods, measuring equipment and proficiency test items to be used, are adequately defined, documented and understood;

  2. the proficiency testing provider has the capability and resources to meet the requirements; and

  3. the proficiency testing scheme is technically appropriate.

NOTE 1 This review is particularly important when a customer requests a proficiency testing scheme to be created for a specific purpose or for a different level or frequency of participation from that normally offered.

NOTE 2 This review can be simplified when the proficiency test scheme is fully described in a catalogue or other notice, and the participant is enrolling for a routine shipment.

    1. Records of such reviews, including any changes, shall be maintained. Records shall also be maintained of pertinent discussions with a customer relating to the customer's requirements, or the results of the work during the period of execution of the contract, or both.

    2. The review shall cover all aspects of the request, including any work that is subcontracted by the proficiency testing provider.

    3. The participants and other customers, as appropriate, shall be informed of any deviation in the contract or agreed proficiency testing scheme design.

    4. If a request or contract is amended after the proficiency testing scheme is underway, the same review process shall be repeated and any amendments shall be communicated to all affected personnel.

  1. Subcontracting services

    1. When a proficiency testing provider subcontracts work, the proficiency testing provider shall demonstrate that the subcontractors' experience and technical competence are sufficient for their assigned tasks and that they comply with the relevant clauses of this International Standard and other appropriate standards.

    2. The proficiency testing provider shall not subcontract the planning of the proficiency test scheme (see 4.4.1.2), the evaluation of performance (see 4.7.2.1) or the authorization of the final report (see 4.8.1).