| 5.B |
Documentation of MOT and test tool validation
|
| 7.D |
Checking the testing or validation environment
|
| 5.B.2 |
If, for a given MOT or test tool, there is no suitable validation service available outside the testing laboratory to which accreditation is applicable, and there is no suitable reference implementation that could be used by the testing laboratory to validate the MOT or test tool, then the testing laboratory shall define and document the procedures and methods that it uses to check on the correct operation of the MOT or test tool, and provide evidence that these procedures and methods are applied whenever the MOT or test tool is modified.
|
| Committee |
ICT/1
|
| 9.A |
The validation concept
|
| 7.D.1 |
The laboratory shall make it clear to the client that it is their responsibility to ensure that all equipment supplied by them is available and operating correctly. The laboratory shall ensure that pre-test or pre-validation checks are performed on the equipment, to establish its suitability for purpose.
|
| 7.C |
Remote testing over a network
|
| 5.B.1 |
The procedures for carrying out MOT and test tool validations shall be documented by the laboratory.
|
| 7.B |
Identification of environment
|
| 7.D.4 |
For all software testing and validation, it shall be ensured that any files containing old results or old test programs on the SUT cannot be confused with the current test programs and test or validation results.
|
| 5.6 |
ISO/IEC Guide 25 subclause 5.6 is interpreted in 4.B.
|
| 7.D.3 |
A validation laboratory, or a testing laboratory which conducts its own MOT or test tool validations, shall ensure that the correct version of each reference implementation or other means of validation is used and that they have not been modified in any way that might lead to incorrect validation results.
|
| 9.7 |
No IT&T specific interpretation is required for this subclause.
|
| 9.6 |
ISO/IEC Guide 25 subclause 9.6 is interpreted in 9.A.2, 9.B.1 and 9.E.
|
| 9.5 |
No IT&T specific interpretation is required for this subclause.
|
| 9.4 |
No IT&T specific interpretation is required for this subclause.
|
| 7.6 |
ISO/IEC Guide 25 subclause 7.6 is interpreted in 11 .A.
|
| 9.3 |
ISO/IEC Guide 25 subclause 9.3 is interpreted in 9.A.2 and 9.B.1, and referenced in 5.B.2.
|
| 7.5 |
No IT&T specific interpretation is required for this subclause.
|
| 9.2 |
ISO/IEC Guide 25 subclause 9.2 is interpreted in 5.B and 9.D.1.
|
| 7.D.2 |
A testing laboratory shall ensure that the correct version of each MOT and each test tool is used and that they have not been modified in any way that might lead to incorrect test results.
|
| 9.1 |
ISO/IEC Guide 25 subclause 9.1 is interpreted in 5.A, 9.A.2, 5.B.2 and 9.F.1.
|
| 7.A |
Separation of data partitions
|
| 6.1 |
ISO/IEC Guide 25 subclause 7.2 is interpreted in 6.A.
|
| 8.4 |
ISO/IEC Guide 25 subclause 8.4 is interpreted in 8.A, 8.C, 8.D, and 9.A.2.
|
| 8.3 |
No IT&T specific interpretation is required for this subclause.
|
| 8.2 |
ISO/IEC Guide 25 subclause 8.2 is interpreted in 5.A, 5.B.2, 8.B, 8.C, 9.A.1 and 9.B.1.4.
|
| 8.B.2 |
Thus, the testing laboratory shall document and use adequate procedures, for use whenever any MOT or test tool is suspected or found to contain errors which make it defective or unfit for use. These procedures shall include establishing that there is a genuine error, reporting the error to the appropriate maintenance authority, withdrawing the MOT, test tool or test case(s) from service, as appropriate, correcting the errors, and then re-validating the MOT or test tool, as appropriate.
|
| 8.1 |
ISO/IEC Guide 25 subclause 8.1 is interpreted in 9.B.1.
|
| 6.3 |
No IT&T specific interpretation is required for this subclause.
|
| 8.B.3 |
Similarly, a validation laboratory shall document and use adequate procedures, for use whenever any reference implementation or other means of validation is suspected or found to contain errors which make it defective or unfit for use. These procedures shall include establishing that there is a genuine error, reporting the error to the appropriate maintenance authority, withdrawing the reference implementation or other means of validation from service, as appropriate, correcting the errors, and then re-checking the reference implementation or other means of validation, as appropriate.
|
| 6.2 |
ISO/IEC Guide 25 subclause 7.2 is interpreted in 6.A.
|
| 4.A.1 |
If a commercial implementation (not designed to be a reference implementation) is used by a testing laboratory or validation laboratory as a reference implementation for the purposes of MOT validation within the laboratory, then the adequacy of the technical coverage with respect to the other implementations that are available shall be kept under review by the laboratory, in consultation with the accreditation body. If it is agreed that the technical coverage has become inadequate compared to other implementations, then the commercial implementation should be replaced, within a time period to be agreed with the accreditation body, by one of the following, as appropriate: another implementation with better coverage; a set of implementations from different suppliers; or an implementation which is designed to be a reference implementation.
|
| 7.B.1 |
The laboratory shall obtain from the client and record, as far as possible, the correct and complete identification of the test environment within the SUT (or the validation environment within the MOT). The client shall take responsibility for this identification. In addition, the laboratory shall record the values of any parameters that are used to control the test environment within the SUT during testing (or to control the validation environment within the MOT during validation).
|
| 4.2 |
ISO/IEC Guide 25 subclause 4.2 is interpreted in 4.A and 4.B.
|
| 4.1 |
No IT&T specific interpretation is required for sub clause 4.1 of ISO/IEC Guide 25.
|
| Publisher |
BSI
|
| Title |
Information technology. Interpretation of accreditation requirements in ISO/IEC Guide 25. Accreditation of information technology and telecommunications testing laboratories for software and protocol testing services
|
| 8.B.1 |
If a defect in an item of equipment is such that it impacts only a few specific test cases, then those test cases shall be taken out of service. These test cases shall not be put into service again unless revised procedures are introduced for analysing the results of those test cases to overcome the defect.
|
| Descriptors |
Laboratory testing, Conformity, Calibration, Telecommunication, Laboratory accreditation, Definitions
|
| Identical National Standard Of |
ISO/IEC TR 13233:1995
|
| ISBN |
0 580 26595 1
|
| 7.B.2 |
The test (or validation) environment outside the SUT (or MOT) shall be sufficiently recorded and controlled by the laboratory to ensure, as far as possible, correct and complete identification of the test (or validation) environment outside the SUT (or MOT) at any time.
|
| 7.4 |
ISO/IEC Guide 25 subclause 7.4 is interpreted in 7.A.
|
| 5.A |
Maintenance procedures
|
| 5.A.2 |
A validation laboratory, or a testing laboratory which conducts its own MOT or test tool validations, shall have procedures defining the checking to be performed whenever major or minor changes are made to the reference implementation or other means of validation, in order to ensure that harmonisation is maintained as appropriate with other validation laboratories and that correctness is maintained with respect to the relevant standard (s) or specification (s).
|
| Pages |
56
|
| Status |
Definitive
|
| Publication Date |
1996-11-15
|
| 7.C.1 |
For the purposes of this Technical Report, remote testing is testing which is conducted over a network, with the SUT connected to but separate from the network, and the network being part of the test environment.
|
| 7.A.3 |
Insofar as the IUT is integrated in a system also provided by the client, the laboratory shall ensure that there is no interference from other activities in the environment part of the SUT which could affect the test or validation results, and shall instruct the client to take the necessary actions to achieve this.
|
| 7.A.2 |
The testing laboratory and client shall agree in writing what constitutes the IUT and what constitutes the environment within the system under test (SUT).
|
| 7.A.1 |
There shall be a clear separation of test tools, the implementation under test (IUT) and the rest of the environment, even when part of that environment is within the same computer system as the IUT.
|
| 8.D |
Identification of equipment
|
| 8.B |
Procedures for handling errors in the MOT and test tools
|
| 7.3 |
No IT&T specific interpretation is required for this subclause.
|
| 5.2 |
ISO/IEC Guide 25 subclause 5.2 is interpreted in 5.A and 5.B.
|
| 7.C.2 |
In the context of remote testing over a network, the laboratory shall use test methods and procedures that ensure that the behaviour of the network does not invalidate the test or validation results.
|
| 8.A |
Means of testing and test tool validation records
|
| 5.A.1 |
A testing laboratory shall have procedures defining the checking to be performed whenever major or minor changes are made to the MOT or other test tools, in order to ensure that harmonisation is maintained as appropriate with other testing laboratories and that correctness is maintained with respect to the relevant standard(s) or specification(s).
|
| 6.A |
Maintaining Competence
|
| 4.B |
Proficiency testing
|
| 4.A |
Use of commercial reference implementations
|
| 6.A.1 |
The testing laboratory shall have a procedure to define how to maintain competence of its test operators, particularly in the absence of clients for a given testing or validation service.
|
|
Nevertheless, the requirements from ISO/IEC Guide 25 quoted here are applicable to software and protocol testing. When taken together it can be seen that they require some means of checking the validity of the MOT and test tool Standard Number BS ISO/IEC TR 13233:1995 Title Information technology. Interpretation of accreditation requirements in ISO/IEC Guide 25. Accreditation of information technology and telecommunications testing laboratories for software and protocol testing services Status Definitive Publication Date 1996-11-15 Identical National Standard Of ISO/IEC TR 13233:1995 Descriptors Laboratory testing, Conformity, Calibration, Telecommunication, Laboratory accreditation, Definitions ICS 35.020 Information technology (IT) in general, 03.120.20 Product and company certification. Conformity assessment Committee ICT/1 ISBN 0 580 26595 1 Publisher BSI Pages 56
|
| 8.A.1 |
A testing laboratory shall maintain a record of all MOT validations and re-validations carried out.
|
| 1 |
Scope
|
| 8.C |
MOT and test tool maintenance
|
| 8.A.2 |
If the MOT validations are carried out by the testing laboratory itself, its records shall include the reasons for the test cases being run, date, environmental information if appropriate, and a summary of the results obtained plus the detail of any discrepancies from the expected results. When MOT validation is made by the testing laboratory itself using a reference implementation, the requirements also mean that the testing laboratory shall document fully the expected results (i.e. previously obtained results) from using the full conformance test suite to test the nominated reference implementation.
|
| 5.5 |
No IT&T specific interpretation is required for this subclause.
|
| 8.C.1 |
A testing laboratory shall keep a record of the design and maintenance authority for each MOT, test tool and executable test suite.
|
| 8.A.3 |
If the MOT validations are carried out by one or more external organisations, the testing laboratory shall keep in its records the validation reports, each including the identity of the organisation(s) carrying out the validation, date(s), and results of the validation.
|
| 7.2 |
ISO/IEC Guide 25 subclause 7.2 is interpreted in 7.A, 7.B, 7.C, 7.D, 11.A.2 and 12.A.
|
| 5.4 |
No IT&T specific interpretation is required for this subclause.
|
| 8.C.2 |
A validation laboratory, or a testing laboratory which conducts its own MOT or test tool validations, shall keep a record of the design and maintenance authority for each reference implementation and each tool used in other means of validation.
|
| 7.1 |
No IT&T specific interpretation is required for this subclause.
|
| 5.3 |
No IT&T specific interpretation is required for this subclause.
|
| 9 |
Validation and traceability (Measurement traceability and calibration)
|
| 4.B.1 |
If a laboratory claims to offer a harmonised testing or validation service, it shall provide evidence of its participation in the relevant inter-laboratory comparisons to ensure that the declared harmonisation is achieved and maintained.
|
| 2 |
References
|
| 8 |
Equipment and reference materials
|
| 5.1 |
No IT&T specific interpretation is required for this subclause.
|
| Standard Number |
BS ISO/IEC TR 13233:1995
|
| 7 |
Accommodation and environment
|
| 9.A.1 |
In IT&T, the concept of MOT validation is used to meet the ISO/IEC Guide 25 requirements for verification of equipment.
|
| 6 |
Personnel
|
| 5 |
Quality system, audit and review
|
| 4 |
Organization and management
|
| 3 |
Definitions
|
| ICS Codes |
35.020 - Information technology (IT) in general
|
