Appendix D to Part 236 - Independent Review of Verification and Validation
49:4.1.1.1.30.11.125.1.67 : Appendix D
Appendix D to Part 236 - Independent Review of Verification and
Validation
(a) This appendix provides minimum requirements for independent
third-party assessment of product safety verification and
validation pursuant to subpart H or subpart I of this part. The
goal of this assessment is to provide an independent evaluation of
the product manufacturer's utilization of safety design practices
during the product's development and testing phases, as required by
any mutually agreed upon controlling documents and standards and
the applicable railroad's:
(1) Railroad Safety Program Plan (RSPP) and Product Safety Plan
(PSP) for processor based systems developed under subpart H or,
(2) PTC Product Development Plan (PTCDP) and PTC Safety Plan
(PTCSP) for PTC systems developed under subpart I.
(b) The supplier may request advice and assistance of the
reviewer concerning the actions identified in paragraphs (c)
through (g) of this appendix. However, the reviewer shall not
engage in any design efforts associated with the product, the
products subsystems, or the products components, in order to
preserve the reviewer's independence and maintain the supplier's
proprietary right to the product.
(c) The supplier shall provide the reviewer access to any and
all documentation that the reviewer requests and attendance at any
design review or walkthrough that the reviewer determines as
necessary to complete and accomplish the third party assessment.
The reviewer may be accompanied by representatives of FRA as
necessary, in FRA's judgment, for FRA to monitor the
assessment.
(d) The reviewer shall evaluate the product with respect to
safety and comment on the adequacy of the processes which the
supplier applies to the design and development of the product. At a
minimum, the reviewer shall compare the supplier processes with
acceptable validation and verification methodology and employ any
other such tests or comparisons if they have been agreed to
previously with FRA. Based on these analyses, the reviewer shall
identify and document any significant safety vulnerabilities which
are not adequately mitigated by the supplier's (or user's)
processes. Finally, the reviewer shall evaluate and document the
adequacy of the railroad's
(1) RSPP, the PSP, and any other documents pertinent to a
product being developed under subpart H of this part; or
(2) PTCDP and PTCSP for systems being developed under subpart I
of this part.
(e) The reviewer shall analyze the Hazard Log and/or any other
hazard analysis documents for comprehensiveness and compliance with
applicable railroad, vendor, supplier, industry, national, and
international standards.
(f) The reviewer shall analyze all Fault Tree Analyses (FTA),
Failure Mode and Effects Criticality Analysis (FMECA), and other
hazard analyses for completeness, correctness, and compliance with
applicable railroad, vendor, supplier, industry, national and
international standards.
(g) The reviewer shall randomly select various safety-critical
software, and hardware modules, if directed by FRA, for audit to
verify whether the requirements of the applicable railroad, vendor,
supplier, industry, national, and international standards were
followed. The number of modules audited must be determined as a
representative number sufficient to provide confidence that all
unaudited modules were developed in compliance with the applicable
railroad, vendor, supplier, industry, national, and international
standards.
(h) The reviewer shall evaluate and comment on the plan for
installation and test procedures of the product for revenue
service.
(i) The reviewer shall prepare a final report of the assessment.
The report shall be submitted to the railroad prior to the
commencement of installation testing and contain at least the
following information:
(1) Reviewer's evaluation of the adequacy of the PSP in the case
of products developed under subpart H, or PTCSP for products
developed under subpart I of this part, including the supplier's
MTTHE and risk estimates for the product, and the supplier's
confidence interval in these estimates;
(2) Product vulnerabilities, potentially hazardous failure
modes, or potentially hazardous operating circumstances which the
reviewer felt were not adequately identified, tracked, mitigated,
and corrected by either the vendor or supplier or the railroad;
(3) A clear statement of position for all parties involved for
each product vulnerability cited by the reviewer;
(4) Identification of any documentation or information sought by
the reviewer that was denied, incomplete, or inadequate;
(5) A listing of each applicable vendor, supplier, industry,
national, or international standard, procedure or process which was
not properly followed;
(6) Identification of the software verification and validation
procedures, as well as the hardware verification validation
procedures if deemed appropriate by FRA, for the product's
safety-critical applications, and the reviewer's evaluation of the
adequacy of these procedures;
(7) Methods employed by the product manufacturer to develop
safety-critical software;
(8) If deemed applicable by FRA, the methods employed by the
product manufacturer to develop safety-critical hardware by
generally acceptable techniques;
(9) Method by which the supplier or railroad addresses
comprehensiveness of the product design which considers the safety
elements listed in paragraph (b) of appendix C to this part.
[75 FR 2720, Jan. 15, 2010]