Appendix E to Part 240 - Recommended Procedures for Conducting Skill Performance Tests
49:4.1.1.1.34.7.137.1.97 : Appendix E
Appendix E to Part 240 - Recommended Procedures for Conducting
Skill Performance Tests
FRA requires (see § 240.127 and § 240.211) that locomotive
engineers be given a skill performance test prior to certification
or recertification and establishes certain criteria for the conduct
of that test. Railroads are given discretion concerning the manner
in which to administer the required testing. FRA has afforded
railroads this discretion to allow individual railroad companies
latitude to tailor their testing procedures to the specific
operational realities. This appendix contains FRA's recommendations
for the administration of skill performance testing that occurs
during operation of an actual train. It can be modified to serve in
instances where a locomotive simulator is employed for testing
purposes. These recommended practices, if followed, will ensure a
more thorough and systematic assessment of locomotive engineer
performance.
The Need for a Systematic Approach
There are numerous criteria that should be monitored when a
designated supervisor of locomotive engineers is observing a person
to determine whether that individual should be certified or
recertified as a qualified locomotive engineer. The details of
those criteria will vary for the different classes of service,
types of railroads, and terrain over which trains are being
operated. At a minimum, the attention of a designated supervisor of
locomotive engineers should concentrate on several general areas
during any appraisal. Compliance with the railroad's operating
rules, including its safety directives and train handling rules,
and compliance with Federal regulations should be carefully
monitored. But, in order to effectively evaluate employees, it is
necessary to have something against which to compare their
performance. In order to hold a locomotive engineer accountable for
compliance, a railroad must have adequate operating, safety and
train handling rules. Any railroad that fails to have adequate
operating, safety, or train handling rules will experience
difficulty in establishing an objective method of measuring an
individual's skill level. Any railroad that requires the evaluation
of an individual's performance relative to its train handling rules
needs to have established preferred operating ranges for throttle
use, brake application, and train speed. The absence of such
criteria results in the lack of a meaningful yardstick for the
designated supervisor of locomotive engineers to use in measuring
the performance of locomotive engineers. It also is essential to
have a definite standard so that the engineer and any reviewing
body can know what the certification candidate is being measured
against.
Evaluating the performance of certain train operation skills
will tend to occur in all situations. For example, it would be rare
for a designated supervisor of locomotive engineers to observe any
operator for a reasonable period of time and not have some
opportunity to review that engineer's compliance with some basic
safety rules, compliance with basic operating rules, and
performance of a brake test. As the complexity of the operation
increases, so does the number of items that the operator must
comply with. Higher speeds, mountainous terrain, and various signal
systems place increased emphasis on the need for operator
compliance with more safety, operating, and train handling rules.
Accounting for such variables in any universal monitoring scheme
immediately results in a fairly complex system.
FRA therefore recommends that designated supervisors of
locomotive engineers employ a written aid to help record events and
procedures that as a minimum should be observed for when conducting
a skills performance test. FRA is providing the following
information to assist railroads in developing such a written aid so
as to ensure meaningful testing. When conducting a skills
performance test, a designated supervisor of locomotive engineers
should be alert to the following:
- Does the employee have the necessary books (Operating Rules,
Safety Rules, Timetable, etc.)? - Are predeparture inspections
properly conducted (Radio, Air Brake Tests, Locomotive, etc.)? -
Does the employee comply with applicable safety rules? - Does the
employee read the bulletins, general orders, etc.? - Enroute, does
the employee: - Comply with applicable Federal Rules? - Monitor
gauges? - Properly use the horn, whistle, headlight? - Couple to
cars at a safe speed? - Properly control in train slack and buff
forces? - Properly use the train braking systems? - Comply with
speed restrictions? - Display familiarity with the physical
characteristics? - Comply with signal indications? - Respond
properly to unusual conditions? - At the conclusion of the trip,
does the employee: - Apply a hand brake to the locomotives? -
Properly report locomotive defects?
Obviously, the less sophisticated the railroad's operations are,
the fewer the number of identified practices that would be
relevant. Hence, this list should modified accordingly.
The Need for Objectivity, Use of Observation Form
It is essential that railroads conduct the performance skills
testing in the most objective manner possible, whether this testing
is the locomotive engineer's initial qualification testing or
periodic retesting. There will always be some potential for the
subjective views, held by the designated supervisor of locomotive
engineers conducting the testing, to enter into evaluations
concerning the competency of a particular individual to handle the
position of locomotive engineer. Steps can be taken, and need to be
taken, to minimize the risk that personality factors adversely
influence the testing procedure.
One way to reduce the entry of subjective matters into the
qualification procedures is through the use of a document that
specifies those criteria that the designated supervisor of
locomotive engineers is to place emphasis on. The use of an
observation form will reduce but not eliminate subjectivity. Any
skill performance test will contain some amount of subjectivity.
While compliance with the operating rules or the safety rules is
clear in most cases, with few opportunities for deviation, train
handling offers many options with few absolute right answers. The
fact that an engineer applies the train air brakes at one location
rather than a few yards away does not necessarily indicate a
failure but a question of judgment. The use of dynamic braking
versus air brakes at a particular location may be a question of
judgment unless the carrier has previously specified the use of a
preferred braking method. In any case the engineer's judgment, to
apply or not apply a braking system at a given location, is subject
to the opinion of the designated supervisor of locomotive
engineers.
A railroad should attempt to reduce or eliminate such
subjectivity through use of some type of observation or evaluation.
For railroads developing any evaluation form, the areas of concern
identified earlier will not be relevant in all instances. Railroads
that do not have sophisticated operations would only need a short
list of subjects. For example, most smaller railroads would not
require line items pertaining to compliance with signal rule
compliance or the use of dynamic brakes. Conversely, in all
instances the observation forms should include the time and
location that the observer started and ended the observation. FRA
believes that there should be a minimum duration for all
performance skills examinations. FRA allows railroads to select a
duration appropriate for their individual circumstances, requiring
only that the period be “of sufficient length to effectively
evaluate the person.” In exercising its discretion FRA suggests
that the minimums selected by a railroad be stated in terms of a
distance since the examination has to be of a sufficient duration
to adequately monitor the operator's skills in a variety of
situations. FRA also suggests that the format for the observation
form include a space for recording the observer's comments.
Provision for comments ideally would allow for the inclusion of
“constructive criticism” without altering the import of the
evaluation and would permit subjective comments where merited.