What Leads to Practical Exam Failures: Threat and Error Management

I am often asked what areas applicants struggle the most on, and where most failures occur. While the answer to this question ranges dramatically from the type of exam to the student, it usually begins with a threat that leads to an error, and eventually an undesired aircraft state and/or exceedance from the ACS or PTS standard. While the concept of Threat and Error Management is somewhat foreign to the new private pilot, it applies very well to our daily operations and I see it in action on a regular basis as a designated pilot examiner.

I will start by discussing the definitions:

Threats are events that occur beyond the influence of the pilot(s), increase operational complexity, and which must be managed to maintain the margins of safety.

Errors are defined actions or inactions by the flight crew that lead to deviations from pilot intentions or expectations. 

Undesired Aircraft States are pilot induced aircraft position or speed deviations, misapplication of flight controls, or incorrect systems configuration, associated with a reduction in margins of safety.

While the entire Threat and Error Management concept is lengthy, I will simplify it for this brief discussion. Threats exist everywhere in our operation. Weather, a busy controller, a busy airport, and heavy training traffic are just a few that we often encounter on practical exams. These are threats that can lead to errors if not caught. As part of my briefing to applicants, I often tell them that “perfection is not the standard.” Even as airline pilots we are not immune to making mistakes and errors. However, we are trained to use our toolbox which includes procedures, crew resource management, and aeronautical decision making to keep us from ending up in an Undesired Aircraft State (UAS). While the UAS definition can vary, most deviations from the ACS standard for the purposes of the practical exam are considered a UAS. The importance of this model is to realize that uncorrected errors lead to a UAS, and recovery is paramount to avoid exceeding the standards. An uncorrected UAS will likely lead to an accident, incident or violation; and for practical exam purposes, a notice of disapproval.

That being said, the reason I bring this up is because the areas I see applicants struggle the most on are threats that lead to errors. Sometimes this is because of a lack of training, nerves, fatigue, or a plethora of other factors. 

I will bring up an example to illustrate this point:

On a practical exam, an applicant failed to descend on the glideslope on an ILS approach. Realizing the mistake after trying to descend and recognizing the approach was unstable due to being so high, the applicant used good judgement and decided to execute a missed approach. However, the applicant had never practiced a missed approach from an altitude above the minimums on the plate, and was confused about when to turn and what altitude to fly. In this scenario, a busy controller and task saturation led to a late descent, which resulted in an error of being too high and unstablized, and the lack of knowledge and training led to an undesired aircraft state of flying past the missed approach point at an inappropriate altitude.

The ACS is meant to be scenario based, but in real life instrument procedures are rarely like we practice. I have rarely gone missed off an instrument approach due to weather. After all, we normally don’t attempt an approach when the real-life weather is below minimums. There are exceptions to this rule, but generally we would look at diverting when weather drops below minimums. I have been instructed to execute a missed approach by ATC more times than executing a missed approach due to weather.

If applicants apply the Threat and Error Management concepts to their single pilot operation and practical exam, they would see improvement in their performance. Understanding the threats they encounter and having tools to recognize and recover from errors is key to mitigating problems before they become tolerance exceedances or undesired aircraft states.

The key takeaway from this article is that real life (and even practical exams) are not always like the training environment. We try to do our best to put controls in place to make the evaluation as easy as possible; but we can’t control variables such as busy controllers, traffic, and other situations that lead less than desirable conditions. By recognizing these threats an applicant can work to increase their situational awareness and develop methods to reducing errors and recovering from a UAS. It’s usually threats that lead to errors and errors that lead to UAS that make a practical exam unsatisfactory. This applies to many of the practical exams I have given, and my hope is that applicants can prepare for real world flying in their training. After all, my job is the gatekeeper before the real world, and the ACS demands that the applicant is prepared to exercise those privileges at the end of the practical exam.

If you want more reading on Threat and Error Management, check out:

Free FAA Safety Presentation on Threat and Error Management (for WINGS Credit): https://www.faasafety.gov/gslac/ALC/course_content.aspx?cID=556&sID=983

https://skybrary.aero/articles/threat-and-error-management-tem

Sarah is currently a FAA Safety Team Lead Representative, NAFI Master Instructor, Gold Seal flight instructor, and 737pilot for a Major U.S. airline. Sarah holds an ATP, CFI, CFII, MEI and has flown over 7300 hours. She holds a pilot license in 4 different countries (USA, Canada, Belize and Iceland – EASA) and has flown over 150 different types of airplanes in 25 different countries including oceanic crossings in small aircraft. Since aviation for work isn’t enough, she also lives in a hangar home on the west side of Houston! Although much of her flying is now professional in nature, she enjoys flying her Super Cub, Patches, on her days off. As a regular attendee of Oshkosh and local fly-ins, she enjoys the company and camaraderie that general aviation brings and is passionate about aviation safety and flight training.

A DPE’s Take on Deciphering the ACS: Emergency Descent

When I begin my briefings with applicants, I often begin with stating that the ACS is the standard. Sometimes I have students and flight schools asking me how I want to see certain maneuvers, and I always point them to FAA published materials to answer their question. There is no certain “way” that I want to see a maneuver – if the maneuver is compliant with the ACS and the applicant demonstrates proficiency, risk management, and good judgement – then it is satisfactory. However, there are times where the ACS does leave a lot to the discretion of the applicant and/or examiner, so I figured I’d discuss the process for determining how to complete a task.

The question came up on a group on social media where a student was asking the community how to perform the Emergency Descent maneuver (PA.IX.A). The student began with saying that his instructor had told him to accelerate to VNE (never exceed speed). He then asked if this was how you were supposed to complete the maneuver, to which I would say as a DPE “it depends.” The ACS task states that the applicant demonstrates the ability to: 

• • PA.IX.A.S1 Clear the area. 
• • PA.IX.A.S2 Establish and maintain the appropriate airspeed and configuration appropriate to the scenario specified by the evaluator and as covered in POH/AFM for the emergency descent. 
• • PA.IX.A.S3 Maintain orientation, divide attention appropriately, and plan and execute a smooth recovery. 
• • PA.IX.A.S4 Use bank angle between 30° and 45° to maintain positive load factors during the descent. 
• • PA.IX.A.S5 Maintain appropriate airspeed +0/-10 knots, and level off at a specified altitude ±100 feet. 
• • PA.IX.A.S6 Complete the appropriate checklist.

The emphasis on “how” really falls under skills section 2, where the ACS directs the applicant to: “Establish and maintain the appropriate airspeed and configuration appropriate to the scenario specified by the evaluator and as covered in POH/AFM for the emergency descent.” One important thing to note is that the evaluator should be providing a scenario which requires the use of this maneuver. If the examiner doesn’t specify, then it might be prudent of the applicant to ask since the emergency may impact the configuration chosen by the applicant. That being said, many small general aviation airplanes do not have a specific emergency descent procedure detailed in the POH/AFM. So the question becomes, how do we perform the maneuver without direct guidance from the manufacturer? 

While not an all-encompassing list, the accomplishment of any task generally falls within the following guidance or priority: 

1. Airplane manufacturer specifies through the POH/AFM.
2. Approved procedures by the operating certificate holder (such as a 121 or 135 operator, or 142 training center approved syllabus).
3. Recommended procedures through FAA published material (Advisory Circular, Airplane Flying Handbook, etc).

When we don’t have options 1 or 2, we then must revert to option 3. There are several resources for the maneuver through FAA published material.

The Airplane Flying Handbook, Chapter 17, discusses the maneuver: “Emergency descent training should be performed as recommended by the manufacturer, including the configuration and airspeeds. Except when prohibited by the manufacturer, the power should be reduced to idle, and the propeller control (if equipped) should be placed in the low pitch (or high revolutions per minute (rpm)) position… The pilot should not allow the airplane’s airspeed to pass the never-exceed speed (VNE), the maximum landing gear extended speed (VLE), or the maximum flap extended speed (VFE), as applicable. In the case of an engine fire, a high airspeed descent could blow out the fire. However, the weakening of the airplane structure is a major concern and descent at low airspeed would place less stress on the airplane. If the descent is conducted in turbulent conditions, the pilot must also comply with the design maneuvering speed (VA) limitations. The descent should be made at the maximum allowable airspeed consistent with the procedure used. This provides increased drag and, therefore, the loss of altitude as quickly as possible. The recovery from an emergency descent should be initiated at a high enough altitude to ensure a safe recovery back to level flight or a precautionary landing.”

Advisory circular AC 25-20 also discusses emergency descent procedures specific to pressurized aircraft. While pressurized aircraft operations are generally not evaluated through this maneuver on a private pilot practical exam, it’s important that an applicant understands WHY we would use an emergency descent procedure and can accomplish it based on the scenario presented by the evaluator. As mentioned above, turbulence and type of emergency (such as a fire) can and will dictate the speeds and configuration to use.

So the question still persists: what speed and configuration to use when we don’t have manufacturer’s guidance?

Many students often don’t read the foreword and after words of the ACS. In the very first page after the table of contents, the ACS goes on to say:

“Safe operations in today’s National Airspace System (NAS) require integration of aeronautical knowledge, risk management, and flight proficiency standards…The ACS integrates the elements of knowledge, risk management, and skill listed in 14 CFR part 61 for each airman certificate or rating…The flight portion of the practical test requires the applicant to demonstrate knowledge, risk management, flight proficiency, and operational skill in accordance with the ACS.”

At the end of the ACS, there is a paragraph on Aeronautical Decision-Making:

“Throughout the practical test, the evaluator must assess the applicant’s ability to use sound aeronautical decision-making procedures in order to identify hazards and mitigate risk. The evaluator must accomplish this requirement by reference to the risk management elements of the given Task(s), and by developing scenarios that incorporate and combine Tasks appropriate to assessing the applicant’s risk management in making safe aeronautical decisions.”

As an examiner, I’m looking for mastery of the ACS task through evaluation. Risk management and decision making is paramount to the successful execution of any task, and I look for that in every maneuver. If an applicant chooses a configuration and airspeed that is appropriate for the scenario according to the manufacturer, approved course/operation, or FAA published guidance; then that maneuver will be satisfactory as long as all of the elements of the ACS task are met.

References:

Click to access AC25-20.pdf

Click to access 19_afh_ch17.pdf

Click to access private_airplane_acs_change_1.pdf

Sarah is currently a FAA Safety Team Lead Representative, NAFI Master Instructor, Gold Seal flight instructor, and 737 pilot for a Major U.S. airline. Sarah holds an ATP, CFI, CFII, MEI and has flown over 7300 hours. She holds a pilot license in 4 different countries (USA, Canada, Belize and Iceland – EASA) and has flown over 150 different types of airplanes in 25 different countries including oceanic crossings in small aircraft. Since aviation for work isn’t enough, she also lives in a hangar home on the west side of Houston! Although much of her flying is now professional in nature, she enjoys flying her Super Cub, Patches, on her days off. As a regular attendee of Oshkosh and local fly-ins, she enjoys the company and camaraderie that general aviation brings and is passionate about aviation safety and flight training.

Designated Pilot Examiner Hot Spots: Where Applicants Struggle

I often get asked on what areas do I see people struggle the most. While performance can vary greatly from geographic region to flight schools to airplanes, I do see some common themes that make the exams more challenging for the applicants, and have even resulted in notices of disapproval. I am creating a blog to highlight some “hot spots” where applicants struggle.

1. The #1 issue that I see comes from EFBs. Most of my applicants use an EFB. They often flight plan their cross country on it, and use it for the oral exam when I ask questions. However, while most of them can navigate to a point, often times the applicant doesn’t know how to use functions other than looking up frequencies or navigating. For example, when someone plans a cross country on their EFB through a restricted area or MOA, I often ask them the altitudes and hours of the airspace, in addition to any requirements to fly through it. This has often stumped the applicant, not realizing that holding their finger on it will give them the answer most of the time. Often times I see them revert back to paper charts in this instance. Additionally, locating the AF/D, locating weather charts on the EFB and layers to show SIGMET/AIRMET, and finding regulations are some of the other top areas that people struggle with. The DPE recommendation: If you plan to use an EFB, know how to use it!
2. Another top issue that I see comes from only having a rote understanding of a topic. The ACS is specifically designed to be scenario based. Most people can answer a question when asked directly, such as how many landings do you need to be current. However, when presented with a scenario, they struggle to come up with the answer. While the published oral exam guides are great, insist that your CFI (or if you are a CFI) ask you scenario based questions instead of just asking for a regulatory answer.
3. Understanding maintenance and PIC responsibility. Most applicants can answer questions such as what inspections are required, but there seems to be a general lack of understanding that while PIC, you are responsible for airworthiness. Sometimes the applicant has never seen an airplane logbook until the day of the test. An FAA Safety Briefing put it so elegantly: “Who is responsible for the airworthiness of an aircraft? It is tempting to say it’s the mechanic who worked on the airplane, but in fact, 14 CFR 91.403(a) says the owner/operator is primarily responsible for maintaining the aircraft in an airworthy condition. This includes Airworthiness Directive (AD) compliance, and 14 CFR section 91.7 says no person may operate a civil aircraft unless it is in an airworthy condition.”

These are just a few trends that I’ve seen from oral exams, and I will publish more on the flight portion shortly. One of the biggest ways to impress your examiner and set the mood for the exam is to come prepared and follow all instructions on what to do/bring! Chasing down paperwork or showing up without a flight plan completed are ways to show that you’re not ready for a practical exam!

Sarah is currently a Designated Pilot Examiner, FAA Safety Team Lead Representative, NAFI Master Instructor, Gold Seal flight instructor, and 737 pilot for a Major U.S. airline. Sarah holds an ATP, CFI, CFII, MEI and has flown over 7300 hours. She holds a pilot license in 4 different countries (USA, Canada, Belize and Iceland – EASA) and has flown over 150 different types of airplanes in 25 different countries including oceanic crossings in small aircraft. She continues to stay involved in general aviation through her leadership roles and volunteering for different aviation organizations. Although much of her flying is now professional in nature, she enjoys flying her Super Cub, Patches, on her days off. As a regular attendee of Oshkosh and local fly-ins, she enjoys the company and camaraderie that general aviation brings and is passionate about aviation safety and flight training.

Why aviation insurance rates are going up and what YOU can do to get better rates

I recently did some research for an upcoming aviation safety presentation on the top 10 causes of aviation insurance claims. I had the opportunity to sit down with a major carrier’s claims and underwriting departments to get some answers.

Insurance carriers insure risks across the spectrum – anything from single engine planes used for pleasure all the way up to airline and space. When a loss occurs in space, for example, it affects everyone’s rates. We went from a soft to a hard market, and the market correction has been challenging for everyone.
Over the past few years we have been continuing to see a rise of general aviation accidents. When I asked the insurance company what they think is the leading cause of claims, they said recent experience, proficiency and time in make and model.

This brings us to our challenge: you need experience to get experience. For the most part, insurance companies are not allowing people without make and model experience to be insured without training. This doesn’t mean you fly without insurance – and I highly recommend you don’t fly without insurance! – you just need to get proficient and get training prior to solo flight.
Feel free to send any questions or comments!

Video:
https://www.youtube.com/watch?v=d6kpT-ZcJ58&t=2s

Sarah is an aviation insurance agent for Clemens Insurance Agency and is also a FAA Safety Team Lead Representative, NAFI Master Instructor, Gold Seal flight instructor, and 757/767 pilot for a Major U.S. airline. Sarah holds an ATP, CFI, CFII, MEI and has flown over 6500 hours. She holds a pilot license in 4 different countries (USA, Canada, Belize and Iceland – EASA) and has flown over 147 different types of airplanes in 20 different countries including oceanic crossings in small aircraft. She is the owner and chief pilot of FullThrottle Aviation; which started out in 2013 as a small flight school and grew to an international business with over 20 pilots moving airplanes around the world today. She continues to stay involved in general aviation through her leadership roles and volunteering for different aviation organizations. Although much of her flying is now professional in nature, she enjoys flying and instructing in her Super Cub, Patches, and her Cessna 170, Stanley, on her days off. As a regular fly-in attendee of Oshkosh, she enjoys the company and camaraderie that general aviation brings.