- The Cautionary
- Posts
- Learning AI #26
AI and the dangers of skills erosion
At 35,000 feet over the Atlantic, the autopilot on Air France Flight 447 disconnected. Four minutes later, the aircraft was in the ocean.
It was June 1, 2009. The flight had left Rio de Janeiro for Paris and flown into a system of thunderstorms. Ice crystals blocked the plane's airspeed sensors. The autopilot lost a reliable airspeed reading and did what it was built to do in that situation: it disengaged and handed control back to the pilots.
What happened next is one of the most studied four minutes in aviation history.
The aircraft climbed briefly toward 38,000 feet, then stalled. The nose pitched up, the wings stopped producing lift, and the plane began falling toward the ocean.
The correct response to a stall is to push the nose down and rebuild airspeed. The pilot flying pulled back on the controls instead, and kept pulling back trying to get the nose up, as the stall warning sounded more than 75 times. All 228 people on board were killed when the aircraft hit the water.
The captain was on a scheduled rest break when the crisis began. He returned to the cockpit about a minute and a half later to find the aircraft already falling at close to 10,000 feet a minute, two first officers giving him conflicting information, and no clear picture of who was doing what to the controls.
Pilots on Air France 447 were confused on what to do once the autopilot handed them control.
The investigation turned up something more specific than mechanical failure.
Three trained pilots, with thousands of flight hours between them, could not recognize a stall or recover from one by hand.
In normal operation, the autopilot does almost all of the flying. Over a career, that adds up to thousands of hours in the cockpit without actually flying the plane, but watching displays, managing systems, and trusting automation that rarely asks for help.
The physical, instinctive feel for an aircraft approaching a stall, the kind earlier generations of pilots built through years of manual flight, fades. Nobody decides to let it fade. It happens on its own, because the automation is good enough that the skill is never called on.
The systems on Air France 447 worked exactly as designed. The autopilot correctly identified that it could not trust its own data and handed control to a human, which is the entire point of keeping a human in the loop. The human it handed control to had quietly lost the skill the system was counting on him to keep.
Bottom line
You are not flying a plane. But if you use AI to write your emails, build your spreadsheets, draft your code, or make your first pass at a hard decision, you are using the same approach the airline industry used on its pilots. The tool gets better. The skill it replaces gets weaker, quietly, with nobody deciding to let it happen.
That is not an argument against using AI. It is an argument for knowing which of your own skills you are letting go soft, and checking, every so often, that you could still do the job by hand if the system disengaged on you at 35,000 feet.
Things I think about
The blue whale is the largest animal ever known to have lived on Earth, including dinosaurs.
Free gift for my subscribers
Here is the link for a free audiobook chapter from my new book, No Adult Left Behind. AI is Learning. Are You?
There is no login or anything like that, just click and listen. The chapter is all about how to use prompts with AI. A good starting point for anyone who is serious about learning AI. (The audiobook is narrated by my AI-cloned voice.)
**********