夜色视频

A Boeing 737 scheduled for a cabin modification sits in an aircraft hangar, seating and panels removed. A maintenance engineer wearing an augmented reality (AR) headset with a visor that stretches across his field of vision can immediately access all the information needed to route and install new looms and equipment. A virtual model of what he should see once the modification has been completed overlaps what he鈥檚 actually looking at. Step by step, each phase of the modification is described, visualized with any necessary tooling called out. Any mistakes are quickly identified before the next task is started. The use of the AR headset ensures quality is built in as this complex task proceeds, and accurate recording of work completed facilitates a comprehensive and smooth handover should the modification run across multiple shifts.

Meanwhile, across the airport a pilot sits in a Cessna 172 getting ready for her pre-flight checks. She too slips on an AR headset, which begins to visually guide her through her checklist. Once she鈥檚 all set and on the runway, she clears her view so that what remains are the visual cues necessary for take-off and any warning messages that should be accounted for. Despite the dark and rainy conditions, enhanced vision systems overlay a crisp image of the surrounding areas and enable a confident, on time departure.

Welcome to the near future of augmented reality and avionics.

Why Augmented Reality, Why Now

Virtual reality may be all you鈥檙e hearing about lately, but thanks to AR鈥檚 broader industrial applications, it鈥檚 more likely to be the medium that will rewrite our digital futures. By all estimations,  鈥� three times that of virtual reality. And that money has an effect on the smart glasses that will drive the changes to come.

鈥淚t鈥檚 accelerating the process of getting them ready for all of these difficult use-cases that are coming to the market,鈥� says Vincent Higgins, CEO of , which specializes in industrial applications of AR and VR.

What鈥檚 also accelerating the process is good technological timing. 鈥淲e鈥檝e got the convergence of mobile computing power, display technology, sensors 鈥� all of the things that you need to really make the technology work are now starting to come into play,鈥� says Barry Po, Senior Director of Product and Business Development at , another maker of industrial AR. The effect of that convergence is that, the better the technology works, the more it鈥檚 going to become dependable enough for industries to start seeing the benefits of AR.

The aviation industry is especially poised to benefit from the rise of augmented reality. In fact, AR has already made its first steps into piloting and maintenance. Companies like Optech4D and NGRAIN, among many others, have begun previewing how aviation will undergo a major AR upgrade within the next five years.

The Potential of Augmented Flying

The arrival of AR headsets brings one immediate upgrade to the aviation industry: A shift from the forward looking-only head-up displays in commercial airline cockpits, to the omni-directional abilities of head-mounted displays (HMD) like Microsoft HoloLens. Now pilots will have the ability to look wherever they choose, at any time, and have digital information appear naturally in their field of vision. What they鈥檒l be able to do with that is being spearheaded by innovators like 脕kos Mar贸y and Franz Viertler.

脕kos Mar贸y is the CEO of , a Hungarian company (with a San Diego outpost) currently developing smart glasses which pilots would use not only to augment take-off and landing, but flight time too. With a headset on, a pilot could check their flight path, receive hazard warnings or be shown distances to objects, such as nearby aircraft. Franz Viertler, a system engineer at , is working on something similar, albeit more specialized: Helping helicopter pilots to fly in low-visibility weather conditions with AR that projects digital topographical information (mountains, houses) in the pilot鈥檚 field of view, so they can 鈥渟ee鈥� what鈥檚 ahead.

Both entrepreneurs share an ultimate goal, one that will influence the augmented piloting road ahead; that is that AR-assisted flying will empower pilots to make safer and more informed decisions, especially in challenging circumstances. 鈥淭he bottom line is safety,鈥� says Mar贸y.

The AR Piloting Challenges Ahead

Safety may be the ultimate goal, but there鈥檚 still work to be done before it gets there and is (potentially) widely adopted. 鈥淭he performance is already there, but you have to ensure the reliability and the safety of the system itself,鈥� Viertler says. In this case, that means improvements in the real-time information the HMD draws from to plot a pilot鈥檚 course. What if a week-old database update doesn鈥檛 show a construction crane that was set up the day before a pilot flies?

And in Mar贸y鈥檚 case, potential use on commercial airlines means standardization, which the company is , Mar贸y says. The company is seeking a 鈥渕inimum set of requirements in terms of quality, reliability and interoperability with existing procedures and systems onboard鈥� 鈥� but the goal isn鈥檛 just to conform to standards, but also potentially innovate them. 鈥淲e really are trying to play around with things and explore new approaches,鈥� he says.

There are still user-experience design questions that need to be finessed through that exploratory process: How will AR info appear in different lighting environments? Do existing aviation colors and symbology need to be adopted or evolved? How can visual clutter be avoided so as not to obscure a pilot鈥檚 view?

That鈥檚 why both entrepreneurs say there鈥檚 still a good deal of testing to be done. But while their expectations are reasonable, there is still optimism. 鈥淚t will take more than three to five years to really have the systems where we want to have them, but in the next few years I think we鈥檒l already bring them into reality and demonstrate that it is working there,鈥� says Viertler.

The arrival of AR headsets brings one immediate upgrade to the aviation industry: A shift from the forward looking-only head-up displays in commercial airline cockpits, to the omni-directional abilities of head-mounted displays (HMD).

Augmenting Aircraft Maintenance and Inspections

One of the appeals of adapting AR to piloting is making a complicated job a little easier. NGRAIN鈥檚 Barry Po believes the same mission should be applied to airplane maintenance. 鈥淲e have to stop thinking of aerospace engineers and technicians as tradespeople and start thinking about them a little bit more like they鈥檙e fighter pilots. They鈥檝e got incredibly complex jobs where sometimes they don鈥檛 have a whole lot of time to make a decision,鈥� he says.

He鈥檚 not the only one who believes AR can make that responsibility a little easier to handle. Several companies are currently developing AR options to streamline aircraft maintenance in ways that don鈥檛 just benefit the workers. 鈥淲e think maintenance is something that is going to have a pretty dramatic return on investment for airlines and aircraft manufacturers, while pretty clearly demonstrating the power of the technology,鈥� says Mark Cheben, the global marketing director of .

Like many of his colleagues, Cheben stresses that the main ROI of augmented reality is hands-free movability and real-time information, but the technology is being used in a variety of ways 鈥� often in conjunction with artificial intelligence, the Internet of Things, sensors, 3D scanning and video conferencing. Optech4D, for example, has what it calls 鈥淭he Safe Beacon,鈥� which is a sensor that can be attached to a specific piece of equipment in a plane that, when approached with an AR-enabled tablet or headset, will trigger information about the equipment.

The company also has something called 鈥淩emote Expert,鈥� a form of teleconferencing that uses smart glasses鈥� front-facing cameras to enable an expert at headquarters to see through an operator鈥檚 glasses and push through information 鈥� documents, annotations, e.g. 鈥� to guide the inspection or maintenance process.

As for NGRAIN, it has technology that uses artificial intelligence and 3D graphics to scan environments and conduct visual inspections that could identify scratches, dents and other anomalies that need looking after.

Many of these applications are still in their early stages, but nonetheless they foreshadow the immense potential ahead. Po says NGRAIN鈥檚 AR could scan, identify and track problems throughout an aircraft鈥檚 life 鈥� all with just smart glasses 鈥� and help airlines and manufacturers make more informed decisions about significant overhauls. It would enable, as he puts it, 鈥減roactive, not reactive, maintenance.鈥�

Will Augmented Reality and Aviation Take Off?

The promise of such a fluid, streamlined, AR-enabled future 鈥� in both piloting and maintenance 鈥� is tantalizing, but the technology, while promising, still has a ways to go.

What鈥檚 more, airline companies may lean towards a wait-and-see approach to gauge whether the ROI of augmented reality is enough to embrace widespread, or even partial, adoption. But those taking these early steps will see advances in the next few years. 鈥淭he hardware is increasingly going to get to a point where it is mature enough to be deployed in a manufacturing setting or in a real-world setting. That鈥檚 not necessarily a place where it is today for every scenario, but one would expect that in short order that鈥檚 where it will be,鈥� says Po.

In other words, it may be worth second-guessing an inclination to write off augmented reality and aviation as hype. 鈥淧eople need to understand that it鈥檚 not just a cool technology,鈥� says Higgins of Optech4D. 鈥淏ig companies don鈥檛 buy 鈥榗ool.鈥� They buy things that have a return on their investment.鈥� If even a fraction of the ambition of these early explorers are realized, it鈥檚 not hard to imagine companies will be looking to buy.