RuAviation

The use of deep neural networks in aircraft vision systems is becoming more and more relevant in connection with the development of technology and the requirements for safety and efficiency of flights. The results of experimental studies conducted at the State Research Institute of Aviation Systems (GosNIIAS) show that the integration of intelligent algorithms allows to increase situational awareness of the crew, as well as to reduce the risks associated with the influence of the human factor.

To ensure situational awareness of the crew of prospective civil aircrafts, the Institute’s specialists propose integration of vision systems consisting of multispectral optoelectronic systems, onboard high-performance computers and functional software. One of the tasks of such systems is runway detection for automatic landing without the use of ground systems.

The use of highly sensitive sensors of different spectral ranges and physical nature, as well as a new generation of algorithms for processing visual information of zakabin space will make it possible to realize the function of automatic runway detection and provide data correction of the global navigation satellite system. GosNIIAS specialists have formulated a method involving the use of neural networks based on the YOLO family architecture with subsequent refinement of runway image angular points using the MnasNet architecture.

In addition, intelligent means of crew support provide automation of aircraft movement to the place of parking or takeoff. To solve this problem, it is also proposed to use vision systems with the implementation of neural network algorithms for recognizing runway and taxiway markings.

Also, within the framework of work on the development of vision systems for aircraft, specialists of the Institute have developed an algorithm for creating a three-dimensional model of a low-altitude suburban area based on images of terrain maps without vector markings. The scientific results obtained by GosNIIAS specialists will become the basis for intelligent crew support systems. The technology demonstrator underwent flight experimental studies on a flying laboratory in Novosibirsk.

In the course of the tests, new vision algorithms were worked out, as well as field data was collected to train a neural network, which in the future will allow the aircraft to automatically detect and recognize the runway. “The flights were conducted in order to evaluate the developed technologies for intellectualization of onboard complexes of advanced aircraft and to collect video materials to form a training and test data sample, which will be used for further training of neural network algorithms for technical vision,” said Sergey Khokhlov, General Director of GosNIIAS.

Technical vision is a complex system of visualization and image processing of stationary and moving objects. The airplane – flying laboratory was equipped with highly sensitive sensors of different spectral ranges and video fixation means. In its turn, the software and hardware solution provides intelligent information support for the crew, forming images of the cockpit space.

In the course of research the performance of intelligent algorithms for detection and recognition of marking elements and symbolic elements of the runway, as well as obstacles on the flight field was tested. The developed algorithms will reduce the load on pilots at the stages of takeoff and landing, as well as increase the efficiency of aircraft onboard equipment complexes, said Sergey Khokhlov.