Vision-aided Sensing Pipeline with AI-based vertiport detection for Precision Navigation in UAM Approach and Landing Scenarios

This paper presents an original vision-aided, multi-sensor navigation architecture to support approach and landing procedures of Vertical Take-Off and Landing aircraft to vertiports in Urban Air Mobility scenarios. The architecture relies on an Extended Kalman filter integrating inertial measurements with positioning information provided by GNSS or GNSS/visual processing, depending on the distance from the landing pad. Indeed, visual-based measurements allow improving the accuracy and the integrity of the navigation filter output, especially at close distances from the landing areas. The visual processing pipeline relies on a two-camera configuration designed to ensure continuous coverage of the landing pad along an approach trajectory which is compliant with the existing guidelines. Regarding the landing pad, an original configuration with multi-scale fiducial markers is considered, which is obtained modifying the one proposed by the European Aviation Safety Agency so as to enable both manned and unmanned operations. From the algorithmic perspective, different processing blocks are customized and combined within the sensing pipeline paying particular attention to the timely detection and removal of anomalous measurements as well as to the compliance of the estimated uncertainty with navigation performance requirements during landing operations. Landing area detection is entrusted to an ad hoc Convolutional Neural Network trained on the basis of the operational domain. Then, custom motion-aware image processing methodologies are used to detect, match, and track visual landmarks which are then used to solve a perspective-n-points problem through an iterative non linear least squares strategy. Analysis of residuals, as well as covariance-based checks on the difference between predictions and estimates, are used before feeding the visual solution into the filter, thus providing protection against outliers in the visual-based pose estimation process. Performance assessment is first carried out by means of numerical simulations within a high-fidelity virtual environment able to realistically reproduce aircraft dynamics, sensors’ operation, illumination and weather conditions. Such analysis is then complemented by tests conducted on real images taken during an experimental flight test campaign with a small unmanned aerial vehicle.