1. Abstract

     We present a new dataset, called Air2Land(A2L), for advancing the state-of-art in object detection and 6D pose estimation in the context of UAV auto landing research. It meets vision and robotics for ground-based vision guidance system having the multi-modal data obtained by different sensors, and pushes forward the development of computer vision and robotic algorithms targeted at visually assisted UAV landing. The dataset, contains sequential stereo images and other sensor data (UAV pose, PTU angles) simulated in various climate conditions and landing scenarios, can be employed for tasks including UAV tracking, keypoint localization, pose estimation and action recognition, etc. In addition to providing plentiful and scene-rich data, our dataset covers high risk scenarios that are hardly accessing in reality.

2. Our Air2Land dataset

Notice: You can download the dataset (only partially provided) using Baidu Drive (size: 2.01GB, passward: 6tk1).

2.1 Overview infomation of the Air2Land dataset

2.2 The terrestrial stereo guidance system for UAV safe landing

    In our guidance system, two PTUs (pan-tilt unit) are symmetrically located on both sides of the run way with each camera fixed on. The computing unit processes the images captured by cameras during UAV landing and estimates the its spatial coordinate and posture. The motion parameters are wirelessly feedback to the on-board autopilot to assist its safe landing.     As shown in Figure 2, the Pixhawk autopilot runs the actual airborne flight control code, which is responsible for the control of the drone during the loading and landing of the mission route, the monitoring and feedback of the drone flight status, the analysis and execution of operating instructions, etc. QGroundControl is a ground station for UAV flight protection, mainly responsible for flight mission planning, digital map display, data link transmission, and comprehensive data analysis. ROS connects various components to realize the transmission of messages on different topics, and the 3D simulator performs simulation and display of the scene.     The built simulation system includes the physical controller and professional dynamics simulation software into the loop, which retains the kinematic characteristics of the simulation data to the greatest extent.

2.3 Typical samples from Air2Land

    The UAV present a very diverse range of perspectives and scale variations from view of terrestrial cameras.

3. Experiments

3.1 Keypoint localization experiments

    The most widely used pose estimation method is to apply the pin-hole camera model to solve the correspondence between 2D image points and 3D spatial points as a PnP problem. After detailed consideration, we selected the left wing tip, left tail tip, front gear, nose, right wing tip, and right tail tip of the UAV as feature points, and verified the localization accuracy of a key point recognition network -- Stacked Hourglass network under different weather conditions.     We use images generated under sunny, cloudy, light rain, moderate rain, light snow, moderate snow as the training set, and the testing set is composed of images under heavy snow, heavy rain, and foggy weather, both of which cover the early morning, midday, and evening time periods.

3.2 Experimental results

    To compare localization performances, we use percentage of correct key-points (PCK) as evaluation metric. Detected point is considered correct if the distance between the predicted and the ground true is within a certain threshold (threshold varies). Drawing on this metric, we choose the bounding box of the drone's nose as a benchmark, and set the correct threshold of prediction to 50% of the length of the rectangular box, which we also call PCKh@0.5.

4. Other