Challenge being tackled: Challenge number 3 was proposed by the University of Twente, and it was the Human Operator interface for FA-IUAS. The main objective was to develop a high-TRL operator interface for the FA-IUAS to unlock the capabilities of fully actuated aerial platforms in autonomous contact-based NDT inspections.
Description of the solution: A two-part technology solution that enables users to easily monitor and guide an autonomous contact NDT with an aerial vehicle by integrating the autonomy stack and the web application-based user interface.
The autonomy stack consists of four major modules. The first module is a mission planner in the form of a simple finite state machine (FSM) that allows manual control over the UAS or autonomous exploration of the environment, navigating to the inspection location, and performing the contact NDT. The second is a localisation module that utilises state-of-the-art SLAM to enable aerial platform localisation based on the sensor measurements (IMU, LIDAR).
Based on the performed localisation, we can ensure autonomous operation in GNSS-denied environments (i.e., enclosed spaces) such as ballast tanks. Besides the mission planner and the localisation module, the autonomy stack consists of the chosen trajectory planner, which can generate trajectories for energy-efficient exploration and inspection tasks. The fourth module of the autonomy stack is the adaptive impedance control module, which enables contact-based NDT inspection of the chosen inspection points using external wrench estimation.
Together, these modules enable autonomous aerial platform operation in GNSS-denied environments and ensure efficient exploration and inspection using mission planning, localisation, trajectory planning, and FA-UAS contact control. The web-based user interface consists of the following UI functional modules: UAV state selection, sensor feedback visualisation, and measurement visualisation module. The state selection module enables users to toggle between states, such as EXPLORE, INSPECT, and RETURN.
The sensor feedback visualisation module allows users to visualise the camera stream, the estimated location of the aerial platform in the 3D map of the environment, and the chosen or autonomously planned inspection points. These modules are crucial for the user to monitor mission execution actively. Besides functionalities used during the non-contact tasks, the
measurement visualisation module will offer live visualisation of the acquired measurements with NDT sensors. By integrating the autonomy stack and the UI, we will enable autonomous NDT operation of the FA-UAS in enclosed environments like ballast tanks and cargo holds.
This approach reduces the time needed for operator training and transitions human surveyors from dirty and dangerous jobs to supervisory roles. It also increases overall inspection value and quality by allowing the operator to monitor real-time progress and adjust the mission parameters as needed. The modular and adaptable nature of the interface enables it to be applied to other industries as well, making the inspection process more efficient and sustainable, while simultaneously increasing European competitiveness and resilience.
About the provider: Crobotic Solutions is robotics start-up developing AI enabled robotics solution with advanced autonomy for real-world environments.
Check their LinkedIn here.
The Team behind the scenes:
Filip Zorić – CEO
Ana Milas – COO
Marko Križmančić – CTO
