WP3 focuses on AI applied to EO system and computer vision applications. The work proposed, based mostly on a Deep learning approach,
may require larger amounts of training data, to perform well data management issues including how to acquire large datasets and how to
improve the quality of large amounts of existing data become more and more relevant. Moreover, the results of computer vision tasks based
on artificial intelligence (AI) training could depend from the data : they must be sufficiently diverse, well balanced, appropriate to
the context and unbiased, in order to avoid problems such as artificial “AI bias”.
Accurate data collection techniques in the era of Big data gives motivation to conduct as a first step a comprehensive survey of the data
collection literature on different tasks appropriate to the TeamAware project that will be merged with data acquired in the specific test
bed proposed in TeamAware project.
The main reason that leaded to this activity is because the right data contribute to generate the appropriate approach to guarantee the
quality of the results. Indeed, good processing is about using the right data and algorithms at the right time.
Our work in this perimeter is to identify and to review relevant open source datasets useful to train model adapted to manmade and natural disaster
scenarios. In order to cover the end-user operational requirement 3 main applications are expected to be addressed by the Visual Scene Analysis System (VSAS) system:
Victim detection : The aim is to create a dataset for research and rescue applications. The categories we are looking for to detect victims include images of
people in various poses: lying down, sitting, and falling etc. from different distances and different angles, where some images would show the whole human body,
and others would show parts of the human body. In addition, there would be images captured in a controlled environment and “in the wild” conditions. Finally,
the images containing other spaces and rooms with no human would be gathered.
Considering the natural disaster demonstration, the VSAS system will be deployed in two phases:
a drone equipped with an RGB-D camera (Depth Sensor - RGB camera) will enter into the disaster zone for a first appreciation of the situation and to look for
stranded passengers. The video footprints are sent and will be analysed into the command-and-control station. Thanks to the AI algorithms, previously trained
with a victim detection dataset, a first evaluation of the victims will be performed.
After this first overview, an LEA wearing a helmet with an infrared (IR) camera and a standard grey camera will enter in the underground tunnel to rescue
victims and to inspect in deeper the environment: other victims could be hidden by debris.
In the human-made disaster scenario, the helmet and the drone will be deployed and it will use to identify injured people in an demolished abandoned building
Situational awareness using visual segmentation: A part of the reviewed datasets that could be a valuable resources to develop and train deep learning
algorithms for detection and recognition tasks, are based on videos/images captured by drones and wearable cameras, which are the main source of video footage
that will be used in the project. Beside to that, were also identified dataset that includes semantic and geometric indoor data in 2D, 2.5D, and 3D domains,
as well as their instance-level annotations. Apart from these images and information, there are also point clouds and raw 3D meshes registered and semantically
annotated.
Damage assessment: The datasets merge photos of real damages after catastrophic events and synthetic images of damaged structures. In particular the one
composed by photos of reals cases, are constituted by annotated images, in which several structural elements and structural damages are identified. For both
datasets, the event of reference is mostly a natural disaster, in particular earthquakes but also ground deformations (such as landslides, settlements or
subsidence) that may jeopardize the structural safety. The use of the datasets may also be extended to those manmade disasters that could cause damages to
structural components or to those conditions in which a suitable maintenance level for infrastructure operability is not guaranteed.
According to these 3 main tasks, D3.1, that was recently submitted, proposes a survey on exploitable dataset in the context of the TeamAware project and a perspective
of data collection based on synthetic generation and a methodology for collection of data in the test sites that will be forecasted in the future.
