Microflown AVISA: General Objectives of WP6
The general objectives of Work Package 6 are:
design of a wearable (compact and lightweight) georeferenced drone equipped with an acoustic sensor;
detection and localisation of gunshots, explosions, in the range of 200 m with the localisation accuracy of 10 meter and direction accuracy within ±5 degrees;
detection and localisation people whistling, and human speech in the range of 50 meter, with the localisation accuracy of 10 meters.
Above mentioned objectives will be achieved by Microflown AVISA with developing the acoustic vector sensor array and CERTH/ITI with developing algorithms, for the detection and localisation of emergency events such as explosions, gunshots and snipers, as well as human voices (e.g., screams asking for help) and whistling in the operations.
The first version of the design of the acoustic event detection system including the role of the ADS in the demonstration scenarios and the communication protocols with the TeamAware platform were determined in D6.1.
The concept of acoustic vector sensor, the way it works and the measurements it provides were explained. Examples of acoustic vector sensors produced by AVISA and some arrays of such sensors were discussed.
The new UAV-mountable array that is to be developed for TeamAware project and the steps to be followed for its build were explained. Besides, a literature review has been conducted and public datasets were selected as a benchmark for the developed algorithms.
For the TEAMAWARE project, as an intermediate step in the ongoing developments of a new sensor node, Microflown AVISA will provide CERTH some prototypes that allow them to acquaint themselves with or sensor technology.
Figure 1:: PCB designs of new UAV Mountable Array The ultimate deliverable is new sensor node (dubbed AMMS 22/8) that is 23 cm in diameter, has 16 channels, a sampling rate of 48 kHz and an ethernet output.
This new sensor node will be the new platform that can handle the advanced beamforming algorithms needed for instance to handle the acoustic reflections that will occur when the sensor node is deployed on a multicopter in a tunnel test scenario.
The details information regarding the first recording using an array of AVISA acoustic vector sensors to be used by CERTH were provided. This data recording performed by AVISA using their existing 3D-sensors. Additionally, a data collection plan to be done in the CERTH premises has been created. Deep learning algorithms for the single audio-based event detection and sound source localisation (estimating the azimuth and the elevation) tasks were implemented. The public datasets used as input to the algorithms and the necessary pre-processing steps were demonstrated. In particular, the DenseNet-121 with spectrogram magnitude representation and the DOANET algorithm with raw audio signals features were applied. Preliminary results of the aforementioned experiments were illustrated, indicating comparable outcomes to those of the state-of-the-art.
The first version of the design of the overlapping acoustic event detection system was presented as a report for D6.2.
The current description of the overlapping algorithm will be subsequently discussed and further refined in the two upcoming iterations of the design report (D6.4 and D6.7). The relation of the particular task to the demonstration scenarios was presented and the communication protocols with the TeamAware platform were also determined in D6.1.
Furthermore, a literature review of overlapping sound event detection methodologies using a UAV as a surveillance system was demonstrated. The FSD50K public dataset using the Scaper library of Python was used for the algorithm's training as well as the data collection plan in the CERTH premises and data provided by the AVISA acoustic vector sensor were described. The pre-processing steps and feature extraction methods were also demonstrated. In particular, the Wavegram-Logmel classifier was implemented to solve the problem of overlapping audio-based event detection. The preliminary results are encouraging for this task since high performance was achieved on unseen during training data.
