Situational Awareness Platform – Part 1: Data Fusion

Within the TeamAware System, the Situational Awareness Platform acts as the central data aggregation system, where every data collected with all the TeamAware Sensor Systems are brought together to one situational awareness picture, which can be accessed by the mission operators directly and by the first responders on field.

Within the platform, all the data and information gained is organized in a common database that contains raw data but also processed data and knowledge gained by fusing some of the data incidents to get easy access to a very complex overall situational picture. Included in the platform is also an AI-based decision support system to help operators and first responders decide which could be the next logical steps to take in a very complex situation.

The first development step in that direction is the database itself (coming up in a later blog entry), but also the data fusion modules integrated into the platform system.

The different data fusion capabilities are set up as singular modules to make them interchangeable and to give the possibility to add further data fusion modules. As new sensor systems are added and make new data types available for the system and as research gets farther in that topic they can be easily added into the system. This structure also serves the purpose to make the information more easily accessible by any information point within the TeamAware system like the operation center or the wearable information systems like the AR glasses or the mobile devices of the first responders themselves.

There are single data type data fusion modules, which are mainly for correlating and synchronizing the same type of data coming in from different sensor systems, mostly geographical and time-related reference data. With this, a Personnel Localization Enhancement (PLE) will be set up to provide very accurate position data on every first responder on the field.

On the other hand, there will also be data fusion of multi-sensor data points with correlation in geographical and time reference stamps. For one this is the MuFASA System (Multimodal Fusion Architecture for Sensor Applications) wherein high-level data and identified events are put into correlation concerning their geographical and timely closeness to each other. It tracks important events via bringing different incidences and alarms together, shows codependences and weights confidences of the different data gained to give a more in-depth situational awareness. This helps to reduce false alarms and aims to vastly increase the confidence of high-level detection events by essentially confirming them through different sensor modalities.

Additionally, a Sensor Information Density System (SID) is set up to keep track of all the incidences and alarms raised and their time and spatial relation as well as their degradation over time or their repeated occurrence. Hereby, the main objective of SID is to work on data of any processing level to provide essential knowledge about sensor coverage and information reliability in an easy but comprehensive way.

Lastly, an Incident Based Path Proposal (IBPP) is calculated in case of a staggered deployment of first responders to give later coming teams or personnel an as-secure-as-possible way to the location they are needed at. This is based on the information gained within SID and MuFASA but also takes into account knowledge of paths taken of first responder groups that have reached the point of interest before.

Through the course of the project, as the system gets set up and further developed, additional ways of combining data to enhance the knowledge base for the operation can and will be implemented within the TeamAware Situational Awareness Platform.