The DUNE

TeamAware Team Monitoring System (TMS)

The Team Monitoring System in TeamAware encompasses two major developments: the Continuous Indoor Outdoor Localisation System (COILS) and the Body Motion Analysis (BMS); the latter has been introduced in the here, therefore the COILS is the focus here.

DUNE in TeamAware

DUNE is an Italian R&D SME operating since 1980 in aerospace, industrial control and telecommunications, with a major focus on sensor technologies, signal processing, localisation, and embedded solutions. Throughout its 41 years history, DUNE has collaborated with several leading Italian Companies, International Academic Partners and Research Agencies to carry out industrial and research Projects, also having 22 years of seamless involvement in European Projects, spanning from aerospace, industrial control, underwater applications, wireless communication, and indoor localisation.

Within TeamAware, DUNE will coordinate the partners’ efforts in WP7 (Team Monitoring System), bringing into the project its 10 years of expertise on inertial localisation, condensed around its (registered) ARIANNA tracking and safety system system, 5 patents on inertial processing, and the DUNE expertise gained in the involvement into R&D on-topic projects, leading two European H2020 (PROTECT, PROTECT-2), leading three Italian projects (RESCUE, PATH-SAFE, EXPLORERS), and participating into one EU FP7 Pre-Commercial Procurement (Smart@Fire) and one Czech TREND initiative (ILOC).

Background of the indoor localisation

Despite the GNSS availability makes “tracking” a trivial task, almost all the safety/security interventions, the Critical Infrastructures (CIs) and urban soft targets are GNSS-denied environments (e.g., indoor and underground areas); therefore, safety/security operators are (almost) never traceable, jeopardising the Situational Awareness and the effectiveness of the operations, thus increasing the loss-probability of human lives and high-value assets. In addition, GNSS is also very easy to jam; a point to be considered when intentional hostile actions are accounted in the scenario.

The impossibility of locating the operators in scenarios where the GNSS signal is absent (e.g., hostile jamming, indoor environments) is still a major cause of severe decrease of the Situational Awareness, thus decreasing the prevention, preparedness, intervention, and post-crisis recovery capabilities.

Given the above, it is not surprising that the IFAFRI (International Forum to Advance First Responder Innovation) has identified 10 “common global capability gaps” (i.e., areas where new or improved technologies could greatly enhance the safety, effectiveness and efficiency of the world’s first responders) and the Capability gap 1 is: “The ability to know the location of responders and their proximity to threats and hazards in real time”.

The issues and the challenge

• Performance: tracking personnel in the absence of GPS is a very hard scientific and technological challenge, therefore most of the solutions offered in the recent past have been basically academic prototypes with little or no chance of commercial success. Only in the last 6 years in USA and Europe several Companies (e.g., Sony, Intel, Navigine, Indoors, Insoft, Qrok, Combain, RadiantRfid, all gathered in the powerful InLocation Alliance) have performed working deployments of infrastructure-based localisation systems (e.g. WiFi, RFID) in airports and malls, insuring accurate tracking over a virtually unlimited time span. These systems are precise, but they are effective only in specific conditions that are not met in CIs or security/emergency scenarios.

  An alternative approach is based on infrastructure-free solutions, solely based on wearable sensors: mainly inertial sensors, possibly augmented by ancillary sensors (compass, altimeter, optic, UWB). Although their main advantage is to get rid of any location-aiding infrastructure (unsuited for safety/security applications), they suffer from a rapid growth of the position errors, making the estimated location useless just after few minutes.

• Suitability: although the infrastructure-based solutions have demonstrated to be able to offer commercial and performing solutions, they are entirely based on the presence of elements that are almost never available in the realm of safety, security and in CIs protection: indeed, first responders face high-risk, time-critical missions and cannot deploy ad-hoc infrastructures during the interventions in unknown areas; many CIs (e.g. power plants) cannot provide the necessary clearance to allow the deployment of location-aiding infrastructures or disclose their floorplans. Also, the “fingerprinting” campaigns in CIs are subject to clearance restrictions and suffer from time obsolescence, thus must be refreshed on a regular (yearly) basis, severely impacting on the OPEX.

• Affordability: disregarding the operational suitability issues above, both the deployment of dense localisation aiding infrastructure or the use of wearable strategic-grade IMUs have the potential to meet the desired performance level, but the resulting acquisition cost (CAPEX) and the necessary maintenance (OPEX) might need a budget largely exceeding the available budget of the potential customers. Any solution aiming at having a real impact on the operation must be financially affordable, both on the acquisition and on the maintenance side.

The challenge: on the one hand, the infrastructure-based solutions can provide good tracking performance, but have little or no room in the CIs protection and safety applications; on the other hand, the infrastructure-free solutions suffer from rapidly degrading performance, making them unsuitable for long term operations; therefore, the key problem is to find a reasonable tradeoff solution capable of insuring sufficient accuracy by solely relying on infrastructure-free solutions, implemented with an operationally suitable and commercially affordable/profitable system.

The TeamAware approach

TeamAware employs a new approach for the seamless continuous outdoor/indoor localisation, based on innovative fusion paradigms (owned by the partner DUNE and widely on-field tested) based on the joint use of wearable IMUs for motion tracking purposes, Compass, Altimeter, GNSS and UWB. The localisation solution is designed to be infrastructure-free and wearable, being also specifically designed to be a “switch-on and forget” paradigm, coping with the operational needs of the responders (e.g., no need of complex calibrations, expert users, cumbersome setup).

In-team situational awareness system will consist of continuous outdoor/indoor localisation system (COILS), health and body posture monitoring subsystems (BMS: Body Motion Analysis). In-team situational awareness system will be a wearable system and will be connected to the cloud-based TeamAware platform via wearable gateways. Fully integrated real-time indoor/outdoor localisation system is required to localise responders in hazardous and risky situations during operations. More specifically, Inertial + GNSS fusion will be used for the outdoor localisation, whereas wireless beacons, RF transceivers and inertial measurement units (along with new multisensory fusion techniques) will be employed in indoor, underground and, in general, in all the environments where GNSS is unavailable, unreliable or intermittent. Furthermore, activity monitoring system (AMS) will consist of health monitoring and body motion capture systems. Health monitoring subsystem will monitor the vital signals of the first responders. Body motion capture will monitor position and orientation of a first responder to detect activity of the responder such as still, moving, holding a hose or lying on the ground. COILS and AMS will build up the Team Monitoring System (TMS).

Key development elements

To meet the best tradeoff between performance, usability, and marketing issues, the COILS system has been developed around the following key elements.

• Infrastructure-free: solely based on wearable sensors, with no need of setting up, deploying, calibrating any external or internal infrastructure (e.g., radio beacons, RFID tags, BT tags).

• User-independent: the system can be shared and switched among different users, with no decrease of performance.

• Switch-on and forget: the system doesn’t need any intervention during the operations, performed by the user (they have their job to do).

• Wearability: the system is pocket-size, lightweight, easily wearable, and not annoying.

• Open Communication: COILS output needs a very low data rate link (e.g., < 1 kbps) and the output interface is rather universal and can accommodate a widespread ensemble of Communication standards (including most of the ones already owned by the users).

• Affordability: the acquisition and maintenance costs must not exceed the (reasonable) budget available on the end-users’ side.

• Hierarchical information management: even in the lack of any information source (e.g., no GNSS, UWB out of range, unreliable altimeter, no floorplans available) the system will guarantee a minimum level of performance, solely based on the wearable IMU; however, as soon as any information source is available (and reliable), the system improves its performance (e.g., when the magnetic information becomes reliable, the system can perform a position-drift compensation).

• Processing topology and offloading: the general “rule” is to process information where is generated; however, this is not straightforward when a location-refinement algorithm needs the information from multiple sources, generated or available in different places (e.g., the inertial information is generated in the foot-mounted sensor, but the digital floorplan is available at the remote ICS). Considering also that the information transfer impacts on the bandwidth requirements of the wireless links, the COILS distributes the processing elements as a reasonable tradeoff between the available computation power, battery consumption, and bandwidth requirements.

COILS sensors and processing

The following figure schematically illustrates the COILS sensors and processing elements: the leftmost orange box represents the wearable elements: IMU unit, altimeter, GNSS receiver, TAG reader (e.g., UWB), magnetometer. The rightmost box illustrates the processing elements (i.e., inertial localisation with compass-based drift compensation, fusion with altimeter data, fusion with the GNSS data, track refinement based on the TAG data, improvement based on the availability of the floorplans).

Albeit the figure illustrates the physical elements (left) and the processing tasks (right), specific choices affect the place where the processing elements are performed; for instance, the inertial localisation with compass-based drift compensation is presently performed inside the foot-mounted sensor unit; the fusion between the altimeter and the inertial vertical position is presently performed at the ICS side, as well as the map-matching track refinement, whereas the GNSS-inertial fusion is placed inside the wearable smartphone. Different choices about “where” a specific processing is located are possible, and no choice is a-priori “optimal”, each entailing pro and cons on the battery consumption, processing power, bandwidth requirements and so on.

Based on the fulfilment of the above elements, DUNE has developed the ARIANNA indoor localisation system (www.ariannasystem.com) that constitutes the backbone of the COILS to be developed in TeamAware. The ARIANNA elements are illustrated in the following figure.

Wearable segment: miniaturised foot-mounted sensor unit (9DOF MEMS IMU), a Control Unit with power supply and communication management, plus an optional smartphone; the drift-compensated inertial track is computed inside the sensor unit; the Control Unit oversees the collection of additional information coming from other (optional) sensors (e.g., GNSS fixes and NMEA data, altimeter, heart pace), format and deliver them to the communication system.

Remote segment: receives all the data from the wearable system, performs additional track-refinement operation, either automatic (e.g., inertial track fusion with the altimeter, fusion with the GNSS fixes) or operator-assisted (e.g., floorplan-assisted track refinement, track deskewing with landmarks); visualises in 2D, 2.5D and in 3D the track with the satellite maps (multiple servers); manages the contextual presence of additional visual information (e.g., photos, audio notes); generates automatic reports of the whole mission.

Examples of on-field results

As a starting point, the development of the COILS system of TeamAware will leverage from the performance achieved by the DUNE’s ARIANNA system, already assessed by countless experiments and trials performed in realistic operational conditions by the end-users (e.g., an Agency of the United Nations operating in Critical Infrastructures, Italian Firefighters, Chinese Firefighters, Taiwanese firefighters, and so on).

The first figure reports the result (in 3D) of a test performed in cooperation with the Italian Firefighters inside an underground cave (GNSS never available), exploiting the presence of four landmarks established during the operations (no deployment of location-aiding elements).

The second figure is relevant to a demonstration session performed in the disaster simulator SISMA-USAR (Urban Research and Rescue) facility of the Italian Firefighters (with real collapsed buildings, collapsed industrial shed and car parking, one large field of debris). In the figure, the yellow squares indicate the indoor areas (no GNSS) and the red path is the estimated track walked by the firefighter. The overlaid image is the firefighter crawling inside the tunnels located under the field of debris.