Newsletter February 2022

Welcome back to our newsletter with February issue! With this issue, we summarize what we achieved in the last three months.

As mentioned previously, November 2021 was a very crucial month. The vertical technical workpackages (WP3 to WP11) have started. All workpackage leaders organized their Kick-off meetings and monthly regular meetings.

Here is the summary of achievements in each workpackage:

• WP2 System Architecture Specification and Design: WP2 has continued its work as the main meeting point for technical discussions between end users and technology providers. On the one hand, it has served to define a baseline of the final demonstrations that will take place in year 3 of the project. This baseline lays the foundations for the work to be carried out in WP13 right after its kickoff. On the other hand it has allowed to outline the main functional and technical requirements of the main elements of the TeamAware ecosystem, concretising crucial aspects that will define their good operation in real-life situations represented by the final demonstration scenarios.

  A crucial work has been to map the gaps that end users usually face in their everyday operations to the functional requirements of TeamAware’s systems, platform, and cloud infrastructure, and their technical requirements. Also, a baseline procedure for evaluate the different components of the TeamAware ecosystem has been put in place. This has entailed the definition of research questions, evaluation KPIs, and links among them and with the user gaps.

  Last, but not least, WP2 has set the legal and ethical principles that will be followed throughout the different stages of the TeamAware project. For this purpose, an analysis of the main legislation in relation to data rights and privacy and fundamental rights has been carried out.

  All in all, WP2 has set the framework in which the rest of the project will operate. After its conclusion in M12 of the project (April), the rest of work packages will build onto the foundations and guidelines set in WP2.

• WP3 Visual Scene Analysis System: After technical meetings with the end users, suitable technologies have been identified. It includes an indoor drone platform and two types of sensors: mini-lidars for obstacle avoidance, and a RGB-D camera for mapping. Mini-lidars are now being studied in order to determine how many of those are needed and how to place them properly on the drone. A huge care is brought to the system’s autonomy, as it is crucial for all the end users. Multiple algorithms are being developed, tested and improved too in order to maximize the system’s resilience. State of the art algorithms are taken into account as well, as the final solution may integrate one of those. Besides that, other discussions are still on going on the segmentation tasks and the identification of the list of elements to detect with end-users. The results of all these elements will feed a dataset on the VSAS system.

• WP7 Team Monitoring System: In the scope of the AMS (Activity Monitoring System), the implementation of the calibration and neural network algorithms have started. Tests are being conducted to obtain calibration parameters and to search for the ideal ways to include them within the measurements and fuse the sensor data coming from IMU. Both CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) based neural network architectures are currently addressed in the activity. The WP7 activities about the COILS (Continuous Outdoor-Indoor Localisation System) has progressed in several ways. The GNSS (Global Navigation Satellite System) has been introduced to be fused with the Inertial and geomagnetic information coming from the foot-mounted IMU (Inertial measurement System). Both a first GNSS quality metric and a preliminary fusion strategy have been designed. Several experimental data (indoor, outdoor, and mixed) have been collected and both the quality metric and the fusion paradigm have been tested (see figure below). In addition, two different GNSS receivers have been employed: 1) the fix (latitude and longitude) coming from a commercial smartphone, with no RTK (Real Time Kinematics) augmentation; 2) the fix coming from a GNSS development kit (Simplertk2b v1 1, with ZED-F9P chip), with RTK augmentation.

  

• WP8 Citizen Involvement and City Integration System: Our work package aims to integrate and involve city Internet of Things devices and citizens that may be affected by emergency situations as part of the overall TeamAware solution. Our work package has now started and we are working on the initial steps in prototyping and design of the detailed solution for these purposes. This includes the integration of relevant publicly available social media messages shared by citizens, continuing work on a custom mobile phone app that we prototyped prior to the project as proof of concept and beginning work on an automatic framework to provide guidance to citizens caught up in emergency situations.

  

• WP9 Secure and Standardised Cloud Network: In this workpackage, we designed and installed initial version of the Secure Cloud Architecture in SRDC premises. The system is based on Kubernetes and we provide necessary installation guidelines; therefore, it can be installed to bare metal servers in any private network. In addition to the architecture, we currently work on security/privacy mechanism for TeamAware. The mechanism will be based RBAC (Role based access control) with the use of Open ID Connect (together with OAuth and User Managed Access specifications) and as the software, we will use open-source Keycloak tool. Furthermore, we are building the domain ontology through which all of the TeamAware components talk. The ontology will be standard-based and we will benefit from widely used emergency management and IoT standards like OASIS Emergency Data Exchange Language (EDXL) and OGC SensorThings API.

  

• WP10 TeamAware AI Platform Software: Within the WP10 work package of the TeamAware project, we worked hard on the further definition and detail planning of the TeamAware platform software design and architecture in conjunction with WP2 and WP9. Together with our partners from the project, we introduced and defined the structure of the relevant data flow and resulting parameters and the connection points to the different platform modules. In regard to the database, the architectural design started to integrate this information in its planning process. The same goes for the data fusion and front-end modules for the operator within the software platform of TeamAware. Within WP10 directly, the planning and design of the data fusion modules has started, which includes robust and detailed review of existing and new data fusion methods and their evaluation towards the integration within the TeamAware ecosystem. Singular modules are currently being evaluated towards their usability in respect to the specific sensors and data present in the project as well as the resulting technical prerequisites and needs. The operator front-end module from WP10 is in its design and planning phase also, which includes major incorporation of the outcome of WP11 to form a coherent access for the end-users to the system and definitions of interfaces for data exchange and communication with other modules within WP10 and the whole TeamAware ecosystem.

• WP11 TeamAware AR/Mobile Interfaces: After the kickoff, on work package 11 we have been focusing on 2 aspects:

  • First, we researched the best suitable technologies for the project and developed mockups for the AR and Mobile user interfaces. Then, we contacted the end users to get their input on this first design. We got valuable information from them that will help us design solutions better suited to their needs. A second iteration of the user interfaces reflecting the input we received is already under development.

  • The second focus of our work has been to start working together with WP9 and WP10 to make the integration with the platform as early as possible. This would provide us with more time to understand what data we have, how we should retrieve and lastly find the best way to present it to the final users. We have already had some productive conversations and more meetings are already planned to continue cooperation between our work packages.

On the other hand we had promising achievements in our dissemination activities. Our partner EUCENTRE published the following journal publication.

Use of UAS for damage inspection and assessment of bridge infrastructures, M.Mandirola, C.Casarotti, S.Peloso, I.Lanese, E.Brunesi, I.Senaldi, International Journal of Disaster Risk Reduction, Volume 72, 1 April 2022, 102824

Our partner AITEX presented our project in their regular magazine.

Furthermore, DUNE prepared an editorial to be published in the "Tecnologia ed Innovazione" magazine (paper and online), Issue March 2022.

We will continue to update you on our exciting development activities, so look out for more information about how TeamAware is progressing in future issues of our Newsletter!