Kicked off in November 2020, the EU-funded XR4DRAMA project recently wrapped up its first year – with a number of impressive technical achievements successfully implemented by a Pan-European consortium.
XR4DRAMA – which resolves to “Extended Reality For Disaster Management And Media Planning” – aims to build digital situation awareness (SA) tools for basically any organization that sends staff to unfamiliar, unsteady, or unsafe locations.
The idea is to improve SA by exploiting multi-modal data and XR technology. In concrete terms, this means:
- collecting geospatial data, all kinds of web content, sensor readings, cultural context etc.
- structuring, interpreting, and ingesting it into a single platform
- creating a “virtual twin” of a preselected environment
- making it accessible and maintainable via user-friendly interfaces that potentially support everything from light-weight mobile devices to complex VR setups.
XR4DRAMA partners
Seven partners have taken on the “better SA via XR” challenge. They all play different roles – and complement each other:
The Autorità di bacino Distrettuale Delle Alpi Orientali (Alto Adriatico Water Authority, AAWA) is an Italian public body that knows a great deal about flood defense, hydrogeological modelling, disaster risk mapping etc. In XR4DRAMA, AAWA is mainly responsible for the disaster management pilots. The Information Technologies Institute (ITI) at the Centre for Research and Technology (CERTH) is XR4DRAMA’s lead coordinator and also in charge of technical implementation. DW Innovation (a special unit of international broadcaster Deusche Welle) takes care of scenario development and user requirements (with a focus on a media planning pilot) and also handles external communications/dissemination. Another partner is German software developer Nurogames – an expert in AR, VR, 3D animation and interfaces. Smartex, also hailing from Italy, creates smart garments, e.g. textiles that can sense the bio signs of a first responder. The Natural Language Processing Group at the Universitat Pompeu Fabra (UPF-TALN) offers their expertise in natural language processing (NLP/NLProc): Disaster managers and media workers may profit from quickly processed natural language info concerning their deployment area – and maybe their stress levels can be detected via their speech? Last, but not least, Greek SME up2metric supports XR4DRAMA by providing the latest in computer vision, i.e. technology that makes cameras and other gadgets see and understand what is going on in a designated location.
Technical achievements so far
Over the course of XR4DRAMA’s first 12 months, the consortium has achieved major achievements. Our company had significant involvement in the following in terms of technical research and engineering:
Backend tools/modules
- recording of drone footage (thousands of images and videos for the media production use case captured on Corfu, Greece)
- establishment of a satellite service able to exploit satellite data from the sentinel hub (true color, all bands, DEM) for specific areas and time spans; the data is turned into a rough 3D terrain model which can be used in the VR or geo services applications
- finalization of a (constantly running) 3D reconstruction service that is able to
○ receive multiple requests (images, videos)
○ generate a 3D model
○ simplify the produced mesh
○ provide the result to XR4DRAMA users
- completion of the geo service modules (initial version, incl. geoserver and GIS); data from OSM, opengov.gr and AAWA are aggregated and organized via categories and subcategories that address the user requirements; existing data can be updated/extended (also via user frontend tools)
User/frontend tools
- development of an AR app (1st version)
○ improves situation awareness of first responders, location scouts, and other users in the field by providing relevant platform data (e.g. geospatial information)
○ allows for bilateral communication: both users in the field and control room staff can send updates
○ offers the following first set of user features: add/edit POIs, upload multimedia files, add comments, manage project, manage user profile
○ offers 2D maps and an initial AR view (e.g. display of POIs, POI info, and navigation on top of real world view)
○ exploits mobile phone sensors (GNSS, IMU, compass) to help users in the field navigate and make sure control room staff can locate them
Outlook on 2022
Upcoming project months will see a lot of system integration, fine-tuning, and – of course – user testing. Laptops, headsets, and smart devices have already been procured and set up to thoroughly check the prototype applications. XR4DRAMA is officially scheduled to run until the end of 2022, with a possible extension into 2023, as the Corona crisis has taken a toll on the proper execution of some of the project’s pilot use cases
Further information on XR4DRAMA
XR4DRAMA official website: xr4drama.eu
XR4DRAMA on Twitter: twitter.com/xr4drama
XR4DRAMA on LinkedIn: linkedin.com/company/xr4drama
