Work package 6
Objective: Smart Data Management
Task 6.1 – Decentralised Virtualized Data Layer
Design and develop a trust-driven decentralized data layer for an open network of autonomous data providers, forming a key part of the Smart Data Management (SDM) system based on Onedata. This layer manages diverse distributed datasets efficiently, enabling processing without replication and providing unified, secure access.
It enhances fault tolerance, scalability, and security, while maintaining decentralization to reduce latency and eliminate bottlenecks. SDM connects to SPICE via the Cognitive Mapper (T5.3) for secure, scalable data handling.
Task 6.2 – Decentralised Data Governance
Develop a federated data governance framework supporting local decision-making and lifecycle management, leveraging European e-infrastructure storage services and IoT/Fog integration.
This system will ensure privacy, security, and compliance while managing diverse datasets. The XaaS marketplace (T7.4), supported by the Unified Knowledge Layer (T4.1, T4.2), will enable decentralized collaboration and access across organizations.
A data anonymization service will balance privacy and information retention for big data processing.
Task 6.3 – Large Language Models for Data Discovery
Leverage large language models (LLMs) integrated with the SDM and UKL (WP4) to automate data discovery, categorization, and semantic search.
LLMs will extract insights from large, raw datasets, revealing hidden patterns and improving contextual understanding. The goal is to make data exploration intuitive, empowering users—regardless of expertise—to derive actionable insights efficiently.
Task 6.4 – Smart Data Movements
Optimize data movement across distributed environments using AI, ML, and predictive analytics (in collaboration with T4.1, T4.2).
The system will anticipate data needs, preemptively relocate data, and select optimal transfer routes based on system load, cost, integrity, and privacy. This ensures efficient, timely, and cost-effective data availability without redundancy or delay.
Task 6.5 – EOSC Integration
Integrate SPICE with the European Open Science Cloud (EOSC) through collaboration with EOSC stakeholders and alignment with EOSC Core and EOSC Exchange.
Focus on data standardization, metadata consistency, and best practices for cross-border collaboration.
Using Onedata as a gateway, the task aims to build a unified distributed data management layer compatible with EOSC, simplifying access for researchers and fostering interoperability across Europe.
Task 6.6 – Market-oriented EU Common Data Spaces
Integrate SPICE with European common data spaces following IDSA and Gaia-X standards.
Initially, SPICE will act as a Resource Owner, later evolving into an autonomous Provider. The work includes researching Blockchain, smart contracts, ontologies, and developing a Connector aligned with the Eclipse Data Space Components project.
Together with T6.5, this will enable full integration with EU data spaces, supporting the European Data Strategy and ensuring the platform’s long-term sustainability for both industry and research communities.