A Chain of Trust against Fake Multimedia Content
Project
One of the latest challenges in the area of visual misinformation is the appearance of DeepFakes, i.e. AI-generated synthetic media using deep learning to create fake videos by swapping the face of a person with the face of another person.
Although the scientific community is making progress in implementing tools that can help identify DeepFakes, we are still far from having methods that can automatically recognise any forgery. Even more challenging is the “arms race” nature of the problem, i.e. when new ways appear, new synthetic media may be generated that could circumvent them. To equip the fact-checking community with the necessary knowledge and resources to understand and counter the problem of DeepFakes, we propose the creation of a readily accessible web-based portal consisting of:
- DeepFakeChain (DFC): a blockchain and related smart contracts for indexing content jointly identified as DeepFake by the required number of moderators, together with annotations that could help other users understand how it is possible to visually identify the forgeries in the cases that this is possible.
- The DeepFake annotator: a tool that will allow authenticated users to annotate areas and/or time segments of candidate DeepFake material and which will enable other users either to confirm/reject its claims or even reuse these findings when working on other cases.
- The search engine: an area where users can upload video or image content to check if visually similar material is already present in DFC.
- The DeepFake checker: a service where users can submit video or image content to receive an automatic reply on the possibility of this being a DeepFake.
Team
CERTH (Centre for Research and Technology Hellas) – ITI (Information Technologies Institute) participate in the proposed project through the Multimedia Knowledge and Social Media Analytics Laboratory (MKLab), and specifically the Media Verification Team (MeVer), which has significant experience in the development of multimedia content analysis technologies from social networks and the application of network analysis techniques, signal processing and artificial intelligence. CERTH-ITI is one of the leading research institutes in Europe engaged in research in the fields of Information Technologies and Telecommunications, employing almost 400 researchers and supporting personnel.
The participating laboratory (MKLab) has significant experience and scientific expertise in web and social media mining and the discovery and extraction of multi-modal data from heterogeneous Internet sources such as web resources, social networks and open data repositories. MKLab has been involved with leading roles in over 100 projects, many in the media domain. The Media Verification (MeVer) team is the developer and owner of multimedia forensics tools widely used in research against disinformation.
In the context of several EC-funded projects (REVEAL, InVID, WeVerify and AI4Media), MeVer has been developing tools for image and video analysis that greatly assist in the understanding of forgeries in multimedia content (information and demos are available at https://mever.iti.gr/web/resources). Some of these are already available through the InVID-WeVerify plugin to more than 70,000 users worldwide (as of May 2022), including journalists, fact-checkers and human rights defenders.
Established in 2019, ZELUS IKE is a Greek ICT start-up focusing on developing safe and innovative products based on research strength activities towards top-class, robust and user-driven and validated solutions. Empowered by an experienced, strong and skilful team, ZELUS currently delivers services for Digital Forensic Investigations and Analysis (both cyber and physical), Threat Hunting and security-related Situational Awareness, targeting decision-makers in the field and spanning from collection and analysis systems to network security services and privacy-preserving systems. Following a concise strategic plan, ZELUS aims to extend their know-how in diverse domains and test the proposed solutions in real-life environments to penetrate to the market with industrial-tested and user-assessed products. Relevant products include the SmartViZ toolkit, which allows enhanced analysis of large volumes of data via intuitive visualisations for IT and non-IT experts to discover explanations about patterns, behaviours, correlations of data items, hidden relationships and insights, and the Digital Forensics Toolkit (DFT), a versatile solution for digital forensics investigations, analysis of digital evidence and advanced visualisations for cyber security incidents, situation and risk awareness.
Dr Symeon (Akis) Papadopoulos
Project Lead
Dr George Kordopatis
Researcher/developer
Ms Olga Papadopoulou
Demonstration and training
Dr Evangelia Kartsounidou
Evaluation
Ms Stella Markopoulou
Software Developer
Blockchain Experience
The blockchain expertise lies within ZELUS, a Greek ICT start-up established in 2019 to deliver innovative products and services in cybercrime and forensic investigation. The current product portfolio includes SmartViZ, a toolkit for enhanced analysis of large volumes of data via intuitive visualisations, and the Digital Forensics Toolkit (DFT) for digital forensics investigations, analysis of digital evidence and advanced visualisations for cyber security incidents, situations and risk awareness.
The company works with BlockChain capabilities to develop a content repository on cyber-attacks and response actions, empowered with blockchain-based security on attempts to compromise such content. Within DFCLab, we aim to expand our expertise with blockchain and distributed ledger technologies to support channel management, smart contract execution and moderators’ reputation scoring when checking and annotating multimedia online content for deepfakes