Project: Multi-modal event classification of social media content where training data is limited and/or emerging

Key information:

Abstract: 

Event extraction and classification is important for many applications, including in the defence industry, who use event extraction as a tool for open source intelligence (OSINT). The task is applicable for journalism, as breaking news event extraction and verification must be done within 20 minutes to have a chance of beating other news agencies to publish the story. A major challenge in both of these applications is the lack of annotated data as the event is happening and data is emerging.

Detection has traditionally based on a single type of data (text, audio, visual etc.), however there has been a recent increase in excitement with approaches using multiple modalities of data. The world around us, and by extension, events, contains many different forms of data, thus to understand this, we must consider all information to achieve optimal performance.

This research aims improve detection and extraction of events from multimodal data, focussing primarily on text and audio. We will be developing few-shot learning approaches, allowing for improved results with little training data. Lifelong learning will also feature in approaches, modelled initially by class incremental training, to address the need of changing event types.

We have shown that catastrophic forgetting is an issue with current deep-learning event extraction models. We are now investigating novel Large Language Model (LLM) training regimes for text-only and audio-enhanced models for event extraction in continual learning environments.