MINDS CDT doctoral researcher | University of Southampton | MINDS CDT, 2022
Helping large language models learn new information without forgetting what they already know
Rudra Mutalik is a MINDS postgraduate researcher working on continual learning for large language models: how AI systems can keep taking in new information over time without losing what they already know.
Research at a glance
| Research focus | Continual learning for large language models |
| MINDS theme | Embedded Artificial Intelligence |
| Technologies | Large language models, transformer models, parameter-efficient fine-tuning |
| Wider applications | Keeping language models up to date in fast-moving fields such as medicine, law and current affairs |
| Current status | Doctoral researcher |
| Due to graduate | 2027 |
Updating AI without starting again
The problem Rudra is tackling is known as catastrophic forgetting. When a language model is updated with new data, whether to learn about recent events, a new specialist domain or updated facts, it tends to overwrite previously learned knowledge.
The standard response is to retrain large portions of the model. That process is slow, expensive and energy-intensive, which makes it impractical for fast-moving fields like medicine, law or current affairs.
Rudra’s research investigates how the knowledge inside a language model changes as it learns something new, and develops methods that let models update what they know without destabilising what they have already learned.
Why continual learning matters
Lower cost of staying current
Methods that update a model without extensive retraining could reduce the computing time and cost needed to keep it up to date.
More reliable knowledge over time
Reducing catastrophic forgetting may help updated models keep the knowledge people already rely on, supporting more consistent behaviour.
More sustainable AI
Relying less on large-scale retraining has the potential to reduce the energy used in maintaining large models.
Research approach
Rudra studies how the internal representations inside transformer models shift and drift as they learn new tasks. Understanding that drift helps identify where and why forgetting happens.
The work draws on parameter-efficient fine-tuning, which adjusts a small proportion of a model’s parameters rather than the whole model, alongside representation analysis and evaluation in both text-only and multimodal settings, where models work with images as well as text.
Research and outputs
Featured output: CPIQA climate research dataset
Presented at the ClimateNLP Workshop at ACL 2025, CPIQA: Climate Paper Image Question Answering Dataset for Retrieval-Augmented Generation with Context-based Query Expansion introduced a multimodal question-answering dataset built from 4,551 climate research papers, containing 54,612 question-answer pairs. The dataset tests how well AI systems can answer questions using both the text and the figures in scientific papers.
The dataset and code are openly available:
CLPsych 2024 shared task
Rudra co-authored Extracting and Summarizing Evidence of Suicidal Ideation in Social Media Contents Using Large Language Models for the CLPsych 2024 shared task, which focused on using language models to identify indicators of risk in online posts. The team’s submission achieved the highest score in the task.
Research profiles
- Rudra Mutalik on ORCID
- Rudra Mutalik on Google Scholar
- Rudra Mutalik on GitHub
- Rudra Mutalik’s research website
Public engagement
Alongside his research, Rudra takes part in public outreach, demonstrating an LLM-enabled talking robot and a gesture-controlled DJ system to general audiences at public events.
The DJ system, DJ Air, began as the winning hackathon idea at the 2025 MINDS and SustAI Innovation Camp. Rudra was on the original team and wrote much of the main software. Built with MINDS postgraduate researchers Keniel Peart and Epifanios Baikas and SustAI’s Theo Parpounas, DJ Air won Best Demonstration at the ECS Student Showcase and was demonstrated to the public at the British Science Festival 2026.
Related stories
- Mixing music with a wave of the hand: the gesture recognition behind DJ Air
- Robots, rivers and neurons: our AI researchers meet the public at the British Science Festival
- MINDS researcher part of winning team at international computational linguistics workshop
Connect with Rudra
After his PhD, Rudra is planning to move into a research position in industry. He is due to graduate in 2027.
To discuss Rudra’s research or potential collaboration, connect with him on LinkedIn or visit his research website. For general enquiries about MINDS research, contact the MINDS team at mindscdt@soton.ac.uk.