11-03-2025
Photo: Amazon Science
The research proposal on how to make Large Language Models (LLMs) more trustworthy, by David Semedo and João Magalhães, professors at the Department of Computer Science at NOVA FCT and researchers of the Multimodal Systems group at NOVA LINCS, was selected for the Amazon Nova AI Challenge.
The team includes the students Diogo Silva (team leader) Rafael Ferreira, and Diogo Tavares, along with new key additions—João Pedro Soares, Daniel Pina, Iago Paulo, Artur Horal, and Henrique Paz.
This edition is dedicated to the theme "Trusted AI: advancing secure, AI-assisted software development to build safer, more reliable applications". Building on previous challenges—where innovative task-assistant LLMs were developed and key security concerns were addressed—this challenge aims to achieve security by design in Generative AI.
The research challenge focuses on secure code generation with LLMs that are resilient to attacks, with the participation of 10 universities worldwide: Carnegie Mellon University, University of Illinois Urbana-Champaign, Czech Tech University, University of California at Davis, Virginia Tech, University of Texas at Dallas, Columbia University, University of Wisconsin, Purdue University, and NOVA FCT.
NOVA FCT’s team will explore methods to push LLMs to their limits, probing their weaknesses and evaluating how trustworthy they truly are, towards identifying ways to improve their trustability. In 2023, the team developed an award-winning LLM-based system to assist in manual tasks.
