03-10-2023
Amazon has just announced the results of the Alexa Prize TaskBot Challenge 2, with the Department of Computer Sicence/NOVA LINCS team, from the NOVA FCT, taking first place. This is the second edition of the international competition that seeks to develop state-of-the-art conversational AI solutions.
The Alexa Prize TaskBot Challenge 2 is a university competition that asks students from around the world to solve one of conversational AI's most difficult problems: creating conversational AI experiences that respond to users' ever-changing needs while completing complex tasks. The hope is that TaskBot, Alexa's conversational agent, will communicate effectively and be able to guide the user through manual tasks, from cooking to making origami.
TaskBot is the first conversational AI challenge to incorporate multimodal experiences. This year, the competing teams were challenged to find solutions that incorporated more visual resources into each conversation. The Portuguese team, with the nom de guerre "TWIZ", mentored by Professor João Magalhães and David Semedo led by student Rafael Ferreira, developed an advanced and intuitive multimodal solution: language model processing techniques are integrated with video dialogue models. For example, to complete a complex manual task, this solution makes it possible to follow the instructions in detail in text and video.
“User dialogues in the Alexa TaskBot are unique, shedding a new light into the execution of manual tasks,” said Rafael Ferreira, the TWIZ team lead. “Leveraged by these dialogues, we learned that using TWIZ allowed us to steer conversations in a more contextual and insightful way.”
In the final phase of the competition, five university teams were selected based on customer feedback and the scientific merit of the technical articles produced, among other criteria, as finalists to participate in the final live interactions phase of the TaskBot Challenge 2. Alexa consumers interacted with the taskbots with the phrase "Alexa, let's work together" on their Amazon Echo or Fire TV devices. The evaluations and customer feedback helped the university teams improve their bots as the competition went on.
The winning NOVA FCT team is composed of Rafael Ferreira, Diogo Tavares, Diogo Silva, Rodrigo Valério, Inês Simões, Vasco Ramos, David Semedo, and João Bordalo, master's and doctoral students at NOVA. Last year, the same team took part in the first edition of the Amazon competition and came in second place.
