NOVA among 10 Universities selected by Amazon to investigate A.I. Multimodal conversational


The team coordinated by João Magalhães, professor at the Department of Computer Science  researcher at NOVA LINCS at the NOVA School of Science and Technology (FCT NOVA), formed by Rafael Ferreira, Diogo Tavares, Diogo Silva, Hélder Rodrigues, Gustavo Gonçalves, Mariana Bonito, Frederico Vicente, Rui Margarido, Paula Figueiredo and David Semedo is among the 10 international groups selected by Amazon USA to investigate “multimodal conversational AI”, within the scope of the international competition “Alexa TaskBot Challenge.”

Conversational assistants are increasingly ubiquitous in our society. Assistants like Amazon Alexa, Apple Siri, Google Assistant and Microsoft Cortana support us in a vast number of tasks, such as booking trips, online shopping, searching for information, music, controlling our home, among others. The future generation of conversational assistants will be multimodal and will have a considerably greater connection to the physical world than the previous generation. Assistants will be competent to guide consumers in manual tasks, involving visual and linguistic communication between assistant and consumer. This new level of sophistication demands largely unexplored AI methods, planning and knowledge of the world. This is the challenge that Amazon has proposed to Universities around the world. From the competition, only ten teams were selected: six from the United States of America, three from Europe (two from the United Kingdom and FCT NOVA, the only one from the European Union) and one from Asia.

The School has contributed to this advance through the research carried out in the group led by João Magalhães. This achievement reflects the quality and value of the research done at the NOVA School of Science and Technology. The line of research developed at the School has been bearing fruit in the area of ​​intelligent conversational agents in domains as general as Wikipedia and in specific domains such as online shopping, in partnership with Farfetch. The versatility of the investigated methods, the support of natural language, images/video and general data evidences the NOVA team as a very rich group with complementary skills.