Josep Blat and Vanesa Daza coordinate PRESENT, new H2020 RIA with leading players in entertainment, media and advertising

Josep Blat and Vanesa Daza coordinate the new Research and Innovation action PRESENT (Photoreal REaltime Sentient ENTity), selected in call H2020/ICT/2018/3, and which currently enters into negotiation.

PRESENT is a proposal for a three-year Research and Innovation project to create virtual digital companions––embodied agents––that look entirely naturalistic, demonstrate emotional sensitivity, can establish meaningful dialogue, add sense to the experience, and act as trustworthy guardians and guides in the interfaces for AR, VR and more traditional forms of media.

There is no higher quality interaction than the human experience when we use all our senses together with language and cognition to understand our surroundings and––above all—to interact with other people. We interact with today’s ‘Intelligent Personal Assistants’ primarily by voice; communication is episodic, based on a request-response model. The user does not see the assistant, which does not take advantage of visual and emotional clues or evolve over time. However, advances in the real-time creation of photorealistic computer generated characters, coupled with emotion recognition and behaviour, and natural language technologies, allow us to envisage virtual agents that are realistic in both looks and behaviour; that can interact with users through vision, sound, touch and movement as they navigate rich and complex environments; converse in a natural manner; respond to moods and emotional states; and evolve in response to user behaviour. In addition, the innovative blockchain technology will be used to take care of digital identities, in order to guarantee a greater degree of trust and security of the environment.

PRESENT will create and demonstrate a set of practical tools, a pipeline and APIs for creating realistic embodied agents and incorporating them in interfaces for a wide range of applications in entertainment, media and advertising. The international partnership includes the Oscar-winning VFX company Framestore; technology developers Brainstorm, Cubic Motion and IKinema; Europe’s largest certification authority InfoCert; research groups from Universitat Pompeu Fabra, Universität Augsburg and Inria; and the pioneers of immersive virtual reality performance CREW.