The Japan Times - Augmented books, Wallace and Gromit show VR future in Venice

Running alongside the world's oldest film festival, Venice Immersive is tucked away on a former quarantine island that transforms each year into a showcase for the latest frontiers of entertainment.

This year showed how quickly the tech is evolving.

Some experiences had users interacting with the virtual environment using hand controllers. Thus "Wallace & Gromit in The Grand Getaway" plunged them into the world of the famous animated duo.

Players become Gromit, helping him fix his hapless owner's contraptions and rescue them from an accidental trip to Mars.

"The interactivity in those worlds is increasingly precise and diverse," said Venice Immersive co-curator Michel Reilhac.

"Makers are finding ways to hijack the technology and use it in really unexpected ways."

Another experience used VR helmets to put several people at once in the studio of Spanish architect Antoni Gaudi, allowing them to snoop around his workshop and watch as his famous Barcelona cathedral rose up spectacularly around them.

AI was an inevitable talking point, with one experience using a mix of two AI apps, ChatGPT and Midjourney, to ask users about their deepest thoughts before creating a bespoke story and images based on their answers.

- 'Leap of faith' -

One of the most technologically impressive was "Jim Henson's The Storyteller".

Visitors don augmented-reality glasses to watch a 3D film come to life on a special book they hold in their hands, moving through different chapters as they turn the pages.

It is the latest innovation from VR pioneers Felix and Paul Studios, who have created immersive tours of the International Space Station, the Obama White House and LeBron James's training sessions.

The interactive book was another "leap of faith", co-founder Paul Raphael told AFP.

"We wanted to realise the dream of what an augmented book could be," he said -- but that required "pushing the technology so much further".

Cameras in the glasses read the surface of the pages and track their position in real time, which the algorithm, designed from scratch, uses to calculate where to overlay the constantly moving 3D images.

"The performance and speed at which it needs to happen is kind of insane," said Raphael.

With the emergence of new headsets from Apple and other companies, he believes augmented books could soon become widely available.

"It's early days and there's so much ground to cover," he said.

"Even after 10 years, it feels like we could do this our whole lives and still just scratch the surface."

- 'No longer solitary' -

The festival highlighted social experiences, particularly VR Chat, an online platform allowing users to meet and play in virtual worlds.

"VR immersive is no longer a solitary experience," said Reilhac.

"It's gained a social dimension -- and that's where it will find its 'killer app' that wins over a much bigger audience."

As the tools evolve, creating these virtual worlds has become much easier, with free templates available for newcomers to use.

"There is so much available technology now that it can be easily adopted by people who are not professionals," said co-curator Liz Rosenthal.

They welcome the recent shift of attention from the metaverse to artificial intelligence.

"The hype has moved on to AI, which is great, because it's weeded out the people who were into immersive just for the hype," said Reilhac.

As the tech evolves, there is a "greater depth of quality in the creative side," added Rosenthal.

"People are here because they're passionate. It's an exciting time to be in this space."

M.Matsumoto--JT