​This ‘Pokémon Go’ Hack Lets Pokémon Interact with the Real World


Everyone’s already crazy about Pokémon Go, the augmented reality phone game that superimposes virtual characters on top of the real world environment. But what if the Pokémon could go one step further and interact with the authentic environment? Could Zubat flutter through leaves in a tree?

By Madison Margolin | MOTHERBOARD

Abe Davis, a researcher at MIT CSAIL (Computer Science and Artificial Intelligence Laboratory), developed a technique called „interactive dynamic video” that allows people to „reach in“ and interact with objects in the video frame. With the new technology, outlined in a report with researcher Fredo Durand and doctoral student Justin G. Chen, the character Charmander jumping in a real life puddle would create a splash on screen.

Interactive Dynamic Video works by capturing and using the tiny object vibrations—almost undetectable movements—in order to create video simulations that users can interact with on their devices. And it makes the line between virtual reality and actual reality even more blurry.

„This technique lets us capture the physical behavior of objects, which gives us a way to play with them in virtual space,“ Davis said. „By making videos interactive, we can predict how objects will respond to unknown forces and explore new ways to engage with videos.“

read more