On Tuesday, Niantic, the company behind Pokémon Go, announced the roll-out of a hyper-detailed mapping system they hope will pave the way for the expanded use of augmented reality (AR) in everyday life. Niantic is an organization that has been gathering spatial data in highly-trafficked places around the world for many years. It now wants to make these detailed maps available to developers who can use them to build their games, history lessons, and other audiovisual experiences. However, privacy experts have concerns about AR, and the implications for public surveillance.
AR consists of digital elements that appear within real-world environments, which can be seen either through one’s phone or smart glasses (like the defunct Google Glass). Snapchat and Instagram filters allow you to change your voice, appearance or even the scenery.
John Hanke from Niantic is adamant that AR will always be part of everyday life and in more advanced ways. You might see the directions of your destination overlayed on streets, using smart glasses, or view a building to determine its past and present architectural forms.
His company’s new 3D map will allow developers to see exactly how their users move through the world and view their surroundings, which Hanke says is a key step towards “opening this science fiction reality we dream about. We can start putting virtual things into the world that are attached to the right parts of that physical world,” he said in an interview.
This map was made by combining more than 100 million video clips from Niantic users (who shared their screens), surveyors, and developers. The most developed parts of the map are in dense city centers where they’ve collected the most data, including San Francisco, London, Tokyo, and New York City.
Independent AR developers will now have access to the technology—starting free in beta—on a platform called Lightship. There they might create virtual animals that hide behind park benches; plaques for public statues delving into forgotten histories; or mobile characters “painted on the ground showing the path to your destination,” says Hanke.
One of the first projects created with the new map’s technology is a collaboration with the artist JR, called JR Reality. JR is well-known for putting giant photographs of people into public spaces. Niantic’s technology allows users to add their own JR-style portraits or voice messages to specific public locations, adding to those of everyone else who have made the same pilgrimage.
“Have you ever passed somebody on the street and wondered what their story is?” JR wrote in a statement. “It’s time to go outside and explore, and reconnect with one another and show the world your face again. Together we can tell the world your story and meet the amazing people that live in your city.”
JR, an artist shows his AR project JR Reality created using Niantic Technology.
JR Reality
Hanke, meanwhile, is particularly excited about how the map might improve Niantic’s flagship product: Pokémon Go. The viral advertisement that first introduced the game to the public in 2015 featured Mewtwo flying over Times Square. However, most people only view animated graphics from their phones. “This map paves the way for that giant Mewtwo above the crowd: to have the shared experience of many people watching the same thing from different vantage points,” Hanke says. “This is what we’ve been dreaming of since the beginning.”
Pokémon Go was arguably the first successful commercial use of AR. It was downloaded more than 500 million times in its first year, with aspiring trainers hunting Pokémon all over the world. Jacob Navok, the CEO of Genvid Technologies and one of the leading thinkers on the metaverse, says that Niantic’s pedigree means their new map will likely be a sought-after tool for AR developers. “Releasing more, better and cheaper tools to outside developers is key to the growth of the sector,” he wrote in an email to TIME. “Developers who choose Lightship will do so knowing that Niantic understands how to build product and not just APIs [programming code].”
Many privacy watchdogs are concerned about the impact of AR technology on privacy. They worry that companies like Niantic will collect an astonishing amount of information that would be vulnerable to hacks, and that people’s movements could be tracked. “Every image that an AR device records is potentially one court order away from becoming a government tracking device,” says Albert Fox Cahn, the executive director of the Surveillance Technology Oversight Project. He points to search warrants that have collected recordings from Amazon’s Alexa and Ring cameras. Cahn believes that geofence warrants (in which police search for location data on all persons within a particular location in a given time period) could pose a threat to AR users. However, Virginia just declared one of these warrants unconstitutional.
“It’s something we take very seriously,” Kjell Bronder, a product lead for Lightship, Niantic’s developer platform, says regarding privacy. “These players may upload their contributions [to the map], we do everything in the background to anonymize and make sure that a lot of this data is scrubbed.”
Niantic will also launch Campfire, its social media platform. This app is map-based and users can see where other people are located in their area. They can then message, send content, or organize events. “Our big thesis is that the metaverse is something that happens out in the real world,” Hanke says.
Read More From Time
