Fall  >>  2012  

Our Augmented Reality

An Industry Perspective on the Future of AR Technology

FIGURE 1. An AR view of Boston Common in Boston, Massachusetts.
Photo courtesy of Intergraph Government Solutions.

Vice President of Geospatial Solutions
Intergraph Government Solutions
Reston, Va.

In today’s world of technology, it’s certainly exciting to work in the geospatial industry and experience first-hand the vividly colored images and richly detailed 3D models of our planet. However, we also appreciate that there is inherent value beyond just the visual beauty. The processing, analyzing and integrating of this geospatial information brings increased value to many beyond those who work in the industry.

As we examine key trends in our market and in the broader IT sector in general, such as cloud computing, the explosion of social media, and mobile application integration, we see that these trends are collectively driving new and exciting ways in which we take advantage of geospatial information. However, some of the future applications of geospatial information and technology may be processed in a behind-the-scenes way as a "geospatial engine" powering technology from the background. These applications may be processed in such a way that the end users don’t see an image, a map or a 3D model. In some cases, they may not see any visual representation at all. Augmented Reality (AR) operates this way, and continues to grow in potential within our industry.

FIGURE 2. AR depiction of train information at a New York City subway stop, courtesy of Google.
FIGURE 3. AR information can show past and current information on friends in a social network. Courtesy of Google.

Today’s tablets and smartphones are already equipped with many sensors - GPS chips for determining a position on the planet, accelerometers for determining your orientation to your environment and for recording fine movements... Smartphone cameras visually capture our surroundings and use microphones to record voices and other sounds.

Other innovative applications also take advantage of these sensors to make our lives easier. GPS data are integrated with navigation systems. Star gazing apps understand our position on the planet and display a depiction of the night sky right in front of our eyes as we peer upward. Voice recognition allows us to engage in a more conversational experience as software translates our spoken questions to text and sends them directly to search engines.

Cameras also scan barcodes and QR codes that instantly interpret and explain meanings behind a puzzle of lines and numbers. These codes instantly link us to retail or marketing information, or in some cases pass a snapshot to a web service "in the cloud" that recognizes details in the picture. The pictures transmit images, such as a product logo or even someone’s face. Augmented Reality brings all of these sensors, and others, together in an integrated, real-time manner to create a more immersive, productive, and exciting experience.

Essentially, Augmented Reality blends the real and digital worlds. The concept is that your surroundings are instantaneously sensed and processed and a derived digital set of information is transmitted back to the viewer, perhaps through seamless integration into eyeglasses, visors on helmets, vehicle windshields, or even an audio cue. True innovation, then, occurs when the digital information is transparently rendered onto these fields of view and dynamically synchronized with the actual view of scenery. See Figures 1-4.

One industry already does this integration well. Hollywood has already perfected the art of blending the digital and real worlds – even in 3D – whether it is digital dinosaurs integrated into a live jungle or computer-generated imagery (CGI) Transformers bounding through a real city.

Augmented Reality also does that type of hybridization, but with at least two key differences:

  • All individuals have their own unique perspective on the environment at any point in time, so each user will receive slightly different information from the geospatial engine.
  • And, this all happens in real time.

Geospatial information and technology play critical roles in an AR solution. The first role is in the creation of a detailed representation of the environment that viewers may eventually walk, drive, or fly through in the future, such as a mountain pass or an urban cityscape. The second role is fulfilled by a "geospatial processing engine" that provides powerful frames of reference and models to which comparisons of the real world can be made. Once these steps occur, a number of geospatial processes may happen, in order to send some type of derived digital information back to the user – all depending on the user’s needs and interests at that point in time. That information may consist of a digital geospatial "cue" such as a direction arrow or simply a signpost displaying detail about the object currently being viewed.

Figure 4. AR information displayed on a smartphone screen, courtesy of Glogger.
As device form factors shrink from desktop to laptop to tablet to smartphone, it is important to see that the next device actually has no form factor at all. Instead of utilizing a stand-alone computing device with a screen, the information is instead seamlessly integrated into already existing equipment. Even though modern vehicle navigation systems include audible instructions and voice command recognition, which do a great deal to reduce driver distraction to keep their eyes on the road, there is still a moving map display on the console.

Augmented Reality solutions will take this one step further, where no limited dashboard real estate is taken up with any map display. Appropriate details are displayed on the car’s windshield, thereby reducing distraction to the driver. GPS glyphs, such as turn arrows and distance indicators could even be projected semi-transparently on the car’s windshield.

Augmented Reality solutions can enhance much of our daily lives on a personal level, especially since they drive a strong convergence of social media and collaboration technologies. Tiny video cameras built into a pair of eye glasses will allow friends and colleagues to see the wearer’s viewpoint in real time while a conversation occurs via an integrated microphone and speakers. See Figure 5.

A recent online video from Wired magazine highlights "Project Glass" from Google, an exciting prototype of an AR solution that provides a glimpse into the vast possibilities that will someday be available. This video shows a few basic consumer examples of the powerful application of Augmented Reality, and we can really start to see how remote resources can play a more active role in a local activity: http://bit.ly/LLkp6J.

Google’s official page for Project Glass is here, featuring fashion designer Diane von Furstenberg wearing "Glass": g.co/projectglass.

FIGURE 5. Google’s Project Glass uses glasses to bring Augmented Reality information to the wearer. Courtesy of Google.
When we expand beyond the consumer space, we can see there are also many vital applications for field-based activities, such as physical plant inspections, fighting forest fires, and even military operations. An engineer walking through a petrochemical plant could be presented with vital pieces of spatial information and other annotations about the facility displayed on a pair of safety goggles. This allows him to perform his tasks in the most efficient manner possible. A first responder tackling a large forest fire would greatly benefit from having cues displayed on his visor that indicate the location of other fire fighters or key resources in his vicinity. Special Forces personnel that are storming an insurgent’s safehouse could see a continuous display readout of the locations of their teammates and could realize safer and more effective missions, as friendly fire incidents would be reduced and mission objectives could be carried out more safely and quickly.

Augmented Reality solutions will require speed of processing and reliability of wireless connectivity in order to be successful. As Figure 6 shows, this process involves:

  • collecting data in real time,
  • transmitting back to a central geospatial engine, which then
  • compares the input to existing models and datasets,
  • generates artifacts, and
  • sends that information back to a user – all happening in real time.

Initial implementations may be relatively basic – just taking advantage of GPS and accelerometer data – but the real value will be when the vast archive of geospatial information becomes an integral part of the process.

Terrain models based on LiDAR collections are creating a rich and detailed 3D representation of many places on this earth. These models may someday play a central role as part of an AR geospatial processing engine. The models become the baseline to which continuous streams of input from sensors are compared, generating some derived piece of information. This can range from the name or address of a building to the height of a mountain in the distance, and even to details about the subterranean geology.

Below are a few additional examples of how derived geospatial results can enhance an AR solution:

  • Displaying pop-up messages with street address numbers on your windshield as you drive down the street.
  • Displaying elevation contours on the inside of your sunglasses as you bike up a mountain trail or view a ridgeline from a distance.
  • Re-running your last run – as you head out of your house in the morning for a brisk run you can follow the dashed line representing the last route that you ran – a modern version of "follow the yellow brick road!"
  • Displaying location markers on the visor of an explosive ordnance technician, indicating locations of all previous improvised explosive device (IED) placements as he walks down the street.

These are just a few initial examples of how integrating geospatial technology and information into Augmented Reality solutions can enrich the way we live and work. And we are at a stage where geospatial information is truly becoming a vital part of our everyday lives and is playing a role in more ways than ever before.

We are becoming more dependent on geospatial technologies, and in some cases, such as vehicle navigation systems, technologies have become such an integral part of our lives that we take the technology and its daily benefits for granted. Geospatial resources have evolved into an infrastructure - like running water, electricity, or the Internet.

FIGURE 6. The Augmented Reality process, courtesy of Intergraph Government Solutions.
Because of this quickly growing business, those of us in the geospatial industry have a large responsibility to ensure that these resources remain available and that they provide accurate results. Additionally, information initially collected for one specific purpose today may actually have some unforeseen important usage tomorrow – possibly as an integral part of an AR solution.

One of the key steps to take now is to adhere strictly to open geospatial standards for collecting, storing and sharing geospatial information. Open standards allow geospatial information to exist in a vendor-neutral format, to be shared with a wider variety of users, and to play a key role in a wider variety of applications. Open geospatial standards do a great deal to foster collaboration among many organizations and individuals. This collaboration increases the overall diversity of technologies and services, which in turn accelerate innovation collectively within the community.

I attended Hexagon 2012 earlier this year – Hexagon’s annual international conference. During that event, I learned about many of the latest innovations in geospatial technology across the family of Hexagon companies. These included the latest in remote sensing solutions from Leica Geosystems, state-of-the-art LiDAR processing and Dynamic GIS solutions from Intergraph Security, Government and Infrastructure (SGI), and an intelligent, rule-based 3D design environment from Intergraph Process, Power, and Marine (PPM), to name just a few. It was during that conference that I could see more clearly than ever before that our industry was heading toward an exciting future where Augmented Reality would truly become our reality.

Now is a fascinating time for those of us in the geospatial industry. We are continually developing new and improved solutions for collecting, processing, analyzing, visualizing, and sharing geospatial data. These systems are providing great value to a broad set of consumers. Knowing that these awe-inspiring images, 3D models, and other types of geospatial information may have a purpose beyond the current, obvious usage is a responsible viewpoint to take.

It is important to realize that, in some cases, the most valuable result of a process that, in the end, produces a stunning high-resolution image or 3D model may not be the image or model itself at all. Rather, it may be a very simple, but extremely important, artifact derived from automated analysis of that data – an ‘X’ marks the spot, an arrow indicating which way to turn or a distance readout to an entrance of a safe house. As a community, we have a responsibility to set the stage for the future of Augmented Reality through the growth of our existing geospatial technology. We should completely embrace open geospatial standards and store and share data in formats that make it possible to integrate that data with the most diverse set of technologies possible.


Sensors & Systems | Monitoring, Analyzing and Adapting to Global Change | Stay in tune with the transformation. Subscribe to the free weekly newsletter.