Lucerne, Switzerland, at 5cm GSD, taken with a Leica ADS80 camera
Leica ADS80 near-infrared image of Wannengrat, Davos, draped over a digital surface model derived from the same image data using the new Leica XPro SGM module
Trimble Digital Sensor System (DSS), shown here in complete configuration including 60MP WideAngle camera, mounted on DSS Azimuth Mount
Trimble DSS RapidOrtho, sample ortho image showing site of a propane plant explosion in Toronto, Canada. DSS RapidOrtho is used by first responders and the military to prepare decision-ready ortho maps immediately upon landing, and individual orthos within seconds per image.
Vexcel’s UltraCam from Microsoft
Optech’s DiMAC Ultralight digital camera with embedded forward motion compensation
Geospatial Systems Inc.’s KCM-50 full frame 50MP camera module with field-replaceable mechanical shutter
Every aerial or satellite image you see on Google Earth, on The Weather Channel, or in the pages of this magazine was taken with a camera made by one of a handful of manufacturers that specialize in the sensors used for photogrammetry, remote sensing, and mapping — three growing markets that are rapidly converging. Five years ago, the aerial imaging industry was focused mostly on the transition from analog film cameras to digital ones. Since, manufacturers have increased the productivity of their cameras, making significant improvements in area coverage, image quality, and workflow — covering larger areas, at higher resolution, in less time.
The increased footprint saves users money by allowing them to fly fewer lines. While resolution has skyrocketed – for example, medium-format sensors have grown from 16 MP (megapixels) to 60 MP in just five years – manufacturers are focusing more and more on automating the workflow — that is, on providing their customers with more efficient ways to extract, process, analyze, store, and distribute the massive amounts of data that the latest generation of sensors capture.
In the future, some components, such as storage, will continue to shrink in size, while the swath width will continue to increase. The next big transition, however, will be to real-time, in-flight processing — which is especially important for rapid response, emergency management, and intelligence, surveillance, and reconnaissance (ISR) applications. The military has been experimenting with the ability to off-load and process image information in near real-time for situational awareness, and this capability is beginning to expand to the commercial world.
Likewise, video, which has been important in the military and intelligence markets, will eventually make its way to the commercial airborne photogrammetry market. “With the performance of many frame cameras today, the step to video is probably relatively small and there will probably be more activity in this regard going forward,” says Ruediger Wagner, product manager of airborne imaging sensors for Leica Geosystems AG.
The line between small, medium, and large format sensors has shifted over time and will continue to shift or even blur. Some companies advertise medium format cameras that have a larger footprint than their previous large format cameras. The definition was originally based on the size of the sensor: 24 millimeters x 36 millimeters was small format, between that and 60 millimeters x 90 millimeters was medium format, and everything bigger was large format. In the world of digital airborne photogrammetry and “virtual images,” however, often manufacturers do not fully adhere to those original definitions.
Larger format cameras are typically used as stand-alone sensors for traditional wide-area mapping applications, while medium format cameras are often used to augment lidar data. “This could be in the form of an ancillary dataset to confirm lidar classification accuracy or to create colorized RGB (red, green, blue) point clouds,” explains Michael Sitar, airborne products manager for Optech. “Alternatively, they could be delivered as classic orthomosaics, but with lidar providing the digital surface model (DSM) information directly.”
Leica Geosystems, based in Heerbrugg, Switzerland, specializes in airborne imaging and lidar sensors, making metric, large and medium format cameras that are predominantly used in photogrammetry, remote sensing, and all mapping applications, says Wagner. The company’s large format camera works on the “pushbroom” principle used in imaging satellites: it records a continuous strip, rather than single frames. “The sensor features a single rigid lens system (one optical path), very close integration with the GNSS/IMU (global navigation satellite system/inertial measurement unit) system, and thus has very high radiometric and geometric accuracy. Through the use of a tetrachroid prism, it produces co-registered bands at equal resolution, so that our imagery does not require pan-sharpening (which combines the color information from a multi-spectral file with the geometric information from the panchromatic band). It serves both the remote sensing market and the photogrammetric mapping market.” See Figures 1-3.
Leica Geosystems, according to Wagner, is one of the few manufacturers that offer a complete “in-house” aerial mapping solution — including flight management and post-processing software, GNSS/IMU processing, and mounts. “We test our cameras very rigorously for airborne applications,” he says. “We maintain and grow a very global support network and offer an upgrade path, so that our customers can take advantage of new technology and at the same time protect their investment.”
Trimble, based in Sunnyvale, California, has a portfolio of medium format cameras designed for aerial mapping. They include the Trimble Aerial Cameras and Trimble Digital Sensor System (DSS). The former are mapping-grade, metric medium format cameras for collection of high-resolution visible and near-infrared imagery (NIR, RGB and CIR – color infrared). They come in one-, two-, and four-head configurations and can be used stand-alone or integrated with lidar. “The Trimble Aerial Camera is precise and compact, and in fact has the best ratio of MP to weight and size in the industry,” says Adam Evans, product manager for Trimble Applanix.
The Trimble DSS is a complete, turnkey aerial mapping system that is certified as mapping grade by the United States Geological Survey (USGS). Trimble DSS consists of a camera, an inertial navigation system (INS), a GPS receiver, a flight management system, and a complete post-processing workflow. The DSS RapidOrtho workflow, which is designed for rapid response applications, can deliver imagery within only a couple of hours of landing, according to Evans. The DSS DualCam captures both RGB and NIR data simultaneously and the DSS Tactical Mapping system is used to acquire centimeter-level imagery from very high altitudes. See Figures 4-5.
Vexcel Imaging GmbH, Microsoft Corporation’s photogrammetry division, based in Graz, Austria, specializes in frame-based, large and medium format cameras for aerial mapping and surveying. The large format cameras are called UltraCamXp and UltraCamXp Wide Angle; the photogrammetric medium format camera is called UltraCamLp. The company’s large and medium format cameras differ in price and footprint size, thereby addressing different segments of the market, says Jerry Skaw, marketing manager for Microsoft’s photogrammetry products. See Figure 6.
According to Pat McConnell, the company’s North America sales manager, Vexcel Imaging’s cameras feature leading PAN footprint size and offer radiometric dynamic, forward motion compensation by TDI (time delay integration) in all cones, and fast frame rates. “We provide a removable storage system that enables pilots to maximize their flying time,” says McConnell. “Together, these features give the best price per pixel.” His company, he adds, offers a more complete line of cameras than other manufacturers, including loaner cameras, and enables its customers to either trade a camera in or have it upgraded.
Optech, a manufacturer of laser-based survey and imaging instruments based in Vaughan, Ontario, Canada, recently purchased DiMAC, a company that specializes in the design and manufacture of medium and large format cameras. “There is industry recognition that the strong height accuracy available from lidar, combined with the excellent planimetric accuracy of imagery, creates a high-quality end product,” says Sitar. “We are now in a position to offer our clients the benefit of both active and passive imaging solutions in either stand-alone or fully-integrated sensor products, all fully supported by a single vendor. We also incorporate small format interline cameras in many of our products requiring higher frame rates.”
“Our DiMAC cameras,” Sitar adds, “utilize a patented approach to the implementation of forward motion compensation (FMC) to minimize pixel smearing, or image motion blur, caused by the movement of the sensing platform across the target. As charge-coupled device (CCD) pixels get smaller, the percentage of pixel smear increases when flying at speed, all other factors remaining equal. You can use faster shutter speeds to compensate, but this requires a larger aperture to ensure commensurate lighting of the CCD array, which can negatively impact image quality radially. DiMAC cameras allow users to fly faster, lower, and for a longer period of time, because they can utilize slower shutter speeds, which produce better lighting conditions, at the same time as a tighter aperture, which produces better image quality.” See Figure 7.
Geospatial Systems, Inc. (GSI), based in West Henrietta, New York, specializes in airborne survey and mapping (ASM) cameras — specifically, for lidar augmentation, corridor mapping, and natural resources — as well as solutions for ISR for defense and homeland security. “In the ASM market, rather than competing with Vexcel Imaging and Intergraph in the large-format space, we operate in the small- and medium-format niche, where our cameras are typically used in conjunction with lidar,” says Barry Cross, the company’s director of sales and marketing. “In addition to color and panchromatic sensors, we also provide multi-spectral, mid-wave, and long-wave IR (infrared) sensors, which can be deployed alone or integrated in multi-sensor solutions. We build true metric systems on the leading edge of the price/performance curve. Performance and productivity features include our kinematic mounting design, a-thermal lenses, field-replaceable shutters, and DGX control and processing architecture, which integrates with various inertial navigation systems and pre-processes imagery for the photogrammetric workflow. Also, our selection of sensor modules can be combined into solutions for nadir and oblique imaging.” See Figure 8.
There is high demand for imaging solutions with a tightly-integrated flight management system and post-processing software, which significantly improves the workflow.
–Adam Evans of Trimble Applanix
Z/I Imaging — now part of the Security, Government & Infrastructure (SG&I) business unit of Intergraph, which is based in Huntsville, Alabama and was just acquired by Hexagon — has developed, manufactured, and sold digital aerial cameras for nearly ten years. The first generation was the DMC, first sold in 2003, which is still flying, with small investment for enhancements, says Klaus Neumann, the company’s product manager for sensor systems. “We are the market leader in the United States, China, and Japan. Our latest digital mapping camera, the DMC II, is the dream of every surveyor because it has a single, monolithic, ultra-large CCD.” This, he explains, means that image data post-processing does not require CCD stitching or image mosaicking, which requires finding enough features and structures to stitch together seamlessly and compensating for each CCD’s different geometry. “In photogrammetry you want to measure directly from the image.”
The DMC II 140 has 140 MP and the DMC II 250 has 250 MP and a pixel size of 5.6 microns. “Our strategy is to protect the customer’s investment as long as possible. The DMC II has a very high frame rate, an extra large footprint, and the ability to fly at 5,400 feet with a GSD (ground sample distance) of 10 centimeters.” Its main application is large area mapping with engineering precision.
Z/I Imaging’s CCD was custom-manufactured by DALSA using optics custom-developed by Carl Zeiss. This results in extremely good image quality, says Neumann, and can compensate for inaccuracies, while other vendors have to compensate for pressure and other effects. “To complete the system you need good electronics and we have our own developers,” he adds. See Figure 9.
In the past five years, Sitar points out, larger CCDs have enabled increased image resolutions and larger footprints, which enable imagery to be collected more efficiently by flying at higher altitudes for equivalent resolutions. Floating hard drives that required pressurization have been replaced by cheaper, more compact, and more reliable solid state disk storage. “Our core systems would not be possible without the advancements in the core sensor technology,” says Cross.
Two other important developments, says Wagner, have been the international acceptance of direct geo-referencing and increased reliance on distributed processing, both of which have greatly reduced the time it takes to, for example, produce orthophotos or extract DEMs and point clouds from the imagery. “The reliability of the components has improved,” he adds. “Cameras have fewer electronic components. Solid state drives (SSDs) and field-programmable gate arrays (FPGAs) have helped in this regard.”
According to Skaw, a key development was Vexcel Imaging’s release of a fully metric “medium format” camera, at the July 2008 ASPRS conference. “It has the same radiometric and geometric accuracy of our large format camera, because it is based on the same camera design and uses a subset of the equipment and electronics that we use in our large format camera, but for the same price that film cameras would have if they still existed.”
Opinions vary as to the remaining hardware, software, and workflow challenges.
As Wagner sees it, most of the remaining bottlenecks are in basic things that are partially out of the control of the aerial photogrammetry companies — such as the cost of flight operations and aircraft, the weather, air traffic control, and flying height restrictions. “So, our task is to develop sensor systems that are very effective despite that,” he says. “Of course, the investment threshold for many companies looking to invest in digital technology continues to play a role. The IT infrastructure to process the data is as important as the type of camera. In an ideal world, the application or end-use of the data should drive the data acquisition, and in my view there is still opportunity. For example, improvements in the lower-priced medium format camera segment, such as four-band imagery, stable multi-head configurations, or lower-cost sensors for specialized niche applications, could help to close the gap to the high-performing, large-format segment and thus create better access.”
The increases in cameras’ resolutions, explains Evans, challenge manufacturers to supply lenses that will support those resolutions — because the shrinking pixel size requires commensurate improvements in optics. “As resolutions increase,” Sitar explains, “distortions or aberrations from the use of poorer quality lenses become much more apparent. Similarly, surveyors demand shutters with high reliability. It is not uncommon for users to collect more than 100,000 images during a survey season. Having the confidence to know your lens can support your client’s entire survey season is critical. The ultimate goal is to minimize the client’s down time.”
There is high demand, Evans points out, for imaging solutions with a tightly-integrated flight management system and post-processing software, which significantly improves the workflow. System flexibility is particularly important for small and medium-sized mapping companies, which are always looking for ways to cover more ground faster and reduce their operating costs. “Many of these shops,” he says, “have told us that they need to be able to take on very different types of work with complementary sensors — such as 4-band, thermal, and lidar. They are asking us for sensors that provide interchangeable camera heads and focal lengths for different kinds of work. This means configurations with good base-height ratio for stereo work, others that are suitable for flying under clouds, and still others designed for matching the imagery footprint and resolution to a lidar swath.”
“It seems that right now much of the big money is being spent on satellite data,” says Cross, “and this will directly impact the growth of the aerial sector. In the ASM market, capacity for large format digital has been building and, at least domestically, we will really have to see how the large programs such as Clear30 and the Imagery for the Nation initiative play out. The good news is that the value of ‘geospatial’ is now broadly understood and, therefore, continued investments will be more easily justified. This will not only apply to the update frequency on large programs but will also support growth in GSI’s niche sectors, such as lidar augmentation, high-value asset corridor mapping and natural resources, including precision agriculture, forestry, and environment. In this sense, the biggest bottleneck becomes the ability to process, manage, and extract value from the huge volumes of data coming from the variety of sensors. You could say it is increasingly a ‘last mile’ problem.”
By contrast, Skaw sees no technical bottleneck. “Our systems,” he says, “are not overwhelmed by the amount of data generated by the new hardware, because we poured in huge sums of money to accommodate the increases in data to be processed. The only real bottleneck is how much aerial mapping work is available because of the economic downturn. It has been a struggle and there is a lot of apprehension about using capital for upgrades versus creating jobs.”
Finally, Sitar points to the large investment in software, both desktop and in-flight, required for real-time processing for in-air evaluation and coverage verification as a challenge.
In the next three to five years, what single technical development would help the aerial imaging industry the most? A “blue sky” button to eliminate cloud coverage, jokes Wagner! He then lists three more: in-flight, real-time data processing; continued increase in area performance at a lower cost, without a loss in photogrammetric and radiometric performance; and hardware-accelerated processing for data extraction, processing, classification, and data fusion. “There are also promising new technologies and platforms, such as UAVs,” he adds.
“Building real-time, mapping-grade ortho products will be critical to effective emergency response and tactical mapping,” says Evans. “High-accuracy aerial ortho imagery developed with direct geo-referencing does not require aerial triangulation or ground survey points, which can be time-consuming and often dangerous to acquire in rapid response situations.”
Sitar cites the effort to create larger single CCDs that don’t require stitching together multiple images, yet cover the same geographic footprint with similar resolutions or GSD. “Currently,” he points out, “many larger cameras use several CCDs (RGB and B&W) to compile a single large image. This requires stitching of the imagery, color balancing to remove CCD to CCD variations in light sensitivity, pan-sharpening techniques, and very high quality lenses to minimize radial distortions in each sub-image. The end result can be a poorer quality image compared to that which is collected using a single RGB CCD.”
“At the recent MAPPS conference, I heard that the community is hoping for more in-flight processing, so that you have a certain level of product when you land,” says Skaw. “Part of our value proposition is to tap into Microsoft’s software ambitions — e.g. its Dragonfly technology — so that we will have more and more software innovations.”
“With so much enabling technology in place currently — including sensors, computing power, and Internet communications — the single technical development that would most help our industry would be technology emphasis and compliance with open geospatial computing standards and effectively managing and more efficiently deriving real value from the investments that are being made,” says Cross. “There are still huge inefficiencies, and we have yet to achieve real interoperability among the systems that house our investments. So, meaningful advancement of open geospatial standards remains a critical need in delivering maximum ROI, which, in turn, will justify continued investment and growth of the industry. This is a technology development that, unlike others, will require strong policy leadership.”
Two U.S. companies build cameras used on imaging satellites: ITT Corporation, based in White Plains, New York, and Ball Aerospace & Technologies Corp., based in Boulder, Colorado. Manufacturers elsewhere in the world include Elbit Systems Electro-Optics Elop Ltd. in Israel, EADS Astrium SAS (which just acquired Germany’s Jena-Optronik) and Thales Alenia Space in France, and Mitsubishi Electronics Corporation and NEC in Japan.
ITT Corporation currently has sensors or imaging cameras on all the high-resolution commercial remote sensing satellites operating in the U.S. market. The company built the entire imaging cameras for GeoEye’s IKONOS and GeoEye-1 satellites and for DigitalGlobe’s WorldView-2 satellite, as well as the imaging sensors for DigitalGlobe’s QuickBird and WorldView-1 satellites. WorldView-1, which contained the second generation of ITT Corporation’s space-based sensors, was launched in 2007 with the National Geospatial-Intelligence Agency as its “anchor tenant.”
In late 2007, GeoEye first contracted with ITT for long-lead items for GeoEye-2. In August 2010, DigitalGlobe awarded ITT a contract to build the imaging system, which will include a sensor subsystem and an optical telescope unit, for its WorldView-3, high-resolution commercial Earth imaging satellite, anticipated to be available for launch by the end of 2014.
How do space-based cameras differ from aerial ones? One difference is regulatory: GeoEye-1’s resolution is 41 centimeters and WorldView-2’s is 46 centimeters. However, below 50 centimeters, GeoEye and DigitalGlobe are required to resample the imagery to 50 centimeters before selling it commercially. Therefore, much of the very high resolution imagery comes from aerial sensors. However, airborne imaging can’t be flown everywhere because there are some denied environments. This is the primary reason why users of remotely sensed imagery typically choose a combination of satellite and aerial imagery.