Remote Sensing by Remote Control
UAVs Become Invaluable Assets
When Somali pirates hijacked the U.S. freighter Maersk Alabama and took Capt. Richard Phillips hostage in April, a U.S. Navy ScanEagle unmanned aerial vehicle (UAV) built by Boeing’s Insitu unit took video footage of the developing situation. Predator and Reaper UAVs have been in the news recently because the U.S. military has used them to launch missile strikes, such as the one that reportedly killed Osama Bin Laden’s son, against Al-Qaeda and Taliban targets in Afghanistan and Pakistan.
Figure 1: MDA Corporation’s Heron UAV is a Medium-Altitude Long-Endurance System.
Less publicized is the extensive use of UAVs by the U.S. military for “dull, dirty, or dangerous” surveillance tasks for which they are better suited than piloted aircraft. Real-time images and videos are increasingly used for remote surveillance, intelligence gathering, situational awareness, and decision-making. “At the receiving end, the amount of data available is increasing exponentially,” says Kevin Kelleher, airborne integration lead for the National System for Geospatial Intelligence (NSG) at the National Geospatial-Intelligence Agency (NGA).
By associating geospatial information with imagery intelligence, this airborne video surveillance (AVS) technology allows decision makers to view developing situations in their geographic context, track and visualize events as they unfold, and predict possible outcomes.
What UAVs Are and What They Do
Also known as remotely piloted aircraft (RPA), unmanned aerial systems (UAS), and simply drones, these remotely piloted fixed- and rotary-wing aircraft and lighter-than-air and near-space systems are both armed and unarmed. They range in size from hand-launched models that look like toy planes and can weigh as little as 12 pounds to the jet-powered RQ-4 Global Hawk, built by Northrop Grumman Aerospace Systems, which has a 3,000-mile range and operates at about 60,000 feet, and the MQ-1 Predator and MQ-9 Reaper, both built by General Atomics Aeronautical Systems.
The Predator, which can fly at altitudes of up to 25,000 feet, performs surveillance and reconnaissance missions and carries two laser-guided anti-tank Hellfire missiles; it can stay in the air for about 40 hours. The Reaper is a larger and more capable aircraft that can fly at 50,000 feet, carrying up to 14 Hellfire missiles, and using infrared sensors to distinguish the heat signatures of rocket launchers, anti-aircraft guns, and other firepower on the ground.
Today, the U.S. military deploys more than 5,000 UAVs, and daily UAV missions in Iraq and Afghanistan have nearly tripled in the past two years. Military applications include peering over hills or buildings, monitoring the seas, eavesdropping from high altitudes, and assisting in special operations. The U.S. military also uses UAVs to transmit live video from Iraq, Afghanistan, and Pakistan. Traditional roles for military airborne geo-intelligence, Kelleher says, include operational support, battle damage assessment, treaty/inspection monitoring, non-combatant evacuation operations, forensic analysis, and coalition operations; new roles include disaster relief, counter-terrorism/narcotics, and homeland defense.
Direct connection between UAVs and operators on the ground or on aircraft is limited to line-of-sight communication; however, communication relay nodes and satellites enable operators to control UAVs and download data from anywhere.
History of Military UAVs
The first operationally significant U.S. Air Force UAV program was the Lightning Bug, which was routinely used for tactical reconnaissance during the Vietnam War. Israel developed various UAVs in the 1970s, successfully deployed them in the early 1980s, and then began selling them to the United States, which also began to develop new systems such as the Predator.
The United States used UAVs for reconnaissance during the Gulf War, but it was particularly in the skies over the Balkans, Afghanistan, and Iraq that they proved their worth as an intelligence, surveillance, and reconnaissance (ISR) platform. In Kosovo, commanders deployed the Global Hawk in combat, where they were reluctant to risk piloted U-2 spy planes. According to a U.S. Air Force report, “The MQ-1 Predator, armed with the AGM-114 Hellfire missile, continues to be one of the military’s most requested systems.”
Figure 2: Basic interface of BAE Systems’ SOCET GXP Video Analysis software (shown in Figures 2-6)
Figure 3: Selected regions for editing in the Video Analysis software.
Figure 4: This image shows the active film roll window.
Figure 5: This shows enhancements applied.
Figure 6: Video Analysis software allows tracking a vehicle.
In Afghanistan and Iraq, small man-portable, low-altitude, short-range UAVs—including the RQ-11 Raven (weighing only 4.2 pounds), the Pointer, and the Force Protection Aerial Surveillance System (FPASS)—have also played important roles by assisting in providing base security, force protection, reconnaissance, and targeting. Recent advances in sensor miniaturization, electro-mechanical control, aerospace design, and wireless video communication have enabled the production of these micro-UAVs, which are much cheaper and quicker to deploy than traditional ones.
Advantages of Using UAVs for Remote Sensing
For both civil and military missions, UAVs have two key advantages over piloted aircraft. First, they are more efficient because they can be designed without regard to human-factor limit-ations. Second, while they cost about as much per pound as piloted aircraft, they have much lower life-cycle operating and maintenance costs.
From a military perspective, UAVs are “just another source of geo-intelligence data,” says Kelleher. However, four advantages have made UAVs more feasible and attractive to military planners.
First, technological advances have made sensor and weapon payloads smaller, lighter, and more capable and have greatly increased the bandwidth connectivity of the data links used for vehicle command and control, payload command and control, and data transfer. Advances in microprocessor technology and software development have enabled onboard processing of sensor data, while advances in inertial and GPS navigation have enabled robust autonomous flight control systems. New composite materials and improved propulsion systems have resulted in lighter, smaller, and more stealthy airframes.
Second, UAVs are particularly appropriate to the challenges of “asymmetric warfare” against non-state actors. They can operate in “dirty” environments, such as those contaminated by chemical, biological, or radioactive agents. The endurance of large UAVs provides sustained support for “dull” missions requiring greater persistence than that provided by manned aircraft, such as monitoring conditions that rarely change and reporting only items that need attention. Small, quiet UAVs can get close to a target and provide a bird’s eye view.
Third, UAVs with endurance that exceeds human limitations allow commanders to reduce the number of sorties, which translates into fewer flight hours lost due to transit time, less wear and tear on the vehicles, and a sharp drop in the number of accidents. Remote control of UAVs also allows crews to fly operational missions without deploying forward. This reduces support costs and force-protection requirements.
Andrew Carryer, a systems engineer at MDA Corporation (Richmond, B.C., Canada), with the UAV Heron (see Figure 1), points out that the key advantages of UAVs for remote sensing are the time on target, for the military, or the ability to dwell over an area of interest, for civil applications; the ability to quickly re-task the asset to capture fleeting events; and the ability to provide a real-time data feed. Additionally, unlike satellites, piloted aircraft and UAVs can tailor their approach to each target, in terms of altitude, angle, etc.
Fourth, UAVs are much quieter than piloted aircraft, according to Vern Rummel, a UAV expert with BAE Systems’ Unmanned Aircraft Programs (formerly Advanced Ceramics Research). “Targets have no idea that they are being tracked or imaged,” he says. In addition, Carryer points out that UAV image interpreters and analysts can be anywhere in the world.
UAVs are not an alternative to piloted aircraft and satellites for reconnaissance but a complementary tool, according to Carryer. “Each one brings unique capabilities,” he says. “Cross-cueing allows you to bring a full complement to bear.” The trade-off between satellites and aircraft, whether piloted or not, is coverage versus resolution. Satellites provide greater coverage, while aircraft collections provide higher resolution.
Challenges of Using UAVs for Remote Sensing
Military UAV operators, according to Carryer, face two major challenges: adequate access to air space and to radio frequencies. These, of course, are political, legal, and administrative issues, not technical ones. “Frequency allocation in a congested area is a big deal,” says Carryer. “Typically, the people who manage frequencies are not associated with aviation safety. Now a UAV is connected to the ground via a data link, and there are safety issues.”
Figure 7: MicroPilot images for farming from the UAV CropCam include this stitched image of British Columbia (above), taken from an elevation of 2,100 feet, and the NW Quarter of a zero-till farm (left).
While stabilization is a trivial issue for large UAVs, which use the same gyro-stabilized turrets that are mounted on airplanes and helicopters, it is a major challenge for micro-UAVs. These are bounced around by wind and turbulence much more than larger or faster aircraft, Carryer says. Paul Moller, president of Moller International, Inc. (Davis, Calif.), agrees. The key, he says, is to minimize the vehicle’s attitude, rate of angular velocity, and angular acceleration — which requires “very sophisticated power control and millisecond reactions.” Turbulence is more related to the design of a vehicle than to its flight speed, points out Scott Newbern, Raven project manager at AeroVironment, Inc. (Monrovia, Calif.). “Our small vehicles tend to operate at 500 feet above the ground, so they encounter more turbulence and changes in the wind than UAVs flying at much higher altitudes,” he says. “We can get around the shaky video problem through digital stabilization, gimbals, and software tools to stabilize images.” As for the ability to track moving objects, according to Carryer, the only challenge is clouds.
Figure 8: MicroPilot images for farming from the UAV CropCam include this stitched image of British Columbia (above), taken from an elevation of 2,100 feet, and the NW Quarter of a zero-till farm (left).
Trade-Offs Between Large and Small UAVs
Large UAVs can fly much faster and for much greater distances than small ones and can carry weapons. However, they compete for transport space with other assets, such as tanks, forcing planners to make difficult choices, says Rummel. Small UAVs, he points out, can easily be loaded into the back of a light armored vehicle (LAV) and can be launched off the ground or from a weapons mount. Designed exclusively to provide aerial reconnaissance for situational awareness, small UAVs enable companies and platoons, for example, to look over a hill for ambushes or IEDs without having to call for a higher level asset, such as a Predator, according to Newbern. They are a force multiplier, he explains, that allows them to perform missions that otherwise would take a much larger force.
Figure 9: Beaver dam shown with MicroPilot’s CropCam.
The Role of the NGA
While the NGA does have a significant role in exploiting geo-intelligence data, most of the first level of exploit-ation is done by the military in the field, Kelleher explains. However, the services expect the NGA to store the data and to make it available for future use. (“You guys are storing all this stuff, right?”) UAVs are in part respons-ible for the huge growth in the amount of geo-intelligence data collected. The director of NGA sees these data as so important that he appointed a Deputy CIO for Data, Jack Hill. “The analytic resources are limited and the source data is exploding much faster than our ability to exploit it,” says Kelleher. “That is why it is important for us to continue to promulgate standards.”
In order to plug into the existing communications architecture wherever they operate, UAVs must use data standards. NATO, for example, has standards for the formats of still images (STANAG4545, similar to JPEG) and of video streams (STANAG4609), as well as for communications and data links. Within NGA, there are currently three major working groups in place to address standards: the Geospatial Intelligence Standards Working Group, the NGA Interoperability Action Team, and the NGA Standards Board. There are also four chartered focus groups made up of subject matter experts: the National Imagery Transmission Format Standards Technical Board, the Motion Imagery Standards Board, the Community Sensor Model Standards Working Group, and the Metadata Focus Group.
Last summer, President Bush revised Executive Order 12333, which regulates U.S. intelligence activities, to make the director of the NGA the functional manager for geo-intelligence. The NGA does not own or operate any aerial intelligence collection platform, Kelleher says. The agency’s role “is not as much about controlling the activities as about leading them and providing oversight,” he says. The NGA director is also responsible for defining geo-intelligence data standards and tweaking them as the technology advances.
Video Analysis Software
One of the most valuable products that UAVs can provide is streaming video, in real time or near real time. This product also poses one of the greatest challenges. “To be useful, this massive amount of data must be analyzed in near real time,” says Matt Bower, of BAE Systems (Rockville, Md. with headquarters in San Diego, Calif.), a subject-matter expert for the company’s SOCET GXP Video Analysis software tool. The very flexible and dynamic re-tasking of UAVs, he points out, requires analysts to quickly analyze the data stream, which can be part of live operations.
SOCET GXP, he explains, can take in video from UAVs—plus a stream of support data that includes such variables as the platform’s location, its look angle, the temperature, and the wind speed—and display the footprint of the camera’s field of vision on a map. The user can then move the frames of interest into SOCET GXP to create annotations, mark-ups, briefing products, terrain extraction, building extraction, and so on. “You now have all the SOCET GXP functionality in the video,” says Bower, “and can push the data through. You can fuse the video with other reference sources you might have and use it to drive other geospatial software, such as Google Earth.”
The latest version of SOCET GXP enables users to track moving objects they select on screen, as well as to push the video’s telemetery data into a sensor model and use it to extract the coordinates of the moving object and monitor its speed and heading. See Figures 2-6.
These data streams raise concerns about processing power, says Bower. “If you have a video at 30 frames per second, any advanced computation on those frames—even something as simple as sharpening and dynamic range adjustment—could incur a very big CPU processing cost, because you have to re-do the operation for every new frame.”
UAVs can employ various methods to reduce the amount of bandwidth needed to transmit video streams by several orders of magnitude. First, their on-board computers can disseminate only the most pertinent data. Second, they can recognize targets and transmit their coordinates rather than large imagery files. Finally, when they do need to transmit large volumes of data, they can use advanced data compression to reduce bandwidth requirements.
Civilian applications of UAVs range from monitoring crops, weather, coast lines, and borders to surveying crime scenes, assisting in search and rescue operations, and exploring for minerals. Some companies specialize in the production of relatively cheap UAVs for these civilian applications. MicroPilot (Stony Mountain, Manitoba, Canada), builds UAVs that cost less than $7,000 and do not require trained pilots to fly them. According to Pierre Pepin, the company’s vice president of sales and marketing, components for his company’s UAVs “are available online or wherever you have a hobby shop.” The main reason that people buy UAVs, he points out, is that they are cheaper to buy and operate than aircraft, especially in Third World economies.
MicroPilot began to build UAVs for farmers and scientists but now builds them for all kinds of clients. See Figures 7-10 on pages 27 and 31. “One shot some footage with a digital video camera that ended up in a one hour BBC presentation; another one wants to fly over the Galapagos Islands to map invasive species; one uses it to map land mines,” says Pepin.
Figure 10: Electrical tower nest shown with MicroPilot’s CropCam.
One limitation of these low-end UAVs is that they cannot fly when winds are over 30 miles per hour or very gusty, says Pepin. Furthermore, the FAA has banned most UAV flights over the continental United States because, unlike human pilots, UAVs cannot “sense and avoid” other aircraft. However, the University of North Dakota has bought a couple of MicroPilot’s crop cameras and is working with the FAA to develop standards for UAVs.
UAVs as remote sensing platforms have given many research groups the opportunity to acquire data at sufficiently low cost to justify the use of remote sensing in the first place. Once airspace regulations have been adapted to accept them as regular aircraft, these platforms may therefore become the catalyst for many new users and uses of remote sensing.
- “The Rise of the Pilotless Planes: Are Remote-controlled Drones—Used in Iraq and Increasingly in Pakistan and Afghanistan—the Future of Warfare?” The Week, April 9, 2009, www.theweek.com/article/index/95250/The_rise_of_the_pilotless_planes.
- Andrea Shalal-Esa, “Firms Vie for Share of Growing Unmanned Plane Market,” Reuters, Aug. 11, 2009, via www.washingtonpost.com.J. Everaerts, “The Use of Unmanned Aerial Vehicles (UAVs) For Remote Sensing and Mapping,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B1. Beijing 2008 www.isprs.org/congresses/beijing2008/proceedings/1_pdf/203.pdf.
- James Risen and Mark Mazzetti, “C.I.A. Said to Use Outsiders to Put Bombs on Drones,” The New York Times, Aug. 20, 2009.
- The U.S. Air Force Remotely Piloted Aircraft and Unmanned Aerial Vehicle Strategic Vision (2005), www.uavforum.com/library/usaf_uav_strategic_vision.pdf.
- Turner Brinton, “New Aerial Drones to Be Compatible with WGS Satellite Constellation,” Space News Business Report, www.space.com, July 31, 2009.