Rendering a More Realistic Scene

Written by Peter Buxbaum

MT2 2010 Volume: 15 Issue: 1 (February)

With the continued emphasis by the U.S. DoD and other nations’ militaries on realistic simulation scenarios for training and mission rehearsal, the development of image generators that can portray true-tolife situations has become all the more important.

Image generators (IGs) are a component of training and simulation visual systems that create a virtual environment which is as realistic as possible for the task at hand. These might include, for example, out-the-window views on flight simulators, or an entire cast of characters and cultural artifacts for urban operations training. Visual systems are the package of technologies—including image generators, hardware, software, and display systems— that bring these virtual environments to the trainee.

ENHANCED AND NEW IGS

“Image generators are essentially the picture generator portion of a game engine,” explained Phil Perey, technology director at CAE. “They give you a view of the outside world and they do that in a way that is interactive.”

Current image generators operate at 60 Hz, a speed which will be increasing before long, challenging developers to provide fluid graphics with very low latency times for a myriad of environmental permutations including, for example, changes in terrain, weather, and shading. The latest IGs also come equipped with capabilities that emulate views from infrared sensors, night vision goggles, and other such devices.

“Whether you are including dynamic shadowing across entities and terrain, creating breaking waves within a marine environment, or need to provide highly accurate physics-based sensor views,” said Nick Giannias, vice president for research and technology at Presagis, “a powerful image generator is a must so that you can easily include the needed amount of data to render a realistic scene.”

The Air Force acquires image generators as part of its procurement of larger visual systems. But it also develops specific detailed requirements for image generators for specific applications to satisfy user needs.

Timothy Dwyer, a training systems engineer at the 88th Air Base Wing, located on Wright Patterson Air Force Base, develops image generator specifications for training systems for a myriad of aircraft, from fast jets to transport aircraft to unmanned systems, as well as for non-aircraft applications such as air traffic control. The 88th’s customers include the Air Combat Command, the Air Mobility Command, the Air Education and Training Command, the Air Force Special Operations Command, and the Air Force Space Command.

“Our threshold requirements typically call for commercial off-the-shelf equipment and the maximum use of commercial open standards,” he said. “We usually get requirements from users and translate those high level requirements into performance and technical requirements. We tell providers what we are looking for, but don’t try to box them in the solution space too much. The main things are that it has to be doable and affordable and there has to be some competition out there.”

The requirements specified in recently acquired systems include: efficient automated and rapid databases; specifications for urban feature densities; built-in support for infrared and night vision imaging sensors; weather effects such as broken cloud layers, thunderstorms, and lightning bolts; and 3-D damage effects.

One of the key challenges in developing IGs that are up to the high standards expected in military applications is to exploit to the fullest, the capabilities found in today’s computer graphics cards. Image generators are optimized to “rake every last drop of capability out of graphics engines,” said Perey. “Commercial graphics cards serve a purpose,” said Nick Gibbs, senior director, visual systems, Rockwell Collins Simulation and Training Solutions, “but we believe the requirements of training systems exceed what is available in COTS graphics cards. Things like advanced texture, very high performance rates, and extensive use of shader technology can’t be achieved with COTS.”

In response to this capabilities challenge, Rockwell Collins developed a new simulation architecture—CORE, which is built upon Common Open Reusable Elements— that allows the company to enable greater functionality and capability throughout the life of a product. “We have achieved a ten-time improvement over previous products,” said Gibbs. “It gives us the ability to extend that growth path into the future with future software upgrades and no need to change the hardware footprint. This is a significant benefit for every simulator application.”

Rockwell Collins’ new EP-8000 image generator, which was unveiled at the 2009 I/ITSEC, was built on CORE and will be ready for delivery to customers in the second quarter of this year. New features of the EP-8000 include real-world scene density and depth complexity with correlated lighting, atmospherics, and special effects.

“We believe there are requirements better served by special purpose applications that we can provide through software in our new architecture,” said Gibbs. “One of the design goals of the product is to preserve the simulation environment as a whole without wholesale disruption by the complete reuse of interfaces, databases, connections.”

Another advantage to Rockwell Collins’ new IG is the ability to define unlimited levels of shading. “Shaders are a very important part of simulation applications,” said Gibbs. “Our architecture enables us to provide innovative and unique applications of shader technology not immediately available through COTS hardware.”

Shader development also figured prominently in the latest IG from Aechelon Technology Inc., according to the company’s co-founder and director of programs Javier Castellar. “We adopted our shader-based rendering for both sensors and visible spectrum from day one of our PC-based systems [in the early 2000s] and have transitioned more and more capabilities from CPU [central processing unit] to GPU [graphics processing unit],” he said. “Our sensor implementation is now on its fifth generation of shader code. The single most valuable early design decision was providing a sensor independent architecture capable of supporting both current and future sensor capabilities.”

Aechelon’s latest IG, pC-Nova v5.0 was released last December during I/ITSEC. “We have focused considerable R&D resources on supporting per-pixel display system integration and calibration for both OTW [outthe- window] and NVG [night-vision goggles] stimulated display systems,” said Castellar. “Another area of improvement is accurate representation of sensors operating under impaired conditions, as well as narrow wavelength energy representations.”

Aechelon has received contracts for the supply of pC-Nova for the Marines Corps’ CH-53E Super Stallion helicopter, several U.S. Coast Guard programs, the MH-60S Knight Hawk Multi-Mission naval helicopter, the Marines Corps’ MV-22 Osprey containerized flight training device, U.S. Army CH-47F Chinook helicopter, as well as several ongoing fixed wing programs.

SDS International developed its AAcuity line of image generators on open standards, ensuring cost effectiveness through the reuse of existing databases and commonality with other simulations during distributed mission operations activities. AAcuity can provide scene generation for flight, space, ground, and sea-based training applications.

“AAcuity IGs are optimized to run virtually any OpenFlight, Paging OpenFlight, and TerraPage database,” said Ed Bryan, a company vice president. “AAcuity is designed to seamlessly operate using terrains built with SDS’ GeoSource-RTB Rapid Terrain Generation tool to support area-specific training, pre-deployment familiarization, and mission rehearsal activities.”

The PC-based system provides the required 60 Hz performance for flight applications and can portray large 500-square nautical mile areas. Features include day and night scenes, realistic weather, variable time-of- day and wind effects, light-points, and night vision and infrared sensor usage. AAcuity also provides physics-based per-pixel lighting— which provides appropriate illumination at each rendered pixel of an image—for lighting sources that include directional sunlight, point lighting, and spot lighting. It provides real-time dynamic shadows that shadow terrain and objects, detail texturing and bumpmapping, and moving-trees and grass. It also has the capability of real-time portrayal of damage to 3-D military operations in urban terrain buildings, directional terrain cratering, and complete set of maritime features. AAcuity also supports its dismounted infantry for ground-based applications.

“AAcuity was selected by the U.S. Air Force Research Laboratory as the first PC-IG suitable for use on their highfidelity F-16 Viper simulations,” said Bryan. “AAcuity IGs are also currently used on all Meggitt Training Systems training systems, a number of forward air control training systems including the Indirect Fire-Forward Air Control Trainer, and the Special Operations Command’s Special Operations Air Ground Interface Simulator.” It has also been chosen by the Army and Marine Corps for use in Stinger missile simulation training domes.

Presagis’s Lyra 3.0 is a COTS IG which combines the utility of open standards such as Common Image Generator Interface, OpenFlight, and FlightICD with the flexibility to create unique applications with the underlying Vega Prime visualization toolkit. “The IG is highly compatible with existing and customized simulations,” said Giannias. “To help developers address the complex needs for sensor simulations, Presagis also offers Lyra Sensors, which add the ability to easily simulate infrared, night vision goggles, and other sensor data easily.”

Presagis Lyra has integrated features with other Presagis tools. For example, Lyra is integrated with Presagis’ Vega Prime Marine. “This integration enables developers to create high-fidelity and dynamic scenarios that include features such as dynamic shadows and dynamic sea states,” said Giannias. “These features can then also be simulated in the sensor views as well.”

Presagis has provided Lyra to a number of diverse applications. Boeing uses Lyra in the weapon system simulation for the Apache AH-64D helicopter simulator. The Israeli Air Force uses Presagis Lyra along with other Presagis tools and its own existing assets to develop geo-specific simulation features in an immersive training environment. Lockheed Martin has upgraded two training systems for the U.S. Air Force using Lyra. The F-16 cockpit simulator uses Presagis Lyra in 11 out-the-window channels and three sensor channels.

At MetaVR, Inc., the key technologies incorporated into its image generators are patented textured paging and geometry representations, according to Richard Rybacki, the company’s chief technology officer and co-founder. “We have some unique ways to address geospatial data that don’t create problems of scale,” he explained. “The IG can address databases as large as 10 terabytes and come up with images as sharp as one meter resolution without having to store lot of indexing information.”

MetaVR’s IG, the Virtual Reality Scene Generator (VRSG), has been on the market for 13 years. VRSG’s features include dynamic lighting and time-of-day conditions for sky visuals, as well as support for simulating the sea floor, water, and coast line of shore-line geographies. The IG supports a variety of weather condition simulations, lighting conditions, and sensor usages. VRSG can manage thousands of human character entities and display hundreds in the field-of-view.

At the 2009 I/ITSEC, MetaVR demonstrated VRSG’s ability to stream simulated UAV camera video to receiver devices to mimic the communication configuration used by real joint terminal attack controller systems. The company also demonstrated views from a small unmanned ground vehicle (SUGV). The SUGV simulation displayed three VRSG views: an out-the-window IG view, a monocular view simulating decoded feed from an SUGV, and the UAV camera payload station view. All of these demonstrations accessed MetaVR’s new database covering 9,600 square kilometers of the Afghan province of Kabul, including a high-resolution village with over 500 buildings, based on the small village of Khairabad in the southern part of the province.

CAE recently introduced its Medallion- 6000 series image generators. “The product uses the latest commercial graphics to bring to market features such as immersive weather environments [including] broken clouds and advanced lighting effects,” said Perey.

CAE Medallion-6000 maintains full database and interface compatibility with earlier Medallion iterations, making it suitable as a complement for current CAE customers. The IG’s features include a full suite of special effects, including tracers, missile trails, explosions, smoke, rotor downwash, conforming craters, and bullet impact. Maritime environments can be simulated up to sea state six with 3-D portrayals of ship wakes, swell and wind lanes. It also comes equipped with a correlated sensor suite. Operating on a visual system based entirely on COTS workstation components, the IG conforms to industry standards such as Windows XP, OpenGL, and OpenFlight.

FORECASTS

Looking to the future, IG developers have their work cut out for them. Dwyer and his team have developed a long laundry list of features they expect to see in future IGs.

“Future requirements will include true dynamic shadows,” said Dwyer. “These are useful for navigation and long duration mission training where targets, threats, and landing zones are in shadow.”

Yet further down the road, in the 2015 to 2020 time frame, the Air Force is going to want still more detailed and realistic weather effects, higher visual acuity displays, capabilities for tens of thousands of coordinated and networked entities, and physics-based damage effects assessments resulting from precision weapon impact and explosions.

But perhaps the biggest and most basic enhancement to Air Force image generator specifications will be its much higher speed. “We have research from the Air Force Research Laboratory that says that to achieve the visual acuity of the average Air Force student, which is 20/15, we need to consider running image generators and display systems at higher than 60 Hz,” said Dwyer.

How high? The goal right now, said Dwyer, is to double the speed to 120 Hz. ♦

Back to Top