Creating High Fidelity Virtual Training in Night/Restricted Conditions
Written by Erin Flynn Jay
MT2 2010 Volume: 15 Issue: 3 (May)
The U.S. Air Force’s current efforts to increase the rigor and fidelity of simulator training scenarios in restricted or night missions depend on the specific platform.
There are technology limits to creating a higher fidelity training environment in restricted or night missions. The government/ industry team has projects under way to expand the envelope of technology in this training environment and better allow crews to train as they will operate.
ONE SERVICE’S PERSPECTIVE
Many operational platforms carry equipment to provide an augmented visual capability for nighttime operations, such as night vision goggles (NVG) or forward looking infrared (FLIR) pods. “It is often required that simulators be equipped with similar systems, in order to train the operation of the equipment, as well as conduct simulated missions in which such equipment is employed,” said Tony DalSasso, chief, Air Combat Engineering Division for 677th Aeronautical Systems Group. “The specific requirements vary by platform.”
The U.S. Air Force’s current efforts to increase the rigor and fidelity of simulator training scenarios in restricted or night missions depend on the specific platform.
Some technology restrictions exist in developing a high fidelity training environment for these conditions. “For one, visual display devices are designed to operate in the visible range of the electromagnetic spectrum, and are not optimized for the infrared wavelengths in which night vision equipment operates,” said DalSasso. “The physical proximity of a simulator display to the operator’s eyepoint is incompatible with the optical designs used by operational night vision equipment, which are designed to view objects at greater distances.”
Night vision equipment is susceptible to artifacts that are not easily simulated. Source material for visual models and terrain databases often does not contain sufficient detail for infrared simulation, such as internal heat sources, heat transfer properties, etc. However, DalSasso concluded it should be noted that workarounds do exist in most cases, and night training in simulators is generally considered effective.
HIGH FIDELITY SYSTEMS MODELING
CAE employs the latest generation of liquid crystal on silicon (LCOS) projectors and image generation systems to realistically replicate the difficult operating conditions of restricted visibility, precipitation, brownout and whiteout conditions, as well as NVG operations. The precipitation and brownout models use a particle-based modeling approach that allows thousands of particles (e.g. snowflakes, raindrops, dust) to be realistically displayed to the aircrew or soldier being trained. “Although more rudimentary representations were available 10 years ago, today’s high-fidelity systems are approaching photo-perfect modeling,” said Philippe Perey, technology director for CAE.
An example of a program performed by CAE related to restricted visibility training came after a request by the U.S. Army to help train Apache helicopter pilots prior to deployment to Iraq. CAE initially developed and deployed a brownout model for the CAE Medallion visual system being used in the Apache combat mission simulators to help train pilots being deployed into desert environments during the early phase of operation Iraqi freedom. Further, evolutions of this brownout model are now provided in all simulators equipped with a CAE Medallion visual system, said Perey.
Operating an aircraft or armored vehicle or any other weapon system in restricted conditions, including use with NVGs, is an accident-prone environment that requires special training. The crew can easily lose situational awareness, resulting in aircraft drift or improper user inputs that can quickly result in an accident. “This margin of safety is substantially reduced for military covert operations where special operations forces must move in quickly, fly or hover at very low altitudes and operate in unknown environments,” said Perey. “The state of the art in CAE simulators allows these conditions to be reproduced for the training and mission rehearsal environment.”
Although NVGs provide improved visibility, the goggles themselves limit the field of view seen by the pilot and may provide incorrect depth perception when observing scenes with different colored lights. For example, Perey said a distant red light will appear much brighter (and hence nearer) than a close blue light. These effects can be reproduced in the simulator environment when using high-quality projection systems.
The starting point for high fidelity simulation is a correlated database throughout the simulator. This includes the data required by all the simulation “clients,” such as the out-the-window visual scenery, the infrared (IR) sensors, the NVG as well as the mission functions. CAE has been a strong promoter of open standards such as the Common Database (CDB) that enable perfect correlation across all the simulator clients while enabling content reuse across simulators, even when provided from different manufacturers.
Most simulators today use a “stimulated” approach to NVG training. Perey said this means the actual NVG units are employed during simulator training. These units are very sensitive to light, and require special design and manufacturing considerations to ensure a “light-tight” cockpit environment, as well as special cockpit filters as used in the actual aircraft.
Most NVGs are sensitive to a light spectrum that overlaps the lower end of the visible spectrum, but also includes portions of near-IR bandwidth. This must be correctly modeled within the image rendering.
CHALLENGES TO NVG SIMULATION
Perey said two key challenges to obtain high-fidelity stimulation of NVG in the simulator are:
1. Proper modeling in the image generation and database attribution to ensure that materials have correct behaviors when optimized for NVG use. This difference in surface reflectance, known as albedo, affects the appearance of objects such as tree foliage and other vegetation.
2. Projectors with a high contrast, low black levels and sufficient IR energy to properly stimulate the goggles. Although the newer generation of COTS fixed-matrix projectors provides acceptable contrast and black levels, they still lack sufficient IR energy to force the goggles to ‘bloom’ as much they do in the real world. Several projector manufacturers have proposed and demonstrated optimized projectors that include more IR energy. Manufacturers include RDE, Barco and Christie Digital.
Several new technologies are being introduced in the market or currently in the research and development phase. A technology called enhanced vision system is a realtime display of the IR scene in the cockpit and it has been integrated into a number of aircraft, such as the Gulfstream G150, Perey said.
CAE is developing a more sophisticated system for use in aircraft called the Augmented Visionics System (AVS) that provides real-time situational awareness in extremely low visibility conditions. Using state-of-theart sensing and simulation capabilities, CAE’s AVS allows pilots to see through harsh conditions, providing a “clear day” 3-D virtual representation of the environment around them.
AVS uses the CDB format that was originally designed for mission rehearsal needs for the U.S. Special Operations Command. “It allows for changes to be made to the synthetic environment without lengthy reprocessing. AVS incorporates advanced sensing technology such as LiDAR [light detection and ranging] and IR to gather data on the surrounding environment,” Perey said. “This data is then used to update the terrain elevation and obstacle information in the synthetic virtual environment and display it in real time.”
CAE’s AVS successfully completed field trials earlier in 2010; however, the system is still in the testing and evaluation phase. CAE will be continuing with research and development throughout the year.
DIGITIZED VIRTUAL TERRAIN BOARD
Night Readiness LLC offers a Virtual Terrain Board (VTB) training system. The initial development concept was to replace the old physical, static terrain board. “Those boards ran anywhere from around $50k [50,000] to $150k and once you bought it, you were stuck with it. They also occupied a large space in your training facility, which could then only be used for that one purpose,” said Stephen Hatley, president of Night Readiness. “Once a student saw it, they had seen it. They were somewhat limited on the number of viewers, what could be shown, etc.”
So Night Readiness took that concept and digitized it, putting it into a virtual world. Their system is a very small footprint, classroom/portable training system that can be tweaked and modified to fit the particular user’s needs. “We use very high resolution, physics-based databases to portray just about any type of environmental flight (or operating) regime that one might possibly encounter,” said Hatley. “We have mountains, desert, water, rural, urban, etc., and can infinitely adjust the ephemeris data—i.e. moon elevations, azimuth, percent illumination— to show the nuances and misperceptions and illusions associated with interpreting terrain through NVGs.”
Night Readiness uses a unique stimulate approach in that students actually use real NVGs to view a large black screen for viewing the databases using COTS hardware. The stimulate approach allows for viewing of the scene unaided (out-the-window) as well. “Of course, we also have the simulate version as well, which is more suited for refresher/ re-currency training where you do not wish to use the actual NVGs. With a ‘light-tight’ classroom, our system allows the use for other purposes as well as the use of the highend projector,” said Hatley. “This allows for one stop training in one classroom for NVGs, then the next day the classroom can be used for other purposes.”
As it is a training system, Night Readiness has developed an instructor guide to ensure all training objectives are met. As far as training objectives go, Night Readiness has covered the learning objectives from all the services, i.e. Air Force, Army, Navy and Marine Corps. It is packaged and sold as a turnkey system that includes all necessary hardware, peripherals, software, installation and training of the instructor cadre. The VTB system is controlled by the Department of State International Traffic in Arms Regulations, so Night Readiness is required to obtain an export license prior to sale.
Night Readiness’ current version available today is their 2.0 software release. “The basic package includes all necessary hardware, software, peripherals, installation and training with enough ‘generic’ databases to cover all necessary learning objectives,” said Hatley. “However, we can also customize scenes/databases for specific customers (i.e., we have airfields), but if you wanted to see a particular airfield in your AOI for instance, we can do that. However, there would be additional development charges depending upon the complexity and support provided as far as imagery collection.”
Night Readiness is in the final stages of development of their next software release, which should be out this spring. “The main changes/additions will be more dynamic shadowing/illumination effects—i.e. can fast forward time or reverse to immediately show the effects of a rising/setting moon, percent illumination, etc.,” said Hatley. “We are adding weather effects, such as decreased visibility, clouds, lightning, etc. As opposed to the 2.0 system where you see aircraft frozen in the scene, we will have 3-D moving models in a ‘fly through’ database. There will also be a new NVG compatible instructor touchpad controller/interface. These are some of the features which will be incorporated. As such, the 3.0 will be better suited for more advanced training applications.”
Hatley said what is truly unique about their system is that they offer the user the capability to use actual NVGs to view their databases. So, instead of replicating the goggle scene as if you were looking through an NVG, they actually do it. This is much more immersive training and allows the user to view the unaided, out-the-window scene as well. Night Readiness materially codes their databases so they are extremely accurate when viewed through the NVGs.
ATMOSPHERIC NIGHT SIMULATION
Falcon simulates sensor viewing under various environmental conditions. Where Falcon excels is its ability to bring accurate atmospheric transport (via MODTRAN) to a real-time rendering simulation, said Tim Palmer, president of TransLucid Inc. The result is Falcon’s atmospheric simulation at night or under any restricted visibility is both accurate and fast.
The human eye, a sensor in its own right, is limited in both bandwidth and sensitivity, especially in low light conditions. “The eye cannot perceive low to moderate thermal emissions or easily resolve low light contrast scenery. Thus the eye, as good as it is, cannot always be reliable for tasks ranging from collision avoidance in a driver simulation to visual target detection and identification,” said Palmer. “Night vision goggles mitigate this inherent weakness of the eye by amplifying low-level light scenes and projecting them in a way that the eye can detect more easily.”
Night vision is not simply making the image a monochrome green color, and FLIR simulation is not simply making the image black and white. The challenges are to use proven computation models such as MODTRAN that are known to be accurate, but apply them in a clever way that allows for real-time image generation with minimal time spent “initializing” and “updating.”
In addition, Palmer said attention must be paid to the temperatures of objects in the simulated scenes, particularly for infrared sensors. These thermal emissions are not typically seen in a visual image generator. Determining the temperatures of simulated objects is not always easy and straightforward because there are many environmental and structural components that sometimes must be inferred or even guessed.
Luckily the coming of age of General Purpose Graphics Processing Units (GPGPU) has brought a miniature super-computing platform of sorts to training simulations. Palmer said this extra computational power enables Falcon to provide a more responsive sensor simulation platform that includes more realism and accuracy than ever before.
TransLucid recently released Falcon version 1.2, which includes extensive support for real-time sensor post-processing effects. “We plan to continue to advance the state of the art in sensor simulation by taking advantage of the latest graphics card and CPU processing capabilities. Our goal is to provide easy to use, high fidelity, fast initialization, realistic sensor simulation capabilities to our customers,” said Palmer. “Specifically, look for new features such as real-time, 3-D heat transfer, built-in sensor fusion capability, GPGPU based radar (including SAR) signature generation, expert systems for on-the-fly model material classification, automatic seasonal signature variations, and hyperspectral terrain material classifications.” ♦