Developments in 20/20 visual acuity
The pursuit of 20/20 visual acuity is complex, involving research in multiple disciplines. Research continues in the visual pipeline technology areas. While progress has been made, it will take more effort to build an affordable eyelimiting visual display system that is capable of running in real-time.
By Erin Flynn Jay
The pursuit of 20/20 visual acuity is complex, involving research in multiple disciplines. Research continues in the visual pipeline technology areas. While progress has been made, it will take more effort to build an affordable eye-limiting visual display system that is capable of running in real-time.
20/20 visual display resolution is the benchmark from which visual systems are judged since this is what a person would be able to see in the real world. An eye-limiting display is commonly characterized as having a resolution of 2 arcmin/optical line pair (OLP).
20/20 display resolution is usually expensive to achieve and should only be included in a display specification when the added expense can be justified by the added training value, said Paul Lyon, resident simulation visualization expert for Barco, a global technology company that designs and develops visualization products.
Applications for 20/20 display resolution include training where targets must be accurately detected and identified at “real world” distances. An actual 20/20 acuity visual system would give more positive training than a lower fidelity visual system with artificially enhanced objects and targets to compensate for the lower fidelity.
“The importance of an eye-limiting display is dependent on the training objectives and the subsequent visual system performance requirements,” said Ron Wolff, chief engineer, Naval Air Warfare Center Training Systems Division, Visual and Sensor Simulation, Orlando. “Not all systems require eye-limiting resolution. However, if the training task requires eye limiting resolution, than anything less than an eye limiting resolution will reduce the overall training capability of the system from its intended purpose.”
For example, under ideal viewing conditions, a visual system with a resolution of approximately 6 arcmin/OLP would require the object of interest to be three times closer before detection when compared to a visual system with 2 arcmin/OLP of resolution. Depending on the training objectives, this could have a significant impact on the training effectiveness and the ability to provide a fair fight in non-homogenous networked systems.
High Resolution Market
The market for high resolution visual display systems is primarily being driven by the simulation industry and the digital cinematic industry, according to Wolff. However, the simulation industry is capitalizing on the trend of higher and higher resolution display panels found in consumer projection systems utilizing digital light processing (DLP) and liquid crystal on silicon (LCoS) technologies. There are also several research efforts developing LASER projection systems and small business innovation research (SBIR) efforts to provide low-cost eye-limiting display resolution systems using hardware and software approaches to provide low-cost geometry warping and edge blending techniques to form seamless high-resolution multi-projector imagery.
SBIR projects are looking into methods for creating and processing high dynamic range imagery for sensor simulation. Image generators and the continuing evolution of shader technology are continuing to improve the overall processing capability and performance of the graphics pipeline. All of these efforts together will ultimately lead to cost-effective eye-limiting visual display resolution systems, Wolff predicted. Air Force Research Laboratory (AFRL), Mesa recently completed an advanced technology demonstration (ATD) with Evans and Sutherland, with the support of Air Combat Command, of a high-resolution laser-projection system that is capable of presenting spatial detail very close to the limit of what can be seen by an average pilot, said Marc D. Winterbottom, visual science program manager, AFRL, Mesa.
In addition, AFRL, Mesa is working with NASA-Ames Research Center to evaluate projector technologies to support a simulator designed to correlate pilots’ visual capabilities with their performance on relevant simulated flight tasks. The projector technologies being considered for this latter project have the potential of presenting visual detail in excess of the 20/20 standard. AFRL, Mesa also works with the Air Force procurement agencies, as well as a number of domestic and international industry partners to continuously improve visual display systems for simulator applications.
20/20 visual resolution is a matter of having sufficient pixels per angular field of view (FOV). For small fields of view, like the viewing block of a tank, 20/20 resolution is easy to obtain now. For a flight trainer with a 360-degree FOV, typically over 200 million pixels are required. Barco has designs today for 20/20 displays, but as the fields of view become large, the initial cost and cost of operation of these visual systems becomes great, said Lyon. Barco has bid large FOV 20/20 displays in response to recent government request for proposals (RFP), but in most cases after the actual costs of such a display were understood, a lower resolution display was selected.
DoD visual simulation systems encompass the entire visual pipeline which includes: the environmental database, the image generator (IG), the projection system, and the display system. “While each system component can be specified and tested independently, it is the total system performance that we are most interested in since the overall system will only perform as well as the weakest link,” Wolff said. “Various techniques for testing visual display systems have been around for years and have been proven to be reliable and consistent. However, as we move away from CRT [cathode ray tube] projectors and move towards digital projectors and higher resolution imagery our techniques and tools will also need to evolve to meet these new technologies.”
Pixel Displays
Lyon said the key to cost effective and maintainable “massive pixel” displays to support 20/20 acuity is to reduce the amount of required display and IG hardware to do this. On the display side, this will be accomplished by using projectors with much higher pixel capabilities. Eight to 10 million pixels per projector is the current state-of-the-art. A 32-million pixel LCoS projector prototype has been demonstrated but no production is planned at this time.
Barco has designs where around 10 32 million pixel projectors provide 20/20 display resolution. This is a reasonable number of projectors. IG technology will need to evolve to drive a 32 million pixel projector. Using 16 2-megapixel IG channels is not a reasonable approach. A much higher pixel density per IG channel will be needed. This is not going to happen in the near future, Lyon said.
When defining a 20/20 visual display resolution, it is not enough to discuss the total number of pixels or the arc minutes per optical line pair, Wolff said. Issues such as throughput, update rate, spatial resolution, temporal resolution, grey level, dynamic range, brightness, contrast, and the effects of speckle as digital projectors tend towards small pupil sizes must all be addressed. A deficiency in any one of these areas could limit the overall theoretical eye-limiting visual display system performance. A digital display system may have a spatial resolution of less than 2 arcmin/OLP but have a temporal resolution of over 10 arcmin/OLP.
The pixel-count of the display is usually the primary parameter of interest, said Winterbottom. However, if the bandwidth of the projector is not sufficient, individual pixels may not be distinguishable. A display is typically described as “20/20” if it can present 60 visually distinguishable pixels per degree of visual angle (a resolution of 1 arc minute per pixel).
Many displays accept a video signal with more pixels than they can actually resolve, noted Winterbottom. A bandwidth limited display may reduce the contrast of small features to such a degree that they are no longer visible to the observer. This is an important consideration for simulator displays because the ability to perform relevant visual tasks such as identification of distant aircraft, which may consist of only a few pixels, may become limited by the display system.
When a feature has few pixels, such as a gun turret or distant aircraft at identification range, most displays demonstrate reduced contrast that can also depend on orientation, Winterbottom added. In this situation, performance is also simulator-limited. A horizontally oriented feature often has higher contrast than a vertically or diagonally oriented feature, resulting in a difference in performance that is not evident in the real world. Therefore, to match simulator performance to that in the real world, either the number of pixels or how well they can be distinguished must be increased. Both solutions increase cost and one of the tasks is to examine the cost-benefit ratios of each.
Two arcminutes per optical line pair in an angular FOV is accepted as 20/20 acuity in the simulation and training market. Two arcminutes per pixel should not be confused with two arcminutes per optical line pair. Lyon said these are two entirely different definitions. The larger the FOV the more pixels are needed.
Even though 20/20 acuity is accepted as 2 arcminutes per optical line pair resolution, these are not equivalent. Recent government RFPs have switched to using “2 arcminutes per optical line pair” instead of 20/20 acuity to avoid confusion. Lyon thinks this is a good idea.
Quest for 20/20 Acuity
The quest for 20/20 visual display resolution is a complex issue involving research in multiple disciplines. There is ongoing research in each of the visual pipeline technology areas. “Significant progress is being made in each area but it will take the system integration of all these components to ultimately build a true eye-limiting visual display system that is capable of running in real-time and is affordable,” Wolff said.
“We have pilots flying around trying to do what are eye-limiting tasks with systems that are five times worse than eye-limiting,” said Stephen Gersuk, vice president, technical marketing, Quantum3D, Inc., a provider of commercial-off-the-shelf, open-architecture, real-time visual computing products and solutions. “They don’t provide enough sharpness in the scene to allow them to do some of these tasks.”
The Lockheed Martin Aeronautics Division’s F-22 Advanced Combat Simulators facility, Marietta, Ga., has taken the rear-projection mosaic approach now common to fast-jet simulators to a new level: instead of putting one projector behind each facet they’ve put four. “It’s a little crowded but you get extremely high resolution,” Gersuk said. “That’s a system that is deployed today and is in the process of being upgraded even further to have higher resolution yet.” Lockheed Martin has demonstrated how this can be done.
More recently, programs like the F-16 Mission Training Center re-compete have come out and specified something about “half eye-limiting resolution” that requires that bidders provide the technology insertion path to get to eye-limiting resolution over the course of a program, Gersuk added. People who are doing fast jet simulation are moving in that direction.
To achieve eye-limiting resolution displays, databases must provide higher resolution elevation data and terrain imagery; model fidelity must improve; IGs must have the capacity to process larger databases with higher resolution textures and more pixels at 60 Hz to 180 Hz; projection systems must have the ability to display more pixels at sufficient brightness and contrast as well as support higher update rates; geometry warping and edge blending techniques must improve for multi-projector systems; and screen materials must be optimized for eye-limiting displays to maximize contrast and minimize potential speckle effects.
Winterbottom noted it is not necessarily a technical challenge to achieve 20/20 resolution in a small FOV. Many desktop displays are capable of near one arcminute/pixel resolution with typical viewing distances. However, achieving 20/20 resolution in a flight simulator with very wide FOVs is very challenging because very large numbers of pixels are required. To achieve 20/20 resolution in the ATD for a 360 degree FOV, 160 million pixels are required (eight 20 million pixel projectors). Commercially available high definition (HD) displays by comparison are only two million pixels. Thus the number of pixels required for high resolution simulation is orders of magnitude greater than that required for commercial or entertainment purposes.
Challenges to 20/20 Resolution
From today’s technology standpoint, Lyon said that using head-tracked and eye-tracked area-of-interest displays is the most economical way to provide 20/20 resolution. These displays only need to supply high resolution in a smaller area for any given instant. The problem with these types of displays is that special “eye tracking” sensors must be mounted on a helmet and are intrusive to the simulation environment. “Eye tracking” has still not gained acceptance in the pilot training community. Eventually, the 200 million pixels needed for 20/20 acuity in a 360-degree display will become affordable and maintainable.
Terrain elevation data and imagery data resolution requirements can vary drastically depending on the training platform and visual requirements (e.g., fixed wing platform, rotary wing platform, urban operations, sensors, etc.). High resolution imagery required to achieve 20/20 visual display resolution requires significantly larger databases and more texture memory and processing capability than for non-20/20 visual display systems.
The requirements for advanced sensor simulation (e.g., night vision goggles, forward-looking infrared, etc.) require high resolution, high dynamic range material texture maps. Wolff said this ultimately presents an enormous challenge to the image generator and leads to additional demands on the projection system to provide sufficient dynamic range to effectively display the high dynamic range imagery. High-frequency cultural features such as power lines, radio towers, guide wires, small radio antennas, ship masts, visual landing cues, detailed model feature, etc. are especially demanding and can all present unique challenges in achieving 20/20 visual display resolution.
The task to be trained is the primary determinant of the fidelity required for an observer-limited simulator, Winterbottom said. Daytime target identification, for instance, would require high-resolution displays to produce acquisition ranges that match those of pilots in the real world. On the other hand, both pilot acuity and display-resolution requirements will be reduced when simulating, for instance, a night mission.