.jpg)
The Changing After Action Review
Written by Erin Flynn Jay
MT2 2009 Volume: 14 Issue: 6 (November/December)
There are two primary types of AAR capabilities being utilized within the live training environment; theater-based and field-based. The primary difference between these capabilities is the physical location and capacity. “Live training environment AARs are multimedia presentations delineating training results as compared to tactics, techniques, and procedures [TTP] doctrine,” said Colonel Fred Mullins, project manager for Training Devices at U.S. Army’s PEO STRI. “The data content presented within the AAR is highly coupled to the particular training instrumentation system,” e.g., MOUT systems are more heavily video-oriented, while homestation training is more digital data related to player event data and activities. The data content of the AAR presentation is scaled to the appropriate level and content of the audience, training doctrine and commander’s intent.
The primary focus of the AAR capabilities is to provide near real-time training feedback of exercise execution to the training unit to enhance readiness and survivability. AAR capabilities will vary depending on the training system (from score cards on small arms ranges, to theater AARs conducted at the brigade or above level), Mullins told MT2. The focus is to ensure that the appropriate training results/data/feedback information is distributed to the lowest level required to enhance the Army’s readiness and performance at every echelon.
As the Army’s mission changes, so does the AAR that the training system provides. “As the Army focuses more on STX and MRE training events, the AAR capabilities are being more honed to support field and unit [squad] training feedback sessions,” Mullins said. “The focus of these AARs is related to individual performance and improvement. Higher echelon AARs are still conducted, but their focus is equally adjusted.”
The Army is pursuing methods and technologies to create common AAR solutions (within a product line framework and common architecture) that will continue to allow them to present the appropriate data to the appropriate unit, without the need to reinvest in the AAR hardware with each change of mission.
CUBIC’S TRAINING CENTER
The AAR capabilities included in Cubic’s combat training center and homestation training systems have been developed over a number of years and include a range of features that address multiple requirements. “The AAR capabilities support simultaneous exercises, scalable architecture in excess of 2,000 players, compatibility with LVC [live, virtual, constructive] training requirements, 3-D visualization, and presentation tools based on COTS software,” said Ray Barker, executive vice president, Business Development & Strategy, Cubic Defense Applications Inc.
Cubic is focused on customer requirements that place a premium on user-friendly features, improved graphics and rapid AAR development. Cubic’s exercise control systems integrate CTC environmental features including video, smoke, sound, interior and exterior weapons engagements, including shoot-through-the-wall, as well as high-quality 3-D representation of players in GPSdenied areas.
Cubic’s AAR capabilities are based on the same technical foundation and are typically tailored to site-specific needs, Barker concluded.
OTHER STRATEGIES
Calytrix’s Mentor supports AARs in service, joint and combined (multi-national) events. Mentor is an objectives-based performance assessment tool for developing and managing individual and team/collective training exercises and knowledge; capturing and recording training results; and generating assessments, analysis reports and conducting an AAR.
Steve Salmon, training systems principal, in the company’s Training Systems division, observed that throughout the conduct of a training activity, the system assists the exercise controllers and evaluators by providing up-to-date feedback on performance measures, enabling the activities to be adjusted to maximize the training objectives. “Throughout the exercise, detailed evaluation/assessment occurs, enabling informative feedback to all participants. Through the use of Mentor, an effective AAR can be planned and conducted. Evaluation outcomes, measured against the training objectives, enable the next activity to be planned cognizant of the strengths and weaknesses of the previous training activity,” he pointed out.
Mentor was the designated AAR tool for the Australian Joint Combined training Capability (JCTC) proof of concept. Populating the tool with a series of performance objectives, Mentor was used to conduct detailed evaluation of the combined air operations center and the combined special operation task force in the conduct of air support to a special forces raid. Evaluators were distributed between Canberra and Townsville, Australia, and made detailed comment on the ability of each group to achieve the required objectives. Salmon continued, “By importing specific video images, screen shots and combat net radio communications, the evaluators were able to back up their subjective assessments with objective data. The aggregated results were used to develop the AAR and assisted with the post activity report.”
Mentor is provided with a standard set of reports as part of the base package. These reports cover a range of areas and presentation styles to suit most common applications. Reports can be simply generated from within the Mentor program and allow for the production of “take home packs,” and on screen reports, as well as the ability to quickly publish reports to a nominated server. The Mentor report suite is typically produced in HTML format to allow use of interactive report features and the rapid recovery and review of any supporting data. This data (artifacts) is attached by assessors/ observers, and any electronic format can be attached for later review as part of the analysis/AAR process.
Over the years, Aptima Inc. has developed AAR applications for a variety of domains. The firm has worked with the fighter pilot community (within the Navy and Air Force) and with ground operations (with the Army and Marine Corps). Many of the AARs Aptima has developed focus on combining different types of performance data collected from different, geographically separated training locations you would find in a distributed training exercise, said Emily Wiese, product manager, Performance Measurement Technologies at Aptima.
“For example, we frequently combine performance data calculated from simulator data and collected by observer-instructors with the more standard AAR functions such as video and sound replay,” said Wiese. “Viewing all of this performance data in a cohesive visualization allows instructors and trainees to obtain more insight into specific areas of performance decrements than video alone currently allows. In this way, specific areas of improvement are identified and can be remedied.”
To develop these AARs, the firm uses Aptima’s Feedback Toolkit, a development capability that allows their scientists and engineers to more rapidly develop AAR applications that are customized to Aptima customers’ individual training needs. “We feel strongly that there is no ‘one-size-fits-all’ AAR application. Not everyone needs the ability to view a replay of the training scenario or requires an analysis of communication between team members, for example,” said Wiese. The Feedback Toolkit allows Aptima to work with customers to identify their performance measurement and AAR requirements and develop an AAR application that addresses their training needs.
TESTING AAR APPLICATIONS
Aptima is currently working with the Air Force Research Laboratory to test an AAR application that combines performance data collected by instructor-pilots through observation, calculated from simulator data, and collected directly from trainees through selfreport questions. “This work takes our previous distributed AAR work in a new and novel direction by more closely integrating the different types of performance data and directly linking the data to defined competencies,” Wiese said. “This application will be used in the context of distributed simulation- based fighter pilot and air battle manager team training.”
In nascent work with the Naval Air Warfare Center Training Systems Division, Aptima is developing a common operating picture for performance measurement data that will display individual and team performance data to instructors in real-time, as a simulation-based training exercise progresses. “While many aspects of this work are exciting, the real-time component is pushing the envelope by investigating how quickly we can provide useful performance data to instructors,” said Wiese. “Our theory is that the sooner instructors are able to see detailed trainee performance data and trends, the sooner they will be able to adjust the scenario to address performance deficiencies or perhaps make the training scenario more difficult.”
The biggest improvements in AARs are currently coming not necessarily from the inclusion of higher quality replay systems or the addition of sound—by and large, these capabilities exist already. Rather, the biggest leaps forward are coming from the inclusion of instructional theory, e.g., how and when to include performance measurement data, into these technologies to enable instructors to make better use of the AAR technologies provided to them, said Wiese. For example, a high fidelity replay system can be used to bring the instructor and trainees back into a critical moment during the training scenario. However, using these systems by themselves is still fairly inefficient. Instructors must remember when those critical moments occur in the scenario and remember what aspects of performance they wish to discuss during the debrief.
Aptima is moving beyond standard analyses of who-talked-to-whom and for how long, and getting to the point to where they can automatically analyze the content of those communications. Aptima’s ability to do this will have a huge impact on its ability to automatically assess performance associated with team communication and collaboration.
SAAB INSTRUMENTATION SYSTEMS FIELDED
Saab has a large number of instrumentation systems fielded with a wide variety of users from above the Arctic Circle in Finland to the deserts of Saudi Arabia as well as most military forces in Europe and the United States. Each user has its own language, data format and methods of conducting training and after action reviews requirements. Saab’s emphasis on AAR capabilities has been the development of universal tool sets to support the wide variety of user requirements, said Bob Clydesdale, business development manager for Saab Training USA. The entire software suite that plans, executes, analyzes and provides the AAR is WINEXCON II. There are five major tool sets in this suite broken down into the following areas: AAR Tools, Report Generator, Evaluation Tool, Import/ Export Tools, and the AAR Composer. AAR Tools evaluates maneuver training, gunnery exercises and an open protocol system called AAR Viewer. AAR Viewer can be used on any computer for take home packages for the units after they leave the AAR site. “The Report Generator is a standard set of reports for normal information almost any commander wants to see such as time to engage, rounds fired per engagement, friendly fire events, accuracy of fire, casualties and status of all units in various pie, bar or line charts,” said Clydesdale. The Evaluation Tool manages training objectives and provides semi-automatic evaluations of the training results versus the objectives.
The last two tool sets relate to the import and export of data and the composition of the AAR. There are data lists needed before an exercise that will provide a more effective AAR. “The order of battle or unit lists of all players with their name, vehicle number, rank, position, unit, weapon, assigned player ID and corresponding table of organization of unit assignments are almost always input in the planning stage,” Clydesdale said. “The correlated data allows the instrumentation system to automatically collect and report the actions of every player in the exercise.”
Saab often supports exercises involving more than 2,000 live instrumented players in Europe and the use of this universal import data tool allows the rapid input of most data sources in a day as part of the equipment issue. The same principle applies to the export function. Data collection is often required by simulation or analysis systems remote from the live instrumentation exercise controller. This tool supports rapid dissemination of data for other AAR systems.
TRAINING INNOVATION
The final AAR tool in the suite is the Composer, which structures the review with a set of templates and digital, video and audio data sources. Clydesdale said the rapid composition of the AAR by synchronizing video images of engagements, digital information on player status and firing events, collected audio of voice and digital communications from command nets or remote data sensors is the next innovation in training that will enhance the fidelity and utility of the training exercises. “Nothing motivates troops in the field to perform at the highest level than the knowledge that they will be able to see and hear a replay of their actions in the context of the larger operation,” he said. “They will experience this realistic AAR while the live experience is still fresh in their mind and they are still in the training area.”
The Marine Corps recently fielded Instrumented Tactical Engagement Simulation System (I-TESS) using Saab live instrumentation with these AAR tools. The live players will be tracked from open terrain through a large number of urban warfare sites with video and digital systems monitoring every action and rapidly collating the real-time data into a multimedia AAR that can zoom down to the actions of an individual or assemble the reports for entire companies and battalions. The next large Joint Forces Command exercise is “Bold Quest,” held at Camp Lejuene in November 2009 with Marine and Army units using the I-TESS instrumentation system and newly fielded AAR tools.
There are some video capture and integration AAR innovations fielded now in Europe that will be introduced to the United States in some developing programs like the Army’s Urban Operations Training System and the Marine Corps’ next-phase Instrumented Tactical Engagement Simulation System II. Mobile, battery-operated cameras capture the data for real-time display through a wireless video network with the recordings automatically merged with exercise events recorded digitally. “This video integration capability is unique in its ability to rapidly integrate a large number of video inputs from cameras in a wide variety of locations over a large exercise area with no operator in the loop. The digital instrumentation system can trigger video capture of developing situations such as engagement in Figure 1 or soldiers clearing a room in a MOUT mission,” Clydesdale concluded. “The instrumentation system can also identify the units or individuals captured by the synchronized digital/ video and superimpose the identity in video screen for the AAR.” ♦