Document Type


Publication Date





Numerous previous studies have shown the positive effects that simulators provide in training aircrew for both basic maneuvering and more advanced team combat skills. As technology evolves, we have the ability to connect simulators separated by great geographic distance and often in different countries, thus allowing an even greater number of aircrew to train together as a team. In addition, live-fly training events provide an excellent opportunity to train aircrew. However, these large-scale distributed and live-fly training events must be examined for effectiveness. To accomplish this, training assessors must use a common set of data collection instruments, and follow a common protocol that says when to use them, and who should use each one. This is particularly important in distributed simulation exercises where training participants (and assessors) may be located at a number of geographically dispersed sites. The Air Force Research Laboratory along with its international partners has developed a set of knowledge, skill and experience based surveys that can be used to evaluate large scale distributed training events. In addition, we have developed a protocol that specifically describes when each survey should be given, and to whom. These events pose unique challenges in data collection; therefore, in addition to traditional paper surveys, we have developed a website and “minibrowser” that can run on a stand alone, non-networked computer to collect data from participants. These tools, protocols and data collection methods have already been successfully used at a number of large scale distributed events. This paper will present data and results from US and multinational applications of the methods and tools, discuss feedback and lessons learned. We will discuss and highlight the latest versions of the tools, protocol and methods we have developed, and present the future directions we hope to take with this research