Iraqi Checkpoint training using virtual humans to learn cultural gestures
The Challenge
Approximately 90 different language courses are taught annually in Department of Defense. At the Defense Language Institute alone, foreign language courses are conducted at the basic, intermediate and advanced level to over 5,400 military and civilian department personnel. Additional courses are provided by Special Operations Forces and by each of the military academies.
The Department of Defense’s Joint Advanced Distributed Learning (JADL) Co-Lab needed to develop the Iraqi Checkpoint course to help military and civilian personnel learn Iraqi Arabic. A difficult aspect to “textbook” language training is that the meaning of words often depends on the cultural context. Depending on culture and gestures, when someone says “no” it may mean “no”, or “yes”, or “maybe”. Their body language, including gestures, can provide additional clues. Unless a soldier knows the cultural protocol and gestures used by the other person, they may interpret these incorrectly. This can result in misunderstandings or worse.
On-the-job-training (i.e. at the checkpoints) could prove a dangerous way to learn Iraqi Arabic and culturally appropriate behaviors, so the JADL Co-Lab wanted to take advantage of a virtual simulation solution.
The Solution
Using X3D and the Humanoid Animation standard, VCom3D in collaboration with the Naval Postgraduate School (NPS) and Defense Language Institute Foreign Language Center (DLI-FLC) created virtual humans in 3D simulations that combined gestures, actions, facial expression, visual cues and lip-synchronized speech. The student learns by performing progressively more difficult tasks while communicating with virtual human characters that speak Iraqi Arabic and exhibit culturally appropriate behaviors. The course is interactive and high-resolution, and it is designed to operate either over a very low bandwidth connection or as a stand-alone product.
Vcommunicator Studio automatically provides lip-synchronization to any language, using only a voice recording and transcript. The transcript is then presented along a time-line, allowing the author to rapidly add gestures, facial expressions, and eye-gaze appropriate to the character’s culture and to the situation at hand. After the author refines the animations in a preview window, all behaviors are exported as a plain text or XML file for editing or reuse.
X3D and H-Anim allows the human characters and behaviors to be reused in other instructional content.