Graphical Simulation
Corde Lane


A graphical simulation was developed to allow an operator to visualize the telerobot and worksite in a three-dimensional environment. Several windows are provided to allow simultaneous viewing of multiple views. One stereo and three monoscopic views from over 50 predefined virtual cameras may be selected; views attached to each arm, to the vehicle body, and the worksite improves situational awareness by providing several frames of references which the world may be observed. Every view is completely reconfigurable allowing the operator to freely move about the virtual environment and adjust the field of view. Telemetry data, either from a training simulation or from sensors on the vehicle, is used to update the simulation moving the robotic system and highlighting changed states. Using a keyboard and mouse, the operator uses the windows on the perimeter to change control parameters and monitor the vehicle's state. The operator can use a variety of input devices to directly interact with the graphical simulation and the vehicle moving each of the manipulators to perform the specified maintenance task. The graphical simulation not only can be displayed from a standard computer monitor, but stereo is provided using a head mounted display or LCD shutter glasses.

Although the graphical simulation replaces actual video during computer simulations, the graphical simulation can be used to augment live video during robotic operations. The graphical simulation is updated by telemetry data; this data can either come from a simulation or from sensors on the vehicle. During vehicle operations the graphical simulation can provide additional virtual views of the robot. Operators can move to an infinite number of locations using the virtual environment and use that vantage point to assist them with their task. The virtual environment can also be used to perform simple functions when actual video is not provided. For example, during one test, the live video satellite feed was lost. However the data feedback was still streaming, allowing the operators to continue performing basic manipulator checkouts without video.

The capability to display the actual position of the robot within the graphical simulation has proven helpful in many circumstances. The ability to augment, and even replace live video, has improved operator's situational awareness. However, a graphical simulation has the advantage of displaying things that could never be observed from live video images. The graphical simulation can easily highlight items to grab the operator's attention to important information. Telemetry data can warn of impending problems, which can be displayed within the virtual environment. An error condition can cause a manipulator to flash until the operator addresses the condition. Virtual graphics filters can be used to observe power consumption, temperature values, and global status for each manipulator joint by providing a gradient of colors to indicate various levels.

Multiple Movie Links can be see in the Movie Gallery.

Many Screenshots of the graphical sim are located in the Image Gallery.

Related Works:

Lane, J. Corde, Craig R. Carignan and David L. Akin. "Advanced Operator Interface Design for Complex Space Telerobots". IEEE International Conference on Robotics and Automation: Vehicle Teleoperation Interfaces Workshop. 2000.

Carignan, Craig R., J. Corde Lane, and David L. Akin. "Real-Time Simulation Of A Free-Flying Robotic Vehicle". AIAA Modeling and Simulation Technologies Conference. 1999.