It may seem at first that adding various thuds, clicks and bangs when the user hits our virtual wall is a cute feature that does not enhance the measured metrics of the project. However, there are a few papers that show the that auditory feedback has an influence on what the user feels.

The Effect of Auditory Cues on the Haptic Perception of Stiffness in Virtual Environments by DiFranco, Beauregard and Srinivasan is one such paper. In this experiment, subjects were asked to rank how hard they thought various virtual walls were while using the PHANToM device available commercially. Unknown to the subject, in one test, the walls were simulated exactly the same. The only difference was the sound played. The sound cues were about 0.18 seconds long and were played the moment of impact. The subjects were allowed to tapped as often as they liked to determine the hardness ranking of each type of wall. After analyzing the data, the researchers found that “when the surface was paired with sounds associated with tapping on harder surfaces it was generally perceived to be stiffer than when it was paired with sound stimulus”. For example, the same wall was ranked a hardness of 2.3/10 when the sound of a strike on fabric was played verses a reported hardness of 8.6/10  when the sound of a strike on metal was played. “Interestingly, several of the subjects of Experiment 1 remarked afterward that they felt a physical difference in the stiffness associated with different sounds, even though, unknown to them, the stiffnesses were always the same”.

This effect was consistent among experienced and non-experience users for one wall, however, when a random wall was paired with a random sound non-experience users’ judgment was correlated heavily with the sound. Experienced users, on the other hand, could feel an extremely spongy surface and see that the hard tapping noise does not match. However, as long as the subject is not “concentrating on ranking surface stiffness carefully, auditory cues will be useful in augmenting the haptic display of stiffness”.

The lack of tactile feedback (our device merely stops the users, kinesthetic feedback) can be augmented by using other sensory modes. Visual modes, auditory modes  and other aspects of the virtual environment can be controlled. Thus, “there are techniques and or methods that could compensate for limits in the performance of haptic interface hardware”. As long as the sound we play is what the user expects for what they are felling, this auditory enhancement will remain in effect throughout a 2 month long rehabilitation attempt.

Furthermore, O’Malley and Gupta say on page 59 that “due to the amplifiers and actuators that provide the force sensations for the operator” there are unwanted audio sensations that should be masked with headphones.