Inclusive Technologies
This project is supported by the Monash Assistive Technology and Society Centre. The project brings together the Department of Human-Centred Computing and the School of Languages, Literature, Cultures and Linguistics at Monash University.
Academic research around tactile signing remains relatively scant. Understanding the nature of backchannel feedback signals in tactile Australian Sign Language (Auslan) is crucial for enhancing the quality of deafblind interpreting services and will also contribute to the broader understanding of haptic feedback in tactile sign languages worldwide.
In any communication, both spoken and signed, the backchannel feedback from the receiver to the sender is vital. In visual sign languages, feedback is given with head nods and shakes, indicating that the receiver is engaged, agreeing, or disagreeing. This feedback enables the signer to monitor the conversation and fulfil the receiver’s needs. However, this feedback is not available to deafblind signers. The precise feedback conventions used vary across tactile signed languages, but all use a variety of taps and squeezes, which are unique to tactile signing. Tactile Auslan signers have been observed to use a variety of finger taps and squeezes to provide tactile feedback. The timing of these taps is crucial, and they may be combined with other body movements such as head nodding to communicate effectively.
The project’s objective is to develop technologies for detection of finger taps and head nods in tactile Auslan interactions. Finger taps detection will be further divided into 1) the detection of one-finger and four-finger taps and 2) the automatic discovery of other types of taps that function as tactile feedback. This will be the first attempt at computational modelling of backchannel feedback in tactile Auslan interactions. The technology will support the analysis of tactile signed interactions and assist in understanding the conversational processes and structures involved. The technology will detect various types of tactile feedback and will enable the study of the subtle differences in meaning that are possible (e.g., speed and number of repetitions given to taps subtly changes the meaning).