This project is supported by the Telematics Trust. The project brings together the Department of Human-Centred Computing and the School of Languages, Literature, Cultures and Linguistics at Monash University.
Background
Australia faces a shortage of Australian Sign Language interpreters leaving Deaf people isolated and unable to access critical services. Improving Australian Sign Language training is key to ensuring more people pass interpreting exams and enter the workforce as confident and diverse communicators. Most signers in the Australian Sign Language-using community use fingerspelling and signs as part of their communication, where, on average, signing includes about 10% fingerspelling. Therefore, fingerspelling is a core component of Australian Sign Language, used to render proper names and other English words for which there is no lexicalised sign or to clarify a message if the addressee does not understand a sign. Fingerspelling also forms many lexical signs where initials or other letters in the English word form part of how the sign is made.
Aims
The project aims to design and develop technologies for the automatic generation of Australian Sign Language fingerspelling. The project will generate knowledge by creating the first dataset of high-quality motion-captured 3D fingerspellings and the transitions between them, enabling further advancements in this research area. The project outcomes will also include the invention of the first Australian Sign Language fingerspelling generator capable of synthesizing all 26 letters (A-Z) of the English alphabet and the associated transitions, representing a substantial advancement towards automated Australian Sign Language translation. The generator will be made freely available on Auslan Signbank: Australia’s premiere Australian Sign Language website and dictionary.
Investigators
Steering Committee
- Fiona Marke (Melbourne Polytechnic)
- Tracy Pearman (Melbourne Polytechnic)
- Rachel Miers (Melbourne Polytechnic)
- Wendy Devlin (Melbourne Polytechnic)
- Olivia Beasley (Expression Australia)
- Paula Thornton (Deaf Australia)
- Elizabeth Levesque (Victorian Deaf Education Institute)
Contributors
- Jim Smiley
- Michael Neylan
- Andre Ky Pham
Outcomes
Dataset
We are in process of recording the first dataset of high-quality motion-captured 3D Australian Sign Language fingerspelling. The participants (including both left- and right-handed) fingerspell a specifically designed word list
Web Application
The WebGL application will allow for the conversion of a text prompt into an animated, interactive 3D model fingerspelling the prompt. The application will be deployed on the Auslan Signbank: Australia’s premiere Australian Sign Language website and dictionary. Below is a prototype of the application. Type a word in the search box and press ↵ Return to convert into 3D fingerspelling (use to zoom, pan and rotate).
Smartphone Application
The iOS application will allow for the conversion of a text prompt into an animated, interactive 3D model fingerspelling the prompt. The application will be deployed on the Apple App Store.
