TED Conversations

a mum...,

This conversation is closed.

To enable sign language to be translated for non speakers and vice versa.

Using the gesture interactive technology on 'Signers' fingers to be read by a camera and translated into words by a program that enables non signers to understand and interact without having to learn sign languages ( of which tree are more than one, I believe).

  • thumb
    Dec 19 2013: That's a really great idea!

    I Googled it and there seems to be quite a few big companies working on it, here's an article on Microsoft and the challenges that this technology faces. http://blogs.technet.com/b/inside_microsoft_research/archive/2013/07/16/digital-assistance-for-sign-language-users.aspx

    There seems to be a bunch of apps with pictures of sign language so that you can do it manually, letter by letter but that's not really that good.

    Also Google Hangouts now has an app for calling in an interpreter if you need one, which is a kind of neat way of bypassing the technical issues, but then again you don't get privacy and you both need to be in front of a computer.

    I think that it will be here within 2 years tops.
  • Dec 21 2013: I believe that this "translation of motion into either sound or written language should be eventually achieved by the pattern recognition technique adapted to motion image instead of a still picture. It seems to me that sign language may be even easier that trying to interpret speech sound because that could be very different based on the speaker's pitch and accent, while the sign language has less variation than that of speech.
    In my opinion, the problem of the slight variation in the posture of the facial, arms and fingers must be normalized first, then the signs may be reasonably interpreted on the normalized frequency, the extension and the moving speed. In other words, we could always record the signer's interpretation, then "standardize" the signer's facial, arm and finger dimensions and the speed of the movement,, then start to interpret the sign language word block by word block. with the synchronized continuous motions of the signer or the sign interpreter.
    There will be a time lag at the beginning, but with the modern computer speed, it could easily catch up as the signer action continues on. Actually when we watch the language captions on the TV broadcast, you probably sometimes notice the time lag as well. In other words, the sign-to-translated-language, either in written or voice would first be normalized by the computer to shrink or magnify the size of the signer (his face, arm and fingers) and also to speed up or slow down the movement speed so that a standard version of the sign language translation could be applied by looking for the meaning with images of every nth (say 5th frames) frame, then reaches the interpretation of the combination of these consecutive motion images.