Alibaba brings visual-recognition capability to its smart speakers

Alibaba AI Labs has launched - the updated human - computer interaction system, AliGenie 2.0 that will enable Tmall Genie X1, the Labs' first smart speaker with visual- recognition capability, to scan and read over 100 sets of children's books as well as 40,000 medicine boxes. AliGenie is the AI platform powering Tmall Genie.

The visual-recognition capability is achieved through "Genie FireEye", a new function in Tmall Genie's mobile app. The function is activated by scanning a QR code that comes with a phone holder "XHolder", which is available for RMB 49 each. It can then use the front-facing camera in a user's cell phone to scan the cover and read the children's books by leveraging the image-processing and visual-recognition capabilities of AliGenie.

The function also makes Tmall Genie a good companion for the elderly, as medical information such as side effects and expiration date can be retrieved and read out by scanning over 40,000 medicine boxes. Alibaba AI Labs has partnered with publishers in China and AliHealth, Alibaba's healthcare subsidiary on the project.

In addition, Genie FireEye enables Tmall Genie to interact with users visually through more than 20 simulated animations of facial expressions. A user can touch the "eyes" and "nose" of Tmall Cat, the screen image when the visual function is activated, to generate feedback and interaction. Users can enjoy a personal, multi-module interaction, including voice, visual and touch interaction with the machine.

As of today, over 2 million units of Tmall Genie has been sold in China since its launch in July 2017. Tmall Genie has been connected to more than 45 million smart home devices and performed over 900 million tasks. AliGenie has partnered with over 100 brands in the airline, education, hotel, nursing home, auto, home-appliance and music and entertainment areas in China.