A Step by Step Guide to Voice Enabled Device Testing

It has been said that devices cannot do everything that humans can do. However, the devices that we use in our daily lives have been evolving over the last couple of decades. We have seen significant changes in them in terms on functionality, connectivity, and size. One of the biggest challenges, however, has been the size of the device as more efforts are put in to achieving a smaller form factor.

A few years ago, a new challenge emerged: the device cannot communicate like humans. This led to standalone devices being transformed into connected devices with added voice enabled operations.

How are Voice Enabled Devices Helping Human?

Initially, human touch was required to perform any action on the device. Now, with voice enabled devices and IoT technology, humans can give a command to operate the device over voice. These newly invented devices convert human voice into device action, sends command to another device over the internet, and get the desired action performed. These devices can not only interact with humans, but can also interact with other devices over the internet.

The Major Challenge of Voice Enabled Devices Testing

Day-by-Day usage of voice enabled devices around the world is increasing rapidly. Having more than 500 countries supporting more than 1000 languages with different accents, genders, and voice modulations based on age group, makes it challenging to verify voice enabled devices. It is almost impossible to test these devices with so many different combination and permutations in a short span of time. So, let us see how we can automate the testing of voice enabled devices.

Automating Voice Enabled Device Testing

To avoid manual testing efforts, we need to design an automation solution which can be used to test these devices using different languages. The easiest option is to work with frameworks that can help develop automation scripts with such voice integrated devices.

As of now, there are no open-source frameworks available in the market that provide all of the features required to test integration with voice enabled devices. The challenges here are how to give a command to the device in different languages, how to read the response from the device, and test the expected output.

• To give a command to a device without manual efforts, one needs to identify a command in text format. Convert the text in audio format.
• Play the audio so that voice enabled devices can listen and process.
• Wait for a response from the device, record it in an audio file, and as the last step you need to convert this audio into the text to match against the expected format.

Each device testing procedure will have custom requirements; hence, the framework has to be modular. To get the solution, we need to design a modular and scalable framework where each step of this solution can be implemented by open-source or paid libraries available in the market.

Real Life Scenario for E2E Testing of Home Automation Product

Home Automation System consists of various devices that can be operated through the web using REST services. Security cameras, lights, thermostats, sensors, and doorbells are few examples of home automation devices. For example, the end user who is the homeowner can turn on or off the light remotely using internet portals. Some of the systems provide integration with third party partners like Alexa, Google, etc.

Companies provide devices that can listen to the human voice and perform the action requested by the user. So, considering light as a home automation product and Alexa as a third-party partner to home automation system provider, we want to test whether light can be turned on/off through Alexa or not.

To automate the E2E scenario, we need to perform the below steps using the above discussed automation framework.

1.Prepare the Alexa command to turn on the light in the English language.

2.Convert above command to mp3 file

3.Play an mp3 file near the Alexa device using the speaker attached to the automation machine.

4.Record the response from Alexa in the wav file.

5.Convert audio file to the text which could be “Ok. Turning on the light” or “light is turned on”.

6.Verify the converted text with the expected result set.

7.In verification, just one step ahead, we can also verify the actual IoT light status using:

a.REST API can be used to fetch the light status from the home security system

b.Web automation of web security portals can be done to verify the light status

c.If the light status is being stored in cloud DB, we can fetch the data from DB to verify the status

Conclusion

Using the above steps, one can not only test the system integration or end-2-end testing with just one voice enabled device, but can also test the system by combining multiple clients or devices. Users can perform an action on one of the device/product using Alexa and verify it’s status using Google or Portal or vice versa. For example, user asks Google to Turn On the light and then fetch the light status using Alexa/Customer Portals.

eInfochips is a preferred partner for product companies requiring comprehensive test coverage from devices to applications. eInfochips offers extensive cost and effort savings through test automation, SDET (Software development engineer in Test), Shift-left testing, and DevOps.