Thanks to this technology, you can not only automate tests but also speed up other processes in the project. In this article, you will learn why automation is so important, why we decided on such a solution, and how we dealt with the problems we encountered along the way. You will also learn how to use Appium and AWS Device Farm in other ways and how we would like to develop our solution further.
Purpose of the application
We create applications that collect data from the phones of patients with neurological diseases. This data is then analyzed and used to determine the effectiveness of the drugs. This is a more effective and cheaper method than clinical tests carried out during hospitalization for several weeks.
The applications present the patient with several tasks to check his health every day. A few examples of tasks include: short conversation recording, facial recognition, simple memory games, and polls.
It takes a long time to test especially games that do not have a predefined scenario, and it is also a big challenge for automated tests.
Besides, the schedule for patients provides for several different lengths of such an examination, depending on the patient’s condition: from 14 to 42 days.
The patient has a different set of tasks for each day. On the first day of the research, the user has to complete all possible tasks, but in the following days, there are much fewer tasks, they are set in a specific order and with a different time limit. In the case of manual tests, it is very time-consuming and burdensome, because you have to make sure that all games are successful, and check that the sets of tasks agree with the individual days for which they are prepared. In this case, automated tests are an ideal tool that supports manual tests. Once designed, the automatic test, which goes in the same sequence when we program, allows us to thoroughly check the entire testing process.
Support manual tests with automation
Automated scripts were programmed with Java. We use Maven for project building and Appium for communication with mobile devices. We do not use emulators for testing, but real phones with both Android and Apple software. To solve the problem of the diversity of devices available on the market, we use AWS Device Farm, which allows you to run tests on a large pool of physical devices. These devices can be chosen freely, paying attention to the type and version of the operating system, it is very important because some applications support only some versions of the OS, both in the case of Apple and Android.
To use the pool of AWS Device Farm devices, we pack our tests into JAR and send them to the farm’s servers together with the application binary file. Thanks to this, everything works locally and there is no problem with long-distance communication or various geolocations.
The report we get after performing such tests shows how the devices coped with a given test – how long it lasted when it was started, and how it ended. After selecting a specific device, we have access to a movie recorded of the entire test, screenshots, log of the tests themselves, log Appium, and Farm log. Based on this data, we can find out what went wrong and report errors or correct the automatic code. The most common errors are not software errors, but compatibility problems with a specific phone model, or that in the resolution of a given screen you cannot see everything that the application is supposed to show.
More than testing…
The applications are available in different countries and different languages. Translating the application requires a contribution in the form of screenshots from the entire application, not just texts, as is the case in web applications. This enables the appearance and format of the translated text to be verified. Phrases and sentences in different languages have different lengths, so you need to check that the application wraps it properly and that the subtitles appear correctly on the screen.
We currently support 27 different applications with a maximum of 25 languages.
What does the translation process with Appium look like?
We create the so-called deck – a set of all screenshots available in the application generated in the form of a ppt presentation. This method is most useful for business and translators. The translated text comes back to us and is implemented by the programmers.
In an application with a new language version implemented, the entire deck must be made again, which is the basis for the final verification of whether the texts are correct and whether they display well in the application.
Before we introduced the automatic method of creating screenshots, the whole process was done manually: The person responsible for translations had to take screenshots from the entire application, in this case from the entire duration of the patient schedule (up to 42 days!). This person had to make sure not to miss any test, try different variants, entering correct and incorrect data, to take screenshots of all possible application screens. Such a process for one application took up to 8 hours and was associated with the risk of overlooking some screens inadvertently.
Using the technology: Java, Appium, maven, and JUnit, we were able to execute special scripts that were used to collect screenshots. Thanks to them, we were able to automatically collect all screenshots from one application within 1 hour, which is multiple accelerations of work.
Further development of the solution
The next step in improving our solution was to automate the launch of the entire deck generation process for a given application. In the Bitrise system, which is used to build applications, we have added a task in which we decide for which application, in what version and language we want to generate screenshots. Using the Bitrise API, we download the IPA or APK and send it to AWS Device Farm using their API. Then we download the automatic test code which we pack into JAR and also send it to the device farm using the AWS API. The next step is to run a task that uploads the application to a given pool of devices and runs the script. After completing its operation, the same tool using the API, this time GDrive, sends a set of all screenshots divided into directories to the appropriate folder. The last step is to run a Python script that generates ppt files from assemblies.
So the only entry into the process is the selection of the application, version, and language. After an hour, we get a ready ppt presentation at the exit, which we can send straight to the translator or business to verify the translation implementation. Reducing this time from 8 hours is a huge gain for the project.
Summary
Automation with Appium is a powerful tool that can be used not only to automate regression tests. It is also perfect for supporting the translation process, testing the appearance of the application, preparing data for publication or user manuals, or supplying data to the demo application. The use of services that give API access to the pool of physical devices (such as AWS Device Farm) significantly increases our possibilities of automation, and thus shortening the time of business processes in the project.
