A rundown of trends in 2018 and what to expect in 2019.One of the ongoing trends in the QA industry for the last few years has been Test Automation and Continuous Testing, and this trend is going to continue in 2019 as well. While CI/CD, DevOps and Test Frameworks will remain prominent themes in the coming year, several new technologies are affecting what we test and how we test.
Expect to see more open-source testing frameworks in JavaScript land, more artificial intelligence (AI) capabilities embedded in the tools you use and more innovation coming from commercial tool vendors. Another continuing trend is a combination of functional testing with performance testing — (think of it as Selenium combined with your Jmeter tests). Also, expect to see lots of new development in Behavior Driven Development (BDD Testing), and how it is adopted in an Agile organization. Automatic test scenario generation is another area which we are working on with several of our clients.
Here is a full rundown of the state of the software test automation.
IoT Testing
IoT (Internet of Things) is affecting the testing field significantly. Traditional methods of automation like Selenium are rendered useless in an embedded environment. We are seeing more and more Python and C/C++ based test frameworks that perform unit testing, integration testing, and system testing. Most test frameworks are testing APIs exported by these embedded libraries, where quite a few of them are calling into the embedded code to perform unit testing. This requires specialized test engineers with significant Software Development experience — and we see more Software Developers will be deployed to automation testing roles. Python is probably the language of choice for IOT test framework development — because of its ability to call into C code directly with ctypes package.
Another new trend is the DevOps environment for IoT is starting to get standardized. So far, we have seen mostly ad-hoc implementations of a CI environment — but we have been working on standardizing the IoT CI environment. We have pre-built solutions in place for build management, tests management, image loading, deployment of IoT images on different devices, A/B Testing for IoT devices with different builds etc.
Continuous Testing
Continuous Testing is another trend from last year that still continues to date. We have seen an explosion of DevOps and CI/CD frameworks in the past, and this year the trend continues with newer frameworks like Nevercode and Codefresh.
Another trend in Continuous Testing is AI based risk assessment for each release. Previously, this operation was manually performed to determine which releases can be deployed for an application. We have implemented a couple of CI/CD platforms that perform automatic AI based A/B deployment of the application.
AI Based Testing
AI based testing methodologies have become more than just a buzzword and entered mainstream testing practices. AI and automation are two parallel aspects of testing — automation is used for functional testing and AI is used for Visual Testing. AI based visual testing — including the look and feel testing and giving a quick run-down of visual changes per build is an immensely helpful method for release validation. We have worked on implementing Applitools based visual testing solutions at different clients in Denver.
Though strictly speaking, Visual Testing is not AI based currently. The image comparison algorithms are traditional gradient classification based, but most in the industry refer to this as AI based testing.
Few other unique tools that we have worked on can automate many tasks intelligently.
- Test Suite Optimization: we have developed few tools that analyze log patterns and identify which test cases are repetitive or duplicate.
- Defect Identification using Log analysis: Highlights software defects based on log analysis.
- Automatic Test Scenario Generation similar to Swagger.
Open-source Testing Frameworks
One of the growing trends we have been seeing over the past few years — has been the move away from traditional enterprise Testing solutions like HP QC, ALM, UFT, IBM etc. We are seeing increased adoption of Opensource testing platforms across organizations of all sizes. We have personally migrated several of our clients’ Test Frameworks from HP QC/UFT towards other open-source solutions. Though there is coding involved with these open-source solutions, they are highly customizable and maintainable over the long run. We predict that these open-source solutions would continue to gain more traction as we progress into 2019.
Merging of Agile and DevOps
The key principle of DevOps is that the Development team, Test team, and Operations Team collaborate to get out the software releases seamlessly. It means centralized or segregated QA departments are now having to merge with development and ops teams to provide testing services on-demand for various releases. Testing is becoming more progressive, iterative and integrated with the application development and deployment processes.
We are seeing an adoption of BDD based testing mechanisms that allow iterative testing for new features developed over a sprint cycle. BDD stands for Behavior Driven Development, which itself is derived from Acceptance Test Driven Development (ATDD). BDD forces teams to come up with Test Scenarios along with the requirement gathering. The test scenarios are immediately written down and checked in into CI system to force the CI system to show failures for these scenarios. The goal of the development and QA teams during the Sprint now becomes making these scenarios pass. This new mechanism of testing framework development — is novel in its approach and well suited in an Agile environment. We are seeing a large number of our clients are moving to BDD based test development in their Agile practices.
Performance Testing to Performance Engineering
One of the key trends in testing has been the continuous shift of Performance testing role into a full-fledged Performance Engineering role. Performance engineering now includes not only the testing aspects but also monitoring the performance of the system, automatic scaling of resources, A/B testing, ELBs, database optimizations, bottleneck identifications, and monitoring. Several cloud-based tools are now available to accurately monitor various performance parameters on different cloud resources and a dashboard monitoring of all resources with alerts has been one of the main parts of our work at various clients.
Micro Services Testing
As more and more applications are moving towards the micro-services model, the test architecture is also moving towards the micro-services testing model. Previously QA for the product followed a black-box testing model, but now, with microservices testing, we are moving towards a gray-box testing model.
Micro-services testing includes API Testing, Database Testing, Auth Service/Search service testing etc. We can call this testing model as more of a component-testing model instead of testing an integrated product.
Micro-services testing allows us to catch issues in advance and prior to the big-bang integration of all the changes. It is still a level higher than unit-testing, as components have to be completely defined and testing is based on the external APIs of these components.
Testing as a Service (TaaS)
Testing as a Service (TaaS) or Managed QA Services is an outsourcing model where testing activities for the organizations are performed by an external team rather than employees. The external team in many cases is an offshore team, but we have had some instances where we have developed on-shore development teams for initial phases of automation development and project hand-over, followed by an offshore team for QA maintenance.
The advantages of Testing as a Service (TaaS) include:
- Support for on-demand testing resources. Testing is a cyclical activity where resource utilization is not constant. Testing as a Service would mean that clients would only have to pay for the hours when the test resources are in use.
- Lower costs due to offshore resources: Testing as a Service also helps in reducing costs for the organizations due to lower labor costs associated with offshore organizations.
- Automation Services are included: Testing as a Service includes Test automation frameworks, CI/CD frameworks, and performance testing and monitoring, thus reducing the varied costs to the organization.