Automation testing is a great long-term QA strategy. It is also complicated and sometimes isn’t entirely practical. In this post we review some prerequisites to think about when analyzing the potential for automation.
Assessment
Prior to the implementation of any automation effort, a professional assessment of the potential effort needs to be completed. This assessment would be done by a test automation expert along with availability to and the interaction of associated application experts. This assessment should include a thorough investigation of existing manual test cases to be automated, test data, environments, associated databases, test tool compatibility, etc. This assessment should reveal what tests are the best candidates for automation as well as the estimated effort.
Pre-existing Tests
Automation does not preclude manual tests or manual testing. One must avoid the “silver bullet” mentality that automation is going to replace manual testing or creation of manual tests. It simply is not true. All new functionality in an application is initially tested manually which means manual tests are still required. Once the application is deemed stable and correct the manual tests become candidates to be converted to automated tests.
There is a set of criteria that all existing manual test cases must satisfy in order to create ideal candidates for automation.
Independence
Tests best suited for automation should be able to run as a standalone test. This means no other (dependent) tests or application experts are needed to accomplish the test execution. If the test is to be automated, the automation expert should be able to follow the test easily without needed extra dependencies not directly attached or associated with the target test.
Repeatable & Reliable
The test cases should be easily repeatable and reliable in the test environment. This means the test results are consistent and the application functionality targeted by the test is currently not in a state of change. Regression tests are ideal candidates for automation.
Environment
The testing environment must be available for test automation development and execution. This should be a dedicated testing-only site. A potentially unstable environment should be avoided (such as the developers’ sandbox). All environment variables need to be available to the expert doing the automation assessment. Such variables could include the application under test (AUT), application and test database(s), middleware/drivers, hardware, etc.
Data
All test data associated with target test cases must be available prior to implementing the automation effort. There should be solutions for the creation and population of data that tests are dependent upon.
Application Expert Availability
Existing AUT experts such as test analysts, users, database administrators, developers and associated project managers should be available based on the test automation engineers needs.