Working as a freelancer has me on the hunt for a fitting role more often than when I was on a payroll. This arrangement also allows me to talk to many people about their needs and wants in the software testing arena, especially when it comes to automation of these tests. Over the last year, it became evident that the business awareness of the benefits of automating your functional tests grew tremendously. Good for me, more work! To make sure we all talk about the same thing when we have a conversation about test automation, here are a few definitions and itemisations with regards to my wonderful profession.
It is put in the form of a FAQ, so I can shamelessly re-use it in my LinkedIn profile.
What is a test?
A test is a series of steps taken against the target (or surrounding) system to mimic real-world and designed situations including asserts to ensure one-selves that the system is behaving as desired.
What is an automated test?
Like above, with the addition that all steps and asserts are executed by the computer in a script without human interference or interaction.
What is test automation?
Like above, with the addition that all pre-test activities, test execution and reporting is executed by the computer without any human interference or interaction. This as a whole can be then added to continuous integration solution such as Jenkins.
But wait, then what is a tester? Or testing?
A tester does testing, he will have a series of (sometimes pre-defined) steps that he executes by hand to mimic real world usage and visually check the result to see if it matches his expectations. Testing is slow, error-prone and pointless unless done as a final step by business users on a fully integrated environment using real-world data.
So you can do some test scripting, right?
Test scripting is taking the above tests and writing a single-layer script to execute the otherwise manual test or some part of it. As the test was originally designed to allow for the human to catch deviations outside of the strict steps, and these unscripted observations are not in the script, this form of test automation is often seen as low-coverage and brittle. Upside is that the tester is now able to run more tests than before, downside is that it still requires manual interaction either for setup, starting the tests or checking reporting. Personally, I stay away from test scripting as has low coverage and is not fully automated.
So who will do the test automation? That is a tester, right?
Honestly, writing your tests in such a way that they are robust and can be executed many times without manual interaction or monitoring is rather different from classic testing. Your tests will be smaller, more numerous and atomic in nature(checking just one thing), they will re-use steps written before and have navigation checks (for example if you are on the correct page) and waits.
In my experience, most of the automation is done in a framework allowing for one or two very specific interfaces, such as a web application and a Rest endpoint. For extension of the capabilities and embedding of the framework into the CI/CD setup, the help of a software engineer is needed.
We also need a software engineer?
Depending on the complexity of your systems you need a Software Engineer in Test paired with one or more Test Automation experts. The SE will setup the test framework, enable and enhance the interfaces to the system under test, ensure all starting conditions are met using automation, automate reporting and embed the tests into the CI/CD setup. Everything considered, the SEiT will build an application (using frameworks) specifically for the testing of a complex system.
So there we have it, the scale from tester to engineer and of course it is not a hard either/or and more a gradual scale. The key take from this scale is that a tester and a software engineer in test are very different jobs with very different skills. Asking your engineer to do manual testing would be the same as going grocery shopping with an excavator. It is possible, but there are vehicles much better suited for the task as there are tasks much better suited for an excavator.
