From Tester to Software Engineer in Test

Working as a freelancer has me on the hunt for a fitting role more often than when I was on a payroll. This arrangement also allows me to talk to many people about their needs and wants in the software testing arena, especially when it comes to automation of these tests. Over the last year, it became evident that the business awareness of the benefits of automating your functional tests grew tremendously. Good for me, more work! To make sure we all talk about the same thing when we have a conversation about test automation, here are a few definitions and itemisations with regards to my wonderful profession.

It is put in the form of a FAQ, so I can shamelessly re-use it in my LinkedIn profile.

What is a test?
A test is a series of steps taken against the target (or surrounding) system to mimic real-world and designed situations including asserts to ensure one-selves that the system is behaving as desired.

What is an automated test?
Like above, with the addition that all steps and asserts are executed by the computer in a script without human interference or interaction.

What is test automation?
Like above, with the addition that all pre-test activities, test execution and reporting is executed by the computer without any human interference or interaction. This as a whole can be then added to continuous integration solution such as Jenkins.

But wait, then what is a tester? Or testing?
A tester does testing, he will have a series of (sometimes pre-defined) steps that he executes by hand to mimic real world usage and visually check the result to see if it matches his expectations. Testing is slow, error-prone and pointless unless done as a final step by business users on a fully integrated environment using real-world data.

So you can do some test scripting, right?
Test scripting is taking the above tests and writing a single-layer script to execute the otherwise manual test or some part of it. As the test was originally designed to allow for the human to catch deviations outside of the strict steps, and these unscripted observations are not in the script, this form of test automation is often seen as low-coverage and brittle. Upside is that the tester is now able to run more tests than before, downside is that it still requires manual interaction either for setup, starting the tests or checking reporting. Personally, I stay away from test scripting as has low coverage and is not fully automated.

So who will do the test automation? That is a tester, right?
Honestly, writing your tests in such a way that they are robust and can be executed many times without manual interaction or monitoring is rather different from classic testing. Your tests will be smaller, more numerous and atomic in nature(checking just one thing), they will re-use steps written before and have navigation checks (for example if you are on the correct page) and waits.
In my experience, most of the automation is done in a framework allowing for one or two very specific interfaces, such as a web application and a Rest endpoint. For extension of the capabilities and embedding of the framework into the CI/CD setup, the help of a software engineer is needed.

We also need a software engineer?
Depending on the complexity of your systems you need a Software Engineer in Test paired with one or more Test Automation experts. The SE will setup the test framework, enable and enhance the interfaces to the system under test, ensure all starting conditions are met using automation, automate reporting and embed the tests into the CI/CD setup. Everything considered, the SEiT will build an application (using frameworks) specifically for the testing of a complex system.

So there we have it, the scale from tester to engineer and of course it is not a hard either/or and more a gradual scale. The key take from this scale is that a tester and a software engineer in test are very different jobs with very different skills. Asking your engineer to do manual testing would be the same as going grocery shopping with an excavator. It is possible, but there are vehicles much better suited for the task as there are tasks much better suited for an excavator.

Manual testing is stealing from the future.

Often when arguing with testers about what to automate and what to test with manual scripts, my point of view is very simple. You will automate ALL the things. The tests, the builds, the testing of the tests on the builds, code quality tooling, the deployment further down the chain, the testing on the chain and if all is green after this firework, the deployment to production and turning it on for people to see. This is not some strange future alien technology thing, it is real and companies that are willing to invest are already doing this, keeping them in front of the hordes of competitors.

The most common arguments against doing full automation, always come down to the same things.

You cannot automate everything.
Yes, you can, but it might not be easy and it is very possible that your current testing workforce is simply not capable of pulling this off. Running manual tests is very tempting when you have been doing it for years, test automation might be scary, threatening or complex for those stuck in their ways. To these, I would like to say: either adapt by learning new skills or get out of the way please, you are holding up progress.

Test automation takes time to get set up.
Exactly, investment in time is needed, all the more reason to start today! Every day you wait is a day lost in the face of a deadline that is always around the corner.

Automating things is expensive / slow / we don't have the people for it.
Seriously, this is the best you can do? When you fire 10 manual testers and hire 2 test automation engineers who are worth their salt, you will have money left over. It is fast in execution and after setup can be repeated every build without added cost. And if you have problems recruiting people who can do this job, either train your workforce or offer higher compensation. We are talking software QA here, you want to cut corners on that and deliver buggy software? That is fine, but do not expect sympathy from anyone who did not want to work for ultra-low wages.

Our project has two weeks of regression testing and bi-monthly releases, there is no time to do any automation.
Change asks for investments. Either you do less manual testing and move people to the automation team or you spend money to speed up regression testing in the future.

And that last point is where the title comes from, all manual testing you do are taking time and people away from a better future. For every manual test executed, it could also have been automated. Once automated, the next run is without cost to your testers, so they can automate another and another and work away on that huge pile of technical QA debt you build up over the years.
When writing tests in an automation framework, your people become more trained in doing so, asking them to put the automation effort aside and do a quick manual test, or long manual test period, will hinder them in acquiring the skills to write testcases faster.

If you want to do test automation, accept these points:

• A test automation engineer has higher pay than a manual tester, test automation is hard. Not every one is suited for it.
• Test automation takes some time to get started, when running it can run a zillion time without added effort.
• Automated testcases are pieces of software that fit into the project, when the project delivers code, some tests will change and require maintenance.

Test automation is the future and the future has arrived. Please don't steal from your own future.

Introducing FitNesse, the testing framework

Talking about test automation often is a discussion about tooling, what tool is best for what purpose. Normally I do not take part in this, in my view a tool is a tool is a tool.
Knowledge on what tools are available and what each has to offer is a necessity, you need that to be able to fill your personal toolbox, but more important is the ability to pick up a tool an know how to use it reasonably well in a matter of days. Each place you work at has it's own demands on environment, the tools already in place and the personal preference of the people who make up the organisation. All this means that knowing all tools a bit, know a few well and have the ability to ramp up your skills fast is far more important than picking a tool and sticking with it.

Yet, this post is about the specific tool FitNesse because I think it is underestimated in the market for a variety of reasons, not in the last place because it is relatively unknown in my local market and it is user friendly. It took me a while to understand how user-friendliness can be detrimental to the popularity of any tool/framework but I think I have it figured out. Because FitNesse allows you to set up your tests in a way that you can explain to business users what you did and what your data points are in a way that is editable to them, there is some obscurity in the step to the actual (java) testing code. This is a good thing, the java code is not causing static in gaining the business trust that the tests are performing. On the other hand, those who are writing the tests like to be more on top of the code and view the format FitNesse offers as a hindrance. This is a bad thing, as it displays a misunderstanding of the framework FitNesse offers.

So let's go into that a bit more. According to the wikipedia page, FitNesse is "a webserver, a wiki and an automated testing tool for software". The rest of the page is technically true but somewhat confusing, it took me a while to figure it out myself. This is another reason FitNesse is under represented, the documentation and publications could use some work.

So here is my explanation of FitNesse to other testers and developers: "It is a stand-alone, no server needed, testing framework that allows you to write both very transparent test scripts in a table based format and set up data driven testing using javacode, whatever your preference." The use of the term framework is to emphasize you can go whatever direction you want when writing your java tests while the framework will take care of data input and output validation, reporting and integration with CI/CD tooling.

And there you go, FitNesse is a tabular view on either testdata and outcome predictions and with the right plugins, a tabular view on your testscripts.

To make the distinction more clear:

• FitNesse provides you with a framework to give a tabular view on your input/output tables, while your java-based testscript is somewhat hidden

OR

• FitNesse (with the right plugins) provides you with a wiki-based platform to give you transparent test scripting as well as input/output tables.

These two implementations are not exclusive, you can even mix and match in the same testscript if you desire to do so, but realize that whatever you do, you create a presentable test set that is build upon a mature framework.

In the first case, your testscripts require more than basic java knowledge and a thorough understanding of test automation, where it's strengths and weaknesses lie. In the second case, java is less important, some programming concepts are still required but this goes for the entirety of the test automation role.