On Agile, SCRUM and agility

These are interesting times for me, the transition from a management position where a team would do all the work and my role was mostly to shield them from the politics and planning problems, to a role where my production is determined by the amount of tests automated and the strength of my code.

After joining a new team a while back, I could not help but notice the wide variety of methods, tools and general ways of working across the several team members tasked with testing. While trying to understand what was going on and how to connect to the team effort in the best way possible, this variation struck me as a Bad Thing. It prevented me from following the determined path and fitting in, it obstructed adherence to the standards and it didn't allow me to meet expectations by replicating what others did. All because there was not one way to do things.

So the manager in me did the thing managers do, which is to create structure where there is none. I lobbied with my colleagues for some standards in ways of working, for detailed documentation on methods and for a singular environment to the benefit of all, to get clarity, conformation and unity. Most agreed with me that for newer folks and transparency's sake, guidelines and standards would be a Good Thing.  There was one guy who pointed out the principle from the Agile Manifesto that says: "People and Products over Tools and Processes." as some sort of warning not to go overboard.

Somehow I ran into a wonderful 2014 blog post by Dave Thomas Time to Kill Agile in which he argues that the word Agile has been hijacked by consultants who have something to sell that is not software development and that will be more about their Tools and Processes than anything you want to achieve in the first place. Then the weekend started and I promised my fellow team members to think over the best way forward to reach the goal of uniformity and standards.

On Saturday evening (I know, I know), I send out the following email:

Hi All,

A few thoughts to begin with:

Individuals and Interactions over Processes and Tools
Working Software over Comprehensive Documentation
Customer Collaboration over Contract Negotiation, and
Responding to Change over Following a Plan

The above are the practices from the Agile Manifesto and we need it to keep things in perspective. From my background, I have a tendency to write everything down into minutiae and setting an approach that leaves little room for creativity and personal preferences from those who have to work with it. In earlier roles this was a boon, now it is a burden. I wish to apologize if I go back to what I know, the transition to real agility surprises me from time to time.

What the practices tell us is that basically every team member can work the way he likes the most, instead of tying things down and stifle creativity.

The difficulty starts when one is no longer operating in a vacuum, some things will have to be agreed upon and synchronized across the various members of this team. So my proposal is that we as the testers find some sort of common ground, agree on that and write it down in whatever wiki/forum/repo is best.

These are the topics I think we should agree upon:

·         How to deal with old tests and old code

·         What toolset is preferred (what fits the client as well as the team)

·         How to set up new tests

·         what is the definition of done

·         What quality standards should tests meet before we go to production

·         But mostly: how will we work together

Looking forward to have this conversation with you, afterwards I will put up the things we are willing to put down.

Thinking about the why of this made me realize that the deep core of the wish for standardization came from my own desire to have a benchmark to which I am measured. I want to do a Good Job so what are you guys doing, if I do that well, will you like me? And somehow, by trying to add value I was busy removing value in the form of creativity and freedom from people. It can be traced back to a long history of workplaces where freedom and creativity is shunned and conformity and standards are celebrated, as well as my personal need for approval, to do a good job.

The take away is of course that any person who complains about a lack of standards and guidelines in whatever software development team they join, are in the core people who want to do a good job and are uncertain on how to move forward OR are genuine in their desire to kill off freethinkers. The first group is easily helped out, the second only need to see your back as you walk out the door to a better place.

Keep an eye on the tooling

Over the last few months I have been really busy at work with a Java/Selenium/JUnit/FitNesse based framework and I think this is awesome and fun to do. On the other hand, it is an established framework and most of the work has been building new testcases with the available building blocks at hand, using the tooling and techniques already there.
Nothing wrong with that, it enables one to hone skills in a very specific department and some people love being a specialist.

Just that I am curious and now that I understand these workings, I want to know what else is in the market so I made a list of tools to learn. In an earlier post there were a few of these but I want to be inclusive and be sure most is covered. Also this is a reminder to myself that all tooling is relevant and I believe that of the things listed, an automation tester should at least know the basics, the internal architecture and how to set it up in your environment.

For the above goal I build separate projects, all using the code base I have for the MantisBT automated test suite. Whenever it is working to an extend that is satisfying to me, I upload to github so check there for examples. Also, after uploading a new blog post will appear here detailing my sources and how I went about setting everything up so hopefully someone else can use it to their benefit.

The latest list is here, this excludes anything I am already familiar with. Also I try to only list open source tools as I really like the philosophy behind it.

Test Frameworks
Geb, an automation framework using Groovy for headless browser testing
Cucumber, a framework for Behavior Driven Development Test Automation (that is a lot of capitalization). Uses Gherkin as a scripting language.
JBehave comes with a description similar to Cucumber. Cucumber has a higher market relevance so I'll look into that first.
Mockito is a framework for Test Driven Development and mocking in Java.
Spock / JUnit / TestNg are testframeworks for unittesting Java (and other) development. Looking into the differences and specific use for each is part of the fun.
Protractor, an AngularJS test framework build using Selenium libraries. This is not likely relevant anytime soon but good to have listed here.


Continuous Integration Tooling
Jenkins. The leading open source build automation server. Nuff said.
Bamboo. Not open source tool build by Atlassian who are best known for JIRA. There is a small demand in the market for this, just like there is for the HP ALM scripting language UTF. I am listing them here to show I did not forget about them but will most likely skip them both.
Gradle, an open source build automation tool with a company behind it that offers consulting and services.
Docker,  an application container engine
Chef, infrastructure automation tooling
Puppet, also infrastructure automation tooling but more of an enterprise feel
SaltStack platform or Salt is a Python-based open-source configuration management software and remote execution engine.

Languages / Protocols
YAML, A straightforward machine parsable data serialization format designed for human readability and interaction with scripting languages such as Perl and Python.
JSON (JavaScript Object Notation) is a lightweight data-interchange format.
Groovy is an object-oriented programming language for the Java platform.
PERL is a family of high-level, general-purpose, interpreted, dynamic programming languages
JavaScript  is a high-level, dynamic, untyped, and interpreted programming language.
PHP is a server-side scripting language designed for web development but also used as a general-purpose programming language.
Ruby  is a dynamic, open source programming language with a focus on simplicity and productivity.
Python is a widely used high-level, general-purpose, interpreted, dynamic programming language.

Test Tooling (aka Misc)
HTMLUnitDriver This is currently the fastest and most lightweight implementation of WebDriver. As the name suggests, this is based on HtmlUnit.
JMeter Graphical server performance testing tool, for both static and dynamic resources
SQuirreL SQL as an alternative for TOAD in talking to your SQL server.
SonarQube is an open source quality management platform, dedicated to continuously analyze and measure source code quality

Challenge yourself, get some experience as a bonus.

Having done more than a bit in the way of setting up a development environment and teaching yourself some java and selenium webdriver, it is time to get your hands dirty on more Java.
As it turns out, Selenium Webdriver itself is not so very difficult, it is the Java surrounding it that makes it hard to work on, to work with. Sometimes you need to check if something already exists and if not, create it. You will need to work with variables, with loops, arrays and all other kinds of seemingly basic stuff that you somehow still need to know.

In my last post, I explained how to gain the basic knowledge on Java and teach yourself all this. But now what? Having been doing other things / stuff for a decade means I am nowhere good enough to consider myself a valuable part of a team of developers after three months of self-study on java. I need practice, it needs to be somewhat of a learning curve and it needs to be somewhat anonymous so nobody can laugh at my pathetic attempts on coding.

Lucky for us, there are a ton of websites to aid us and I want to dive a bit deeper into a few of them. In the java post I already mentioned a few and I'll repeat them here:

Kattis. This is a collection of coding challenges, going from not so complex to really frigging hard. You gain points which is fun as you can pass others. I am currently hovering around rank 1000 and my code is getting sloppier every time I try something. This is most often because I have no idea how to approach a problem which can be caused by having a lack of knowledge on Java. You can use other languages as well, the interface is very useful and they have a ton of problems you can solve. Go here if you want to race others for imaginary internet points.

Hackerrank is similar to Kattis in some ways, such as points and ranks but with a few very nice additions and some things lacking. The idea is similar, but it is split up in domains that allow you to earn points each instead of one problem to solve in the language of your choice. Nice if you want to focus on other things than just math problems and Kattis has many of those.

Project Euler is very obviously not designed but it is well written and has another 545 problems you can work on.

Topcoder seems nice but has a hard time actually offering beginner challenges. This is nice if you have more experience and want to know how good you are compared to others.

These are websites that allow you to upload your solution, some snippet of code, after which you are a bit better at writing short bits of code for a very specific purpose.

Looking at the goal of all this is to land a job doing something we like to do and doing it for money in such a way that we still like what we are doing. A job you enjoy will leave you energized at the end of the day which in turn will make you happier which in turn will make your family happier which will make this world a better place. Just try to enjoy your job, if not for yourself, do it for the puppies.
To get that job that we expect to enjoy we need to prove to a prospective employers that:

• We know what we are talking about
• We can work as a developer in a team
• We have some real experience

While currently there is a huge global shortage of technical IT people, developers and testers alike, you still need to have some merit to land a job doing this. 
The best way to gain experience is write actual code for an actual project and there are several ways to go at that. Let's get the first one out of the way:

• Land a job as a very junior developer at a company willing to hire a mid career guy with no experience. Yeah, not likely unless you are happy to work for free which is fine if you have a ton of savings and no family to support.
• Do all the challenges mentioned above and open up your repo on Github. This will help a bit but you still have to prove you can work in a dev team writing useful software.
• Contribute to open source projects. There are guides here and here and a website called CodeTriage here. 

Because we are learning Java with test automation as the goal, it should not be to hard to start somewhere. In fact, I am building a testset for an open source project myself, you can find the repo here on Github. It needs work, feel free to contribute :-)

Selenium, finally some progress

After getting the hang of Java and being able to produce some workable code, I wanted to go ahead and automate some tests. After all, 6 weeks had passed in my new career adventure and not a single testcase had been automated. It was about time!
With some help of smart googling, I ran into this blog and training series helping me use Selenium, JUnit and IntelliJ, the all in one package I was looking for. Really useful as it will teach you how to use Maven within IntelliJ, what the JUnit framework can do for you and after this setup how to build some basic Selenium Scripts. He changed the layout since I used it and moving from one post to the next became weird, so let me build it out for you here.

Newbie friendly guilde to using Selenium in IntelliJ.
Part 1: Install Maven and do some basic setup.
Part 2: Install IntelliJ
Part 3: Get Testing

These are the basics and I don't want to steal this guys pageviews so go here for a full overview. Posts I can recommend are:
Firebug and Firepath, very useful with Selenium. These will allow you to find out what that element you are trying to locate actually is called and how to refer to it.
Using Assertions. It is nice you can click a button but is the result what you expect it to be?
His series on waits is useful knowledge as well. In fact, I would recommend you check out what classes and methods (you remember this from Java, right?) are being offered in the greater Selenium package. To do some straightforward reading on this you go here, it took me some searching as for some reason it is not that clear and open in the Seleniumhq.org documentation. Things are a bit scattered as technical documentation has a tendency to do but realize the following:

With Selenium Webdriver you can do whatever a user can do in a browser screen in terms of clicks, movements, scrolling, typing and whatever you can think of. Even better, this can be done in multiple browsers, across platform and with a wide variety of recordings such as screenshots, textfiles and reports.

The commands allow you to do all that, you just will have to figure out how to tell Selenium to do exactly what you want in the browser. This sounds like quite some effort to get the hang of and it is so I am using as many things I can find to make my life easier.

The one I like best is using the Selenium IDE to record and then export to Webdriver/Java/JUnit. Some people, maybe even most people, would consider this cheating but allow me to explain.

The Firefox addon SeleniumIDE which we installed a few posts back allows me to record in any webpage what I click, where I type and so on. There is a certain order to things and it can be found in the Options screen under the tab Location Builders, like this:





This lists what it will pick first, what second and so on. Basically, if it cannot find a locator of a certain type for the object you click, it will move one down and in the end the Xpath remains which will always work but is a real pain to maintain.

So what I do is the following.
Open website and open Selenium IDE addon.
Click record on the Selenium IDE window. Do not forget this, else you have to redo.
Record whatever you are doing.

Your window will now look like this:


There are some actions that I now want to extract from this to use in my Java/Selenium Webdriver/JUnit test scripting and here is how to do that:



Yes, we can actually export that to the exact setup we are using, so let's go and do just that, put it in a nice place where you can find it such as the desktop and do not treat it as anything as a super temporary file.

Now open Notepad++ and navigate to the super-temporary-file.java on your desktop. There will be a ton of code that you really don't care about in the grand scheme of your laziness so we ignore that and look up the selection bit, highlighted here:


Look at that, you copy/paste this to your java code in IntelliJ or Eclipse or whatever you are using and edit it how you want it to behave but it saves so much time in trying to figure out the most optimal way to pick an element. Also screw that the picture is to wide, we need all that booty to show a point!

Hoping I angered many purists and made a lot of lazy people happy, I will move to Git next time as Git is a good thing to have, know and use.