Agile Testing

A co-worker was recently asked “Who will be doing the system testing” on his project and this brought up the topic of Agile testing.  He sent an email asking “When doing a project using Agile and Scrum, how do you develop the system tests?  Do you script scenarios based on the stories?  The tasks?”

 

So in my reply, I was able to get a bit of a two for one deal out of it.  I was able to reply to his question and create a blog post as well.  Here is a paraphrased version of my reply:

----------------

“When doing a project using Agile and Scrum, how do you develop the system tests?  Do you script scenarios based on the stories?  The tasks? “

From the stand point of Agile engineering best practices, quality needs to be built into the system.  TDD is popular among the XP crowd.  Automated testing and continuous integration is key in Agile.  With the rapid pace of iterations, it is difficult to go back and manually test previously covered areas, so the automation of testing provides repeatable and repeated verification of correctness.

Using utilities to check for test coverage of code is helpful.  Sound testing at the engineering level is the first and best line of defense.  If something broken, stop the work until it is fixed.  Remember, a user story is not complete until it has been tested.  So yes, craft tests around the user stories, but also have the automated tests hit as many areas as possible.  Using tools such as nUnit for unit testing, Selenium for UI or interface tests, Fit/FitNess for acceptance tests, and LoadRunner for performance testing are all ideas in the automated test toolbox.  There will certainly be a bit of an engineering cost to implementing these tools, but I know there is research out there that does show that these engineering costs are paid back in savings in the quality of the code/project.

 

Specifically on this project, my thought is :

• Ensure the team has a responsible level of code coverage through unit testing

1. Consider using TDD (though I have not personally had great luck with it, but I’m no expert in it)

• For each story, design a responsible set of test scripts for the story.  Hopefully these scripts are tests that can be automated.

1. Work with the tester and the team to identify what responsible set of test scripts mean
2. Work with the tester and the team to identify how to (and who will) manually test the non-automated tests

“Who will be doing the system testing?” 

The question “Who will be doing the system testing?” is a bit confusing.  By system test, do we mean a traditional test of the overall system?  Does this question ask who will be creating the test scripts and/or who will be performing any of the manual testing?  In terms of who should be writing the test scripts, I think that answer varies, but it ultimately should most likely be a team effort.  It varies due to the specifics of the project, domain, and available resources.

In this case where there may only be one person devoted to testing, if this is a highly skilled person, they can own the testing strategy and create testing stories and strategies which the team (along with this person) can execute.  If the person is not able to fill that role, the team will need  to determine the test strategy and work with the testing resources to implement and execute the testing strategy.

The concept of having a person(s) coming in fresh at the end of a project (or even project cycle) to test seems ineffective.  In the past I have seen this as just an exercise in helping the tester understand the system being developed and how to use the system.  The tester may stumble upon on few defects  or issues here and there, but from cost benefits standpoint, the returns seems quite low.  Having resources that are integrated with the team and developing tests iteratively with the team seems ultimately more efficient.