Test automation sounds "sexy" and attractive to any group/team leader, or to any QA engineer who wants to make an improvement in his testing capabilities. But taking the wrong approach could lead to making some fatal mistakes, which eventually might cause unnecessary waste of resources to the organization, and frustration to all involved. To avoid this, simple rules must be followed, and relevant (truthful) information should be presented to the decision makers. The purpose is to successfully adopt automation tests, and make them an integral part of the development process.
It's Overwhelming !
Implementing software automation testing, might seem a little bombastic to any newbie. The wide range of concerns starting with 'where to begin?', right to the parts of 'what to automate?', 'whether' and 'how to automate setup and configuration procedures?', 'how to check the results on each milestone?', 'logging the tests', and notification issues, such as 'the form of the reports', are all a great deal to handle when you start from scratch.
Not to worry, Rome wasn't built in a day, and neither did any of the greatest existing automation solutions. Don't let the huge amount of issues and tasks, deviate you from your main objective, which is to give a reliable picture of the current status of the tested application. You have to start somewhere, and better sooner than later, since the development is waiting for no one, and new features are piling up all the time.
The Big Question
An important thing before we start diving into the solution, is that it all comes down to one significant question: "What do you want?" Or in other words what is our ideal state that we want to be in (...testing wise)? Surely I'm not going to leave this one open. My answer is: "everything!", I want it all. I want the setup, the configuration, tests execution, checkups, and reports, all fully automated.
As soon as a new build is ready, or even after every commit/check-in (new Dev. code merged into the application), full scale testing procedure should be executed, including sending results notifications. All of these, in my ideal world, would be done without any human touch. This is what we should see (or aspire for) at the end of the road.
Know Your Resources
Sorry for asking this, but what makes you think that manual testing skills, would be adequate for automation system implementation?
Moving to automation would definitely require from you, as a leader/manager, to re-evaluate seriously the capacity and abilities of the resources at your disposal. Automation is a different concept than manual testing execution. It involves different tools and frameworks, and it surely requires different skills than just being capable of doing manual tests (no disrespect for all you, manual testers out there).
If you're going to use your current staff, there's a learning curve you should be aware of. If it's outsource people, or new skilled automation developers you are going to hire, then there's another learning curve to take into advise- the curve of knowing the application under test. As capable and talented your new workers are, they couldn't write automated tests to a system they know nothing about.
In many cases though, some of the manual testers show interest in making their life easier. They may start with scripts to automate their preparation and configuration tasks, but they are the ones that usually open the door for other automation tasks to be accomplished. Encouraging these guys, and giving them all they need to fulfill their curiosity and potential (time, guidance, compensation etc.), would be a big step toward achieving your automation goals.
These were some insights regarding the human resource aspect of starting to automate. But what about the time resource? I have already mentioned the learning curves, but even if you have a very talented new team of experienced automation programmers, that know all there is to know about automation, and all there's to know about the tested application, even then, if automation is done correctly, testing infrastructure must be built. It should be assembled after careful & thorough design, and after proper brain storming. This takes time. In fact, sometimes weeks, or even months will pass from the time one starts writing automation, until he actually sees his first test-flow execution.
Choose Your Tool Wisely
This is critical !!
If you don't want to write automated tests over and over, and change frameworks every month, this must be done properly. Characterize the application which is under test, and analyze the tests procedures. Examine all interfacing angles of the application being tested, and list the tools you currently use to check them. This could include the UI of the end user, admin console interface, the application's APIs, and more.
The chosen automated tests framework must be able to support all (or most) of the listed interfaces, including the future outlook and vision of all interfaces to be used. If all tests are done using WEB browsers, than 'Selenium Web-Driver' might be the best tool to use. If the system under test has some windows apps that are to be tested, then almost certain that 'QTP' is your tool. There are many more options, but these ones I personally worked with.
Sometimes it's good to know (and acknowledge) that you don't know. It would be wise to ask the counseling of experts, especially in this matter. The required knowledge won't be in the organization, since you have just started automating. The automation development framework/tool that you choose, will be used for a long time, and would be the base ground for all test flows developed in the future, so it must be chosen after a serious consideration & research.
One principle to follow, is to set one 'marshaling' language/program/script, from start to end. Meaning it would be best to have a single point of management interface, where all actions and all references of a test flow, will start from, and return to. Same script/language code, which would start the setup procedure, will execute the test, and would also send the report at the end. One, single 'Main'-like code, which would make a coherent, organized marshaling point, through which all test related actions could be controlled from.
Another issue to consider when choosing development framework, is the R&D background and knowledge. This has extra importance for teams new to automation and code development in general. If R&D is using Java, than Selenium Web-Driver over Java would be preferable. If it's .NET, than you might consider using C# platform. The support system of the application developers is important, not only when you're stuck and need advice (for this, online professional forums would do the work), but also for code & design reviews.
Just to set things straight-changing of code is done all the time. Refactoring and re-designing are done continuously. So no one says that once you set the automation framework design, then that's it. But it would be extremely beneficial to all, if before writing the first line of automation code, one would think it through, consult, and do some research, in order to minimize design mistakes, which might cost dearly.
Prioritize & Focus
So, after exhausting so many clichés, you should divide your overall execution into segments. Basically, full test flow, end to end, has the following parts (roughly):
• Setup of the testing environment (including DB preparation)
• Application configuration
• Execution
• Checking actual outcome (response) in comparison to the expected results
• Reports
First thing you need to know is that you must adhere to the current way you test today. As mentioned before, with so much to do in order to achieve a reliable automation system, a focus and prioritization is desperately needed. Furthermore, many times quick results are to be presented (to a stressful management), to justify the efforts and the resources allocation. Generally speaking, all automated tests include two combined elements: simulating a scenario (execution), and checking the results (the response of the application under test). So, from the five segments mentioned above, there is no doubt that the core two, are the scenario 'execution', and 'checking the actual outcome'. If you want to start somewhere, start there.
Some Principles To Live By
Whether it's automation tests for any kind of GUI application, or for other SW component, always strive to generate robust and scalable automation code for maximum control in the future, and in order to keep your independence and flexibility.
For starters, keep the following rules:
- Design the automation framework in a way that utilities are written once for common use
- Design the framework so that object repositories are shared and maintained in one place (per version)
- Plan before writing each test
- Have design & code reviews to be part of the automation development process- for constant improvement
- Keep the automation knowledge in-house (not in the hands of out-source people), and share it
- Use parameters to support multiple configuration & setups execution of single code
- Make your tests resilient to app changes, by robust object identification (RegEx and other techniques), and object synchronization.
It's only fair to mention the following sources (there are many more), that each expressed it's own take on the same issue:
Good Practices For Automating Functional Tests
10 Tips you should read before automating your testing work
6 Tips to Get Started with Automated Testing
Getting started with automation testing
