Oooh Automation Testing. One of the fields that now has become popular as some of the most recent frameworks provide a more complete set of APIs for testing it’s core components and also some software products came on the market to try to provide the most easy way to handle this high value functionality.
Both have pros and cons (as usual) , and here is the time where we put in the ring these two kind of tools in order for you to compare and see which is the one that better suits your needs.
For this match Im going to compare the tools Selenium and Test Complete (which I think are the most popular) on different aspects to consider when choosing an automation solution for an application.
Easy of use
Selenium as well as Test Complete provide a feature called recording, which when activated it starts to track all the whole actions that the user is performing on the browser.
This feature is good for people that doesnt have a lot of technical skills for doing test automation, because they can record simple tests and add them to the test stack.
Selenium provides a recorder called Selenium IDE which is a Firefox plugin only, which also is not too powerful as it generates a bunch of output for complex testing scenarios, leaving as your only choice to code this types of tests with the WebDriver API.
Test Complete offers more enhanced recording capabilities and tooling to manipulate the actions captured by it.
With its “keyword testing” approach, it maps the components with some “alias” through a NameMappingFile which can be replaced as needed if the UI changes some time in the future. Also, while recording, you have the chance to check for specific data into the page through checkpoints, which are similar to the F12 Developer Tools element inspection experience for choosing the element to be checked inside the DOM.
For me, Test Complete wins this round as a simple user can have a good set of easy tests without ever having programming skills at all.
Selenium provides its WebDriver API to code tests in a bunch of different languages including C#, Java and Ruby. So by itself it provides a lot of customization capabilities and also the power that the these programming languages have to create powerful automation frameworks.
In the other hand, Test Complete just provides the ability to create scripts within its own product , with JScript or VBScript.
The IDE capabilities for scripting are not as good as you find on Visual Studio or IntelliJ , but you can access to the whole Test Complete libraries and use them with these languages. The thing is that the languages are too old and its difficult to create a coded framework within the tool as the language capabilities are not as powerful as the most modern ones.
So if is not more than clear, Selenium wins here.
Every stakeholder in your team wants to see the results of your automation efforts so you have some visibility on the team and they can consider your work as high valuable, and what a better solution that showing them a report of your test executions.
Selenium doesn’t have the ability to generate test reports by itself , but I know they are some third party tools that with some hooks on the code you can create a very decent report.
I came to generate my own reporter by writing an html document from C# with its I/O API. So it depends how complex you want to get.
With TestComplete you have a more than good reporting capabilities. It logs every action and callstack that you can get. Also it captures screenshots and attaches them to the report which in Selenium you have to built that from the scratch.
For me,Test Complete wins this reporting round as it’s a plug & play feature and you dont have to worry on investing time and money to build this required capabilities for your automation scripts.
Every automation engineer wants to reach to this step, integrating its automation efforts as a part of a building process provided by some CI products around there like TFS,Jenkins and TeamCity.
As Selenium tests are written against a test framework which is what the CI server uses to execute your tests, Selenium can be integrated as a part of a build process , or even in the cloud with Sauce Labs.
On the other hand, Test Complete also offers support for integrating your tests into this CI systems and on the cloud with its tool TestExecute.
So for this round, as both offer the same capabilities, I declare a tie between the two.
So this round is one of the most difficult ones as it is an important aspect that everyone considers for taking one technology over another.
Selenium as you may know is totally free. You are able to use the technology for create tests as your own without spending a cent for using it , which is totally great. The problem comes when building the automation scripts for your application.
Building an automation framework is not easy task, which may require you to probably invest almost the same amount of money in development time, than the purchase of Test Complete license.
Test Complete price is high. Among all the features that it provides, probably can save you a lot of development time as some of them are built in, but there is not something that a good selenium framework can have.
So for me this round goes for Selenium.
The automation field is as important part of the development lifecycle and the creation of an automation framework should be considered as equally important as any kind of application.
These two powerful tools, provide a more than an acceptable testing environment to bring more quality to the application that the whole team is working on. The choice as always, depends on your team capabilities , going from technical knowledge to budget.
But the most important thing here is to bring test automation to your current project so you can be able to gain some time on important issues like regression and cross browsing testing, because a lot of rework comes when you try to track this important aspects, and of course, no one likes reworking on something right?
Thanks for reading!