Friday, October 28, 2011

Selenium WebDriver tips 2

1. "Xpath expression cannot be evaluated or does notresult in a WebElement" error

During our recent automation project we've been encountering really irritating error using WebDriver
"The xpath expression '//given xpath' cannot be evaluated or does notresult in a WebElement"

This error was appearing randomly, for example script was working for 10 times but failed on the 11th. Eventually after several experiments we were able to narrow the problem to following factors:

- this problem appears on the IE only(IE8 in our case)
- this problems appears when WebElement is identified using XPath selector
- this problems appears often shortly after webpage is loaded

Possible Solutions:

Get rid of the XPath selectors
This is great idea and Alister Scott presents sound arguments for this here. This approach however is not always applicable in our context. We know that sophisticated XPath selectors can be brittle but having to automate application based on Microsott SharePoint there is not much choice

Add pause before first FindElement (with XPath selector) method
This solution worked out however you need to remember about adding this pause every time new page is opened. Because in general we don't like pauses in our automation scripts for many reasons we were looking for something else.

Update current FindElement method
We found this solution indirectly thanks to Alister Scott and his SpecDriver example on github. Because this problem appears usually at the first attempt to find element using XPath it's easy to update FindElement method and create your own where you can catch the exception for given number of times. An example in C#:

for (int i = 1; i <= 10; i++)
{
try
{
return driver.FindElement(selector);
}
catch (OpenQA.Selenium.WebDriverException e)
{
Console.WriteLine("Error " + i + " raised: " + e.Message);
}

It really works and often above loop doesn't need more than two iteration to return correct WebElement.

2. WebDriver loses focus on IE8

Another issue which appears randomly. Fortunately we were able to identify one of the reason of this issue - Microsoft Office Communicator. In order to run tests using WebDriver on IE (Windows 7, IE8 in our case) Microsoft's IM needs to be switched off if not then WebDriver randomly switches from IE to IM window during script execution.

Alek

Thursday, October 27, 2011

Testing the application displayed in Chinese (中國)

One of the reason I am the software tester is the need to learn something new everyday. For every new project there may be different testing missions, different stakeholders, different technologies - different project context. My latest project was completely different from previous because user names were displayed in local alphabets - eg. Chinese. It was great that I've had the opportunity to learn new alphabet. After couple days I was able to spot some character patterns however I have also realized that I can not see problems in places where I were not looking for. It seemed that it was much easier and faster to test when user names were presented in English. When I complained about this to one of my colleague he suggested to translate those names to more readable format and thanks to this we've found FoxReplace.
"This tool lets you replace text in web pages. You can define a substitution list and apply it automatically or at your own discretion, or make individual substitutions."

Fortunately e-mail addresses of our Chinese users were stored in database using English alphabet, I created substitution list where for each name written with Chinese characters I could assign replacement name based on email login eg. '亞歷山大' -> 'Alexander'.



Thanks to this substitution list I was able to replace all names on the page and easily find (and remember) any user name I was looking for.



Alek

Tuesday, October 25, 2011

Selenium WebDriver tips

Since the beginning of my current automation project we've been encountering issues that seemed to be real showstoppers. These issues were not only time-consuming but could also stop the whole idea of automated checks. Fortunately thanks to the Selenium WebDriver community we were able to handle every problem challenge. Here are some tips that were useful in our context:

1. HTTP Basic Authentication with Selenium WebDriver
Our automated checks needed to be run against web server with HTTP Basic Authentication.

Firefox Authentication Window


IE Authentication Window


Basically the authentication prompt is not available from the Selenium WebDriver. One of the way to handle this limitation is to pass user and password in the url like like below:
http://user:password@example.com

Before we can pass username and password in the url we need to make sure that given web browser let us use this feature.

Firefox
You need to change the following flag:
browser.safebrowsing.malware.enabled

You can change the state of this flag in the default Firefox profile using about:config service page (double click on the flag to change it's state) From now on the Firefox should let you go through http authentication using name and password in the url. (Note that if you are using Selenium WebDriver 2.6 or higher this flag should be disabled by default)

Internet Explorer
Because of the security reason IE by default has this feature disabled. To overcome this limitation you need to update a windows registry as described in this article : http://support.microsoft.com/kb/834489
You need to go to regedit and look for the key

HKEY_LOCAL_MACHINE\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE

Then you need to add a DWORD 'iexplorer.exe' with '0' value in this key:


This functionality should be available immediately after restarting IE browser

In case your authentication server requires username with domain like "domain\user" you need to remember about adding double slash “\\” (or "%5C" symbol in case of IE) to the url you use in Navigate().GoToUrl() function

http://localdomain\\user:password@example.com
http://domain%5Cuser:password@example.com



2. Same Protected Mode across all zones on IE 8
To be able to run WebDriver on IE you need to equalize the Protected Mode settings for all zones in IE8. It's easy when you are in charge of your computer security policy, however it might be difficult when your machine works under global company policy. In that case you may be not able to change this settings even with local administrator rights.



One of the way we found useful is to update these settings through the windows register in following register keys:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones

and equalize values for 0x2500 position in all zones.


Value: 3 (Protected Mode off)
Value: 0 (Protected Mode on)


[UPDATE 1]
It seems there is another way to overcome this IE limitation, however we haven't tried it yet - http://www.qaautomation.net/?p=373

[UPDATE 2]
Above tips were tested on : WebDriver 2.6, Windows 7, Firefox 7, IE8

[UPDATE 3]
Thanks to Damian Galek:

Another way of bypassing Selenium limitation, regarding not being able to reach Windows Authentication prompt, that actually worked for us, could be using SendKeys class (one of the class in System.Windows.Forms.dll library – link). It can be used to type whatever we need into any Windows-based application. So in our solution something similar to thefollowing code was used to enter any desired users credentials and accept authentication prompt:

Driver.SwitchTo().Alert(); #Web driver function
System.Windows.Forms.SendKeys.SendWait("Down}");System.Windows.Forms.SendKeys.SendWait("Down}");System.Windows.Forms.SendKeys.SendWait(username);System.Windows.Forms.SendKeys.SendWait("TAB}");System.Windows.Forms.SendKeys.SendWait(password);System.Windows.Forms.SendKeys.SendWait("{Enter}");Driver.SwitchTo().DefaultContent(); #Web driver function

This may not be the cleanest solution out there but it works.


Alek

Friday, October 21, 2011

"What If..." a new ebook for software testers

Recently I had a pleasure to review an ebook written by Ajay Balamurugadas. This ebook titled "What If..." is the set of tips for Software Testers.
In 22 Chapters Ajay presents his real-life testing experience. This is great book for new testers however more experienced testers should find this ebook interesting too. From this ebook you can learn new techniques, tools and different ways to sharpen testing skills. In addition this ebook presents list of many links to useful websites, blogs and suggests testing books worth to read. I really liked this ebook. Below my first testimonial I had pleasure to write to for this ebook :)
Ajay writes "I wished that someone had warned me beforehand." I can't agree more I wish someone gave me the book of tips about software testing at the beginning of my career or at least taught me to use "what if" question more often. If you aim at skills improvement in software testing and you want to find out the ways you can do this, I strongly recommend this book.


Alek

Tuesday, October 18, 2011

My story with software testing (checking) automation (part 2)

Part 1 of this story.

How to automate ?
Being the student of context-driven school I know that there are no best practice for automation checking. We needed to consider our context before we started to try with any of the available tools or technologies.
Things we tried to take into account :

- Which layer of the application we are able to use for automation?
- Which automation tool will easily handle web application based heavily on AJAX
- What types of web browsers we need to cover?
- Does anyone from the team has some previous experience with automation tool?
- Is our automation going to be a part of CI?
- Are we allowed by company policy to use any open-source tools?
- What are the programming skills among the team.
- Which programming language will be the easiest to learn for new team members?
- Who will be using our automation scripts besides testers and developers?
- How often we have changes in core functionality and UI and what approach and what tools will best deal with such situation?
- Is there possibility to reuse automation scripts or framework for user acceptance/validation part?
- Can we have tools that will let the stakeholder be involved in creating scenarios for automation checking scripts?
- What kind of report we want to deliver at the end of the day?

Thanks to above questions we could easily start looking for things that would best fit our context.

Tools
We started from QTP but this tool was not able to deal with our web application (or we gave up to early). Then we gave a chance to Selenium 1.0 and Watir but we had a problems with automating some sophisticated HTML 5 elements. Eventually we tried Selenium WebDriver and it was it! We were able to manipulate almost all of the application's elements. We could also use C# programming language. In addition Selenium seemed to have a big and active community which is crucial in any open-source project. Selenium WebDriver works also on our required web browsers : Internet Explorer and Firefox. Having the tool selected we could start developing the automation framework.

Approach
We decided to develop the application using Page Object pattern which seemed promising. We liked the idea of separating checks(assertions) from the UI interacting elements. Secondly with Page Objects we could easily split the work among the team members . But most important was the fact that using Page Object pattern in case of UI change we needed to apply the fix just in one place because there was no duplicated code.

Going further
The above approach to automation was great but it was not enough. To crate checks we needed someone who has a basic programming skills. In addition with that framework we were not able to involve stakeholders in user acceptance/validation part. We started to looking for ways to overcome this limitation. We found the Cucumber. Then we went to Specflow which aims for .net projects and the GIVEN-WHEN-THEN story has begun...

Alek

Monday, October 10, 2011

My story with software testing (checking) automation (part 1)

It's started as usual. There is a web based software product with many manual test cases already written. There is a need in project to shorten the regression time for each iteration. There is a few days free where testers wait for the new build. There is a company orientation toward software automation (no matter what, let's automate !). So eventually there must appear someone powerful in project who doesn't know testing at all and suggests to automate manual testing. Keeping in mind that it is very easy to automate some activities on web page (Selenium IDE, QTP recording, etc.) it was a great recipe for automation testing disaster. However this time based on successes and failures of other teams we wanted to start differently and finsish successfully.

Why to automate at all?
We started with this question. What is the reason for automating anything? Can we really automate manual testing? According to great blog post from Micheal Bolton, we should switch labels and use automation "checking" word instead of "testing". But is "checking" word as catchy as "testing" for people who drive the project? It is not and thanks to this new label it was easier for us to explain that what we automate can never be replaced with human "testing". As testers our goal is to provide information about the product. What information we can deliver using automation?

1. We can verify if product being checked automatically is in same shape as it was at the time of automation scripts being written (but only to the extent that the check scripts verify this product).
2. We can reuse the automation scripts on every regression phase
3. We can free testers from repetitive tasks and given the time for real testing.
4. We can use automation to support our manual testing. To check things that are too complicated for human activity or simply too boring for human (read more on Alan Page blog post) It means that we can challenge our previous assumptions about the product and learn new things about the product being tested.

So we decided that we would try to cover with automation some of the regression part. We wanted to have more time for testing and for creating new testing ideas. We also wanted to use automation framework for creating tests that have never been executed before on our software like testing with more types of inputs, with more combinations, with repetitive activities.

What to automate?
Before we started with automation we had already manual regression test cases created. We decided to use them as background for regression automation. Should we really use manul testing for automation? According to one of the lesson from Lesson Learned In Software Testing it can be trap and we should avoid it. But as I mentioned before we wanted to use them as a background only. Based on the manual regression test cases we created a list of things that needed to be obligatory verified during regression. However this list of things to check wasn't our end goal. We needed also to think about the ways of verification some test cases, about the oracles. How we will know that something works or not or better how the machine will know that this works or not. And this discussion leaded us to the tools and question with "how" - how we will do this?.