Software Testing: February 2013

Wednesday, February 27, 2013

6 Testing Mistakes to Avoid Like the Plague

It’s okay to make mistakes,” they’ll tell you. “As long as you learn from your mistakes.” Not the worst advice you could get, but a wiser person would tell you that it’s far better to learn from someone else’s mistakes. Wouldn’t you agree?

If so, you’re in luck, because organizations all over the world continue to make the same software testing mistakes over and over again – all so that you don’t have to. While there are far too many to list here, we wanted to share a few of the more common gaffes. Here they are, in no particular order:

Mistake #1: Testing too late
How do you know if you’ve waited too long to test your apps? One sign would be if users are publically complaining via social media about defects they found on their own. When it comes to quality, the sooner you can involve the test team, the better off you are going to be. As you’ll see throughout this post, many of the great testing blunders occur for this very reason. Teams that see testing as the “last line of defense” not only misunderstand the role of QA, but they put themselves and their business in dangerous territory. Brian Marick explains the advantages of testing early:
Tests designed before coding begins can improve quality. They inform the developer of the kinds of tests that will be run, including the special cases that will be checked. The developer can use that information while thinking about the design, during design inspections, and in his own developer testing. Early test design can do more than prevent coding bugs.

Mistake #2: Testing with amateurs
In an ideal world, every testing project would have two types of experts: a testing expert and a domain expert. The reasons for having the testing expert on hand should be obvious (to readers of this blog anyway). The great testers understand, regardless of the product, where flaws and vulnerabilities are likely to be found. They are, as James Bach has said, professional skeptics. Domain experts on the other hand might not know much about testing per se, but they will immediately pick up on inconsistencies and shortcomings in the product from a feature/functionality perspective. Unfortunately, many companies are making the mistake – the BIG mistake – of having neither a testing expert nor a domain expert on their team. Don’t be one of them.

Mistake #3: Testing without a scope
“Just look for bugs and tell me what you find.” Famous last words! While this type of “improv” testing can often yield quality results, it should never be standard operating procedure. Unfortunately it is inside many companies. A note to testers: If this is the direction you’re asked to go in, here is some good advice from Jon Bach:
There are *always* requirements, but they aren’t always written. They are both implicit and explicit. Some are hidden, some may be obvious. Some take time to emerge, others hit you right away. You find them and they find you. You’ll know you found one when you sense a user has an unmet need.
Techniques for testing without requirements:

Ask stupid questions
Read the FAQ
Look at competing products
Read out-of-date documents

Invite a developer to the whiteboard, then diagram your assumptions of the architecture or some states
Testing is an active search, not just for bugs, but for requirements. Approach your requirements-gathering the same way you approach testing. You have to go find the bugs – they often won’t reveal themselves to you on their own.

Mistake #4: Testing “one and done”
A lot of time and energy goes into a software launch. So it’s only natural to relax a bit when the application is released to users. It may seem as though all is said and done from a testing perspective, but it ain’t. As we’ve seen time and again, many of the more serious bugs and issues crop up well after a product launches, often times through no fault of the engineering team. Maybe the application isn’t synching with a recently updated third party app. Maybe a browser setting changed. The list is endless. Whatever the circumstance, companies would do well to avoid the mistake of failing to run regression tests.

Mistake #5: Testing in a controlled environment
These days, users are likely to consume your application under all sorts of various conditions; an endless combination of devices, operating systems, browsers, carriers and locations. Why then do so many companies only test their applications within the confines of a highly controlled test lab? Why are they not moving a portion of their testing into the real world? There was a time when companies could get away with 100% on-site testing – and many did. Unfortunately, many continue this practice (old habits die hard, after all) resulting in production bugs that could have easily been found by testers in-the-wild.

Mistake #6: Testing too fast/slow
The Agile testers can’t seem to catch their breath, while the waterfall testers look like they’re about to die of boredom. Our last testing mistake has to do with the pace of testing – specifically, why many organizations can’t seem to keep it consistent. Much of the problem (again) boils down to the misperception of QA. When testing is seen as the last step in a long process (usually with a looming deadline) it’s natural for testers to feel rushed. When the role of QA is properly defined and agreed upon, testing will never be rushed and it will never slow to a crawl. Instead, it will become an integral part of the organization, not merely a department being waited on.

Of course these mistakes are all matters of opinion, and so we’d love to hear yours. What are the biggest testing mistakes in your view? Please share your thoughts in the comments section below.

Wednesday, February 13, 2013

Selenium interview questions with answers

1.    What is Selenium?
Selenium is a suite of tools for browser automation. It is composed of "IDE", a recording and playback mechanism, "WebDriver" and "RC" which provide APIs for browser automation in a wide variety of languages, and "Grid", which allows many tests using the APIs to be run in parallel. It works with most browsers, including Firefox from 3.0 up to 7, Internet Explorer 8, Google Chrome, Safari and Opera 11.5

2.    Describe technical problems that you had with Selenium tool?
As with any other type of test automation tools like SilkTest, HP QTP, Watir, Canoo Webtest, Selenium allows to record, edit, and debug tests cases. However there are several problems that seriously affect maintainability of recorded test cases, occasionally Quality Assurance Engineers complain that it takes more time to maintain automated test cases than to perform manual testing; however this is an issue with all automated testing tools and most likely related to improper testing framework design. Another problem is complex ID for an HTML element. If IDs is auto-generated, the recorder test cases may fail during playback. The work around is to use XPath to find required HTML element. Selenium supports AJAX without problems, but QA Tester should be aware that Selenium does not know when AJAX action is completed, so ClickAndWait will not work. Instead QA tester could use pause, but the snowballing effect of several 'pause' commands would really slow down total testing time of test cases. The best solution would be to use waitForElement.

3.    What test can Selenium do?
Selenium could be used for the functional, regression, load testing of the web based applications. The automation tool could be implemented for post release validation with continuous integration tools like Jenkins, Hudson, QuickBuild or CruiseControl.

4.    What is the price of Selenium license per server?
Selenium is open source software, released under the Apache 2.0 license and can be downloaded and used without charge.

5.    How much does Selenium license cost per client machine?
Selenium is open source software, released under the Apache 2.0 license and can be downloaded and used without charge.

6.    Where to download Selenium?
Selenium can be downloaded and installed for free from seleniumhq.org

7.    What is the latest version of Selenium?
The latest versions are Selenium IDE 1.3.0, Selenium Server (formerly the Selenium RC Server) 2.8.0, Selenium Client Drivers Java 2.8.0, Selenium Client Drivers C# 2.8.0, Selenium Client Drivers Ruby 2.8.0, Selenium Client Drivers Python 2.8.1, Selenium Grid 1.0.8.

8.    What is Selenium IDE?
Selenium IDE is a Firefox add-on that records clicks, typing, and other actions to make a test cases, which QA Tester can play back in the Firefox browser or export to Selenium RC. Selenium IDE has the following features: record/play feature, debugging with step-by-step and breakpoints, page abstraction functionality, an extensibility capability allowing the use of add-ons or user extensions that expand the functionality of Selenium IDE

9.    What are the limitations of Selenium IDE?
Selenium IDE has many great features and is a fruitful and well-organized test automation tool for developing test cases, in the same time Selenium IDE is missing certain vital features of a testing tool: conditional statements, loops, logging functionality, exception handling, reporting functionality, database testing, re-execution of failed tests and screenshots taking capability. Selenium IDE doesn't for IE, Safari and Opera browsers.

10.    What does SIDE stand for?
Selenium IDE. It was a very tricky interview question.

11.    What is Selenium Remote Control (RC) tool?
Selenium Remote Control (RC) is the powerful solution for test cases that need more than simple browser actions and linear execution. Selenium-RC allows the developing of complex test scenarios like reading and writing files, querying a database, and emailing test reports. These tasks can be achieved by tweaking test cases in your preferred programming language.

12.    What are the advantages using Selenium as testing tool?
If QA Tester would compare Selenium with HP QTP or Micro Focus SilkTest, QA Engineer would easily notice tremendous cost savings for Selenium. In contrast to expensive SilkTest license or QTP license, Selenium automation tool is absolutely free. It means that with almost no investment in purchasing tools, QA Team could easily build the state of the art test automation infrastructure. Selenium allows developing and executing test cases in various programming languages including .NET, Java, Perl, RubyPython, PHP and even HTML. This is a great Selenium advantage, most likely your software developers already know how to develop and maintain C# or Java code, so they transfer coding techniques and best practices to QA team. Selenium allows simple and powerful DOM-level testing and in the same time could be used for testing in the traditional waterfall or modern Agile environments. Selenium would be definitely a great fit for the continuous integration tools Jenkins, Hudson, CruiseControl, because it could be installed on the server testing box, and controlled remotely from continuous integration build.

13.    What is Selenium Grid?
Selenium Grid extends Selenium RC to distribute your tests across multiple servers, saving you time by running tests in parallel.

14.    What is Selenium WebDriver?
Selenium WebDriver is a tool for writing automated tests of websites. It is an API name and aims to mimic the behaviour of a real user, and as such interacts with the HTML of the application. Selenium WebDriver is the successor of Selenium Remote Control which has been officially deprecated.

15.    How many browsers are supported by Selenium IDE?
Test Engineer can record and playback test with Selenium IDE in Firefox.

16.    Can Selenium test an application on iPhone's Mobile Safari browser?
Selenium should be able to handle Mobile Safari browser. There is experimental Selenium IPhone Driver for running tests on Mobile Safari on the iPhone, iPad and iPod Touch.

17.    Can Selenium test an application on Android browser?
Selenium should be able to handle Android browser. There is experimental Selenium Android Driver for running tests in Android browser.

18.    What are the disadvantages of using Selenium as testing tool?
Selenium weak points are tricky setup; dreary errors diagnosis; tests only web applications

19.    How many browsers are supported by Selenium Remote Control?
QA Engineer can use Firefox 7, IE 8, Safari 5 and Opera 11.5 browsers to run actuall tests in Selenium RC.

20.    How many programming languages can you use in Selenium RC?
Several programming languages are supported by Selenium Remote Control - C# Java Perl PHP Python Ruby

21.    How many testing framework can QA Tester use in Selenium RC?
Testing frameworks aren't required, but they can be helpful if QA Tester wants to automate test cases. Selenium RC supports Bromine, JUnit, NUnit, RSpec (Ruby), Test::Unit (Ruby), TestNG (Java), unittest (Python).

22.    How to developer Selenium Test Cases?
Using the Selenium IDE, QA Tester can record a test to comprehend the syntax of Selenium IDE commands, or to check the basic syntax for a specific type of user interface. Keep in mind that Selenium IDE recorder is not clever as QA Testers want it to be. Quality assurance team should never consider Selenium IDE as a "record, save, and run it" tool, all the time anticipate reworking a recorded test cases to make them maintainable in the future.

23.    What programming language is best for writing Selenium tests?
The web applications may be written in Java, Ruby, PHP, Python or any other web framework. There are certain advantages for using the same language for writing test cases as application under test. For example, if the team already have the experience with Java, QA Tester could always get the piece of advice while mastering Selenium test cases in Java. Sometimes it is better to choose simpler programming language that will ultimately deliver better success. In this case QA testers can adopt easier programming languages, for example Ruby, much faster comparing with Java, and can become become experts as soon as possible.

24.    Have you read any good books on Selenium?
There are several great books covering Selenium automation tool, you could check the review at Best Selenium Books: Top Recommended page

25.    Do you know any alternative test automation tools for Selenium?
Selenium appears to be the mainstream open source tool for browser side testing, but there are many alternatives. Canoo Webtest is a great Selenium alternative and it is probably the fastest automation tool. Another Selenium alternative is Watir, but in order to use Watir QA Tester has to learn Ruby. One more alternative to Selenium is Sahi, but is has confusing interface and small developers community.

26.    Compare HP QTP vs Selenium?
When QA team considers acquiring test automation to assist in testing, one of the most critical decisions is what technologies or tools to use to automate the testing. The most obvious approach will be to look to the software market and evaluate a few test automation tools. Read Selenium vs QTP comparison

27.    Compare Borland Silktest vs Selenium?
Check Selenium vs SilkTest comparison

28.    How to test Ajax application with Selenium
Ajax interview questions could be tough for newbie in the test automation, but will be easily cracked by Selenium Tester with a relevant experience. Read the detailed approach at Testing Ajax applications with Selenium in the right way

29.    How can I learn to automate testing using Selenium?
Don't be surprised if the interviewer asks you to describe the approach for learning Selenium. This interviewer wants to hear how you can innovative software test automation process the company. Most likely they are looking for software professional with a good Selenium experience, who can do Selenium training for team members and get the team started with test automation. I hope this Selenium tutorial will be helpful in the preparation for this Selenium interview question.

30. What are the main components of Selenium testing tools?
Selenium IDE, Selenium RC and Selenium Grid

31. What is Selenium IDE?
Selenium IDE is for building Selenium test cases. It operates as a Mozilla Firefox add on and provides an easy to use interface for developing and running individual test cases or entire test suites. Selenium-IDE has a recording feature, which will keep account of user actions as they are performed and store them as a reusable script to play back.

32. What is the use of context menu in Selenium IDE?
It allows the user to pick from a list of assertions and verifications for the selected location.

33. Can tests recorded using Selenium IDE be run in other browsers?
Yes. Although Selenium IDE is a Firefox add on, however, tests created in it can also be run in other browsers by using Selenium RC (Selenium Remote Control) and specifying the name of the test suite in command line.

34. What are the advantage and features of Selenium IDE?
a. Intelligent field selection will use IDs, names, or XPath as needed
b. It is a record & playback tool and the script format can be written in various languages including C#, Java, PERL, Python, PHP, HTML
c. Auto complete for all common Selenium commands
d. Debug and set breakpoints
e. Option to automatically assert the title of every page
f. Support for Selenium user-extensions.js file

35. What are the disadvantage of Selenium IDE tool?
a. Selenium IDE tool can only be used in Mozilla Firefox browser.
b. It is not playing multiple windows when we record it.

36. What is Selenium RC (Remote Control)?
Selenium RC allows the test automation expert to use a programming language for maximum flexibility and extensibility in developing test logic. For example, if the application under test returns a result set and the automated test program needs to run tests on each element in the result set, the iteration / loop support of programming language’s can be used to iterate through the result set, calling Selenium commands to run tests on each item. Selenium RC provides an API and library for each of its supported languages. This ability to use Selenium RC with a high level programming language to develop test cases also allows the automated testing to be integrated with the project’s automated build environment.

37. What is Selenium Grid?
Selenium Grid in the selenium testing suit allows the Selenium RC solution to scale for test suites that must be run in multiple environments. Selenium Grid can be used to run multiple instances of Selenium RC on various operating system and browser configurations.

38. How Selenium Grid works?
Selenium Grid sent the tests to the hub. Then tests are redirected to an available Selenium RC, which launch the browser and run the test. Thus, it allows for running tests in parallel with the entire test suite.

39. What you say about the flexibility of Selenium test suite?
Selenium testing suite is highly flexible. There are multiple ways to add functionality to Selenium framework to customize test automation. As compared to other test automation tools, it is Selenium’s strongest characteristic. Selenium Remote Control support for multiple programming and scripting languages allows the test automation engineer to build any logic they need into their automated testing and to use a preferred programming or scripting language of one’s choice.

Also, the Selenium testing suite is an open source project where code can be modified and enhancements can be submitted for contribution.

40. What test can Selenium do?
Selenium is basically used for the functional testing of web based applications. It can be used for testing in the continuous integration environment. It is also useful for agile testing

41. What is the cost of Selenium test suite?
Selenium test suite a set of open source software tool, it is free of cost.

42. What browsers are supported by Selenium Remote Control?
The test automation expert can use Firefox, IE 7/8, Safari and Opera browsers to run tests in Selenium Remote Control.

43. What programming languages can you use in Selenium RC?
C#, Java, Perl, PHP, Python, Ruby

44. What are the advantages and disadvantages of using Selenium as testing tool?
Advantages: Free, Simple and powerful DOM (document object model) level testing, can be used for continuous integration; great fit with Agile projects.

Disadvantages: Tricky setup; dreary errors diagnosis; can not test client server applications.

45. What is difference between QTP and Selenium?
Only web applications can be testing using Selenium testing suite. However, QTP can be used for testing client server applications. Selenium supports following web browsers: Internet Explorer,

Firefox, Safari, Opera or Konqueror on Windows, Mac OS X and Linux. However, QTP is limited to Internet Explorer on Windows.

QTP uses scripting language implemented on top of VB Script. However, Selenium test suite has the flexibility to use many languages like Java, .Net, Perl, PHP, Python, and Ruby.

46. What is difference between Borland Silk test and Selenium?
Selenium is completely free test automation tool, while Silk Test is not. Only web applications can be testing using Selenium testing suite. However, Silk Test can be used for testing client server applications. Selenium supports following web browsers: Internet Explorer, Firefox, Safari, Opera or Konqueror on Windows, Mac OS X and Linux. However, Silk Test is limited to Internet Explorer and Firefox.

Silk Test uses 4Test scripting language. However, Selenium test suite has the flexibility to use many languages like Java, .Net, Perl, PHP, Python, and Ruby.

47. What is the difference between an assert and a verify with Selenium commands?
Effectively an “assert” will fail the test and abort the current test case, whereas a “verify” will fail the test and continue to run the test case.

48. If a Selenium function requires a script argument, what would that argument look like in general terms?
StoreEval(script, variable) and storeExpression(expression, variableName)

49. If a Selenium function requires a pattern argument, what five prefixes might that argument have?
glob, regexp, exact, regexpi

50. What is the regular expression sequence that loosely translates to "anything or nothing?"
(.* i.e., dot star) This two-character sequence can be translated as “0 or more occurrences of any character” or more simply, “anything or nothing.

51. What is the globbing sequence that loosely translates to "anything or nothing?
(*) which translates to “match anything,” i.e., nothing, a single character, or many characters.

52. What does a character class for all alphabetic characters and digits look like in regular expressions?
[0-9] matches any digit
[A-Za-z0-9] matches any alphanumeric character
[A-Za-z] matches any alphabet character

53. What does a character class for all alphabetic characters and digits look like in globbing?
[0-9] matches any digit
[a-zA-Z0-9] matches any alphanumeric character
[a-zA-Z] matches any alphabet character

54. What must one set within SIDE in order to run a test from the beginning to a certain point within the test?
Set Toggle BreakPoint.

55. What does a right-pointing green triangle at the beginning of a command in SIDE indicate?
Play Entire Test Suite

56. Which wildcards does SIDE support?
*, [ ]

57. What are the four types of regular expression quantifiers which we've studied?
Ans : * quantifier: 0 or more of the preceding character (or group)
+ quantifier: 1 or more of the preceding character (or group)
? quantifier: 0 or 1 of the preceding character (or group)
{1,5} quantifier: 1 through 5 of the preceding character (or group)

58. What regular expression special character(s) means "any character?"
(.*)

59. What distinguishes between an absolute and relative URL in SIDE?
Absolute URL: Its is base url and this represent domain address.
Relative URL: (Absolute URL + Page Path).
Open command uses Base URL (Absolute URL) to navigate web page.

60. How would one access a Selenium variable named "count" from within a JavaScript snippet?
${count}

61. What Selenese command can be used to display the value of a variable in the log file, which can be very valuable for debugging?
echo()

62. If one wanted to display the value of a variable named answer in the log file, what would the first argument to the previous command look like?
echo()

63. Which Selenium command(s) simulates selecting a link?
click, clickandWait, ClickAt, ClickAtandWait, DoubleClick, DoubleClickandWait, doubleClickAt, doubleClickAtandWait

64. Which two commands can be used to check that an alert with a particular message popped up?
The following commands are available within Selenium for processing Alerts:
• getAlert()
• assertAlert()
• assertAlertNotPresent()
• assertAlertPresent()
• storeAlert()
• storeAlertPresent()
• verifyAlert()
• verifyAlertNotPresent()
• verifyAlertPresent()
• waitForAlert()
• waitForAlertNotPresent()
• waitForAlertPresent()
The AlertPresent() and AlertNotPresent() functions check for the existence or not of an alert – regardless of it’s content. The Alert() functions allow the caller to specify a pattern which should be matched. The getAlert() method also exists in Selenium RC, and returns the text from the previous Alert displayed.

Sunday, February 3, 2013

QTP interview questions and answers

Software Testing