How to evaluate QA tester's work?

Tricky Test manager interview question may sound like : Imagine being a test manager and trying to evaluate the performance of your QA testers. As we all know staff performance reviews is a one of the responsibilities of QA Managers. Software quality assurance engineers do a wide variety of tasks, such as bug hunting, bug reporting, test planning, and test tool development. To fully evaluate the work of the QA tester, Test Manager would evaluate the quality of work on each of the tasks. As I noticed during several interviews, some Test managers prefers to focus on quantitative measures like number of bug reports or number of bug reports returned as duplicates, but in my opinion any performance review that is based on number, can be easily gamed. The successful QA manager would focus on qualitative measures while providing performance evaluation for QA Testers like quality of test plans, test cases and bug reports. Another measuremnets could be the ability of SQA Engineers to research and acquire a new testing skills, attitude towards coworkers, ability to manage time effectively, capacity to respond to changing priorities.

What is a good ratio of QA tester to developer?

It depends - is a correct answer on this no right or wrong Test Manager interview question. In my humble opinion, there exist no testing industry standards that cover the ratio of quality assurance testers to developers. In order to provide meaningful answer on the open ended interview question QA manager needs to know specific context. I would expect that the candidate for Test Manager position would ask these questions back:

How good a developers? Do they prefer to throw badly written code over the wall and blame QA testers for missed bugs and features? In this case QA Manager would need to hire more testers.

How much company is going to pay for mistakes discovered by customers? Probably it would be wise to spend more on hiring proper QA team members and save on the future expense of fixing bugs in the production environment.

How is the company budget? Could the company afford to hire additional QA Tester instead of another developer?

What do you do when you get bored?

Test interview question should show the real value of interviewee. One of this not puzzle interview questions is asking the prospective candidature about dealing with utter boredom. Occasionally testers get involved in the never ending testing of the same module of the same application. How should SQA Engineer adjust working behavior to continue successfully work on software project?
One of the ways it to exploit variation, for example stop testing the same modules, using the same methods. Try to a new ways to test the application.
The tester could also put the computer to work. Try to learn some scripting languages like Perl, Python or just shell and apply that in your testing to reduce monotonous and repetitive tasks.
Work together with software developer, asking a user about work flows or pitfalls, or business analyst to understand what is going wrong and it will make your job more interesting.

QA interview - testing toaster

How would you test a toaster, vending machine, electric kettle or ATM? Some QA managers consider this most common interview question as perfect QA interview question. In the same time the test interview question is the easiest one. QA engineer needs to tell just two magic words "requirements" and "specifications" and then approach toaster tests as any other usual application under test with different testing types like functionality testing, usability, white box testing, black box testing, performance testing and so on.

How would you test a toaster?

Interview puzzles and riddles is the long going tradition in the software development companies like Google, Facebook and Microsoft. Personally I would not make hiring decision for software test engineer position based on the puzzle interview question like "How would you test a toaster, vending machine, electric kettle, pencil or other appliance?", but some hiring QA managers still think that they can separate the good SQA Engineer from the bad using interview riddles.

Let's say as part of the interview process for Test Engineer position after you successfully answered the interview question about creating test automation framework based on Selenium , explained the difference between white and black box testing, with genuine expression in your voice recited agile manifesto principles, even solved the interview puzzle about one hundred prisoners and when you feel that you almost hired as tester, the interviewer asked you how would you test a toaster.

How would you? Mine answer would be posted after the Labor Day weekend.

Should all testers be developers?

One more QA manager interview question could help to analyze the management abilities of candidate for QA manager position. This interview questions especially critical for the QA teams working in Agile environment.
In certain domains, there is some amount of testing that cannot reasonably be automated in the proper time. This testing requires that QA Engineer eyeballs carefully look at the screen and work through the application under test. It isn't necessary for these QA Engineer to be developers, in fact, it might be better if they aren't programmers, since developers view the world differently than most people. You may want non-programmers QA Engineer for the following mundane tests:

• UI testing
• Usability testing
• Internationalization testing

In the same time QA Manager definitely wants your testers to have the coding skills and be developers when doing the following kinds of tests:

• White box testing
• API testing
• Performance testing
• Database testing

How would you deal with smarty pants SQA Engineer

This is a perfect QA manager interview question. Software quality assurance job market took a dramatic hit early this year, some signs of stabilization are appearing. A few recruiters believe layoffs have slowed. In the same time companies are not ready to begin hiring full-time testers, but many companies started to hire the contract workers to handle quality assurance projects that had been suspended last year. Hiring tester as contractor is always a big hurdle for QA manager, the proper answer on the dealing with smarty pants SQA Engineer would demonstrate management abilities.

There is the carrot and stick approach on resolving the work performance of tester, who instead of doing requested work tries to dig into the code and fix developers bugs in the application code. The carrot approach requires convincing QA Engineer to do something meaningful about the software application quality like discovering the weak point in application logic, set up testing environment for hard to test features, decide when the testing should be completed and finally sign off the production for production. QA Manager could also use the stick approach by asking QA Engineer serious questions about functionality of the new features, testing coverage, number of high severity bug logged, number of usefully executed test cases and when the application under test would be ready for production.