As part of the interview process for members of the Test & Assurance Team it can be useful to check understanding across a number of key area of testing practice.
Below we have an example of the type of quick test you can apply at the end of a face to face interview to assess general understanding. This is broken down into three sections – Testing Concepts, Test Artefacts and Analysis. In this way we get to see the strength areas of the candidate. What should be noted it some questions as about direct knowledge, e.g. do they know the sections of a Test Plan against the standard ISTQB definitions, other questions are looking for the application of general analytical thinking, not direct knowledge of coding for example.
1. Testing Concepts
- What is the main difference between Verification and Validation?
Verification is related to static testing, where as Validation covers dynamic testing.
- Describe the difference between the Black box and White Box categories of testing.
Black Box testing is conducted with no knowledge of the of the application’s code or internal structure. White Box is conducted with knowledge of the code or internal structure.
- Is it possible to use both Black Box and White Box Testing with Verification?
Yes, as documentation reviews are static inspection or ‘desk checks’ we might consider these Black Box and as Walkthroughs are done with the author these might be thought of as White Box.
- Name some types of testing that belong under the Functional testing group.
Types of Acceptance testing such as Alpha, Beta, UAT and OAT would fit under Functional Testing – though we’d expect some cross-over into Non-Functional also. It’s not common to assign Test Types as such as the focus is Features and Behaviours.
- Name some types of testing that belong under the Non-Functional testing group.
Any Test Types that are related to general characteristics and attributes of the application under test such as Accessibility, Localisation and Performance.
- Summarise how Performance, Load and Stress testing types relate to each other.
Load and Stress testing are testing sub-types of Performance testing. Load testing focuses on normal operational profiles of the application where as stress focuses on load outside of the expected operational profile.
- When and against what might we use Static Black Box testing?
When performing an Inspection of Documentation, though ISTQB states this is White Box testing that would more accurately apply to Peer Reviews and Walkthroughs.
- Are all forms of Validation testing dynamic? Explain why they are or aren’t.
Yes, because they involve running the code and when the code isn’t being run we’re doing Static Verification.
- How does Regression Testing fit across the overall testing domain?
Regression Testing is not a specific Test Type but an Approach that covers both manual and automated execution and draws together Test Cases for many Test Types.
- Name the main ‘levels’ of testing as recognised by ISTQB, list any others you recognise.
There are four levels which are; Component, Integration, System and Acceptance.
2. Test Artefacts
- Name some of the documents defined in IEEE 829 and state why they are used.
- Test Plan – For Planning the overall test approach, activities, resources, etc.
- Test Item Transmittal Report – To detail what is being sent for test and its status.
- Test Summary Report – Notify the end of testing and key details about the testing.
- Which documents are used as the basis for Functional Test Cases and why?
The Requirements and Functional Specification are the main documents. These are used because they describe what tasks the customer wants to perform with the application and what functionality the application needs in order to support them.
- State the key items of data that should always be provided in a Bug Report.
Clear summary, accurate description of the issue, environment details, steps to reproduce as a minimum.
- What information does a Test Plan typically provide?
SPACEDIRT – Scope, People, Approach, Criteria, Environment, Deliverables, Incidentals, Risks, Tasks.
- On the Project Plan what phases of testing and key milestones might be shown?
The main STLC phases: Analysis & Planning, Test Case Authoring & Review, Execution & Bug Reporting, Retest & Bug Closure, Reporting & Closure. Milestones could be based on the delivery of key artefacts such as the Test Plan, Test Cases, etc.
- Do Test Cases always need an expected outcome, explain why they do or don’t.
Yes, otherwise we just observe the outcome and guess if it looks correct or not. Defining an expected outcome ensures what we observe is the intended and planned outcome.
- Name three key measures/metrics you would expect to provide a Project Manager.
These can include:
- Total Time Spent on Preparation v Estimated Time
- Number of Test Cases executed against plan
- Days Elapsed from Bug Fix Reporting to Retest
Note: Just about any are valid, we’re looking for ‘context’ such as “…per day” not just “Number of bugs raised” which means very little.
- What artefact(s) would be expected when running an Exploratory Testing session?
Defined Objectives against which to run the testing or possibly a Charter to guide the testing sessions.
- In Scrum, what two lists and what chart are used to define work and track progress?
The ‘Requirements Backlog’ and ‘Sprint Backlog’ are the two lists and the chart to track progress is the ‘Burndown Chart’.
- What Level or Type of testing could a Test Objectives Matrix apply to?
System testing where the features and characteristics of the UI are being checked in volume and the tester wants to use a check-sheet and not a mass of Test Cases.
3. Analysis
- Consider the following fragment of a JavaScript program:
var num = window.prompt (‘Please enter a number between 1 and 10’, ‘ ‘);
If (num >0 && num <10)
document.write(‘The number is less than ten’);
If (num >10)
document.write(‘The number is greater than ten’);
Else document.write(‘Negative number’);Q) What test cases can you identify that would immediately fail against this code?
Answer: One where “num = 10” and another where “num = 0”
- A program validates a numeric field as follows:
Values less than 20 are rejected, values between 20 and 31 are accepted, values greater than or equal to 32 are rejected.
Q) Which of the following input values cover all of the equivalence partitions?
a. 20, 21, 31 b. 14, 20, 31 c. 14, 20, 32 d. 20, 31, 32
Answer: C
- Using the same specifications as the question above
Q) Which of the following covers the MOST boundary values?
a. 19, 20, 21, 32 b. 19, 20, 31, 32 c. 20, 21, 31, 32 d. 20, 21, 30, 3
Answer: B
- Review the following as it might appear in a Requirements document:
If the customer says I want to buy a ring.
Then ask Gold or white gold?
If customer says white gold
Then ask mix with diamonds or pure white gold?
If customer says pure white Gold
Then say £500.
Else £550
Else £400
Q) How many tests do you need to ensure 100% statement and decision coverage?
Answer: 3
- Consider the following code fragment:
READ A, READ B
IF A<0 THEN
PRINT “A is negative”
ENDIF
IF B <0 THEN
PRINT “B is negative”
ENDIF- How many tests are required for 100% statement and branch coverage?
Answer: Statement = 1, Branch = 2
- Consider the following code fragment:
Read A, Read B
If A<0 Then
Print “A is Negative”
Else
Print “A positive or zero”
End if
If B<0 then
Print “B Negative”
Else
Print “B positive or zero”
End if- How many tests are needed to ensure 100% statement and Branch/decision coverage?
Answer: Statement = 2, Branch = 2
- Review the following grades for passing an exam:
Fail: Less than 40%
Pass: Greater than or equal to 40% but less than 50%
Merit: Greater than or equal to 50% but less than 60%
Distinction: Greater than or equal to 60%
Q) Using boundary value analysis, what would be an invalid boundary value for a distinction?
A 59% B 60% C 61% D 100%
Answer: A
- Consider a program which takes “Salary” as input with values 12000…25000 in the valid range. The program calculates tax as follows:
– Salary up to 15000 – No Tax
– Salary between 15001 and 25000 – Tax is 18 % of Salary
– Salary above 25000 – Tax is 20% of Salary
Q) Which of the following values would be valid boundary values?
11999, 12000, 15000, 150001
Answer: 12000
- Name some approaches you could take when conducting test requirements analysis
Use of Conference, Reference and Inference to find test requirements given the information at hand. Deductive and Inductive approaches aided by models such as Fault Tree Analysis.
- Review the specification statement below:
“The application will parse Customer Account data as quickly as possible and use the Optimus-LHC approach.”
Q) What’s wrong with this statement?
The above is a combination of Functional Requirement and Specification, these should be separate as either the required functionality or implementation specifics may change and not affect the other.
