What you need to know right now:
- Avoid starting your screener with personal details.
- Avoid yes/no questions.
- Include questions that help verify responses to earlier questions.
- Use open text field questions to verify further details
- Only ask the things you need to know, and nothing else
For the full details, continue on…
In general, it is helpful to think of screener questions and demographic criteria as a funnel. The first step of the funnel is to ask yourself who the screener should target. This means breaking down what profile you’re looking for by age, location, household income and other criteria.
It is recommended that you think of demographic criteria as the widest part of your screener “funnel.” Demographic criteria consists of a tester’s:
- Country location
- Job title
Once you’ve determined your screener target, you need to start building the screener. You want to start with questions that will start narrowing down the pool of participants so that by the end of the screener your are only left with people who truly fit your criteria.
Tip 1: Avoid starting a screener with Personally Identifiable Information questions (ex: What state do you live in? What year were you born).
Personally identifiable information questions can make participants uneasy about continuing and will lead to a large drop off in responses early on in the screener.
Tip 2: Avoid yes/no questions whenever possible.
Binary questions makes it easier for testers to guess what profile you’re looking for. Yes/No questions suggest what you are looking for and can lead the tester to the passing response. People typically like to please and respond what they think is the correct answer. This is not because potential testers straight out lie (though sometimes they do!), but because people often read questions and say to themselves “Yeah, that’s me, I do that.”
Think of this behavioral tendency as being similar to when someone fills out an online personality test. People tend to fill out such tests based on an ideal version of themselves. Given this behavioral tendency, we recommend avoiding yes/no or leading questions as qualifiers.
For example, say you are looking to speak with small business owners who have at least 1 employee. Your first reaction might to write screener questions like this:
- Do you currently own a small business?
- Yes (MAY SELECT)
- No (disqualify)
2. Do you have more than 25 employees (outside of owner and partners)?
- Yes (disqualify)
- No (MAY SELECT)
The person filling out the screener might look at these questions and say “I did sell some stuff on ebay, so sure that’s me!” Instead we recommend making your screener more opaque. A less obvious set of screener questions might look like:
- Which of the follow best describes your employment?
- I work for the Federal or State Government (disqualify)
- I work full time for a company (disqualify)
- I am a business owner (MAY SELECT)
- I am an independent contractor (disqualify)
- I am unemployed (disqualify)
- I am retired (disqualify)
- I am a student (disqualify)
- I work part time for a company (disqualify)
- Other (MAY SELECT)
2. How many employees does your business have?
- Just me (sole proprietor) (disqualify)
- Me and a business partner (disqualify)
- 2-10 employees (MAY SELECT)
- 11-25 employees (MAY SELECT)
- 25-50 employees (disqualify)
- 50 employees. (disqualify)
The first question narrows down the pool of qualifying participants so that the farther down the screener a tester gets, the more nuanced and detailed the screener should be.
Tip 3: Incorporate questions that help verify the tester’s responses and will qualify/disqualify them easily.
For example, say it is very important that the participants you speak to have an incorporated business and are not independent contractors. In this case, be sure to ask a question that only someone with an incorporated business can answer. In this case, we can use an Employee Identification Number as a qualifying criteria.
Instead of asking if the participants have an Employee Identification Number in a yes/no format, we can ask the following multiple choice question:
- Which of the following does your business have?
- Dedicated checking account (may select)
- Employee Identification Number (MUST SELECT)
- Small business loan (may select)
- Incorporation papers (may select)
- Business credit card (may select)
- None of the above (DISQUALIFY)
In this example, it is acceptable for the tester to have a dedicated checking account, and a business credit card, but the tester MUST have an Employee Identification Number.
Tip 4: Use open text field questions to verify further details
So far they look like the participant is perfect. To verify and take it a step further you can also add a text question that will give you more information on the participant.
In this case you might want to add a question saying:
What is the name of your business? Or: If your business has a website, what is it?
These questions will help when checking on participants profile and further confirm their qualifications.
Tip 5: Ask for what you want to know!
You will need to ask additional questions if you want a detailed understanding of a tester’s demographic criteria. For example, if you want to know exactly what country/state/city your tester is located in, you will need to include a screener question for this information. If you want to know a tester’s exact income (and not just a range), then you will need to ask a screener question.