Unmoderated studies provide a quick and easy way to get user feedback since testers can take it on their own schedule. If you're using Validately's panel, you'll usually get results within hours. 

While setting up an unmoderated study, you'll need to make a few choices:

Do you want to conduct usability testing or a self-service interview?

  • Usability testing is good for when you want the tester to interact with something on the screen, such as a website, prototype, or native mobile app.
  • Self-service "Interviews" are good for when you want to see the participant and/or their surroundings.

Do you want to use Validately's panel of testers or will you bring your own (BYO) users?

  • Validately's panel costs $15 per test with guaranteed results and usually provides results within hours.
  • With BYO you simply send the test URL directly to whoever you want.

Do you want testers to use their desktop or a mobile device?

  • For usability testing it's important to test desktop or mobile separately.
  • For self-service interviews, mobile devices are usually easier to use, especially if you need the tester to show you something.

Step-by-Step Setup Instructions

For this example, we'll setup the following study:

  • Unmoderated usability testing
  • BYO (your own users)
  • Desktop devices


Start by giving the study a name (you can change this later)

Choose Unmoderated / Usability Testing


Choose Test on my own users


Choose With browser bar, and then click on Next:

With or without browser bar?

The No browser bar options means the user will not be able to see or modify the URL of the page they are viewing. Common cases for choosing this: you're testing a prototype, security or privacy concerns.

In most other cases, especially if the user may want to use the browser back button, you'll want to use With browser bar.



On the next page, we'll:

  • Enter the Starting URL for the test
  • Create the Tasks we want the user to perform
  • Add the Questions we want to ask

The Starting URL will simply set the URL the session will begin at.

Tasks

Overview
When conducting an unmoderated usability study, you'll want to provide the test participant with a sequence of tasks you want them to perform.
For example, something like this:

  1. Search for printer ink on Amazon
  2. Compare cartridge capacity between the two best options
  3. Find out if Amazon provides next day shipping for the option that looks the best
  4. See if you can find a better price at a different online site

As the test participant goes through the tasks, the start time and end time of each task are automatically added to the session recording as a time-stamped note:


The start and end times also appear in the video timeline.

When reviewing the session recording, you can give each task a Pass or Fail. These will be used in the Tasks and Questions tab of the study results:

---------------------------------------------------------------------------


Setup

Each task has the following fields/options:

  • Task instructions
  • Start task on a new URL
  • Set Success URL
  • Ask questions after this task

Task instructions

This is what the user sees in their browser during the test.
Here's an example:

Will appear like this for the tester:


When the user clicks on Start Task, the task text will disappear, and the green icon will switch to Show Task:


When the user clicks on Show Task, it will show the task description and a task Complete button.

When the user clicks on Complete, they're presented with the Questions you setup in the Ask question after this task section.

Post Task Questions

After a user completes a task, you can ask them three types of questions:

Open Text Questions
Open ended multi-line text area they can fill out.
For example:

Multiple Choice Questions
List of options they can choose from.
For example:

Please note: At the moment users cannot choose multiple answers.


Sub questions
When using multiple choice questions, you can add question logic by asking sub questions:
 


here is an example:


Rating Questions
This allows users to give a numeric score. You can choose the low score (zero or one) and the high score (3 to 10) and also provide a label for the low and high scores. For example:

Task Success URLs
In some cases, it's possible to define if a tester Passed or Failed a task based purely on them reaching a Success URL.

For example, if the task is to add an item to their cart, in order to pass the task, they must reach a URL that looks something like this:
https://www.amazon.com/gp/cart/view.html?ref_=nav_cart

Please note: If you create a Success URL, the task will automatically be set to Pass or Fail based on if they viewed the URL, but you can manually change it if needed in the session recording notes.

Success URLs also support wildcards.


Follow up questions

You can optionally ask questions after the user is done with all of the tasks.


To continue, please see:

Getting Started with Unmoderated Studies - Part 2

Did this answer your question?