Recruiting participants

Last updated on 2025-11-04 | Edit this page

Overview

Questions

How do I develop an appropriate recruitment strategy based on the chosen target user audience?
How do I find users that I don’t have a connection with already?
How do I manage scheduling and how to coordinate user experience research sessions?
How do I decline users that do not meet my requirements as a chosen user audience?

Objectives

Develop user recruitment messages and other messaging needed for user experience research recruitment
Develop any necessary screeners to ensure you recruit appropriate users
Schedule user experience research sessions
Establish a participant list for longer term testing

Recruitment strategy and refining your user audience

https://sprblm.github.io/devs-guide-to/user-testing/recruiting-testers/

An essential component of doing user experience research is finding, communicating with and spending time with users to understand and learn from their experiences and perspectives. In this episode, we cover the essential aspects of recruitment strategy, messaging, scheduling and optional long term planning.

Make sure that you have reviewed and completed the section ‘Choose and find your target user audience’ in the episode ‘Choosing a research method’.

Once you have your intentions, goals and needs documented, you can translate that information to recruitment messages to potential users.

First, decide how many users you’d like to start with speaking to. There are no wrong answers here, as stated in the ‘Choose and find your target user audience’ section of the ‘Choosing a research method’, this is about you deciding how many users you want to speak to in order for you to feel comfortable with the data you’ve gathered and to use that data to make informed design decisions about your open source science software. You can start with a low number and then increase that should you gather interesting and valid user participants.

It’s also useful to set time-lines for your ‘data gathering’ phase, this helps the potential user participants better align with a timeline in their schedule. When you make timelines flexible to your potential user participants, you can risk the data gathering phase being very long or the user participants having long gaps between the data gathering. This can sometimes mean factors can change your original conditions for your user experience research.

Deciding the duration of each test helps the team and your participants manage expectations. We like to do 20-minute, 30-minute, or 45-minute tests, depending on the size of the task. Remember to give yourself some buffer time and end the test promptly at the time limit to respect the participant’s time. Don’t feel the need to schedule test sessions back to back - give your team a break between sessions.

Testing can be done in-person and remotely. We recommend that everyone’s face is visible (unless anonymity is crucial) and you all can see what’s being tested. If you give a phone or a laptop to the user, sit next to them (if culturally appropriate) so that you can watch the process. If the testing is remote, you can share your screen (the participant verbally gives commands while you perform the operations) or ask them to share their screen (making sure to get their consent in advance).

It’s good to have an idea of what kinds of characteristics and behaviours the users need to have in order to be most useful and informative to you in the test scenario. The balance here is to make sure you are not setting too restrictive characteristics and behaviours so that users self-select out of the running to test. Here we want to reassure users about the eligibility criteria you are setting and that even if they deem themselves ‘less proficient’ that you may still want to learn from them. We’ll explore how to word these needs inclusively and openly when we come to craft recruitment messages and screener surveys.

Discussion

Challenge

Describe the parameters that you’ll start with for your user experience research. Answering the following questions with short, simple answers will help you develop these parameters.

How many days/weeks for gathering data
How many days/weeks for sourcing user participants
How long do you intend for the user experience research session to last? e.g. 20 mins, 60 mins etc.
Should the users be accessible in person, online or both?

Describe the details of what your users need in order to provide you with useful insight to help you make informed design decisions. The following prompts can help you define this but you may also want to define your own criteria.

What kind of scientific/research work should the user be doing?
What kind of software and/or system requirements and proficiency should they ideally have? (remember, if you ask some user to rate their proficiency they sometime self-select out of the process because they consider their skills to be ‘lacking’)
Alternatively, how many times (if any) should they have used a specific technology or software?
What kind of background knowledge should the user have if any? e.g. The user should have experience of using API endpoints, of R statistical programming etc.
What other details are important to your user experience research? e.g. They should work in a lab setting, they should be a current researcher in an institution, they should have published papers etc.

Finding users

When starting your outreach to potential user participants, you can first make a list of possible outreach channels where your community gathers, such as: a specific software or community email listserv, social media followers, academic citations of your software in published papers, bug reports/issues, related communities on Reddit, forums or Discord, your website visitor analytics or submissions. You might also attend relevant conferences or events related to research, programming languages or your open source scientific software and gather interest in person.

When it comes to finding users that you are’t already aware of, such as users outside of your research domain or professional and social circles, creating a lower stakes ‘recruitment’ message can be beneficial to have and circulate. These lower stakes recruitment messages typically omit detailed information about the specific user experience research parameters and are more about expanding your own connections and network.

‘I’m a researcher looking into [specific scientific research subject] at [insert institution, organisation or company] and/or I am building/maintaining [Insert scientific/research software here]. I’m looking to meet and connect with people in [insert region/country] that identity as [insert e.g. new users, researchers in a specific field, people who use OSS, people who code, people learning to code etc.’].’

Finding users that are ‘not online’ already can be really difficult. Some of the people that are systematically excluded from technology, including scientific open source software, are those that have unstable, restricted and/or complete lack of access to the internet or the ‘free and open’ internet. Being aware that across the globe, there will be users in governmental, social or technological ecosystems where access to them will be difficult and their participation in user experience research could put them at risk depending on nuanced and changing attitudes and policies.

Crafting Recruitment messaging

There are two versions of a recruitment message, one that is for large audiences to then contact you specifically and one to one messages that you send directly to a specific person that you believe fit the criteria for your user experience research.

Sample script for a large audience recruitment message Are you familiar with [topic/goal/research question to test]? We want to talk to you. I’m [tool team member name], part of the team at [our tool/software]. We are working on a new feature and we’d like your help to test it out. We’d like to speak with you for [Time interval e.g. 10 - 20 min] and have you try it. Thank you! If you are interested and able to help, send me an email: [email address].

More info Who can be a tester? People who use [tool/software] and/or are familiar with [particular technology]. What is [tool/software]? [One sentence about your tool/software] Do I need to prepare? Nope! Come as you are. Is it private? Of course. We will not be recording voice or video. We will be taking brief notes about what works and what doesn’t in the design so that we can make improvements. We will not share your identity with anyone.*

Alternative sample script for a large audience recruitment message Hello everybody! Have you ever [activity/feature you are targeting]? If you want to help improve [tool/software], read on! My colleagues and I at [tool/project] want to speak for [Time interval e.g. 10 - 20 min] with a few people who have [experience you want to target]. If you’re interested please fill out this short form: [add a link to a form]

Sample script for a one to one recruitment message Do you have [Time interval e.g. 10 - 20 min] and use [software]? We’d like your help! We are about to release a new version of [our tool/software] and want to hear from you. To help us understand your needs and how to improve [our tool/software], we’d like to hear about your experiences with the tool (whether you’ve used it for a long time or just recently started using it!).

We’re booking [Time interval e.g. 10 - 20 min] conversations to better understand how to improve [our tool/software] for everyone. If you’re able to participate, please fill out this simple form [add a link to a form].

Don’t be surprised if after sending these messages you get some messages for clarification. It’s best to not make your messages too long and allow potential users to ask clarifying questions as to their relevance for the user experience research. This back and forth of question and answer can go some ways towards building rapport (which we cover in the section []‘Building trust with users and leading into your user experience research questions’](#) in ‘Conducting Interviews’). It is a good idea to prepare or keep a ‘frequently asked questions’ (FAQ) section for you to either reference in your reply or direct any potential user participants towards. If you wanted to make that FAQ openly accessible on a repository or in an open document online then it can be easily referenced.

Challenge

Using one of the above templates, try to craft your own recruitment message adding in the details that you defined in your recruitment strategy.

For example: Zarah, after speaking to Ester, was inspired and wanted to recruit more people to do user experience with to measure alongside Zarah’s experiences. She sends this to a forum for a conference of plant scientists.

Show me the solution

Hello everybody! Have you ever struggled to get your plant computer vision tools to recognise your images? If you want to help improve the software for plant image vision classification, read on!

I want to speak for 30 minutes with a few people who have been using computer vision models on images and come across challenges with images and recognition accuracy. If you’re interested please fill out this 4 question form I created on google forms so i can contact you. If you have more questions you can head over to my github to see more details.

Thank you Zarah

Discussion

To simulate and test what questions people might have, ask someone else who has explored these topics and to take a look at your draft recruitment message and see what questions they might have for you so you can prepare for those questions.

Alternatively, what kinds of questions would you have for Zarah based on the example draft recruitment message.

Screener surveys and scheduling

A screener (a short request for information and/or submittable form) is an advised but optional part of the recruitment process. Screeners are typically used to be certain that the person you intend to involve in your user experience research meets the criteria of the type of people and experiences that you want to gather data about.

It’s advisable to not make the screener questions too long or have too many of them and be sure that your questions cannot be answered ambiguously. The idea of a screener is to find out whether you want to involve them in your user experience research. Another purpose of the screener is to help you learn a little more detail about the potential user participant in order to make questions and tasks smoother when conducting your user experience research.

Typically a screener that includes: 1. Who you are looking for in terms of what kinds of ‘behaviours’ you want from testers e.g. ‘We want people who use a specific operating system, a specific programming language’ 2. How much time they need to commit 3. How you can follow up and contact them 4. When you expect to conduct testing and any schedule information

A sample screener question set

Are you involved in scientific research that uses technology? Yes/No/Other - explain

How often do you use python and use theusing common line interface (CLI)? Every day / a few times a week / less than once every two weeks

Please enter your timezone/when you’d be free for a 30 min meeting [enter availability or send a scheduling link that connects to a calendar]

Please provide a method to contact you to follow up [enter email/phone number etc]

Thinking longer term and building a user participant list

This lesson is largely about how to do faster and more accessible to non-professionals user experience research. It’s not essential that you look at a longer term strategy for building user participant lists in order to make informed design decisions for your open scientific software.

If you have time, capacity and interest, thinking forward to how these small user experience interventions and explorations build into a structured process or approach can benefit you in the future. Especially if you have interest in making these user experience research processes accessible to designers that may want to offer open source volunteer contributions to the scientific open source software you build/maintain.

Ways of looking to the future can mean: 1. Keeping a private and secure log of the people that have contacted you in accordance with any institutional and regional specific data policies. 2. Sharing what you are able to about your user experience research process openly as per an ‘open source’ approach e.g. keeping track of user experience research conversations in issues/documented in a repository along side notes of the data gathered. 3. Starting manageable community engagement/gathering processes or platforms e.g. starting a forum discussion section and inviting open discussion that abides by a code of conduct. This way you don’t need to spend too much attention on ‘moderation’ of the community that grows. 4. Setting public/openly visible goals for your open source scienctific software so that others that step in to conduct user experience research can know where to focus and/or users can know what you want feedback around from an ad-hoc perspective and can offer that outside of a structured ‘conversation’. These goals don’t need to be long, time-gated or very detailed, but some guidance is better than nothing.

A note on compensation: In an ideal world, you’d compensate people for their time, expertise, and feedback. However, we understand that this is a lean user testing process. Can you offer them something other than money, such as swag (tshirts, stickers, badges), account upgrades for software as a service, or simply just credit and thanks as a contributor. If you can’t offer anything, try to keep the tests on the shorter side and of course, offer your thanks.

Key Points

Key points