So, now let's talk about remote unmoderated testing. So, remote unmoderated testing is great when you've got a short period of time to work with, you've got people who might be dispersed all over the world, and if you're trying to capture a great deal of information. So, this is where you're capturing quantitative data. One of the great benefits of remote unmoderated testing is that you can run tests with hundreds of people. Whereas in the past when you were limited to just a lab setting, you could only interview or test with as many people as you can actually get to the lab and actually test with. Automating that process enables you to do many more tests. Remote unmoderated testing is not for every situation. There are few points that you need to consider to make the choice between doing a remote moderated tests and remote unmoderated test, or continuing to test in the lab setting. So, one is, if you have personally identifiable information or PII, you pretty much don't want to capture that. So, if you're going to require someone to enter anything that is personally identifiable such as their name, their address, you want to take care that information is either not captured or you select a different method. Additionally, sometimes it's very difficult to get around some of the logistical issues around testing remotely. So, for example, if you have special software that people need to install, or you've got special hardware that people need to be able to complete the test, or whatever it is you're testing is only accessible within a firewall. So, you need to be thinking about what can we actually get people to access from outside of our lab setting quickly and easily. Finally, if you need to explore or have dialogue with someone, sometimes it's not so easy to accomplish that using a remote and unmoderated test script. That is because once that script is set and someone is working through it, you can't make changes to it. So, if you need to be able to flex, to be able to explore, you probably want to pick another method where you can actually have a live and interactive conversation. Some of the pros that are a little bit unique to remote and unmoderated testing are that, you can essentially have tests run simultaneously. So, instead of necessarily having to wait for a moderator and a lab to become available, you can actually run tests with hundreds of people at the same time. So, that means that you can get great statistically significant data in a day. It's also relatively cheap because you're not requiring as much time of people typically. The amount of money that people will typically get paid to complete a task is a little bit lower. So finally, one of the additional benefits of remote unmoderated testing is actually a lower chance of bias. That's because sometimes as moderators we are tempted to help people to successfully complete a task that they wouldn't ordinarily complete. Moderated and unmoderated testing can compliment one another. You actually want to think about the types of information that you might want to collect from a qualitative perspective and from a quantitative perspective. So, there may be some cases where you actually use both methods in order to capture information about an experience or a product. One of the major things you'll need to think about and we'll cover this in more detail in another segment is, what platform or what technologies are you going to use to host your studies? So, you need to think about, what's captured? Which at a minimum screen and audio? And what metrics you want to capture? You want to think about, who do you want to be testing with? Is it important for you to have a very narrow set of participants who have a certain set of attributes that correspond to the very narrow audience that you have? Or is it unapproachable experience where pretty much anybody could walk up and use it? Finally, you have to think about what you can test. So, this might mean that in the case as I noted earlier where you have technologies that are only accessible through a firewall or require some installation, you need to be thinking about will the platform enable you to support that sort of thing? Scripts writing is really especially crucial in the case where you are doing a remote and unmoderated tests. And that's because you don't have a facilitator there to get things back on track if someone gets lost. So, you have to be thinking about, how you probe, how you remind people to speak aloud or not, and just with that balances between telling them what to do, when to stop, and how many instructions you need to provide to be able to get meaningful data back. When you're writing a script for a remote and unmoderated tests, you need to be mindful of the fact that after about 15 to 20 minutes, people get tired and they don't tend to do as good a job in terms of completing tasks or answering questions. So, this might mean that you front load some of the really important tasks. Think about how it is that you want to break up a study. So, it might be multiple studies instead of a single hour long study that you would potentially complete in a lab setting. You have to think about how you would handle pre and post test questionnaires to make sure that you are setting some expectations, gathering information, and also making sure that people have the ability to provide you with feedback outside of the structure of the script that you've provided to them. Then finally, one of the major things that you have to think about is always pairing a task with some sort of question or pro if you're doing qualitative remote unmoderated testing, so, that you can get feedback in addition to information about what happened and how people actually completed a task. Who you recruits is less important as I mentioned earlier if you are only exploring the usability of products that anybody could walk up and use. However, obviously, if you're trying to test something that requires additional contexts, or content, or knowledge on the part of your user, then you need to make sure that you're recruiting people who have those attributes. So, essentially that means that you may need to create your own panels so that you've got a group of people that you can work with who you know have those attributes that you work with a panel company in order to help you to find those people, or you make sure that the platform or company that you're working with has a panel that has the people that you need to work with. Remote unmoderated testing can take multiple forms. You'll notice in this list that there are both qualitative as well as quantitative methods. So, you can think about ways of scaling up the amount of research that you're doing with people so that you're understanding sometimes at a more statistically significant level whether or not people are successful, or whether or not they prefer a certain approach. Or you can also dig it and get more qualitative information to get a sense for, what can we do about this particular sticking point that people are encountering in this experience? So, the bottom line is, think about getting creative also with remote and unmoderated testing to get both quantitative as well as qualitative feedback.
