For this module, what we're going to do is a basic anatomy lesson about survey questions. We're going to talk about basic aspects of a survey question that are universal across all survey questions, and then discover the basic formats that every other survey question will follow. Before we do that, I want to just back up and do a shout out again to the book that I'm basing a lot of these lectures of. This is Internet, Phone, Mail, and Mixed-Mode Surveys. The Tailored Design Method by Dillman, Smyth and Christian. There's lots of great survey methods books out there. I highly recommend that you look into that literature. This is for me a canonical book that has been tried and tested over multiple iterations and the authors have done a wonderful job. So, we thank them for sharing with us. So, when you're thinking about writing a survey question, you really want to be aware of what are the basic types of questions? What are my options? So, let's take a step back and think about what's the basic anatomy of every survey question that's out there? If you think about it, every question has three parts. There's a question stem, when survey researchers talk about the question stem, what they're really talking about is what are the actual words in the query or the question that you have? A lot of surveys will then have a piece called additional instructions. Additional instructions could be a little prompt or how you want the respondent to answer. We'll see some examples in a second. Then finally, every survey question has to have basically an answer space or a choice. So, the question stem and the answer space are required for every survey question. What will makes sense to ask a question without providing an opportunity to respond? Additional instructions may or may not appear with a survey question. So, here as an example is all three working together. So, the survey question here is, how many years have you lived in Michigan? Then you can see the additional instructions right below that, please report only whole numbers. For example, if you have lived in Michigan for 20 months please round to two years. Now, this is a way to help the researchers to shape the data further down the road. As people might then put in data like if it were 20 months, they might put in 1.5 years or something like that. The researcher only wants whole numbers, and this is a way to help clarify the question for the respondents. In a way additional instructions like this are a little bit of a crutch. The question is well written enough in the response category and the question stem are really well designed. You should really minimize your need for additional instructions, but we'll get more into that. Here the answer is space comes with a verbal and symbolic instruction. What does that mean? So, the verbal instruction is just this little prompt. Years lived in Michigan. It just mirrors what's going on in the question stem and the symbolic prompt is that little text box that appears next to it. Whether that textbox is short or long of course, means different things to respondents. Keeping the textbox short indicates that you're not looking for a huge number here. So, here's another example of a question stem and answer space. So, the question here is, what is your current marital status? Then the responses are married, living with a partner, divorced, separated, widowed, never married. Because this is a demographic question and we're very used to answering demographic questions. You can see that there is no additional instructions attached to this question stem. It's just a very straightforward nominal response set. So, what if we drill down to just the basics of survey research questions, what are your options? What types of questions can you ask? I think the one that is most famous for people and that you are very excited to use would be open-ended questions. Open-ended questions are survey questions that allow respondents to answer without limiting the range of their responses. It basically provides them with a space asks the question, and the respondents choose what type of information to put into that. Now, this is often used when you can't predefine response categories or for some reason it doesn't make sense to predefined response categories. The last question that we looked at we asked about marital status. You could easily imagine that as a open-ended response said as opposed to a selection list. There are a few problems with open-ended questions to consider. While they're great for allowing freedom for respondents to answer questions in ways that we might not have predicted, they're also much harder on respondents cognitively. They can be easy to write but they require a lot more effort on the part of the respondent to fill out. That means that in terms of compliance with our request to complete our survey, open-ended questions are skipped much more frequently than closed-ended questions are, and sometimes when they're answered responses tend to be satisfying. They tend to be very brief and not to the depth that we would like to see a response. The other issue with open-ended response is that each open-ended response must be data cleaned, coded, and analyzed after data collection. What that means is that you can imagine you have a question like this, what would you say is the primary purpose of your website? People are putting in all sorts of things into this box. You end up with 500 responses. Some skips. Some crazy answers. Some great answers that you really want to capture. You are going to want to find out what are the thematic patterns? What these responses share in common? The other major category of survey question that we have is closed-ended responses. Closed-ended responses are questions like what you see here where we have a question how good or bad was the color scheme on the website? I know it's a silly question, I was just given an example and we go from extremely good to extremely bad. Here what we want to do is we want to limit the response options. We want to get respondents to pick one of these. In some cases multiple of these in order to have easier analysis and no data cleaning at the end of the process. These are cognitively much easier for respondents to answer. So they might go through a much larger set of these and they went through open-ended questions, and because we have to find out these terms and we've defined their response categories. Analysis of these questions is much quicker and more easy than for open-ended questions. There are two major types of closed-ended questions. So, one is nominal questions. Nominal questions sometimes called categorical questions ask respondents to compare a set of categories where there's no inherent order within those categories. Sometimes you want to limit choices, make them pick one out of a set of categories. Sometimes you want them to pick all that apply. But basically a nominal question like this has a set of options you'd provide it to the respondent and they pick amongst them. There's no difference however in how each of these responses are. This is different than ordinal questions. Ordinal closed-ended questions provide an ordered set of options and ask the respondents to place themselves on a continuum for that order. So, for instance, the question here how competitive is the market for your target customer ranging from extremely competitive to not at all competitive? You can see here that what we have is this scale. We have extremely competitive on one side of the spectrum and not at competitive at all on the other side of the spectrum. We're trying to get the respondent to place themselves somewhere along this spectrum. Now this is great for a lot of different reasons for analysis but there are a few issues. One issue is is that it's really hard for respondents to sometimes pick subtle differences between these scales. So, for instance, what's the difference between very competitive and moderately competitive? Those one-off differences between scales can be in constant. They might not be the same actual distance. So, you really want to be careful when you're analyzing ordinal questions like this. So, we often call these scales as a type of data collection. Another thing that you sometimes hear them called are Likert scales, or if you've read this only Likert scales. Now they're called Likert scales because they're named after a guy named Rensis Likert. I like to talk about him because he's another University of Michigan professor. For his graduate work in the 1930s, he proved that five-point survey items scales like this were compelling and could provide as good a data as much broader, more expansive scales that were common at the time. This really simplified data collection and reduce burden for respondents. That allows for us to have a golden age of survey research that was really driven by some of Likert's accomplishments and achievements in survey methodology. Likert himself went on to establish the Survey Research Center here at the University of Michigan and is just a legend in the field of survey research, and is memorialize today by all of us calling his five-point ordinal scales, Likert scales for the most part. So, this is just a really basic introduction to a basic anatomy of survey questions. Questions come in two main parts: a stem and an answer space. Sometimes we add a third piece of explanatory text to help people understand what we're trying to ask. There are two major categories of questions opened and closed-ended. Then closed-ended there typically divided into nominal and ordinal options. In the next lecture, we're going to talk about a more general guidelines for how we write good questions overall.
