What are the characteristics of a good questionnaire?

Research in the internship semester

Carla Bohndick

Written surveys are usually carried out with the help of questionnaires. A questionnaire can be understood as a meaningful collection of questions, either on paper or digitally, which the study participants answer independently and in writing. One advantage is that questionnaires can be carried out in your presence, by post or online. Depending on the purpose, the answer options in questionnaires can be standardized to different degrees. Typically, a stronger standardization is accompanied by a faster and easier evaluation. In contrast, weakly standardized surveys are usually indicated in new and poorly structured research fields in which the exploratory character of the investigation is in the foreground (see chapter Standardization).

A questionnaire usually consists of several parts: In an introduction, the purpose of the investigation is presented. This is often followed by general questions about the person, e.g. B. on gender and age (socio-demographic information) and finally the topics relevant to the research question are dealt with. At the end of the questionnaire, questions about follow-up, e.g. B. Evaluation questions are asked about the survey itself.

Different Response modalities can be used. A distinction can be made between an open answer and the selection of answer options:

In the open response the answer is free, i.e. no answer options are given. An example of an open answer would be the question: In which moments do you like to read? Here it is to be expected that the answers are very little limited by the specifications of the researchers. This is very favorable for exploratory purposes, but makes high demands on the evaluation. Another example would be the age, which can be ascertained with the following question without any answer specifications: How old are you? Here (apart from the legibility of the answers), no particular advantages or disadvantages are to be expected for the answer spectrum or for the evaluation compared to the specification of answer options.

At Questions with default answers the answer options are given, so it is a standardized question format. An example of a question with answer options would be:
Do you have an e-book reader? □ yes □ no

A distinction can be made between single-choice and multiple-choice questions. In the case of the single choice, the most suitable answer should be selected from among various possible answers (e.g. the question about gender). In the case of multiple selection, several alternative answers can be selected. An example is:

What are your reasons for reading?
□ interest in the content of the text
□ interest in the language of the text
□ habit
□ boredom
□ other reasons

Often questionnaires are structured in such a way that various statements are presented that are based on a Response scale, for example from 1 (not correct at all) until 5 (That is completely right) should be assessed. That is why such response scales are also known as rating scales. It is usually assumed that the distances between the selection options are the same and can therefore be regarded as metric (see article on descriptive statistics). The combination of question and answer option is called an item.

Steps in creating a questionnaire

In the following you can follow the creation of a questionnaire step by step. The focus is on the formulation of items for self-assessment. The list can also be useful for creating other items; further information can be found in the literature below.

Step 1: Determine the features to be measured
When creating a questionnaire, you must first think carefully about which characteristics you want to measure. Let's assume that you want to examine the relationship between reading self-concept and reading motivation. This means that you have to assess the characteristic of reading self-concept and the characteristic of reading motivation.

Step 2: research
There are questionnaires for many characteristics that have already been used and for which key figures are available for determining quality. So you should first research whether you can build on the work of others and thus save yourself some effort and better connect to existing research. If you can find suitable questionnaires for your characteristics, I strongly advise you to use them, especially if they have already been used and tested several times. Questionnaires also already exist with regard to the question at hand. For demonstration purposes, however, we also assume that our research was unsuccessful and that there is no preliminary work, so that we have to develop a new questionnaire.

Step 3: formulation of items
First, you need to consider (taking into account current research literature) which indicators point to the expression of these characteristics. Think about how a low or high value of the characteristic you want to measure can be recognized. Since you want to develop a questionnaire, you have to limit yourself to those indicators that can be recorded using (self) information. You should develop several items for each characteristic. For example, you could formulate the following statement for the self-reading concept: Reading is easy for me. One possibility for motivation to read would be: I enjoy reading. In addition, you have to choose answer scales. In the present case, the response scale already presented is 1 (not correct at all) until 5 (That is completely right) imaginable. In principle, other answer scales are also conceivable, e.g. B. to the frequency of 1 (never) until 5 (often).

When formulating the items, you should note the following: Basically, you should try to formulate the statements as clearly, simply and unambiguously as possible. You should always consider whether your questions are being understood the way you intended them to be. Further information on the formulation can be found in Kallus (2010):

  • Is the language level appropriate?
    • Check that your wording is understandable for your target group. You should avoid technical terms, e.g. B. I am intrinsically motivated.
  • Are the references clear and unambiguous?
    • A problematic example would be: I am among the best at reading at my school. It is not clear here to whom the comparison relates. Both older students and teachers could be included in the comparison. A clearer formulation would be: I read well compared to my classmates.
  • Does each item correspond to an aspect?
    • Items that inquire about several aspects cannot be answered unambiguously, e.g. B .: I like reading newspapers and novels. You should separate such items and formulate two items for them.
    • You should also avoid conditions within an item or between items, e.g. B. When it's winter and I have a good book on hand, I sometimes read all day.
  • Are the answer categories appropriate?
    • For example, the answer scale of 1 (never) until 5 (often) not to the item Most of the time I enjoy reading.

Step 4: Formatting the questionnaire and formulating the cover letter
You should put some effort into the formatting, since a clear graphic design makes it easier to understand. Check that all questions are legible and that it is clear which answers belong to which question. In addition, you should write a cover letter that explains the goals of your study and motivates the participants to fill out the questionnaire. Instructions for answering the questionnaire should also be given here. Put the cover letter at the beginning of the questionnaire. If you want to interview minors, you need a declaration of consent from their parents (see also the chapter on research and ethics). You should enclose an information letter with the declaration of consent.

Step 5: Testing the questionnaire
Before you use the questionnaire, you should give it to a few people in your target group to check that everything is understandable. You can use sophisticated procedures (you can find these, for example, under keywords such as cognitive pretest) or simply have 5-7 people fill out the questionnaire. The most important thing is that you get feedback from your target audience. Do this early enough so that you can incorporate necessary changes and try the modified questionnaire again.

After the first operational capability has been ensured with the help of the small sample, the preliminary questionnaire should be used on a somewhat larger sample under real conditions. The data collected is evaluated from the perspective of quantitative quality indicators in order to identify defective items. Before actually using it, we recommend another test run with the improved version.

evaluation

After you have used your questionnaire, the evaluation begins. Depending on the interest in knowledge and the type of question, different procedures are available here. You can evaluate open questions with the help of qualitative content analysis, for example, with closed questions you will probably be interested in the descriptive statistics, relationships or differences. In the case of self-developed questionnaires, in particular, the analysis of the quality characteristics of your own questionnaire is at the beginning of the evaluation.

software

To create a paper questionnaire, text editing programs such as B. Word or similar can be used. Services such as Limesurvey (free of charge) or Questback are available for online surveys.

literature

Aeppli, J. & Gasser, L. (2014). Empirical scientific work. A study book for the educational sciences (3rd ed.). Bad Heilbrunn: Klinkhardt. (Chapter 7)

Bortz, J. & Döring, N. (2006). Research methods and evaluation. For human and social scientists. Heidelberg: Springer.

Kallus, K. W. (2010). Creation of questionnaires. Vienna: Facultas. WUV.

This entry was posted in survey, implementation of bohndick. Permanent link of the entry.