Controlled Randomization of Specific Questions (advanced).

Objective

• Randomize the order of two specific questions (Q1 and Q2) within a group.
• Maintain the fixed order of other questions in the questionnaire.
• Ensure a random display of either Order A (Q1 then Q2) or Order B (Q2 then Q1) for each respondent.

Why use this method?

• For precise control: unlike the "Randomization group name" option which shuffles an entire set of questions, this method ensures that a specific subset follows a predefined and random order.
• For methodological validity: it eliminates position bias (primacy/recency) on key variables while respecting the questionnaire logic.

1. Preparation: Creating the random generator

This step involves creating a variable that will randomly draw the order (either 1 or 2) only once at the beginning of the survey.

1. Create a new question group at the beginning of your survey (before the group containing Q1 and Q2). Name it for example "Variables".
2. Add a new "Equation" type question in this group.
3. Assign it a simple code, for example: Q_Random.
4. In the question settings, go to the "Logic" tab.
5. In the "Equation" field, enter the following code to generate a random number between 1 and 2:

{Q_Random = if(is_empty(Q_Random), rand(1, 2), Q_Random)}

This formula checks if Q_Random is empty. If yes, it generates a random number between 1 and 2. Otherwise, it keeps the existing value.

In the Q_Random question settings, go to the "Display" tab and set "Always hide this question" to "Yes".

2. Implementation: Duplication and condition control

To create the two possible orders (Q1-Q2 and Q2-Q1), you need to duplicate the questions and use the "Condition" field (relevance equation).

Duplicate the questions

1. Duplicate Q1 and Q2. You will then have four questions:
   • Q1 (for order 1) and Q1_R (for order 2, R for Reverse).
   • Q2 (for order 1) and Q2_R (for order 2).
2. Place the questions in the group in the following order:
   • Order 1: Q1, Q2
   • Order 2: Q2_R, Q1_R

Note: The fixed order of non-randomized questions should be placed before or after this block of 4 questions.

Define display conditions

Access the settings of each question under the "Logic" tab. In the "Condition" field (or "Relevance equation"), enter the appropriate display logic:

3. Best practices

• Merge data during analysis: ensure that responses from duplicated questions (e.g., Q1 and Q1_R) are collected under a single column. Use the "Answer storage" or "Save answer as" parameter to force Q1_R to save its response under the Q1 code, and Q2_R under the Q2 code.
• Test your survey: use the "Survey preview" and reload the page several times. The displayed order should randomly alternate between Q1, Q2 and Q2, Q1.
• Document your randomization: note in the survey description or in an appendix file which questions were randomized and according to what logic.

4. Troubleshooting

The order does not change during tests

Solution: Check that the Q_Random question is positioned before the questions to be randomized. Also ensure that you start a new test session for each attempt (clear cookies or use a private browsing window).

Responses are not merged in the export

Solution: Check the "Answer storage" parameter of each duplicated question. If this parameter is not available, you will need to merge the columns manually during data analysis.

Need help?

For any difficulty, contact the LimeSurvey team via: Cellule.Web@ulb.be