Contents:
1. Overview
2. Step 1: Setting up an Authenticity Check API Token in Prolific
3. Step 2: Setting up your study on Prolific to record Prolific IDs via URL parameters
4. Step 3: Setting up Qualtrics to run the authenticity checks
• Loading the tracking on each page of the study
• Enabling authenticity checks on free-text questions
• Apply slightly different code for the final free-text question
5. Step 4: What you should see when you start collecting data

Helpful resources:
1. What is an authenticity check?
2. How to create an API token for authenticity checks

Overview

Our authenticity check is specifically designed to work on Qualtrics questionnaires containing free-text questions.

To get started:

  1. Create an authenticity check API token in Prolific to get access to our API
  2. Set up the study on Prolific to include URL parameters
  3. Add a Qualtrics-specific custom script to each free-text question in your survey that you want to assess. This will track user interactions and send data through to our authenticity check model.

Detailed instructions for each step are provided below.

Step 1: Setting up an Authenticity Check API Token in Prolific

Before you can start implementing authenticity checks into your survey, you'll need to create a new API token on Prolific. We recommend creating one specifically for this use case.

  1. Go to the workspace in which you intend to run your research or AI tasks
  2. Click API Tokens on the left to manage or generate tokens
  3. Click on ‘Create Authenticity Check token’
  4. Enter the name of your new API token to easily identify its purpose later
  5. Click ‘Create token for Authenticity Check’
  6. Finally, you'll be taken back to the API tokens homepage where you can copy the token.

You’ll be able to use this token with any integrations which support authenticity checks (currently Qualtrics & Gorilla only)

Step 2: Setting up your study on Prolific to record Prolific IDs via URL parameters

1. In the Recording Prolific IDs section on Prolific, ensure you select the URL parameters option to record participant IDs.

2. Next, go to Qualtrics, add a block to your survey named ‘Prolific ID’, and create a 'Text entry' question. In the question text, include the following message:

3. Go to 'Survey Flow' and add an 'Embedded Data' element at the beginning of your study. Into the embedded data field, enter 'PROLIFIC_PID' and make sure it says 'Value will be set from panel or URL'. Do the same for ‘STUDY_ID’ and ‘SESSION_ID’. For authenticity checks, it’s important that SESSION_ID is included. It allows us to link the authenticity check to the submission in your study.

4. Now go to the survey question that records the participant's Prolific ID. Right-click on the text-entry field and select 'Default Choices'.

5. When the window opens, click the blue drop-down arrow on the right. Then select 'Embedded data field'. Here, either enter 'PROLIFIC_PID', or click the drop-down arrow to the right of the text entry box and select 'PROLIFIC_PID', then click 'Insert'.

6. You should see a string called '${e://Field/PROLIFIC_PID}' populate the field. Now click 'Save', and you're done!

7. Finally, parameters need to be added to your Study URL in order to record the Prolific ID. Go back to Prolific. In the 'What is the URL of your study?' box, paste your Qualtrics study link in front of the pre-populated parameters.

8. Your 'Study Link' box should now look something like this:

Step 3: Setting up Qualtrics to run the authenticity checks

Loading the tracking on each page of the study

1. Go to the Look and Feel > General section on the left

2. Go to Header input and paste the script below.

<script> window.QUALTRICS_RESPONSE_ID = "${e://Field/ResponseID}";</script>
<script src="https://assets.prolific.com/assets/js/qualtrics/qualtrics.min.js"></script>
<script>window.tracker = MyTracker.getInstance("<your_token>")</script>

3. In the third line of script, replace <your_token> with the token generated on Prolific.

Example: <script>window.tracker = MyTracker.getInstance("123456789")</script>

4. Click save. This will load our tracking script for every page of the study. The next stage is applying it to the questions you want to track.

Enabling authenticity checks on free-text questions

For each question you want to run the check on, you’ll need to add a custom code snippet.

1. Select the first text entry question you want to track

2. Click the JavaScript link on the bottom left. The following modal should pop up

3. Replace the existing code with the Prolific-customised code here

4. Click save. This will mean that this question is tracked

5. Repeat for each free-text question where you want to run an authenticity check

6. If you are only tracking a single free-text question in your survey, paste the code snippet from the next section instead

Apply slightly different code for the final free-text question

✋ IMPORTANT: For the final authenticity checked question, you'll need additional script to ensure the tracker sends information to Prolific upon study submission.

1. For the final question, open up the “Edit Question Javascript” modal, as above

2. Copy and paste the code here

3. If you only want to apply an authenticity check on a single free-text question, add the same script you would apply to the final free-text question. (Again, you can also find the code here

You should now be set up and ready to run your pilot!

Step 4: What you should see when you start collecting data

As your study goes live, you should see a new column called “Authenticity Check” appear on the submissions page.

As submissions come in, results appear in the authenticity check column with the following indicators:

  • Green bar: Indicates all free-text responses appear authentic
  • Red bar: Indicates suspicious patterns were detected in all free-text responses
  • Mixed bar: Shows some questions were flagged (e.g., 50% green, 50% red if 2 out of 4 questions raise suspicion)

Need further help?
Click here to contact us