Jun 01, 2025
Version 1
MindCrowd Cognitive Testing V.1
- 1University of Arizona
Protocol Citation: Bonnie J LaFleur 2025. MindCrowd Cognitive Testing. protocols.io https://dx.doi.org/10.17504/protocols.io.14egnyqw6v5d/v1
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: May 19, 2025
Last Modified: June 01, 2025
Protocol Integer ID: 218551
Funders Acknowledgements:
Precision Aging Network
Grant ID: 5U19AG065169-04
Abstract
This protocol outlines scoring for five cognitive tasks that are given through the MindCrowd website. These include tasks specific to the MindCrowd expanded tasks specific to the Healthy Minds for Life cohort study as part the Precision Aging Network.
Introduction
Introduction
MindCrowd (MC) is an online platform (https://mindcrowd.org/) designed to study cognitive aging
across the adult lifespan. Launched in January, 2013, this internet-based approach overcomes at least some of the major barriers to research engagement, allowing individuals to engage in cognitive testing when and where it is most convenient for them. Participants, ages 18 and older, provide demographic,
lifestyle, health and medical history information, and are asked to complete two cognitive tests – a simple visual reaction time task and a paired associates memory task. In an effort to further increase access, MC
participants are allowed to use the electronic device of their choice (e.g., smartphone, tablet, laptop, or desktop computer). MindCrowd-Expanded (MC-E) is an extension of MC that was developed and launched in June 2022. MC-E includes an expanded set of cognitive tests in five domains: associative memory, information processing speed, and three areas of executive functioning – updating/working memory, inhibition, and set shifting. For each domain, specific cognitive tasks were selected from the experimental cognitive aging literature that have been shown to be sensitive to individual differences in normal aging, both cross-sectionally and longitudinally.
This protocol provides details for scoring for five tests/tasks:
1. Verbal Paired Associate Learning (associative memory)
2. Simple and Choice Reaction Time (information processing speed)
And three executive functioning tasks
3. Keep Track Task (updating/working memory)
4. Flanker Task (inhibition)
5. Letter Number Task (set shifting).
For each task we highlight the purpose of the task, testing protocol, materials, quality control, and variables/scoring.
Verbal Paired Associate Learning (PAL)
Verbal Paired Associate Learning (PAL)
Purpose of Task
Verbal Paired Associates (Wechsler, 1945), assesses associative memory, an episodic memory component involved in the binding of (associating) pieces of information depending on their degree of relatedness of in a single episode (Chalfonte & Johnson, 1996). Specifically, this task is designed to test the ability of individuals to remember pairs of words that are difficult to conceptually combine compared to words that may be semantically easier to combine.
Testing Protocol
Overview
For this test, participants were informed that they were completing a 5 minute memory task, that would involve studying and recalling word pairs. The task consists of 3 study test rounds in which the participants are first shown a list of word pairs and then are tested on their ability to recall
the word pair associations. In each of the 3 study test rounds, an identical list of 12 word pairs was presented (see Table 1). The word pair list contained 9 hard pairs (words with low semantic relatedness) and 3 easy pairs (words with high semantic relatedness).
Test and Practice
Participants are given one practice study-test round consisting of 3 word pairs. First, they
are presented with a list of the 3 word pairs. Then, they are shown the first word in each pair and asked to provide the second word in the pair. Each of the three study-test rounds begins with a study phase in which the word pair list is presented on the computer screen, with each word pair presented for 3000 msec. After the presentation of the list, participants are shown the first word of each pair, one at a time, and are required to type the second word of the pair into an open field textbox.Participants have 8000 msecs to begin typing their response. There is no response limit, as long as the response is initiated within the 8000 msec window.
Materials
Practice
Three word pair trials were included in the practice round (funny – sit, town – house, castle – nose)
Test
Table 1. Word Pairs used in the Study-Test Rounds
| Easy Pairs | Hard Pairs | |
| wash - clothes | truck - week | |
| bird - fly | never - pair | |
| rain - snow | today - brown | |
| hope - glue | ||
| cake - river | ||
| alone - large | ||
| pumpkin - help | ||
| sister - flag | ||
| number - kind |
Figure 1. Depiction of the Verbal Paired Associate
Learning Task Procedure
Quality Control
Quality control was assessed at both the participant and test level. At the participant level, tests did not meet QC criteria if the participant age was outside the allowed range (18-100) at the time of testing,
if participant demographic information was missing, or if the participant sex was not recorded as either male or female. At the test level, attempts were excluded if the test was not completed, if trials were recorded incorrectly due to technical issues, or if too many questions were answered incorrectly or not
answered at all to calculate an accuracy score.
Variables and Scoring
Participant responses were automatically considered correct only if they matched the exact spelling of the target word. In order to account for typos and misspellings, all responses flagged as errors were reviewed by coders. If the coder determined that a response approximated the correct word (ex. “vrown” for “brown”), then it was marked as correct. Two accuracy scores - count of correct responses and percentage accuracy – were computed per trial type (easy, hard) and round (1,2,3), as well as across combined trials and rounds.
Table 2. Word Pairs Variable Dictionary
| Variable Name | Variable Description | Score Calculation Information or Variable Value | |
| WP_R1_All_Trials | Word Pairs Round 1 total number of trials | 12 | |
| WP_R1_All_Correct | Word Pairs Round 1 number of all trials (easy + hard) correct out of 12 | Count of correct responses | |
| WP_R1_All_Perc_Correct | Word Pairs Round 1 for all trials including hard and easy combined percent correct | (Number of correct trials across round 1/ total number of trials in round 1)*100 | |
| WP_R1_Hard_Correct | Word Pairs Round 1 number of hard trials correct out of 9 | Count of correct responses | |
| WP_R1_Hard_Trials | Word Pairs Round 1 number of hard trials | 9 | |
| WP_R1_Easy_Perc_Correct | Word Pairs Round 1 easy trials percent correct | (Number of easy trials correct in round 1/ total number of easy trials in round 1)*100 | |
| WP_R1_Easy_Correct | Word Pairs Round 1 number of easy trials correct out of 3 | Count of correct responses | |
| WP_R1_Easy_Trials | Word Pairs round 1 total number of easy trials | 3 | |
| WP_R2_All_Trials | Word Pairs round 2 total number of trials including hard and easy combined percent correct | 12 | |
| WP_R2_All_Correct | Word Pairs Round 1 number of all trials (easy + hard) correct out of 12 | Count of correct responses | |
| WP_R2_All_Perc_Correct | Word Pairs Round 2 all trials percent correct | (Number of correct trials across round 2/ total number of trials in round 2 )*100 | |
| WP_R2_Hard _Correct | Word Pairs Round 2 number of hard trials correct out of 9 | Count of correct responses | |
| WP_R2_Hard_Perc_Correct | Word Pairs Round 2 hard trials percent correct | (Number of hard trials correct in round 2 / total number of hard trials in round 2)*100 | |
| WP_R2_Hard_Trials | Word Pairs round 2 total number of hard trials | 9 | |
| WP_R2_Easy_Correct | Word Pairs Round 2 number of easy trials correct out of 3 | Count of correct responses | |
| WP_R2_Easy_Perc_Correct | Word Pairs Round 2 easy trials percent correct | (Number easy trials correct in round 2 /number of easy trials in round 2)*100 | |
| WP_R2_Easy_Trials | Word Pairs Round 3 number of easy trials | 3 | |
| WP_R3_All_Trials | Word Pairs Round 2 total number of trials | 12 | |
| WP_R3_All_Correct | Word Pairs Round 1 number of easy trials correct out of 3 | Count of correct responses | |
| WP_R3_All_Perc_Correct | Word Pairs Round 3 all trials percent correct | (Number of correct trials across round 3/ total number of trials in round 3)*100 | |
| WP_R3_Hard _Correct | Word Pairs Round 3 number of hard trials correct out of 9 | Count of correct responses | |
| WP_R3_Hard_Perc_Correct | Word Pairs Round 3 hard trials percent correct | (Number of hard trials correct in round 3 / total number hard trials in round 3)*100 | |
| WP_R3_Hard_Trials | Word Pairs Round 3 total number of hard trials | 9 | |
| WP_R3_Easy_Correct | Word Pairs Round 2 number of easy trials correct out of 3 | Count of correct responses | |
| WP_R3_Easy_Perc_Correct | Word Pairs Round 3 easy trials percent correct | (Number of easy trials correct in round 3 / number of easy trials in round 3)*100 | |
| WP_R3_Easy_Trials | Word Pairs Round 3 number of easy trials | 3 | |
| WP_Total_All_Trials | Word Pairs total number of trials across all rounds | 36 | |
| WP_Total_All_Correct | Word Pairs all rounds number of easy and hard trials correct out of 36 | Count of correct responses | |
| WP_Total_All_Perc_Correct | Word Pairs percentage correct for all trials across all rounds | (Number of correct trials /total number of trials presented across all rounds)*100 | |
| WP_Total_Easy_Trials | Word Pairs total number of easy trials across all rounds | 9 | |
| WP_Total_Easy_Correct | Word Pairs number of easy and trials correct out of 9 | Count of correct responses | |
| WP_Total_Easy_Perc_Correct | Word Pairs percentage correct for total number of easy trials across all rounds | (Number of easy trials correct across all rounds /number of easy trials presented across all rounds)*100 | |
| WP_Total_Hard_Trials | Word Pairs Total number of hard trials across all rounds | 27 | |
| WP_Total_Hard_Correct | Word Pairs Total number of hard trials correct out of 27 | Count of correct responses | |
| WP_Total_Hard_Perc_Correct | Word Pairs percentage correct for total number of hard trials across all rounds | (Number of hard trials correct across all rounds /number of hard trials presented across all rounds)*100 | |
| WP_first_attempt | Word Pairs valid first attempt | TRUE/FALSE | |
| WP_user_device | Word Pairs device task was completed on | DESKTOP/ PHONE/TABLET | |
| WP_game_result | Word Pairs attempt identifier | PGI-XXXXXXX | |
| WP_qc_code | Word Pairs quality control code: Code indicating whether participant passed all quality control criteria. If they didn’t the code indicates why participant failed. | QC Passed OR numeric code indicating reason for QC fail 1 - Test result appears to be a true non-attempt, is not scorable, and should not preclude another attempt 2 - "Age outside allowed range." 3 - Test not completed 4 - "Trials recorded incorrectly due to technical issues" 5 - "Missing demographic information" 6 - Participant has already taken test before this attempt, so this attempt is ignored 7 - Too many keystrokes due to spamming, or too few due to lack of participation 8 - Too many questions were answered incorrectly or not answered at all to calculate an accurate score 9 - Test not taken on desktop 10 - Participant sex is not Male or Female |
References
Chalfonte BL, Johnson MK. Feature memory and binding in
young and older adults. Memory Cognition. 1996 24(4):403–416.
Wechsler, D. (1945). A standardized memory scale for clinical use. The Journal of Psychology: Interdisciplinary and Applied, 19, 87–95.
Simple and Choice Reaction Times
Simple and Choice Reaction Times
The purpose of this document is to describe the Simple and Choice Reaction Time cognitive test part of the Mind Crowd Expanded and Healthy Minds for Life Study in the Precision Aging Network (PAN). Participants could complete the task on any internet accessible device including a desktop/laptop, phone or tablet. P2 participants were instructed to complete the task on a desktop or laptop only.
Purpose of Task
The Simple and Choice Reaction Time task is designed to assess information processing speed (Deary et al., 2011). Simple reaction time reflects how quickly a person can make a response to just one particular stimulus (ex. light flashes -> press the button). Choice reaction time reflects the amount of time it takes a person to make the appropriate response when different stimuli and possible responses are available (ex. green light flashes -> press G key, but if red light flashes -> press R key). Choice reaction time is generally slower than simple reaction time, due to increased cognitive demands producing a latency in response time. There is a trade-off between speed and accuracy that can also affect choice reaction times, as the less accurate one aims to be the faster they can potentially respond.
Testing Protocol
Overview
For this test, participants were informed that they were completing a 5 minute speed task. This task includes two conditions: simple and choice. In the simple reaction time condition participants are asked to
respond to a single visual stimulus in one location on the screen. In the choice condition participants had to choose between four possible responses to a visual stimulus that could appear in one of four possible locations on the screen. For both conditions, the visual stimulus is a large ‘X’.
Test and Practice
In the simple reaction time condition, the letter ‘X’ appears in a white box in the center of the screen at random intervals between 1000 and 3000 msec. Participants are asked to press any letter key as quickly
as possible whenever the ‘X’ appears. In the choice condition, four white boxes aligned side-by-side are present on the screen, corresponding to assigned keys on the keyboard (from left to right, D, F, J, and K). The letter X appears at random intervals between 1000 and 3000 msec and randomly in any one of the four boxes. Participants press the corresponding key as quickly as possible when the X appears. Participants have up to 1500 msec to make their response. Participants first complete the simple choice reaction condition, with 20 trials. Then, participants complete the choice reaction condition, with 40 trials. Prior to commencing each condition participants are given a brief practice round.
Materials
Test
Simple and Choice Reaction Time Diagram
Figure 1. Depiction of the Simple and Choice
Reaction Time procedure.
Quality Control (QC)
Quality control was assessed at both the participant and test level. At the participant level, tests did not meet QC criteria if the participant age was outside the allowed range (18-100) at the time of testing,
if participant demographic information was missing, or if the participant sex was not recorded as either male or female. At the test level, attempts were excluded if the test was not completed, if trials were recorded incorrectly due to technical issues, or if there were too many keystrokes due to spamming or too few due to lack of participation.
Variables and Scoring
Two accuracy scores - count of correct responses and percentage accuracy – were computed per condition (simple and choice). For correct responses only, mean and median reaction times were computed per condition. Standard deviation of the mean reaction times was also computed. The difference between simple and choice reaction times were also computed, for both the mean and median scores.
Table 1. Simple and Choice Reaction Time Variable
Descriptions
| Variable Name | Variable Description | Score Calculation Information or Variable Value | |
| SCRT_Simp_Trials | Simple and Choice Reaction Time number of simple trials | 20 | |
| SCRT_Choice_Trials | Simple and Choice Reaction Time number of choice trials | 40 | |
| SCRT_Simp_Count_Correct | Simple and Choice Reaction Time simple trials number of correct responses | Count of correct responses | |
| SCRT_Simp_Perc_Correct | Simple and Choice Reaction Time simple trials percent correct | (Number of correct simple trials/total number of simple trials)*100 | |
| SCRT_Choice_Count_Correct | Simple and Choice Reaction Time choice trials number of correct responses | Count of correct responses | |
| SCRT_Choice_Perc_Correct | Simple and Choice Reaction Time choice trials percent correct | (Number of correct choice trials/total number of choice trials)*100 | |
| SCRT_Simp_MedRT | Simple and Choice Reaction Time median reaction time for simple trials (correct responses only) | ||
| SCRT_Choice_MedRT | Simple and Choice Reaction Time median reaction time for choice trials (correct responses only) | ||
| SCRT_MedRT_Diff | Simple and Choice Reaction Time difference in median reaction time between choice and simple trials (correct responses only) | Choice median RT – Simple median RT | |
| SCRT_Simp_MeanRT | Simple and Choice Reaction Time simple trials mean reaction time (correct responses only) | ∑(correct simple RTs)/ total number of correct simple scores | |
| SCRT_Choice_MeanRT | Simple and Choice Reaction Time choice trials mean reaction time (correct responses only) | ∑(correct choice RTs)/total number of correct choice scores | |
| SCRT_MeanRT_Diff | Simple and Choice Reaction Time difference in mean reaction time between choice and simple trials (correct responses only) | Choice mean RT – Simple mean RT | |
| SCRT_Simp_SDRT | Simple and Choice Reaction Time simple trials standard deviation (correct responses only) | ||
| SCRT_Choice_SDRT | Simple and Choice Reaction Time choice trials standard deviation (correct responses only) | ||
| SCRT_Simp_Timeout | Simple and Choice Reaction Time simple trials timeout | Responses > 1500ms | |
| SCRT_Choice_Timeout | Simple and Choice Reaction Time choice trials timeout | Responses > 1500ms | |
| SCRT_Choice_ErrorCount | Simple and Choice Reaction Time number of errors choice trials | Count of incorrect key presses in choice trials | |
| SCRT_first_attempt | Simple and Choice Reaction Time valid first attempt | TRUE/FALSE | |
| SCRT_user_device | Simple and Choice Reaction Time device task was completed on | DESKTOP/PHONE/TABLET | |
| SCRT_game_result | Simple and Choice Reaction Time attempt identifier | PGI-XXXXXXX | |
| SCRT_qc_code | Simple and Choice Reaction Time quality control code: Code indicating whether participant passed all quality control criteria. If they didn’t code indicates why participant failed. | QC Passed OR numeric code indicating reason for QC fail 1 - Test result appears to be a true non-attempt, is not scorable, and should not preclude another attempt 2 - "Age outside allowed range." 3 - Test not completed 4 - "Trials recorded incorrectly due to technical issues" 5 - "Missing demographic information" 6 - Participant has already taken test before this attempt, so this attempt is ignored 7 - Too many keystrokes due to spamming, or too few due to lack of participation 8 - Too many questions were answered incorrectly or not answered at all to calculate an accurate score 9 - Test not taken on desktop 10 - Participant sex is not Male or Female |
References
Deary IJ, Liewald D, Nissan J. A free, easy-to-use, computer-based simple and four-choice reaction time programme: the Deary-Liewald reaction time task. Behav Res Methods. 2011 Mar;43(1):258-68.
Keep Track Task
Keep Track Task
The purpose of this document is to describe the Keep Track cognitive test part of the Mind Crowd Expanded and Healthy Minds for Life Study in the Precision Aging Network (PAN). Participants could complete the task on any internet accessible device including a desktop/laptop, phone or tablet. P2
participants were instructed to complete the task on a desktop or laptop only.
Purpose
Executive functioning involves broad-based attentional control or self-regulation of goal-relevant behaviors that is likely subserved by a set of more specific cognitive processes. An influential study by Miyake et al. (2000) identified three relatively independent factors of executive function: set shifting, updating in working memory, and inhibition of prepotent responses. Keep Track (KT; Yntema, 1963) assesses working memory updating, the cognitive process of replacing outdated information with more recent and relevant information, while protecting that information from interference.
Testing Protocol
Overview
For this test, participants were informed that they were
completing a 15 minute task that would test their ability to keep track of
words that belong to one of six categories. In this task, participants are
asked to view a serial list of 15 words and are instructed to remember the last
word(s) in the list belonging to a specific category or categories listed at
the bottom of the screen.
Test and Practice
The task begins with a familiarization phase that is intended to expose the participants to all
stimuli used in the updating task, and to ensure that participants are aware of the words that belong to each category. In this phase, all six categories (Sports, Fruits, Distances, Metals, Relatives, Furniture) are listed at the bottom of the screen, while all 36 words are presented, one at a time, for 1500 msec.
Participants are asked to identify which of the 6 categories each word belongs to by clicking the category associated with the presented word. Participants are informed if they answered correctly or incorrectly.
After the familiarization phase, participants begin the updating task. In this task, participants are asked to view a serial list of 15 out of the 36 words previously shown and are instructed to remember the last word(s) in the list belonging to a specific category or categories. At the end of the list presentation, participants must recall the last word belonging to the category or categories listed at the bottom of the screen and type it into an open field textbox. For example, if the participant is given one category – fruits – then they would need to keep track of the last fruit word presented in the list and recall that word when asked.
Participants are given one practice round of the updating task. For this, participants are asked to keep track of two categories while viewing a list of 15 words. The practice is identical to what participants will experience in the Category 2 condition. In the updating task, twelve lists are administered in total, three per Category (1,2,3,4). For the first 3 lists, participants are tested on one category, and then for the next three lists, participants are tested on 2 categories, and so on. At the beginning of each word list presentation, participants are first oriented to the category or categories at the bottom of the screen, while an asterisk is displayed in the middle of the screen for 5 sec. Next, participants are presented with a list of 15 words and are tasked with remembering the last word in the list that belongs to the 1, 2, 3, or 4 categories listed at the bottom of the screen. The total duration of each list presentation is 5000msec, each word in the list is presented for 1500msec with a 150 interstimulus interval. Once all 15 words in the list are presented, the participant must type in the last word from each displayed category in an open field textbox. Overall, participants have three ‘trials’ (1 per list) for the 1 category condition, followed by 6 trials (2 per list) for the 2 category condition, then 9 trials (3 per list) for the 3 category condition, and finally 12 trials (4 per list) for the 4 category condition.
Materials
Table 1. Keep Track Categories and Word List.
| Furniture | Fruits | Metals | Distance | Sports | Relatives | |
| cabinet | lemon | bronze | furlong | soccer | brother | |
| bed | cherry | steel | foot | football | aunt | |
| dresser | mango | iron | inch | hockey | nephew | |
| desk | pear | copper | meter | baseball | daughter | |
| couch | apricot | platinum | mile | golf | sister | |
| bookshelf | grape | iron | yard | tennis | uncle |
Figure 1. Depiction of the Keep Track Test Procedure.
Quality Control
Quality control (QC) was assessed at both the participant and test level. At the participant level, tests did not meet QC criteria if the participant age was outside the allowed range (18-100) at the time of testing,
if participant demographic information was missing, or if the participant sex was not recorded as either male or female. At the test level, attempts were excluded if the test was not completed, if trials were recorded incorrectly due to technical issues, or if too many questions were answered incorrectly or not
answered at all to calculate an accuracy score.
Variables and Scoring
Participant responses were automatically considered correct only if they matched the exact spelling of the target word. In order to account for typos and misspellings, all responses flagged as errors were reviewed by coders. If the coder determined that a response approximated the correct word (ex. “coppper” for “copper”), then it was marked as correct. Accuracy scores representing the count of correct responses and percentage correct were computed for each category (Categories 1-4). Total percent correct across all categories was also computed.
Table 2. Keep Track Variable Dictionary
| Variable Name | Variable Description | Score Calculation Information or Variable Value | |
| KT_Cat1_Trials | Keep Track Category 1 number of trials | 3 | |
| KT_Cat1_Count_Correct | Keep Track Category 1 count of correct responses out of 3 possible | Count of correct responses | |
| KT_Cat1_Perc_Correct | Keep Track Category 1 percent correct | (Number of correct trials in Category 1/ total number of trials in cCtegory 1) *100 | |
| KT_Cat2_Trials | Keep Track number of trials in Category 2 | 6 | |
| KT_Cat2_Count_Correct | Keep Track Category 2 count of correct responses out of 6 possible | Count of correct responses | |
| KT_Cat2_Perc_Correct | Keep Track Category 2 percent correct | (Number of correct trials in Category 2/ total number of trials in Category 2)*100 | |
| KT_Cat3_Trials | Keep Track number of trials in Category 3 | 9 | |
| KT_Cat3_Count_Correct | Keep Track Category 3 count of correct responses out of 9 possible | Count of correct responses | |
| KT_Cat3_Perc_Correct | Keep Track Category 3 percent correct | (Number of correct trials in Category 3/ total number of trials in Category 3)*100 | |
| KT_Cat4_Trials | Keep Track number of trials in Category 4 | 12 | |
| KT_Cat4_Count_Correct | Keep Track Category 4 count of correct responses out of 3 possible | Count of correct responses | |
| KT_Cat4_Perc_Correct | Keep Track Category 4 percent correct | (Number of correct trials in Category 4/ total number of trials in Category 4)*100 | |
| KT_Total_Trials | Keep Track number of trials across all categories | 30 | |
| KT_Total_Count_Correct | Keep Track count of correct responses across all trials out of 30 possible | Count of correct responses | |
| KT_Total_Perc_Correct | Keep Track total trials percentage correct across all categories | Total number of correct trials across all categories/ number of possible trials across all categories | |
| KT_first_attempt | Keep Track valid first attempt | TRUE/FALSE | |
| KT_user_device | Keep Track device task was completed on | DESKTOP/PHONE/TABLET | |
| KT_game_result | Keep Track attempt identifier | PGI-XXXXXXX | |
| KT_qc_code | Keep Track quality control code: Code indicating whether participant passed all quality control criteria. If they didn’t code indicates why participant failed. | QC Passed OR numeric code indicating reason for QC fail 1 - Test result appears to be a true non-attempt, is not scorable, and should not preclude another attempt 2 - "Age outside allowed range." 3 - Test not completed 4 - "Trials recorded incorrectly due to technical issues" 5 - "Missing demographic information" 6 - Participant has already taken test before this attempt, so this attempt is ignored 7 - Too many keystrokes due to spamming, or too few due to lack of participation 8 - Too many questions were answered incorrectly or not answered at all to calculate an accurate score 9 - Test not taken on desktop 10 - Participant sex is not Male or Female |
References
Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD. The unity and diversity of executive functions and their contributions to complex “Frontal lobe” tasks: a latent variable analysis.
Cognitive Psychology. 2000;41(1):49-100. doi:10.1006/cogp.1999.0734 Yntema, D. B. (1963). Keeping track of several things at once. Human Factors, 5, 7–17.
Flanker Task
Flanker Task
The purpose of this document is to describe the Flanker Task which is a cognitive assessment part of the Mind Crowd Expanded (MCE) and Healthy Minds for Life Study in the Precision Aging Network (PAN). Participants could complete the task on any internet accessible device including a desktop/laptop, phone or tablet. P2 participants were instructed to complete the task on a desktop or laptop only.
Purpose of Task
Executive functioning involves broad-based attentional control or self-regulation of goal-relevant behaviors that is likely subserved by a set of more specific cognitive processes. An influential study by Miyake et al. (2000) identified three relatively independent factors of executive function: set shifting, updating in working memory, and inhibition of prepotent responses. The Flanker Task (Erikson & Erikson, 1974), is used to measureselective attention and inhibitory function.
Testing Protocol
Overview
For this test, participants were informed that they were completing a 3 minute task. In this task, participants are presented with a series of arrows pointing to the left or the right of the screen and their objective is to indicate the direction of the middle arrow. There are two conditions in this task: congruent and incongruent. In the congruent condition, all arrows face in the same direction. In the incongruent condition, the middle arrow points in the opposite direction of the flanking arrows.
Test and Practice
Participants are instructed to press the “J” key to indicate the middle arrow is pointing right, or the “F” key to indicate that the middle arrow is pointing left. A short practice round is given before the test. In the
test, a total of 50 trials are presented, including 25 congruent and 24 incongruent trials. Each trial begins with a star that appears for 1000 msec to orient participants to the middle of the screen, followed by an arrow in the same location facing left or right, with arrows on either side (flankers) facing either the same direction (congruent) or the opposite direction (incongruent) as the middle arrow. Participants have 8000 msec to respond on each trial, after each response participants are reoriented with the star and
the next trial begins.
Materials
Figure 1. Flanker Diagram
Quality Control
Quality control was assessed at both the participant and test level. At the participant level, tests did not meet QC criteria if the participant age was outside the allowed range (18-100) at the time of testing, if participant demographic information was missing, or if the participant sex was not recorded as either male or female. At the test level, attempts were excluded if the test was not completed, if trials were recorded incorrectly due to technical issues, or if there were too many keystrokes due to spamming or too few due to lack of participation.
Variables and Scoring
Two accuracy scores - count of correct responses and percentage accuracy – were computed per condition (congruent and incongruent). For correct responses only, mean and median reaction times were computed per condition. Standard deviation of the mean reaction times were also computed. The difference between congruent and incongruent reaction times were also computed, for both the mean and median scores.
Table 1. Flanker Variable Dictionary
| Variable Name | Variable Description | Score Calculation Information or Variable Value | |
| FLK_Cong_Trials | Flanker total number of valid congruent trials | Total number of congruent trials - congruent trials timeouts | |
| FLK_Inc_Trials | Flanker total number of valid incongruent trials | Total number of incongruent trials - incongruent trials timeouts | |
| FLK_Cong_Count_Correct | Flanker congruent trials number of correct responses out of 25 possible | Count of correct responses | |
| FLK_Cong_Perc_Correct | Flanker congruent trials percentage correct | (Number of valid responses in the congruent trials/ total number of congruent trials)*100 | |
| FLK_Inc_Count_Correct | Flanker incongruent trials number of correct responses out of 24 possible | Count of correct responses | |
| FLK_Inc_Perc_Correct | Flanker incongruent trials percentage correct | (Number of correct responses in the incongruent trials/ total number of congruent trials)*100 | |
| FLK_Cong_MedRT | Flanker congruent trials median reaction time in milliseconds (correct trials only) | ||
| FLK_Inc_MedRT | Flanker incongruent trials median reaction time in milliseconds (correct trials only) | ||
| FLK_MedRT_Diff | Flanker difference in median reaction time between congruent and incongruent trials (correct trials only) | Median incongruent -median congruent (median interference RT) | |
| FLK_Cong_MeanRT | Flanker congruent trials mean reaction time in milliseconds (correct trials only) | ∑(correct congruent RTs)/ total number of correct congruent scores | |
| FLK_Inc_MeanRT | Flanker incongruent trials mean reaction time in milliseconds (correct trials only) | ∑(correct incongruent RTs)/total number of correct incongruent scores | |
| FLK_MeanRT_Diff | Flanker Difference in mean reaction time between congruent and incongruent trials(correct trials only) | Mean incongruent RT - mean congruent RT (mean interfernece RT) | |
| FLK_Cong_SDRT | Flanker congruent trials reaction time standard deviation (correct trials only) | ||
| FLK_Inc_SDRT | Flanker incongruent trials reaction time standard deviation (correct trials only) | ||
| FLK_Cong_Timeout | Flanker congruent trials timeout | Responses > 8000 ms or no response | |
| FLK_Inc_Timeout | Flanker incongruent trials timeout | Responses > 8000 ms or no response | |
| FLK_Cong_ErrorCount | Flanker number of errors across congruent trials | Count of incorrect key presses indicating incorrect arrow direction in congruent trials | |
| FLK_Inc_ErrorCount | Flanker number of errors across incongruent trials | Count of incorrect key presses indicating incorrect arrow direction in incongruent trials | |
| FLK_first_attempt | Flanker valid first attempt | TRUE/FALSE | |
| FLK_user_device | Flanker device task was completed on | DESKTOP/ TABLET/PHONE | |
| FLK_game_result | Flanker unique attempt identifier | PGI-XXXXXXX | |
| FLK_qc_code | Flanker quality control code: Code indicating whether participant passed all quality control criteria. If they didn’t code indicates why participant failed | QC Passed OR numeric code indicating reason for QC fail 1 - Test result appears to be a true non-attempt, is not scorable, and should not preclude another attempt 2 - "Age outside allowed range." 3 - Test not completed 4 - "Trials recorded incorrectly due to technical issues" 5 - "Missing demographic information" 6 - Participant has already taken test before this attempt, so this attempt is ignored 7 - Too many keystrokes due to spamming, or too few due to lack of participation 8 - Too many questions were answered incorrectly or not answered at all to calculate an accurate score 9 - Test not taken on desktop 10 - Participant sex is not Male or Female |
References
Eriksen BA, Eriksen CW. Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics. 1974;16(1):143-149. doi:10.3758/bf03203267
Letter Number Task
Letter Number Task
The purpose of this document is to describe the Letter Number cognitive test part of the Mind Crowd Expanded and Healthy Minds for Life Study in the Precision Aging Network (PAN). Participants could complete the task on any internet accessible device including a desktop/laptop, phone or tablet. P2 participants were instructed to complete the task on a desktop or laptop only.
Purpose of Task
Executive functioning involves broad-based attentional control or self-regulation of goal-relevant behaviors that is likely subserved by a set of more specific cognitive processes. An influential
study by Miyake et al. (2000) identified three relatively independent factors of executive function: set shifting, updating in working memory, and inhibition of prepotent responses. The Letter-Number Task measures an individual’s cognitive flexibility, the ability to shift between tasks (Rogers & Monsell, 1995)
Testing Protocol
Overview
For this test, participants were informed that they were completing a 15 minute task that would test their cognitive flexibility. In this task, participants view letter number pairs and must respond to each pair according to a specific rule. There are two conditions in this task, a ‘same’ and ‘switch’ condition. In the same condition, participants are instructed to follow a single rule guiding their response, while in the switch condition participants must alternate between two rules. In the first phase of the same condition, participants are given one rule to guide their responses for all the trials. In the second phase of the same condition, participants are given another rule to follow throughout all the trials. In the
switch condition, participants are asked to alternate between the two rules that were presented across the same condition phases.
Test and Practice
The general task consists of a letter and a number appearing side-by-side (for example, D2 or E9) in one of four quadrants of the computer screen (i.e., upper left, upper right, lower left, lower right). For the first phase of the same condition, the number phase, 32 number-letter pairs appear alternately in the upper left and right quadrants, one at a time. Participants must disregard the letter and indicate whether the number is odd or even by pressing the “J” key indicating odd or the “K” key indicating even. In the second phase of the same condition, the letter phase, a new set of 32 number-letter pairs appear alternately in the lower left and right quadrants. Participants must disregard the number and indicate whether the letter is a vowel or a consonant. In the switch condition, 128 new number-letter pairs appear in a clockwise rotating fashion, throughout all four quadrants, beginning with the upper left quadrant. Participants now must shift between the two rules, indicating “odd or even” by pressing the “J” for odd or “K” for even when the pairs appear in the top quadrants, and “vowel or consonant” by pressing the “D” key for a consonant or “F” key for a vowel when the pairs appear in the lower quadrants. Participants are given up to 10000ms to respond to each pair. Three practice rounds are given – one for the same condition number phase, one for the same condition letter phase, and one for the switch condition.
Figure 1. Depiction of the Two Conditions of the
Letter Number Task
Figure 2. Depiction of the Same Condition Number Phase
of the Letter Number Task
Figure 3. Depiction of the Same Condition Letter Phase
of the Letter Number Task
Figure 4. Depiction of the Switch Condition of the
Letter Number Task
Quality Control
Quality control was assessed at both the participant and test level. At the participant level, tests did not meet QC criteria if the participant age was outside the allowed range (18-100) at the time of testing, if participant demographic information was missing, or if the participant sex was not recorded as either male or female. At the test level, attempts were excluded if the test was not completed, if trials were recorded incorrectly due to technical issues, or if there were too many keystrokes due to spamming or too few due to lack of participation.
Variables and Scoring
Two accuracy scores - count of correct responses and percentage accuracy – were computed per condition (same and switch). For correct responses only, mean and median reaction times were computed per condition. Standard deviation of the mean reaction times was also computed. The difference between same and switch reaction times were also computed, for both the mean and median scores.
Table 1. Letter-Number Variable Dictionary
| Variable Name | Variable Description | Score Calculation Information or Variable Value | |
| LN_Same_Trials | Letter Number valid same trials | Total number of same (combined letter and number rule) trials - number of timeouts | |
| LN_Switch_Trials | Letter Number valid switch trials | Total number of switch trials (alternating between two rules) - number timeouts | |
| LN_Same_Count_Correct | Letter Number total number of correct responses across both same trials (combined letter and number rule) | ||
| LN_Same_Perc_Correct | Letter Number same trials percent correct | (Number of correct responses for same trials/number of possible responses in the same trials)*100 | |
| LN_Switch_Count_Correct | Letter Number total number of correct responses for the switch trial (alternating between two rules) | ||
| LN_Switch_Perc_Correct | Letter Number switch trials percent correct | (Number of correct responses in switch trials/number of possible responses in switch trials) *100 | |
| LN_Same_MedRT | Letter Number same trials median reaction time (correct trials only) | ||
| LN_Switch_MedRT | Letter Number switch trials median reaction time (correct trials only) | ||
| LN_MedRT_Diff | Letter Number difference in median reaction time between switch and same trials (correct trials only) | MedianRT switch trials – medianRT number/letter trials | |
| LN_Same_MeanRT | Letter Number same trials mean reaction time (correct trials only) | ∑(correct same RTs)/ total number of correct same RTs | |
| LN_Switch_MeanRT | Letter Number switch trials mean reaction time (correct trials only) | ∑(correct switch RTs)/ total number of correct switch RTs | |
| LN_MeanRT_Diff | Letter Number difference in mean reaction time between switch and same trials (correct trials only) | MeanRT switch trials – meanRT number/letter trials | |
| LN_Same_SDRT | Letter Number same trials standard deviation (correct trials only) | ||
| LN_Switch_SDRT | Letter Number switch trials standard deviation (correct trials only) | ||
| LN_Same_Timeout | Letter Number same trials timeout | Responses > 10000ms or no response | |
| LN_Switch_Timeout | Letter Number switch trials timeout | Responses > 10000ms or no response | |
| LN_Switch_ErrorCount | Letter Number switch trials number of errors | Count of incorrect responses to single rule | |
| LN_Same_ErrorCount | Letter Number same trials number of errors | Count of incorrect responses to alternating rules | |
| LN_first_attempt | Letter Number valid first attempt | TRUE/FALSE | |
| LN_user_device | Letter Number device task was completed on | DESKTOP/ TABLET/PHONE | |
| LN_game_result | Letter Number attempt identifier | PGI-XXXXXXX | |
| LN_qc_code | Letter Number quality control code: Code indicating whether participant passed all quality control criteria. If they didn’t code indicates why participant failed. | QC Passed OR numeric code indicating reason for QC fail 1 - Test result appears to be a true non-attempt, is not scorable, and should not preclude another attempt 2 - "Age outside allowed range." 3 - Test not completed 4 - "Trials recorded incorrectly due to technical issues" 5 - "Missing demographic information" 6 - Participant has already taken test before this attempt, so this attempt is ignored 7 - Too many keystrokes due to spamming, or too few due to lack of participation 8 - Too many questions were answered incorrectly or not answered at all to calculate an accurate score 9 - Test not taken on desktop 10 - Participant sex is not Male or Female |
References
Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD. The unity and diversity of executive functions and their contributions to complex “Frontal lobe” tasks: a latent variable analysis. Cognitive Psychology. 2000;41(1):49-100. doi:10.1006/cogp.1999.0734
Rogers, R. D., & Monsell, S. (1995). Costs of a predictible switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124(2), 207–231. https://doi.org/10.1037/0096-3445.124.2.207