Nov 24, 2025
Systematic Scoping Review: Methods of Content and Construct Validity, and Cognitive Testing Among Newly Developed Food Frequency Questionnaire
- Fazle Rabbi1,
- Russell de Souza2
- 1PhD Candidate, Mary Heersink School of Global Health and Social Medicine, McMaster University;
- 2Associate Professor, Mary Heersink School of Global Health and Social Medicine, McMaster University
- McMaster University
Protocol Citation: Fazle Rabbi, Russell de Souza 2025. Systematic Scoping Review: Methods of Content and Construct Validity, and Cognitive Testing Among Newly Developed Food Frequency Questionnaire. protocols.io https://dx.doi.org/10.17504/protocols.io.5jyl88ynrl2w/v1
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: November 24, 2025
Last Modified: November 24, 2025
Protocol Integer ID: 233304
Keywords: FFQ, Food Frequency Questionnaire, FFQ validity, FFQ validation method, developed food frequency questionnaire, food frequency questionnaire, systematic scoping review, scoping review, ffq development, cognitive testing, validity in study, requiring improved reporting standard, improved reporting standard, methodological inconsistency, validity, construct validity, content validity
Abstract
This scoping review will systematically map the methods used to assess and report validity in studies that developed FFQs de novo. By synthesizing how content validity, construct validity, and cognitive testing are operationalized in FFQ development, this review aims to clarify current practices, identify methodological inconsistencies, and highlight areas requiring improved reporting standards.
Guidelines
Inclusion and Exclusion Criteria for the Scoping Review Protocol
Inclusion Criteria
- Population / Study Type: Empirical studies that developed a new Food Frequency Questionnaire (FFQ) from scratch for assessing dietary intake in humans.
- Instrument Development Stage: The FFQ must have been conceptually developed (item generation, food list creation, portion size determination) rather than adapted from an existing FFQ.
- Psychometric Evaluation: Studies that assessed at least one form of validity (content or construct) and at least one form of reliability (e.g., test–retest, internal consistency).
- Content Validity: Studies reporting expert review, cognitive interviewing, pilot testing, or participant feedback for item comprehensibility.
- Construct Validity: Studies reporting statistical or theoretical evidence for construct validity (e.g., factor analysis, hypothesis testing, structural validity).
- Population Type: Any human population (adults, children, specific patient groups, or general populations).
- Study Design: Quantitative, mixed-methods, or methodological studies focusing on FFQ development and psychometric evaluation.
- Publication Type: Peer-reviewed journal articles, dissertations, or reports published in English.
Exclusion Criteria
- Adapted / Translated Instruments: Studies that adapted, translated, or culturally modified an existing FFQ without de novo item development.
- Criterion-Only Validation: Studies that assessed only criterion validity (e.g., correlation with 24-h recall or biomarkers) without content or construct validity assessment.
- Reliability-Only Studies: Studies that measured only reliability without validity assessment.
- Reviews / Protocols / Commentaries: Narrative reviews, systematic reviews, scoping reviews, or editorials without primary data.
- Non-human or Laboratory Studies: Studies involving animal populations or in vitro analysis.
- Incomplete Psychometric Reporting: Studies that developed FFQ but did not report any formal assessment of validity or reliability.
Troubleshooting
Introduction
Food Frequency Questionnaires (FFQs) are among the most widely used tools for assessing habitual dietary intake in epidemiological and public health research. Despite their extensive use, the methodological rigor applied during the development of FFQs- particularly in establishing content validity, construct validity, comprehensibility, and reliability- varies substantially across studies. Many existing reviews focus on criterion validity against reference measures, leaving a critical gap in
understanding how newly developed FFQs are conceptualized, structured, and psychometrically evaluated at the instrument-development stage.
This scoping review will systematically map the methods used to assess and report validity in studies that developed FFQs de novo. By synthesizing how content validity, construct validity, and cognitive testing are operationalized in FFQ development, this review aims to clarify current practices, identify methodological inconsistencies, and highlight areas requiring improved reporting standards.
Methods
Research Questions:
Among newly developed Food Frequency Questionnaires (FFQs), how have the studies assessed and reported content validity, construct validity, and cognitive testing/comprehensibility during instrument development?
Specific Objectives
1. To identify studies that developed Food Frequency Questionnaires (FFQs) from scratch.
2. To examine how content validity has been assessed and reported in these FFQ development studies.
3. To describe methods used for evaluating construct validity of newly developed FFQs.
4. To map the approaches used for cognitive testing or comprehensibility assessment during FFQ development.
Search Strategy
The following electronic databases were systematically searched from inception to November 23, 2025:
- Ovid MEDLINE
- Ovid EMBASE
- Ovid PsycINFO
- CINAHL (EBSCOhost)
- Web of Science Core Collection
- Health and Psychosocial Instruments (HaPI)
No date or geographic limits were applied. Animal-only studies were excluded where applicable.
OVID MEDLINE (Then adapted for EMBASE, PsychINFO, and HaPI)
((food frequency questionnaire* or FFQ or food-frequency questionnaire* or dietary questionnaire* or diet history questionnaire* or diet questionnaire* or diet intake questionnaire*) adj6 (valid* or validat* or psychometric* or "content valid*" or "face valid*" or "construct valid*" or discriminant valid* or convergent valid* or "factor analys*" or "exploratory factor analysis" or EFA or "confirmatory factor analysis" or CFA or PCA or "principal component*")).ti.
CINAHL (EBSCOhost)
TI (("food frequency questionnaire*" OR FFQ OR "food-frequency questionnaire*" OR "dietary questionnaire*" OR "diet history questionnaire*" OR "diet questionnaire*" OR "diet intake questionnaire*") N6 ("valid*" OR "validat*" OR "psychometric*" OR "content valid*" OR "face valid*" OR "construct valid*" OR "discriminant valid*" OR "convergent valid*" OR "factor analys*" OR "exploratory factor analysis" OR "confirmatory factor analysis" OR EFA OR CFA OR PCA OR "principal component*"))
Web of Science
TI = (("food frequency questionnaire*" OR FFQ OR "food-frequency questionnaire*" OR "dietary questionnaire*" OR "diet history questionnaire*" OR "diet questionnaire*" OR "diet intake questionnaire*") AND ("valid*" OR "validat*" OR "psychometric*" OR "content valid*" OR "face valid*" OR "construct valid*" OR "discriminant valid*" OR "convergent valid*" OR "factor analys*"))
AND ("valid*" OR "validat*" OR "psychometric*" OR "content valid*" OR "face valid*" OR "construct valid*" OR "discriminant valid*" OR "convergent valid*" OR "factor analys*" OR "exploratory factor analysis" OR "confirmatory factor analysis" OR EFA OR CFA OR PCA OR "principal component*")
Inclusion Criteria
| A | B | |
| Category | Criteria | |
| Population / Study Type | Empirical studies that developed a new Food Frequency Questionnaire (FFQ) from scratch for assessing dietary intake in humans. | |
| Instrument Development Stage | The FFQ must have been conceptually developed (item generation, food list creation, portion size determination) rather than adapted from an existing FFQ. | |
| Psychometric Evaluation | Studies that assessed at least one form of validity (content or construct) | |
| Content Validity | Studies reporting expert review, cognitive interviewing, pilot testing, or participant feedback for item comprehensibility. | |
| Construct Validity | Studies reporting statistical or theoretical evidence for construct validity (e.g., factor analysis, hypothesis testing, structural validity). | |
| Population Type | Any human population (adults, children, specific patient groups, or general populations). | |
| Study Design | Quantitative, mixed-methods, or methodological studies focusing on FFQ development and psychometric evaluation. | |
| Publication Type | Peer-reviewed journal articles, dissertations, or reports published in English. |
Exclusion Criteria
| A | B | |
| Category | Criteria | |
| Adapted / Translated Instruments | Studies that adapted, translated, or culturally modified an existing FFQ without de novo item development. | |
| Criterion-Only Validation | Studies that assessed only criterion validity (e.g., correlation with 24-h recall or biomarkers) without content or construct validity assessment. | |
| Reliability-Only Studies | Studies that measured only reliability without validity assessment. | |
| Reviews / Protocols / Commentaries | Narrative reviews, systematic reviews, scoping reviews, or editorials without primary data. | |
| Non-human or Laboratory Studies | Studies involving animal populations or in vitro analysis. | |
| Incomplete Psychometric Reporting | Studies that developed FFQ but did not report any formal assessment of validity or reliability. |
Data Extraction Table
| A | B | C | |
| Data Item | Description/What to Extract | COSMIN Domain | |
| Study Identification | Author(s), year, country, population characteristics, study setting | General Study Info | |
| FFQ Purpose & Population | Intended use of FFQ (general diet, specific nutrient/food group), target group | Instrument Description | |
| FFQ Development Process | Item generation, food list development, portion size determination, expert review | Development Process | |
| Cognitive Testing or Comprehensibility | Methods used for pre-testing, pilot studies, cognitive interviews | Content Validity | |
| Content Validity | Expert panel review, relevance, comprehensiveness, clarity, methods used and findings | Content Validity | |
| Construct Validity | Statistical tests (e.g., factor analysis, hypothesis testing), structure validity, results | Construct Validity | |
| Measurement Error (if reported) | Comparison with biomarker or reference, limits of agreement | Measurement Error | |
| Feasibility/Acceptability | Time to complete, burden, participant feedback | Feasibility | |
| Main Findings and Conclusions | Summary of psychometric results and authors’ conclusions | Synthesis | |
| Quality Appraisal (if applied) | Criteria or checklist used to assess study quality | Appraisal | |
| Funding/Conflict of Interest | Funding sources and potential conflicts | General Info |