COREQ+LLM: User manual — item-by-item guidance

User manual

Item-by-item guidance for reporting the use of large language models in qualitative research: what each checklist item asks for, why it matters, and an example of suitable reporting language.

How to use this page Use the guidance below when preparing the methods and reporting sections of a qualitative research article. Each item explains what readers need to know, why the information matters for transparency and reproducibility, and gives an example of suitable reporting language.

Checklist item	How to understand and use this item	Example of reporting
1LLM Operator	Identify the person or people who actually operated the LLM during the study. Report their role in the research team and, where relevant, whether they were qualitative researchers, subject-matter experts, data scientists, research assistants, or part of an interdisciplinary team. The aim is to make clear who made the operational decisions when using the LLM.	Example “The LLM was operated by two qualitative researchers (one with expertise in health services research and one research assistant). A data scientist advised on the API implementation.”
2Credentials of the LLM Operators	Describe the relevant training and experience of the people who designed the prompts and workflows. Include expertise in qualitative methods, the substantive research domain, LLMs/prompt engineering, machine learning, or related technical skills. Report training that is relevant to the reliability of the LLM-supported analysis.	Example “The primary operator had five years of experience in thematic analysis and completed a university workshop on LLM-assisted qualitative research. The API workflow was developed with support from a machine-learning specialist.”
3LLM Role in Analysis	State exactly what the LLM did in the research process. Distinguish between roles such as transcription, data extraction, coding, theme generation, summarisation, comparison, or acting as an additional reviewer. If the LLM performed several functions, describe each one and indicate at which stage it was used.	Example “The LLM was used to generate preliminary codes and to suggest candidate themes. It did not make the final coding or theme decisions.”
4Human-LLM Interaction	Explain how responsibility was divided between researchers and the LLM. Indicate whether researchers directly checked outputs (human-in-the-loop), mainly validated the completed process (over-the-loop), supervised without systematically checking each output, or allowed the process to run without human intervention (out-of-the-loop). Use the category that best describes the actual workflow.	Example “Researchers reviewed every LLM-generated code before it was entered into the final coding framework; the process therefore used a human-in-the-loop approach.”
5Model Selection	Name every LLM used and describe its role. Report whether it was commercially hosted or open source, a standard fixed-weight model or a custom/fine-tuned model, and, for fine-tuned systems, identify the base model and the nature of the customization.	Example “We used GPT-4.1 via API for coding and Llama 3.1 70B, locally hosted, for an independent sensitivity analysis. Neither model was fine-tuned for this study.”
6Model Selection Rationale	Explain why the selected model(s) were appropriate for the study rather than simply stating which model was used. Relevant considerations may include language capability, context-window size, expected analytical performance, reproducibility, privacy, known limitations or biases, infrastructure, and cost.	Example “GPT-4.1 was selected because it supported the study languages and provided a sufficiently large context window. A local open-source model was considered but rejected because its performance on the pilot coding task was lower.”
7Model Version	Report the exact model/version available at the time of analysis. This is important because providers can change models or aliases over time. State whether more than one version was used and, where relevant, report the model’s stated training-data cutoff. If the exact version cannot be identified (for example, for a web interface that does not disclose it), report the date(s) of access instead.	Example “All analyses were conducted with GPT-4.1 (API model identifier: …) between March and April 2026. No model version changes occurred during the study.”
8External Data Integration	Report whether the LLM could retrieve or use information outside the text supplied directly in the prompt, including retrieval-augmented generation (RAG). Identify the external sources, how they were retrieved, and how they entered the workflow. This item is about additional information available to the model, not merely the research data itself.	Example “The coding model used RAG to retrieve passages from the study codebook stored in a vector database. Retrieved passages were inserted into the prompt before each coding request.”
9Tool Access	List tools or capabilities that could influence model outputs, such as web browsing, code execution, calculators, file search, APIs, databases, or other software integrations. Report whether these capabilities were actually used or merely available.	Example “The model had access to a Python code interpreter for descriptive calculations but did not have web access or external database access.”
10Accessing the LLM	Describe how researchers interacted with the model: web interface, desktop/mobile application, API, local deployment, or a custom application. If scripts were used, indicate whether the scripts or relevant implementation details are available as supplementary material or a repository.	Example “All model calls were made through the provider’s API using a Python script. The analysis script and prompt templates are provided in the supplementary repository.”
11Model Parameters	Report settings that can affect model behaviour and reproducibility, such as temperature, top-p, maximum output tokens, frequency/presence penalties, and other relevant parameters. Also report system-level instructions. If a parameter was left at the provider default, say so rather than omitting it.	Example “Temperature was set to 0, top-p was left at the API default, and maximum output tokens were set to 1,000. The complete system prompt is provided in Supplementary File 2.”
12Technical Infrastructure and Environment	Describe the technical environment sufficiently for another researcher to understand how the LLM workflow was implemented. Depending on the study, this may include programming language and version, libraries and versions, operating system, hardware, hosting environment, and dates of access. Focus on components that could affect execution or reproducibility.	Example “The workflow used Python 3.12 with the provider’s SDK version X.Y on Ubuntu 24.04. API calls were made from an institutional server in May 2026.”
13User Prompts	Provide the exact prompts where feasible. If there were many prompts, provide representative examples and explain how they varied. Explain the purpose of the main prompt components so readers can understand how instructions shaped the analysis. Store long prompt sets in supplementary material when appropriate.	Example “For each transcript segment, the prompt instructed the model to assign one or more codes from the predefined codebook and to quote evidence for each assignment. The full prompt template is provided in Supplementary File 3.”
14Language Considerations	State the language(s) used for prompts, source data, model outputs, and any translations. If translation occurred, describe who or what performed it and how accuracy was checked. In multilingual or multicultural studies, explain whether model performance was assessed across languages or cultural contexts.	Example “Interviews were conducted in German, while prompts and outputs were in English. Translations were generated with a separate translation workflow and checked by a bilingual researcher before coding.”
15Prompting Approach	Name and describe the prompting strategy used, for example single-shot, few-shot, multi-turn, segment-wise, iterative, or chain-of-thought-style prompting. Explain how the strategy was applied to the research task rather than simply naming it.	Example “Each transcript was processed segment by segment using a few-shot prompt containing three examples of correctly coded excerpts. The same template was used for all participants.”
16Advanced Prompting	Report additional instructions or contextual materials that shaped the model’s behaviour, beyond the overall strategy named in item 15. This includes personas or roles, required output formats, demonstration examples for in-context learning, codebooks, methodological guidance, or other documents supplied to the model.	Example “The model was instructed to act as a qualitative research assistant and return a JSON object containing the code, supporting quotation, and confidence rationale. The complete codebook was included in the prompt context.”
17Prompt Development	Explain how the prompts were created and whether they were fixed or adapted during analysis, as distinct from items 15–16, which describe what the final prompt contained. Report who developed them, what methodological principles informed them, and whether contextual information was used to dynamically modify prompts.	Example “The initial prompt was developed by two qualitative researchers based on the study’s coding framework. It was standardized before formal analysis and was not changed between participants.”
18Prompt Testing	Describe whether prompts were pilot tested and how their effectiveness was assessed. Report changes made as a result of testing and the criteria used, such as coding accuracy, completeness, consistency, usefulness, or agreement with human coding.	Example “The prompt was pilot tested on 50 excerpts not included in the final analysis. It was revised twice to reduce overly broad code assignments; performance was assessed against expert-coded excerpts.”
19Prompt-Response Iterations	Describe whether the model was prompted repeatedly to refine a single output within one analysis task (for example, asking it to improve one coding decision), and how later prompts depended on earlier outputs. State the stopping rule so readers know when the interaction ended and whether researchers or the model determined that point. This is distinct from whether the model retained conversation history (item 20) or reused outputs across different analytic steps (item 21).	Example “Each transcript segment could undergo up to three refinement rounds. Iteration stopped when the proposed codes matched the predefined coding criteria or after the third round.”
20Session Context	Clarify whether each interaction started from a clean session or whether the model retained previous conversation history. This is a technical property of how the model was called, independent of whether you deliberately iterated on one output (item 19) or reused earlier analytic outputs across workflow steps (item 21). It matters because previous prompts and outputs can influence later responses and therefore affect reproducibility.	Example “Every transcript segment was processed in a new API conversation with no previous messages in the context window.”
21Context Reuse During Analysis	Report whether outputs from earlier analytic steps (for example, a first coding pass) were deliberately passed into prompts for later steps (for example, theme identification), as part of the overall workflow design – distinct from conversational memory within one call (item 20) or iterative refinement of a single output (item 19). State whether the model maintained a running analysis across steps or whether every step was independent.	Example “After initial coding, the list of codes generated for each segment was supplied to a second prompt that asked the model to identify relationships among codes. Each later step therefore reused earlier outputs.”
22Provision of Research Context	Report what the model was told about the study beyond the immediate data being analysed. This may include the research question, objectives, theoretical framework, participant characteristics, study setting, inclusion criteria, or coding framework. State which contextual information was deliberately withheld as well.	Example “The model received the research question, study objectives, and operational definitions of the codes, but it was not given participants’ demographic information unless it appeared in the transcript segment.”
23Analysis Process	Explain how LLM outputs functioned analytically – that is, what evidentiary status they were given, as distinct from item 3, which reports which tasks the LLM performed. Distinguish between outputs treated as data in their own right, provisional interpretations, coding suggestions, summaries, or inputs to a subsequent human analysis. Make clear which outputs contributed to the reported findings.	Example “LLM-generated codes were treated as provisional analytical suggestions. The research team compared them with manual coding and only researcher-approved codes entered the final analysis.”
24Data Processing	Describe any transformation of qualitative data before, during, or after LLM processing. Include segmentation caused by context limits, cleaning, transcription correction, de-identification, chunking, vectorisation, dimensional reduction, or post-processing. Explain why the transformation was made and whether it could affect interpretation.	Example “Transcripts were divided into approximately 1,500-word segments to fit the model context window. Speaker labels and timestamps were retained; no stop words were removed.”
25Output Validation	Describe the concrete procedure used to check LLM outputs for accuracy, usefulness, consistency, and methodological adequacy – distinct from item 4, which reports the general oversight model used throughout the workflow. Report the proportion or sampling strategy for human checks, comparison with other data sources or coders, decision rules for disagreements, and any quantitative agreement statistics used. Also document important failure cases and how they were corrected.	Example “A second qualitative researcher independently reviewed 20% of all LLM-coded excerpts. Disagreements were resolved through discussion. The LLM omitted relevant codes in 7% of checked excerpts, and these cases were used to refine the prompt.”
26Output Variability	Assess whether repeated runs with the same or equivalent input produced stable results. If repeated runs were conducted, report how many, under what settings, and whether findings were reproducible or varied. This is especially important when model settings allow stochastic outputs.	Example “A random sample of 100 excerpts was processed three times. Core codes were reproduced in 86% of cases; 14% showed additional or different secondary codes.”
27Bias Mitigation	Describe how the team looked for hallucinations, systematic omissions, stereotyping, demographic bias, or other distortions, and what was done to reduce them. Report both the procedure and the observed results rather than simply stating that bias was considered.	Example “Researchers compared LLM coding across excerpts referring to different age groups and checked for systematic differences in code assignment. Prompt refinements and human review were introduced when a recurring age-related bias was identified.”
28Reflexivity	Reflect on how using the LLM may have shaped the researchers’ interpretations, coding decisions, or theoretical conclusions. Discuss potential benefits as well as risks, such as anchoring on model-generated themes, automation bias, or changes in what researchers noticed in the data.	Example “The team noted that early LLM-generated themes initially influenced subsequent discussion. To reduce anchoring, researchers completed an independent manual coding pass before reviewing the model’s thematic suggestions.”
29Data Privacy and Security	Explain how participant data were protected throughout LLM use. Report de-identification or anonymisation, what data were transmitted, where inputs and outputs were stored, access controls, retention settings, and whether provider policies allowed data to be used for model training. Include additional security measures relevant to the LLM workflow.	Example “Names and other direct identifiers were removed before API submission. Data were transmitted through an institutional account with training on submitted data disabled, and outputs were stored on an encrypted institutional drive.”
30Conflict of Interest Disclosure	Disclose financial, professional, or institutional relationships between members of the research team and the LLM developer, distributor, or related commercial entities. Include funding, consulting, employment, advisory roles, or other relationships that could influence model selection or interpretation.	Example “One author received research funding from the developer of the LLM used in the study. No author was employed by the company, and the funder had no role in model selection, analysis, or interpretation.”
31Informed Consent	Explain whether and how participants were informed that an LLM would be involved in processing their data, and whether the consent and ethics procedures covered this use. For secondary data, state whether re-consent was considered or required. Where relevant, report whether participants could understand or access the technology sufficiently for meaningful consent or validation.	Example “The participant information sheet stated that de-identified interview transcripts would be analysed with an LLM-assisted workflow. The institutional ethics approval explicitly covered this procedure, and no participant-identifiable information was submitted to the model.”