What content analysis is for
Content analysis is a research technique for making replicable and valid inferences from texts, and other meaningful material, about the contexts in which they were produced and used (Krippendorff, 2004). The texts can be newspaper articles, policy documents, interview transcripts, social media posts, websites or images. What makes it a method rather than a reading is that the categories and the rules for applying them are made explicit, so another researcher could repeat the coding and check it.
Researchers use it to describe the content of communication (how often a newspaper frames migration as a threat), to compare sources or periods (how university mission statements changed after a policy reform), and to infer something about producers or audiences (what a company's reports suggest about its priorities). The inference must stay within what texts can show: texts reveal what was said, not necessarily what was believed or done.
| Variant | What is coded | Typical output | Key source |
|---|---|---|---|
| Quantitative (conceptual) content analysis | The presence or frequency of predefined concepts | Counts, proportions, comparisons across groups or time | Krippendorff (2004) |
| Relational content analysis | Relationships between concepts: co-occurrence, proximity, direction or affect | Co-occurrence tables, networks, concept maps | Krippendorff (2004) |
| Qualitative content analysis: conventional | Categories developed from the data | Descriptive category system | Hsieh and Shannon (2005) |
| Qualitative content analysis: directed | Categories derived from an existing theory, then refined | Support for, or extension of, the theory | Hsieh and Shannon (2005) |
| Qualitative content analysis: summative | Keyword counts followed by interpretation of their use | Interpreted patterns of usage | Hsieh and Shannon (2005) |
| Automated content analysis | Dictionaries, supervised classifiers or topic models applied to large corpora | Scaled counts, classifications, topics | Grimmer and Stewart (2013) |
Content analysis overlaps with thematic analysis, but the emphasis differs. Content analysis aims for systematic, rule-based coding that can be checked for agreement; reflexive thematic analysis develops patterns of meaning through the researcher's interpretive engagement (Braun & Clarke, 2006). Qualitative content analysis sits between the two, distinguishing manifest content (what the text says) from latent content (what it means) (Graneheim & Lundman, 2004).
Running a conceptual analysis
A conceptual analysis establishes whether, and how often, chosen concepts appear in a body of text. The steps below are the core decisions; each should be recorded in your methods chapter.
- State the question and define the population of texts. Which documents, from which sources, over what period? Decide how you will sample them and why.
- Define the units. Sampling units are the documents you select; recording units are the parts you code (a word, a sentence, a paragraph, a whole article). Choose the unit that matches the concept: a frame is usually visible at article level, a term at word level (Krippendorff, 2004).
- Decide existence or frequency. Will you code whether a concept appears in a unit, or how many times? Frequency suits emphasis; existence suits presence across documents.
- Build the category system. Categories should be defined, mutually exclusive at the level you code, and exhaustive (add an "other" category if needed). Decide whether categories come from theory (deductive) or from the texts (inductive), or both (Hsieh & Shannon, 2005).
- Write the codebook. For each category give a definition, inclusion and exclusion rules, an example and a counter-example. State how to treat synonyms, irony, negation and irrelevant material.
- Pilot and revise. Code a small sample with a second coder, discuss disagreements and sharpen the rules.
- Test reliability, then code the full sample. See the reliability section below.
- Analyse and interpret. Summarise counts, compare groups or periods, and interpret the patterns against your question, without claiming more than texts can show.
- 1Question and texts
- 2Units
- 3Categories
- 4Codebook
- 5Pilot
- 6Reliability test
- 7Full coding
- 8Analysis
Running a relational analysis
A relational analysis goes further and examines how concepts relate to one another. The early steps are the same; the difference is in what is coded and how it is analysed.
- Co-occurrence and proximity. Code whether concepts appear together within a defined window, such as a sentence or a fixed number of words, and analyse which concepts cluster.
- Affect or evaluation. Code the direction of a relationship or the evaluative tone attached to a concept, for example whether a policy is described positively or negatively.
- Cognitive or concept mapping. Represent concepts and the coded relationships between them as a network or map, and compare maps across sources.
The choices that matter most are the definition of the window, the rules for coding direction, and how relationships are counted. Statistical summaries such as co-occurrence matrices or network measures are only as good as these rules.
Software can do much of this at scale, from dictionary counts to supervised classifiers and topic models. Automated methods are useful, but they do not replace reading: their categories must be checked against careful human coding before results are interpreted (Grimmer & Stewart, 2013).
Reliability and trustworthiness
Reliability in content analysis means that different coders, applying the same codebook to the same units, reach the same decisions. Two or more coders should code an overlapping sample independently. Percent agreement is easy to compute but misleading, because some agreement occurs by chance, and it becomes harder to interpret as the number of categories grows (Hayes & Krippendorff, 2007).
Hayes and Krippendorff (2007) set out criteria for a good reliability index: it should work for any number of coders, depend on the categories actually used, have meaningful end points, suit the level of measurement and have known sampling behaviour. They argue that Krippendorff's alpha meets these criteria and that commonly used alternatives, including Cohen's kappa, do not. Rather than fixing one threshold, they show how to estimate the probability that alpha falls below a chosen minimum, such as .800 or .667. Krippendorff's own guidance treats .800 as the usual standard and values between .667 and .800 as acceptable only for tentative conclusions (Krippendorff, 2004).
Reporting has improved but remains uneven. A review of 672 content analyses in three leading communication journals from 1985 to 2014 found more use of chance-corrected coefficients over time, and more testing of every variable, with Krippendorff's alpha the most common coefficient in the latest period; progress was uneven across journals and slipped slightly in the most recent period (Lovejoy et al., 2016).
In qualitative content analysis, where interpretation of latent meaning is central, reliability statistics may be less appropriate. Quality is then argued through trustworthiness: credibility, dependability and transferability (Lincoln & Guba, 1985; Graneheim & Lundman, 2004), built in through verification strategies during the analysis rather than checked only at the end (Morse et al., 2002).
Worked example (illustrative)
A doctoral researcher studies how national newspapers framed generative AI in universities in 2023 and 2024. The sampling unit is the article; the recording unit is also the article, because a frame is judged across the whole piece. The codebook defines four frames (opportunity, threat to integrity, regulation, other), each with rules and examples. Two coders independently code 50 articles. They agree on 46, which looks reassuring, but most articles fall into one frame, so much of that agreement could occur by chance. The researcher computes alpha with software, finds it below the chosen standard for the "regulation" frame, rewrites that category's rules, recodes a fresh sample and reports both rounds. These numbers are hypothetical and show the reasoning only.
What to report
| Element | What to state |
|---|---|
| Texts | Population, sampling frame, sampling method, number of documents and dates |
| Units | Sampling and recording units and why they fit the concepts |
| Categories | Source (theory, data or both), definitions and the full codebook in an appendix |
| Coders | Who coded, training, independence and how disagreements were resolved |
| Reliability | Coefficient used, value for each variable, size of the reliability sample |
| Software | Tools used and, for automated methods, how output was validated against human coding |
| Interpretation | How counts or relationships were linked back to the research question |
Journal reporting standards for qualitative research ask for similar transparency about the analytic process and the researcher's role (Levitt et al., 2018; O'Brien et al., 2014).
Common mistakes
- Coding without written rules. If the rules exist only in your head, nobody can check or repeat the coding.
- Reporting percent agreement alone. Use a chance-corrected coefficient and report it per variable (Hayes & Krippendorff, 2007; Lovejoy et al., 2016).
- Mismatched units. Counting words when the concept is a frame, or coding whole documents when the question is about sentences.
- Trusting automated output. Dictionaries and topic models need validation against human reading (Grimmer & Stewart, 2013).
- Over-reaching inferences. Texts show what was communicated; claims about intentions or effects need other evidence.
- Calling it content analysis when it is thematic analysis. Name the approach you actually used and cite its sources (Braun & Clarke, 2006; Hsieh & Shannon, 2005).
For text analysis in health research, see textual data analysis in medical research; for a step-by-step guide to the related method, see thematic analysis step by step.
Key takeaways
- Content analysis makes inferences from texts by applying explicit, checkable coding rules (Krippendorff, 2004).
- Choose between conventional, directed and summative qualitative content analysis according to the role of theory (Hsieh & Shannon, 2005).
- Use a chance-corrected reliability coefficient such as Krippendorff's alpha, not percent agreement alone (Hayes & Krippendorff, 2007).
- Report reliability for every coded variable, a step many published studies still miss (Lovejoy et al., 2016).
- Validate automated coding against careful human reading before interpreting it (Grimmer & Stewart, 2013).
