How-to articleMethodology & analyticsAll disciplines

Content analysis step by step: conceptual and relational analysis, coding rules and reliability

Content analysis turns texts into systematic evidence by coding them with explicit rules. This guide explains what the method is for, how quantitative and qualitative versions differ, how to run a conceptual or relational analysis, and how to show that your coding is reliable.

What you will learn

  • Explain what content analysis is for and how it differs from thematic analysis
  • Choose between quantitative, qualitative, conceptual and relational content analysis
  • Define sampling and recording units and write a codebook with clear coding rules
  • Assess intercoder reliability with a chance-corrected coefficient and report it properly
  • Validate computer-assisted or automated coding against human reading

What you will unlearn

  • Content analysis is just counting words.Quantitative content analysis counts coded categories, and qualitative content analysis interprets meaning; both rest on explicit coding rules.
  • Content analysis cannot be automated.Computer-assisted and automated methods are well established, but their output must be validated against human coding.
  • Percent agreement shows that coding is reliable.Percent agreement ignores chance agreement; a chance-corrected coefficient such as Krippendorff's alpha is needed.
  • Content analysis and thematic analysis are the same.They overlap, but content analysis emphasises systematic, rule-based coding, while reflexive thematic analysis develops interpretive themes.

Skills you will practise

  • Unitising texts
  • Codebook design
  • Coding rules
  • Intercoder reliability
  • Relational analysis
  • Validating automated coding
  • Transparent reporting

What content analysis is for

Content analysis is a research technique for making replicable and valid inferences from texts, and other meaningful material, about the contexts in which they were produced and used (Krippendorff, 2004). The texts can be newspaper articles, policy documents, interview transcripts, social media posts, websites or images. What makes it a method rather than a reading is that the categories and the rules for applying them are made explicit, so another researcher could repeat the coding and check it.

Researchers use it to describe the content of communication (how often a newspaper frames migration as a threat), to compare sources or periods (how university mission statements changed after a policy reform), and to infer something about producers or audiences (what a company's reports suggest about its priorities). The inference must stay within what texts can show: texts reveal what was said, not necessarily what was believed or done.

Table 1. Variants of content analysis and what each produces
VariantWhat is codedTypical outputKey source
Quantitative (conceptual) content analysisThe presence or frequency of predefined conceptsCounts, proportions, comparisons across groups or timeKrippendorff (2004)
Relational content analysisRelationships between concepts: co-occurrence, proximity, direction or affectCo-occurrence tables, networks, concept mapsKrippendorff (2004)
Qualitative content analysis: conventionalCategories developed from the dataDescriptive category systemHsieh and Shannon (2005)
Qualitative content analysis: directedCategories derived from an existing theory, then refinedSupport for, or extension of, the theoryHsieh and Shannon (2005)
Qualitative content analysis: summativeKeyword counts followed by interpretation of their useInterpreted patterns of usageHsieh and Shannon (2005)
Automated content analysisDictionaries, supervised classifiers or topic models applied to large corporaScaled counts, classifications, topicsGrimmer and Stewart (2013)

Content analysis overlaps with thematic analysis, but the emphasis differs. Content analysis aims for systematic, rule-based coding that can be checked for agreement; reflexive thematic analysis develops patterns of meaning through the researcher's interpretive engagement (Braun & Clarke, 2006). Qualitative content analysis sits between the two, distinguishing manifest content (what the text says) from latent content (what it means) (Graneheim & Lundman, 2004).

Running a conceptual analysis

A conceptual analysis establishes whether, and how often, chosen concepts appear in a body of text. The steps below are the core decisions; each should be recorded in your methods chapter.

  1. State the question and define the population of texts. Which documents, from which sources, over what period? Decide how you will sample them and why.
  2. Define the units. Sampling units are the documents you select; recording units are the parts you code (a word, a sentence, a paragraph, a whole article). Choose the unit that matches the concept: a frame is usually visible at article level, a term at word level (Krippendorff, 2004).
  3. Decide existence or frequency. Will you code whether a concept appears in a unit, or how many times? Frequency suits emphasis; existence suits presence across documents.
  4. Build the category system. Categories should be defined, mutually exclusive at the level you code, and exhaustive (add an "other" category if needed). Decide whether categories come from theory (deductive) or from the texts (inductive), or both (Hsieh & Shannon, 2005).
  5. Write the codebook. For each category give a definition, inclusion and exclusion rules, an example and a counter-example. State how to treat synonyms, irony, negation and irrelevant material.
  6. Pilot and revise. Code a small sample with a second coder, discuss disagreements and sharpen the rules.
  7. Test reliability, then code the full sample. See the reliability section below.
  8. Analyse and interpret. Summarise counts, compare groups or periods, and interpret the patterns against your question, without claiming more than texts can show.
  1. 1Question and texts
  2. 2Units
  3. 3Categories
  4. 4Codebook
  5. 5Pilot
  6. 6Reliability test
  7. 7Full coding
  8. 8Analysis
Figure 1. The decisions in a content analysis study, redrawn from the steps above.

Running a relational analysis

A relational analysis goes further and examines how concepts relate to one another. The early steps are the same; the difference is in what is coded and how it is analysed.

  • Co-occurrence and proximity. Code whether concepts appear together within a defined window, such as a sentence or a fixed number of words, and analyse which concepts cluster.
  • Affect or evaluation. Code the direction of a relationship or the evaluative tone attached to a concept, for example whether a policy is described positively or negatively.
  • Cognitive or concept mapping. Represent concepts and the coded relationships between them as a network or map, and compare maps across sources.

The choices that matter most are the definition of the window, the rules for coding direction, and how relationships are counted. Statistical summaries such as co-occurrence matrices or network measures are only as good as these rules.

Software can do much of this at scale, from dictionary counts to supervised classifiers and topic models. Automated methods are useful, but they do not replace reading: their categories must be checked against careful human coding before results are interpreted (Grimmer & Stewart, 2013).

Reliability and trustworthiness

Reliability in content analysis means that different coders, applying the same codebook to the same units, reach the same decisions. Two or more coders should code an overlapping sample independently. Percent agreement is easy to compute but misleading, because some agreement occurs by chance, and it becomes harder to interpret as the number of categories grows (Hayes & Krippendorff, 2007).

Hayes and Krippendorff (2007) set out criteria for a good reliability index: it should work for any number of coders, depend on the categories actually used, have meaningful end points, suit the level of measurement and have known sampling behaviour. They argue that Krippendorff's alpha meets these criteria and that commonly used alternatives, including Cohen's kappa, do not. Rather than fixing one threshold, they show how to estimate the probability that alpha falls below a chosen minimum, such as .800 or .667. Krippendorff's own guidance treats .800 as the usual standard and values between .667 and .800 as acceptable only for tentative conclusions (Krippendorff, 2004).

Reporting has improved but remains uneven. A review of 672 content analyses in three leading communication journals from 1985 to 2014 found more use of chance-corrected coefficients over time, and more testing of every variable, with Krippendorff's alpha the most common coefficient in the latest period; progress was uneven across journals and slipped slightly in the most recent period (Lovejoy et al., 2016).

In qualitative content analysis, where interpretation of latent meaning is central, reliability statistics may be less appropriate. Quality is then argued through trustworthiness: credibility, dependability and transferability (Lincoln & Guba, 1985; Graneheim & Lundman, 2004), built in through verification strategies during the analysis rather than checked only at the end (Morse et al., 2002).

Worked example (illustrative)

A doctoral researcher studies how national newspapers framed generative AI in universities in 2023 and 2024. The sampling unit is the article; the recording unit is also the article, because a frame is judged across the whole piece. The codebook defines four frames (opportunity, threat to integrity, regulation, other), each with rules and examples. Two coders independently code 50 articles. They agree on 46, which looks reassuring, but most articles fall into one frame, so much of that agreement could occur by chance. The researcher computes alpha with software, finds it below the chosen standard for the "regulation" frame, rewrites that category's rules, recodes a fresh sample and reports both rounds. These numbers are hypothetical and show the reasoning only.

What to report

Table 2. Reporting checklist for a content analysis
ElementWhat to state
TextsPopulation, sampling frame, sampling method, number of documents and dates
UnitsSampling and recording units and why they fit the concepts
CategoriesSource (theory, data or both), definitions and the full codebook in an appendix
CodersWho coded, training, independence and how disagreements were resolved
ReliabilityCoefficient used, value for each variable, size of the reliability sample
SoftwareTools used and, for automated methods, how output was validated against human coding
InterpretationHow counts or relationships were linked back to the research question

Journal reporting standards for qualitative research ask for similar transparency about the analytic process and the researcher's role (Levitt et al., 2018; O'Brien et al., 2014).

Common mistakes

  • Coding without written rules. If the rules exist only in your head, nobody can check or repeat the coding.
  • Reporting percent agreement alone. Use a chance-corrected coefficient and report it per variable (Hayes & Krippendorff, 2007; Lovejoy et al., 2016).
  • Mismatched units. Counting words when the concept is a frame, or coding whole documents when the question is about sentences.
  • Trusting automated output. Dictionaries and topic models need validation against human reading (Grimmer & Stewart, 2013).
  • Over-reaching inferences. Texts show what was communicated; claims about intentions or effects need other evidence.
  • Calling it content analysis when it is thematic analysis. Name the approach you actually used and cite its sources (Braun & Clarke, 2006; Hsieh & Shannon, 2005).

For text analysis in health research, see textual data analysis in medical research; for a step-by-step guide to the related method, see thematic analysis step by step.

Key takeaways

  • Content analysis makes inferences from texts by applying explicit, checkable coding rules (Krippendorff, 2004).
  • Choose between conventional, directed and summative qualitative content analysis according to the role of theory (Hsieh & Shannon, 2005).
  • Use a chance-corrected reliability coefficient such as Krippendorff's alpha, not percent agreement alone (Hayes & Krippendorff, 2007).
  • Report reliability for every coded variable, a step many published studies still miss (Lovejoy et al., 2016).
  • Validate automated coding against careful human reading before interpreting it (Grimmer & Stewart, 2013).
Your next step

Want to apply this to your own study?

  1. 1Tell us your topic, data and deadline
  2. 2A subject mentor reviews your work with you
  3. 3You get a plan and a quote
Qualitative methodology

References

  1. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
  2. Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. https://doi.org/10.1016/j.nedt.2003.10.001
  3. Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21(3), 267–297. https://doi.org/10.1093/pan/mps028
  4. Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89. https://www.asc.upenn.edu/sites/default/files/2021-03/Answering%20the%20Call%20for%20a%20Standard%20Reliability%20Measure%20for%20Coding%20Data.pdf
  5. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
  6. Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Sage.
  7. Levitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 26–46. https://doi.org/10.1037/amp0000151
  8. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage.
  9. Lovejoy, J., Watson, B. R., Lacy, S., & Riffe, D. (2016). Three decades of reliability in communication content analyses: Reporting of reliability statistics and coefficient levels in three top journals. Journalism & Mass Communication Quarterly, 93(4), 1135–1159. https://doi.org/10.1177/1077699016644558
  10. Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13–22. https://doi.org/10.1177/160940690200100202
  11. O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/ACM.0000000000000388

How to cite this article

Choose your referencing style, then copy. Check your university's or journal's guide for local variations.

PhD Research Lab Academy. (2020, November 25). Content analysis step by step: conceptual and relational analysis, coding rules and reliability. PhD Research Lab. https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis/

PhD Research Lab Academy. “Content Analysis Step by Step: Conceptual and Relational Analysis, Coding Rules and Reliability.” PhD Research Lab, 25 Nov. 2020, www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis. Accessed [access date].

PhD Research Lab Academy (2020) Content analysis step by step: conceptual and relational analysis, coding rules and reliability. PhD Research Lab. Available at: https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis/ (Accessed: [access date]).

PhD Research Lab Academy. “Content Analysis Step by Step: Conceptual and Relational Analysis, Coding Rules and Reliability.” PhD Research Lab. November 25, 2020. https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis.

PhD Research Lab Academy, ‘Content Analysis Step by Step: Conceptual and Relational Analysis, Coding Rules and Reliability’ (PhD Research Lab, 25 November 2020) <https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis> accessed [access date].

PhD Research Lab Academy. Content analysis step by step: conceptual and relational analysis, coding rules and reliability [Internet]. PhD Research Lab; 2020 Nov 25 [cited [access date]]. Available from: https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis/

PhD Research Lab Academy, “Content analysis step by step: conceptual and relational analysis, coding rules and reliability,” PhD Research Lab, Nov. 25, 2020. [Online]. Available: https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis. [Accessed: [access date]].

@online{what_is_the_purpose_of_textual_content_analysis_in_research_give_a_brief_guide_to_new_researchers_what_are_the_steps_involved_in_content_analysis_2020,
  author = {{PhD Research Lab Academy}},
  title = {{Content analysis step by step: conceptual and relational analysis, coding rules and reliability}},
  organization = {PhD Research Lab},
  date = {2020-11-25},
  url = {https://www.phdassistance.com/blog/what-is-the-purpose-of-textual-content-analysis-in-research-give-a-brief-guide-to-new-researchers-what-are-the-steps-involved-in-content-analysis},
  urldate = {[access date]}
}

This article is for learning. Use it to understand the method and cite the original sources above in your own work; do not copy or submit it as your own.

Content analysis step by stepNeed help applying it to your study?
Qualitative methodology