Statistics For Historians

We are pleased to offer scholars and students of history access to two free online modules providing instruction in digital humanities methods and techniques. This second course introduces historians to statistical methods and quantitative analysis tools that can be used alone or in conjunction with text mining, detailed in the first course. The first course is available here.

Each learning module provides a video explaining the content and some activities to help you develop your skills. It also shows you how you can use statistics and quantitative methods in your own historical research.

These two courses were commissioned as a part of the research of the International Standing Working Group in Medialization and Empowerment at the German Historical Institute London. They are designed to build skills in the digital humanities, and also to help course-takers develop the confidence and ability to use text mining and the quantitative techniques often required to interpret its results in their own historical research.

The course leaders are Dr. Luke Blaxill of the University of Oxford, and Dr. Kaspar Beelen of the Turing Institute. Responsibility for the content lies with the developers.

Unit 1: Introduction to Statistics For Historians

Key Readings

• Konrad Jarausch and Kenneth Hardy, Quantitative Methods for Historians: A Guide to Research, Data, & Statistics (UNC Press, 1991).
• Roderick Floud, An Introduction to Quantitative Methods for Historians (Routledge, 1973).
• Paul Kellstedt and Guy Whitten, The Fundamentals of Political Science Research (Cambridge, 2008).
• Robert Fogel and Geoffrey Elton, Which Road to the Past? Two Views of History (Yale University Press, 1984). [This book is a debate between an outspoken quantifier, and a historian critical of quantification]

Exercises

Alone, or with a friend, consider when you have used quantitative analysis in your own research. If you have, was there anything further you think the data might have been able to tell you? What sort of research questions would a social scientist – working with that same data – have been interested in? If you have not used statistical analysis before, are there any  research questions that you would like to study that might benefit from it?

---

Unit 2: Basic Descriptive Statistics

Key Readings

• Descriptive Statistics Knowledge Base
• Doctoral Journey Descriptive Statistics Resources

---

Unit 3: Statistical Significance & Regression

Key Readings

• Farooq Sabri and Tracey Gyateng, ‘Understanding Statistical Significance: A Short Guide’. Link
• ‘A Researcher’s Guide to Statistical Significance’. Link
• ‘Researcher’s Introduction to Linear Regression’ Link

Exercises

Alone, or with a friend, consider what sorts of historical statistical analysis is made possible with significance and regression? Do you work with incomplete datasets where it would be useful to make projections and predictions for data you do not have? How widespread are the use of these inferential statistics in your field? If they are seldom used, what are the challenges to using them in a field where many scholars are not very statistically minded?

---

Unit 4: Introduction to SPSS Statistics

Key Readings

• SPSS for Data Analysis (a much more extensive video data analysis lecture series). Link
• Jason Delaney, Excel Data Analysis Series (walkthroughs of doing most of this analysis – and more – with Excel rather than SPSS). Link

Exercises

Choose a sample SPSS data study source file (for example from here). Run a basic descriptive statistics command, correlation analysis, and created both a single and grouped scatter. With the former, add a trendline and observe the linear regression equation. You could even perform a regression based on a hypothetical X or Y value.

---

Unit 5: Explorative Data Analysis

This unit is composed of two lectures which will introduce you to DataFrames with Pandas.

Lecture A: Exploring DataFrames with Pandas (Part I)
This session runs in an interactive notebook on MyBinder. Click for more information on how to access and run a notebook. An overview of all interactive materials is available here. 

This notebook introduces the Pandas library and explores tools for working programmatically with tabular data. We have a closer look at realistic and complex metadata derived from the British Library catalogue and demonstrate how you can refine and reorganize information with the goals of studying trends over time.

Lecture B: Exploring DataFrames with Pandas (Part II)
This session runs in an interactive notebook on MyBinder. Click for more information on how to access and run a notebook. An overview of all interactive materials is available here. 

This notebook uses  ‘synthetic’ demographic data about age and gender in late Victorian London. We discuss different types of variables and strategies for visualizing distributions. We proceed with summarising information using descriptive statistics, such as mean and median. From a historical point of view, we investigate whether men are generally younger than women in late-Victorian London.

Key Readings

• Konrad Jarausch and Kenneth Hardy, Quantitative Methods for Historians: A Guide to Research, Data, & Statistics (UNC Press, 1991).
• Roderick Floud, An Introduction to Quantitative Methods for Historians (Routledge, 1973).
• Paul Kellstedt and Guy Whitten, The Fundamentals of Political Science Research (Cambridge, 2008).
• Robert Fogel and Geoffrey Elton, Which Road to the Past? Two Views of History (Yale University Press, 1984). [This book is a debate between an outspoken quantifier, and historian critical of quantification]

Exercises

Integrated into the lectures above.

---

Unit 6: Hypothesis Testing

Distributions and Hypothesis Testing

This session runs in an interactive notebook on MyBinder. Click for more information on how to access and run a notebook. An overview of all interactive materials is available here.

In this section, we move from descriptive to inferential statistics. We assess the statistical ‘significance’ of the gendered differences observed in the previous notebook (on descriptive statistics). We pursue a data-driven and intuitive approach to significance testing. First, We ‘bootstrap’ confidence intervals and then explore permutation for hypothesis testing.​

Key Readings

• Eric Matthes, Python Crash Course: A Hands-On, Project-Based Introduction to Programming (No Starch Press, 2019).
• Nick Montfort, Exploratory Programming for the Arts and Humanities (MIT Press, 2021).
• Al Sweigart, Automate the Boring Stuff with Python: Practical Programming for Total Beginners (No Starch Press, 2019).
• Peter Wentworth, Jeffrey Elkner, Allen B. Downey, and Chris Meyer, ‘How to Think like a Computer Scientist: Learning with Python 3’ (2015).

Exercises

Integrated into the lectures above.

---

Unit 7: (More) Advanced Regression

Lecture A: Further Correlation and Linear Regression

This session runs in an interactive notebook on MyBinder. Click for more information on how to access and run a notebook. An overview of all interactive materials is available here.

This session has a closer look at modelling the relation between different variables. The first notebook (click here) discusses how to compute and interpret correlation coefficients and then continue with a gentle introduction to linear regression. The goal is to understand variation in lifespans in late-Victorian London. We try to understand if residents in more affluent boroughs tend to live longer.

Lecture B: Generalised Linear Models

This session runs in an interactive notebook on MyBinder. Click for more information on how to access and run a notebook. An overview of all interactive materials is available here.

The second notebook on linear regression turns to more advanced techniques: Generalised Linear Models (GLMs). We use GLMs to model and predict count outcomes. We explore two case studies in detail: a) gender bias in university applications and b) gender and participation in the British House of Commons.

Key Readings

• Konrad Jarausch and Kenneth Hardy, Quantitative Methods for Historians: A Guide to Research, Data, & Statistics (UNC Press, 1991).
• Roderick Floud, An Introduction to Quantitative Methods for Historians (Routledge, 1973).
• Paul Kellstedt and Guy Whitten, The Fundamentals of Political Science Research (Cambridge, 2008).
• Robert Fogel and Geoffrey Elton, Which Road to the Past? Two Views of History (Yale University Press, 1984). [This book is a debate between an outspoken quantifier, and historian critical of quantification]

Exercises

Integrated into the lectures above.

---

Downloads

The five ‘Exercise Corpora’ below have been compiled and segmented by us from publicly available textual historical archives. They support the exercises for both courses and can be downloaded in full here.

 

Medical Officers of Health in London, 1848-1972
Medical Officers of Health (MOH) were appointed to investigate the health of the population, sanitary conditions, disease, housing, and clinical services in each London borough. Our corpus enables comparisons between interwar and Victorian MOH, as well as a wealthy borough (Westminster) and a poor one (Poplar).

 

British House of Commons Debates, 1945-2014
Taken from the official record of Parliament (Hansard). Our corpus enables comparisons between parties, ministers, and male vs. female MPs. The debates we have selected concern the issue of abortion law.

 

Heritage Made Digital Newspapers, 1800-1880
Nineteenth-century British articles from numerous newspapers. We have created two sub corpora: all of the articles containing the the word ‘slavery’ and all of the articles containing ‘workhouse’. Subdivided by decades that saw key legislation and campaigns on these issues.

 

British Election Manifestos, 1966-2019
The printed national manifestos of the Liberal, Labour, and Conservative parties in every general election from 1966 to the present. We have set this corpus up to enable comaprisons between parties, and between two key decades: the 1960s (1964, 1966, 1970 elections) and the 1980s (1979, 1983, 1987 elections).

 

The Times Headlines from the 1960s onwards
The Times is often credited as being Britain’s national ‘newspaper of record’. This corpus is setup as a csv file and contains every headline from every day.

---

---