Introduction

This report provides an analysis of BBC Radio 4’s “In Our Time” program, examining episode classifications, speaker statistics, and temporal trends across the program’s history. The analysis covers data collection methodology, classification procedures, overall statistics, and detailed visualizations of episode counts by topic category.

Data Gathering

The data for this analysis was collected through web scraping of the BBC Radio 4 “In Our Time” episode guide. The process involved systematically extracting episode metadata from the BBC website.

Web Scraping. The data collection process (scrape_episodes.py) extracts episode dates (initially in DD/MM/YYYY format from the BBC website), titles, URLs, and speaker/guest information from the BBC Radio 4 “In Our Time” episode guide (https://www.bbc.co.uk/programmes/b006qykl/episodes/guide). The scraper uses Python’s requests and BeautifulSoup libraries, with rate limiting (0.5s for guide pages, 0.3s for episode pages) to be respectful to BBC servers.

Data Structure. The scraping process generated two output formats: JSON format (in_our_time_episodes.json) with structured data and nested speaker arrays, and CSV format (in_our_time_episodes.csv) for easy import. Dates are stored in YYYY-MM-DD format (ISO 8601 standard), with “Unknown” used for missing dates. The dates are converted from the DD/MM/YYYY format extracted from the BBC website to YYYY-MM-DD format for standardized storage and better compatibility with analysis tools. Some episodes may not have speaker information available, especially future/upcoming episodes.

Data Enhancement. After initial scraping, two scripts enhance the episode data: (1) extract_missing_dates.py - identifies episodes with “Unknown” dates and extracts actual broadcast dates by visiting individual episode pages, normalizes dates to DD/MM/YYYY format during extraction, and updates both JSON and CSV files (dates are then converted to YYYY-MM-DD format for storage); (2) extract_holiday_dates.py - specifically targets episodes that aired on special holiday dates (Boxing Day/26th December, Christmas Eve/24th December, New Year’s Day/1st January) by searching episode pages for holiday references and extracting the corresponding year and date. A conversion script (convert_date_format.py) converts all dates from DD/MM/YYYY to YYYY-MM-DD format for standardized storage.

Processing Pipeline. The complete data processing workflow consists of: (1) Data Collection (scrape_episodes.py) - scrapes episode guide pages, extracts dates/titles/URLs (dates initially in DD/MM/YYYY format from BBC website), visits individual episode pages for speaker information; (2) Data Enhancement (extract_missing_dates.py, extract_holiday_dates.py) - extracts missing broadcast dates, including special holiday dates; (3) Date Conversion (convert_date_format.py) - converts all dates from DD/MM/YYYY to YYYY-MM-DD format (ISO 8601) for standardized storage; (4) Speaker Analysis (analyze_speakers.py or generate_all_csv_files.py) - extracts and cleans speaker names, counts appearances, generates statistics; (5) Statistics Generation (generate_statistics.py) - analyzes speaker and episode statistics; (6) Classification and Visualization (analyze_in_our_time.R) - classifies episode titles, generates temporal analysis and facet charts (parses dates in YYYY-MM-DD format); (7) HTML Report Generation (generate_sortable_table.R) - creates interactive sortable HTML table with all episodes, classifications, dates (in YYYY-MM-DD format), and speaker information.

Output Files. The complete data pipeline produces primary data files (in_our_time_episodes.json, in_our_time_episodes.csv), processed data files (in_our_time_speakers.csv, title_classifications.csv, classifications_by_year.csv, speaker_appearance_distribution.csv, speaker_appearances_by_year.csv, in_our_time_plot_data.csv), report files (STATISTICS_REPORT.md, classifications_by_year_table.md, in_our_time_episodes_table.html), and visualization files (three facet plot PNG files).

# Load the episode data
# Note: This handles both cases - when Rmd is rendered from project root 
# (file in current dir) or from working/ directory (file in parent dir)
json_path <- if (file.exists("../in_our_time_episodes.json")) {
  "../in_our_time_episodes.json"
} else if (file.exists("in_our_time_episodes.json")) {
  "in_our_time_episodes.json"
} else {
  stop("Cannot find in_our_time_episodes.json. Make sure you're running from the project root or working/ directory.")
}
cat("Loading JSON from:", json_path, "\n")

## Loading JSON from: ../in_our_time_episodes.json

episodes_raw <- fromJSON(json_path, simplifyDataFrame = TRUE)

cat("Total episodes loaded:", nrow(episodes_raw), "\n")

## Total episodes loaded: 1062

# Deduplicate episodes - remove Archive Episodes and Summer Repeats
# Keep the earliest occurrence of each episode
normalize_title <- function(title) {
  if (is.null(title) || is.na(title) || title == "") return(title)
  # Remove duplicate markers (case-insensitive)
  normalized <- str_replace_all(title, regex("\\(Archive\\s+Episode\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(archive\\s+episode\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(Summer\\s+Repeat\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(summer\\s+repeat\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(Repeat\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(repeat\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(Archive\\)", ignore_case = TRUE), "")
  normalized <- str_replace_all(normalized, regex("\\(archive\\)", ignore_case = TRUE), "")
  # Clean up extra whitespace
  normalized <- str_trim(str_replace_all(normalized, "\\s+", " "))
  return(normalized)
}

# Parse dates for comparison
parse_date_for_comparison <- function(date_str) {
  if (is.null(date_str) || is.na(date_str) || date_str == "" || date_str == "Unknown") {
    return(as.Date("9999-12-31"))  # Put unknown dates at the end
  }
  # Try YYYY-MM-DD format first
  if (str_detect(date_str, "^\\d{4}-\\d{2}-\\d{2}$")) {
    result <- suppressWarnings(ymd(date_str))
    if (!is.na(result)) return(result)
  }
  # Try DD/MM/YYYY format
  if (str_detect(date_str, "^\\d{2}/\\d{2}/\\d{4}$")) {
    result <- suppressWarnings(dmy(date_str))
    if (!is.na(result)) return(result)
  }
  return(as.Date("9999-12-31"))  # Put unparseable dates at the end
}

# Deduplicate
cat("Deduplicating episodes...\n")

## Deduplicating episodes...

seen_titles <- list()
keep_indices <- c()
duplicates_removed <- 0

for (i in 1:nrow(episodes_raw)) {
  title <- episodes_raw$title[i]
  normalized_title <- normalize_title(title)
  date_str <- episodes_raw$date[i]
  date_obj <- parse_date_for_comparison(date_str)
  
  if (normalized_title %in% names(seen_titles)) {
    # We've seen this title before - compare dates
    existing <- seen_titles[[normalized_title]]
    if (date_obj < existing$date) {
      # Current episode is earlier - replace the existing one
      keep_indices <- keep_indices[keep_indices != existing$index]
      keep_indices <- c(keep_indices, i)
      seen_titles[[normalized_title]] <- list(index = i, date = date_obj)
      duplicates_removed <- duplicates_removed + 1
    } else {
      # Existing episode is earlier - skip current
      duplicates_removed <- duplicates_removed + 1
    }
  } else {
    # First time seeing this title
    keep_indices <- c(keep_indices, i)
    seen_titles[[normalized_title]] <- list(index = i, date = date_obj)
  }
}

episodes <- episodes_raw[keep_indices, ]
cat("Duplicates removed:", duplicates_removed, "\n")

## Duplicates removed: 0

cat("Final episode count:", nrow(episodes), "\n")

## Final episode count: 1062

# Display basic information about the dataset
cat("Date range:", min(episodes$date), "to", max(episodes$date), "\n")

## Date range: 1998-10-15 to 2026-02-12

cat("Episodes with speaker data:", sum(sapply(episodes$speakers, length) > 0), "\n")

## Episodes with speaker data: 953

cat("Episodes without speaker data:", sum(sapply(episodes$speakers, length) == 0), "\n")

## Episodes without speaker data: 109

Classification Methodology

Episode titles are classified into categories using a rule-based approach implemented in the classify_title() function. The classification process examines title patterns and keywords to assign each episode to one of ten categories: General Topic (default category), Person (biographical content), Literary Work (books, plays, poems), Historical Event (battles, wars, revolutions), Place/Geographic (locations, regions), Art/Music (artistic works, movements), Scientific Concept (theories, phenomena), Philosophy (philosophical works/concepts), Legal/Political (legal systems, governance), and Religious/Mythological (religious figures, myths).

Classification Rules. The classification function applies pattern matching: (1) Person Detection - checks if the title matches a name pattern (capitalized words, possibly with Roman numerals); (2) Keyword Matching - searches for specific keywords associated with each category; (3) Pattern Recognition - uses regular expressions to identify literary works (quotes, “Part”, “Act”, “Scene”); (4) Default Assignment - episodes that don’t match specific patterns are assigned to “General Topic”.

# Classification function (from analyze_in_our_time.R)
classify_title <- function(title) {
  if (is.null(title) || is.na(title) || title == "") {
    return("Unknown")
  }
  
  title_lower <- tolower(title)
  
  # Person names (usually just a name or "Name - description")
  if (str_detect(title, "^[A-Z][a-z]+(\\s+[A-Z][a-z]+)*(\\s+(Part|I{1,3}|IV|V|VI|VII|VIII|IX|X))?$")) {
    if (!any(str_detect(title_lower, c("the ", "of ", "and ", "battle", "war", "code", "empire")))) {
      return("Person")
    }
  }
  
  # Historical events
  if (any(str_detect(title_lower, c("battle", "war", "uprising", "rebellion", "revolution", "affair", "crisis")))) {
    return("Historical Event")
  }
  
  # Literary works
  if (any(str_detect(title_lower, c("'", '"', "part ", "act ", "scene")))) {
    return("Literary Work")
  }
  
  # Scientific concepts
  if (any(str_detect(title_lower, c("evolution", "hypnosis", "pollination", "bacteriophage", "wormhole", 
                                     "slime", "mould", "habitability", "planet")))) {
    return("Scientific Concept")
  }
  
  # Philosophical works/concepts
  if (any(str_detect(title_lower, c("liberty", "typing", "typology", "philosophy")))) {
    return("Philosophy")
  }
  
  # Places/Geographic
  if (any(str_detect(title_lower, c("trench", "arena", "empire", "venetian", "hanoverian", "korean")))) {
    return("Place/Geographic")
  }
  
  # Art/Music
  if (any(str_detect(title_lower, c("secession", "vase", "soane", "art", "music")))) {
    return("Art/Music")
  }
  
  # Religious/Mythological
  if (any(str_detect(title_lower, c("kali", "god", "goddess", "myth", "legend", "dragon")))) {
    return("Religious/Mythological")
  }
  
  # Legal/Political
  if (any(str_detect(title_lower, c("copyright", "code", "liberty", "civility")))) {
    return("Legal/Political")
  }
  
  # Default
  return("General Topic")
}

Limitations. The classification is based on title text alone, which may not always reflect the full content. Some episodes may be misclassified due to ambiguous titles, and the keyword-based approach may miss nuanced categorizations. Manual review could improve accuracy for edge cases.

# Apply classification to episodes
df <- episodes %>%
  mutate(
    date_parsed = case_when(
      date == "Unknown" ~ as.Date(NA),
      TRUE ~ as.Date(date)  # Parse YYYY-MM-DD format directly
    ),
    year = year(date_parsed),
    classification = sapply(title, function(t) {
      if (is.null(t) || is.na(t) || t == "") return("Unknown")
      title_lower <- tolower(t)
      
      # Person names
      if (str_detect(t, "^[A-Z][a-z]+(\\s+[A-Z][a-z]+)*(\\s+(Part|I{1,3}|IV|V|VI|VII|VIII|IX|X))?$")) {
        if (!any(str_detect(title_lower, c("the ", "of ", "and ", "battle", "war", "code", "empire")))) {
          return("Person")
        }
      }
      
      # Historical events
      if (any(str_detect(title_lower, c("battle", "war", "uprising", "rebellion", "revolution", "affair", "crisis")))) {
        return("Historical Event")
      }
      
      # Literary works
      if (any(str_detect(title_lower, c("'", '"', "part ", "act ", "scene")))) {
        return("Literary Work")
      }
      
      # Scientific concepts
      if (any(str_detect(title_lower, c("evolution", "hypnosis", "pollination", "bacteriophage", "wormhole", 
                                         "slime", "mould", "habitability", "planet")))) {
        return("Scientific Concept")
      }
      
      # Philosophical works/concepts
      if (any(str_detect(title_lower, c("liberty", "typing", "typology", "philosophy")))) {
        return("Philosophy")
      }
      
      # Places/Geographic
      if (any(str_detect(title_lower, c("trench", "arena", "empire", "venetian", "hanoverian", "korean")))) {
        return("Place/Geographic")
      }
      
      # Art/Music
      if (any(str_detect(title_lower, c("secession", "vase", "soane", "art", "music")))) {
        return("Art/Music")
      }
      
      # Religious/Mythological
      if (any(str_detect(title_lower, c("kali", "god", "goddess", "myth", "legend", "dragon")))) {
        return("Religious/Mythological")
      }
      
      # Legal/Political
      if (any(str_detect(title_lower, c("copyright", "code", "liberty", "civility")))) {
        return("Legal/Political")
      }
      
      return("General Topic")
    })
  ) %>%
  filter(!is.na(year)) %>%
  select(year, classification, title, date, episode_url)

cat("Episodes processed:", nrow(df), "\n")

## Episodes processed: 1062

cat("Date range:", min(df$year), "-", max(df$year), "\n")

## Date range: 1998 - 2026

cat("Episodes processed:", nrow(df), "\n")

## Episodes processed: 1062

cat("Date range:", min(df$year), "-", max(df$year), "\n")

## Date range: 1998 - 2026

Overall Statistics

# Overall statistics
total_episodes <- nrow(df)
date_range <- paste(min(df$year), "-", max(df$year))
years_covered <- length(unique(df$year))

cat("=== Dataset Overview ===\n")

## === Dataset Overview ===

cat("Total episodes analyzed:", total_episodes, "\n")

## Total episodes analyzed: 1062

cat("Date range:", date_range, "\n")

## Date range: 1998 - 2026

cat("Years covered:", years_covered, "\n\n")

## Years covered: 29

Classification Distribution. The dataset shows clear patterns in content distribution:

# Classification summary
classification_summary <- df %>%
  count(classification, sort = TRUE) %>%
  mutate(
    percentage = round(n / nrow(df) * 100, 1),
    cumulative_pct = round(cumsum(percentage), 1)
  )

kable(classification_summary, 
      col.names = c("Classification", "Count", "Percentage (%)", "Cumulative %"),
      caption = "Distribution of Episodes by Classification Category") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"), 
                full_width = FALSE, position = "left")

Distribution of Episodes by Classification Category
Classification	Count	Percentage (%)	Cumulative %
Person	477	44.9	44.9
General Topic	415	39.1	84.0
Literary Work	80	7.5	91.5
Historical Event	53	5.0	96.5
Place/Geographic	13	1.2	97.7
Art/Music	10	0.9	98.6
Scientific Concept	6	0.6	99.2
Philosophy	4	0.4	99.6
Legal/Political	2	0.2	99.8
Religious/Mythological	2	0.2	100.0

General Topic and Person categories dominate, together accounting for 84% of all episodes. Literary Works represent 7.5% of episodes, while Historical Events account for 5% of episodes. The remaining categories (Place/Geographic, Art/Music, Scientific Concept, Philosophy, Legal/Political, Religious/Mythological) together represent less than 5% of episodes.

Temporal Trends. The program has maintained consistent production levels over time:

# Episodes per year
yearly_summary <- df %>%
  count(year, sort = TRUE) %>%
  mutate(
    percentage = round(n / total_episodes * 100, 1)
  )

cat("=== Episodes per Year (Top 10) ===\n")

## === Episodes per Year (Top 10) ===

kable(head(yearly_summary, 10), 
      col.names = c("Year", "Episodes", "Percentage (%)"),
      caption = "Top 10 Years by Episode Count") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"), 
                full_width = FALSE, position = "left")

Top 10 Years by Episode Count
Year	Episodes	Percentage (%)
1999	44	4.1
2007	43	4.0
2011	43	4.0
2012	43	4.0
2006	42	4.0
2009	42	4.0
2023	42	4.0
2008	41	3.9
2010	41	3.9
2015	41	3.9

# Plot episodes per year
ggplot(yearly_summary, aes(x = year, y = n)) +
  geom_line(color = "#3498db", linewidth = 1.2) +
  geom_point(color = "#2980b9", size = 2) +
  geom_area(fill = "#3498db", alpha = 0.3) +
  scale_x_continuous(breaks = seq(min(df$year), max(df$year), by = 5)) +
  labs(
    title = "Total Episodes per Year",
    subtitle = paste("Data from", min(df$year), "to", max(df$year)),
    x = "Year",
    y = "Number of Episodes"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(size = 14, face = "bold"),
    plot.subtitle = element_text(size = 11, color = "gray50"),
    axis.text.x = element_text(angle = 45, hjust = 1)
  )

Speaker Statistics. The program features a diverse range of expert speakers:

# Load speaker data if available (check both current directory and working directory)
speaker_csv_path <- if (file.exists("in_our_time_speakers.csv")) {
  "in_our_time_speakers.csv"
} else if (file.exists("working/in_our_time_speakers.csv")) {
  "working/in_our_time_speakers.csv"
} else {
  NULL
}

if (!is.null(speaker_csv_path)) {
  speakers <- read.csv(speaker_csv_path, stringsAsFactors = FALSE)
  
  cat("=== Speaker Statistics ===\n")
  cat("Total unique speakers:", nrow(speakers), "\n")
  cat("Total speaker appearances:", sum(speakers$Appearance_Count), "\n")
  cat("Average appearances per speaker:", round(mean(speakers$Appearance_Count), 2), "\n")
  cat("Maximum appearances:", max(speakers$Appearance_Count), "\n")
  cat("Speaker with most appearances:", speakers$Speaker[which.max(speakers$Appearance_Count)], "\n\n")
  
  # Top speakers
  top_speakers <- speakers %>%
    arrange(desc(Appearance_Count)) %>%
    head(10) %>%
    select(Speaker, Appearance_Count)
  
  kable(top_speakers, 
        col.names = c("Speaker", "Appearances"),
        caption = "Top 10 Most Frequent Speakers") %>%
    kable_styling(bootstrap_options = c("striped", "hover", "condensed"), 
                  full_width = FALSE, position = "left")
} else {
  cat("Speaker data file not found.\n")
  cat("Expected file: in_our_time_speakers.csv or working/in_our_time_speakers.csv\n")
}

## === Speaker Statistics ===
## Total unique speakers: 2400 
## Total speaker appearances: 3167 
## Average appearances per speaker: 1.32 
## Maximum appearances: 13 
## Speaker with most appearances: Laura Ashe

Top 10 Most Frequent Speakers
Speaker	Appearances
Laura Ashe	13
Carolin Crawford	11
Edith Hall	11
Paul Cartledge	10
Simon Schaffer	10
Frank Close	9
Steve Jones	9
Dinah Birch	9
Judith Hawley	8
Angie Hobbs	8

Facet Charts

The facet charts provide detailed visualizations of how each classification category has evolved over time. These visualizations help identify trends, patterns, and changes in the program’s content focus across different years.

# Create data frame for facet plotting
plot_data <- df %>%
  count(year, classification) %>%
  complete(year = min(df$year):max(df$year), 
            classification = unique(df$classification),
            fill = list(n = 0)) %>%
  arrange(classification, year)

# Order classifications by total count (for better visualization)
classification_order <- df %>%
  count(classification, sort = TRUE) %>%
  pull(classification)

plot_data$classification <- factor(plot_data$classification, 
                                   levels = classification_order)

cat("Plot data prepared for", nrow(plot_data), "data points\n")

## Plot data prepared for 290 data points

cat("Classifications:", paste(levels(plot_data$classification), collapse = ", "), "\n")

## Classifications: Person, General Topic, Literary Work, Historical Event, Place/Geographic, Art/Music, Scientific Concept, Philosophy, Legal/Political, Religious/Mythological

Facet Chart 1: Free Y-Scale. This visualization shows each classification category in its own panel with independent y-axis scales, allowing for better visibility of trends within each category regardless of absolute counts.

# Create facet plot with free y-scale
p1 <- ggplot(plot_data, aes(x = year, y = n)) +
  geom_line(color = "#3498db", linewidth = 1.2, alpha = 0.8) +
  geom_point(color = "#2980b9", size = 2, alpha = 0.7) +
  geom_area(fill = "#3498db", alpha = 0.3) +
  facet_wrap(~ classification, scales = "free_y", ncol = 3) +
  scale_x_continuous(breaks = seq(min(df$year), max(df$year), by = 5)) +
  scale_y_continuous(limits = c(0, NA)) +
  labs(
    title = "In Our Time: Episode Counts by Classification and Year",
    subtitle = sprintf("Data from %d to %d (%d episodes)", 
                      min(df$year), max(df$year), nrow(df)),
    x = "Year",
    y = "Number of Episodes",
    caption = "Source: BBC Radio 4 In Our Time | Each facet uses independent y-axis scale"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
    plot.subtitle = element_text(size = 12, hjust = 0.5, color = "gray50"),
    plot.caption = element_text(size = 9, color = "gray60", hjust = 1),
    axis.text.x = element_text(angle = 45, hjust = 1, size = 8),
    axis.text.y = element_text(size = 8),
    axis.title = element_text(size = 11, face = "bold"),
    strip.text = element_text(size = 10, face = "bold"),
    strip.background = element_rect(fill = "#ecf0f1", color = "#bdc3c7"),
    panel.grid.minor = element_blank(),
    panel.grid.major = element_line(color = "gray90", linewidth = 0.5),
    plot.margin = margin(20, 20, 20, 20)
  )

print(p1)

The free-scale facet chart reveals that General Topic and Person categories show the most episodes and maintain relatively consistent production levels. Literary Work episodes show some variation but generally remain steady, while Historical Event episodes appear sporadically throughout the years. Smaller categories (Place/Geographic, Art/Music, Scientific Concept, etc.) show low but consistent representation. Each category’s trend is clearly visible due to independent y-axis scaling.

Facet Chart 2: Common Y-Scale. This visualization uses a common y-axis scale across all facets, allowing for direct comparison of absolute counts between categories.

# Calculate common y-axis limit
max_count <- max(plot_data$n, na.rm = TRUE)
y_limit <- ceiling(max_count * 1.1)

# Create facet plot with common y-scale
p2 <- ggplot(plot_data, aes(x = year, y = n)) +
  geom_line(color = "#3498db", linewidth = 1.2, alpha = 0.8) +
  geom_point(color = "#2980b9", size = 2, alpha = 0.7) +
  geom_area(fill = "#3498db", alpha = 0.3) +
  facet_wrap(~ classification, scales = "fixed", ncol = 3) +
  scale_x_continuous(breaks = seq(min(df$year), max(df$year), by = 5)) +
  scale_y_continuous(limits = c(0, y_limit)) +
  labs(
    title = "In Our Time: Episode Counts by Classification and Year (Common Y-Scale)",
    subtitle = sprintf("Data from %d to %d (%d episodes)", 
                      min(df$year), max(df$year), nrow(df)),
    x = "Year",
    y = "Number of Episodes",
    caption = "Source: BBC Radio 4 In Our Time | All facets use the same y-axis scale for direct comparison"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
    plot.subtitle = element_text(size = 12, hjust = 0.5, color = "gray50"),
    plot.caption = element_text(size = 9, color = "gray60", hjust = 1),
    axis.text.x = element_text(angle = 45, hjust = 1, size = 8),
    axis.text.y = element_text(size = 8),
    axis.title = element_text(size = 11, face = "bold"),
    strip.text = element_text(size = 10, face = "bold"),
    strip.background = element_rect(fill = "#ecf0f1", color = "#bdc3c7"),
    panel.grid.minor = element_blank(),
    panel.grid.major = element_line(color = "gray90", linewidth = 0.5),
    plot.margin = margin(20, 20, 20, 20)
  )

print(p2)

The common-scale facet chart highlights the relative magnitude of categories—the dominance of General Topic and Person categories is immediately apparent. Direct comparison between categories shows that General Topic and Person episodes far exceed other categories. Categories with low counts (Scientific Concept, Philosophy, Legal/Political, Religious/Mythological) appear as flat lines near zero, making their trends less visible, but the common scale provides context for understanding the relative importance of each category.

Facet Chart 3: Bar Charts. Bar charts provide an alternative visualization that emphasizes discrete yearly counts rather than continuous trends.

# Create bar chart version
p3 <- ggplot(plot_data, aes(x = year, y = n, fill = classification)) +
  geom_bar(stat = "identity", position = "dodge", alpha = 0.8) +
  facet_wrap(~ classification, scales = "free_y", ncol = 3) +
  scale_x_continuous(breaks = seq(min(df$year), max(df$year), by = 5)) +
  scale_fill_brewer(palette = "Set3") +
  labs(
    title = "In Our Time: Episode Counts by Classification and Year (Bar Chart)",
    subtitle = sprintf("Data from %d to %d (%d episodes)", 
                      min(df$year), max(df$year), nrow(df)),
    x = "Year",
    y = "Number of Episodes",
    fill = "Classification",
    caption = "Source: BBC Radio 4 In Our Time"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(size = 16, face = "bold", hjust = 0.5),
    plot.subtitle = element_text(size = 12, hjust = 0.5, color = "gray50"),
    plot.caption = element_text(size = 9, color = "gray60", hjust = 1),
    axis.text.x = element_text(angle = 45, hjust = 1, size = 8),
    axis.text.y = element_text(size = 8),
    axis.title = element_text(size = 11, face = "bold"),
    strip.text = element_text(size = 10, face = "bold"),
    strip.background = element_rect(fill = "#ecf0f1", color = "#bdc3c7"),
    panel.grid.minor = element_blank(),
    panel.grid.major = element_line(color = "gray90", linewidth = 0.5),
    plot.margin = margin(20, 20, 20, 20),
    legend.position = "none"
  )

print(p3)

The bar chart visualization provides discrete representation where each year is represented as a distinct bar, making yearly comparisons clear. Years with zero episodes for a category are clearly visible as gaps, and the format makes it easier to identify specific years with high or low episode counts.

Summary. The facet charts reveal several key patterns: General Topic and Person categories consistently produce the most episodes each year; most categories show relatively stable production levels over time; some categories (Legal/Political, Religious/Mythological) appear very infrequently; and the program maintains a consistent mix of content types across its history. Category-specific observations show that General Topic has the highest variability and highest overall counts, Person maintains steady production typically second-highest after General Topic, Literary Work shows moderate and relatively stable production, Historical Event appears sporadically but consistently throughout the years, and small categories (Place/Geographic, Art/Music, Scientific Concept, Philosophy, Legal/Political, Religious/Mythological) appear infrequently but consistently.

Complete Episode List

A comprehensive, interactive sortable table of all episodes with their classifications, dates, speakers, and links is available in the Complete Episode Table. This interactive table provides:

Sorting - Sort by any column (Date, Year, Title, Classification, Speaker Count)
Filtering - Filter episodes by classification, year, or other criteria
Search - Search across all columns including speaker names
Export - Export data to CSV, Excel, or PDF
Episode Details - View episode details including speaker information and direct links to BBC pages
Updated Data - Includes all episodes with complete date information, including recently updated holiday dates (Boxing Day, Christmas Eve, New Year’s Day episodes)

The table includes all episodes with their assigned classifications, making it easy to explore the dataset and verify classifications. The table is generated automatically from the complete JSON dataset with classifications applied using the same methodology described in this report.

Note: The table file (in_our_time_episodes_table.html) is generated automatically when this document is rendered. The table reflects the current state of the episode dataset, including all updated dates.

Conclusion

This analysis has examined the BBC Radio 4 “In Our Time” program through systematic data collection, classification, and visualization. The findings reveal a diverse range of topics covered across 29 years, a classification system that successfully categorizes episodes into meaningful groups, clear patterns in content distribution with General Topic and Person episodes dominating, and consistent production levels across most categories over time. The facet charts provide valuable insights into how different topic categories have evolved, helping to understand the program’s content strategy and audience interests over its long history.

Appendix

Data Sources. Primary data files include in_our_time_episodes.json (complete episode dataset, source: scrape_episodes.py, enhanced by extract_missing_dates.py and extract_holiday_dates.py, with dates converted to YYYY-MM-DD format via convert_date_format.py) and in_our_time_episodes.csv. Processed data files include title_classifications.csv, classifications_by_year.csv, in_our_time_speakers.csv (generated by analyze_speakers.py or generate_all_csv_files.py), speaker_appearance_distribution.csv, speaker_appearances_by_year.csv, and in_our_time_plot_data.csv (generated by analyze_in_our_time.R or generate_all_csv_files.py). Report files include STATISTICS_REPORT.md (generated by generate_statistics.py), classifications_by_year_table.md, and in_our_time_episodes_table.html (generated by generate_sortable_table.R). Visualization files include three facet plot PNG files. Analysis scripts include analyze_in_our_time.R, analyze_speakers.py, generate_statistics.py, generate_html_report.py, generate_sortable_table.R, and generate_all_csv_files.py.

Code References. The analysis draws on the following scripts: Data Collection - scrape_episodes.py (initial web scraping, extracts dates in DD/MM/YYYY format), extract_missing_dates.py (data cleaning, extracts dates in DD/MM/YYYY format), extract_holiday_dates.py (holiday date extraction); Data Conversion - convert_date_format.py (converts dates from DD/MM/YYYY to YYYY-MM-DD format for standardized storage); Data Processing - analyze_speakers.py (speaker analysis), generate_statistics.py (statistics generation), generate_all_csv_files.py (comprehensive CSV generation, handles dates in YYYY-MM-DD format); Analysis and Visualization - analyze_in_our_time.R (statistical analysis, classification, visualization, parses dates in YYYY-MM-DD format), generate_sortable_table.R (sortable episode table generation, handles dates in YYYY-MM-DD format); Documentation - STATISTICS_REPORT.md, README.md, EXTRACT_DATES_README.md, HOW_TO_UPDATE.md.

## R version 4.5.1 (2025-06-13)
## Platform: aarch64-apple-darwin20
## Running under: macOS Tahoe 26.2
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
## 
## locale:
## [1] C.UTF-8/C.UTF-8/C.UTF-8/C/C.UTF-8/C.UTF-8
## 
## time zone: Europe/Paris
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] htmlwidgets_1.6.4 DT_0.33           kableExtra_1.4.0  knitr_1.50       
##  [5] tidyr_1.3.1       stringr_1.5.2     lubridate_1.9.4   ggplot2_4.0.1    
##  [9] dplyr_1.1.4       jsonlite_2.0.0   
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.6       compiler_4.5.1     tidyselect_1.2.1   xml2_1.3.8        
##  [5] jquerylib_0.1.4    textshaping_1.0.1  systemfonts_1.2.3  scales_1.4.0      
##  [9] yaml_2.3.10        fastmap_1.2.0      R6_2.6.1           labeling_0.4.3    
## [13] generics_0.1.4     tibble_3.3.0       svglite_2.2.1      bslib_0.9.0       
## [17] pillar_1.11.1      RColorBrewer_1.1-3 rlang_1.1.6        cachem_1.1.0      
## [21] stringi_1.8.7      xfun_0.53          sass_0.4.10        S7_0.2.0          
## [25] viridisLite_0.4.2  timechange_0.3.0   cli_3.6.5          withr_3.0.2       
## [29] magrittr_2.0.4     crosstalk_1.2.2    digest_0.6.37      grid_4.5.1        
## [33] rstudioapi_0.17.1  lifecycle_1.0.4    vctrs_0.6.5        evaluate_1.0.5    
## [37] glue_1.8.0         farver_2.1.2       rmarkdown_2.30     purrr_1.2.0       
## [41] tools_4.5.1        pkgconfig_2.0.3    htmltools_0.5.8.1

In Our Time: Episode Analysis and Classification

BBC Radio 4 Episode Data Analysis (v5 - with Speaker Talk Lookup)

ML; Cursor AI

2026-02-08