Lets download the Mueller report and do some text mining.
Here is a very nice blog post to follow to get started Text Crunching and Data Munging by Josephine Lukito.
library(pacman)
p_load(tidyverse, pdftools, ggpage, inspectdf)
Download the Mueller report.
The mode “wb” seems to be needed for Windows. This might need to curl on Mac.
# second page text
cat(report[9])
U.S. Department of Justice
Atten1e:y· '>lork Preettet // Moy Cefttttift Material Preteetee Ul'leer Fee. R. Criffl.. P. 6(e)
INTRODUCTION TO VOLUME I
This report is submitted to the Attorney General pursuant to 28 C.F.R. § 600.8(c), which
states that, "[a]t the conclusion of the Special Counsel ' s work, he ... shall provide the Attorney
General a confidential report explaining the prosecution or declination decisions [the Special
Counsel] reached."
The Russian government interfered in the 2016 presidential election in sweeping and
systematic fashion. Evidence of Russian government operations began to surface in mid-2016. In
June, the Democratic National Committee and its cyber response team publicly announced that
Russian hackers had compromised its computer network. Releases of hacked materials-hacks
that public reporting soon attributed to the Russian government-began that same month.
Additional releases followed in July through the organization WikiLeaks, with further releases in
October and November.
In late July 2016, soon after WikiLeaks's first release of stolen documents, a foreign
government contacted the FBI about a May 2016 encounter with Trump Campaign foreign policy
advisor George Papadopoulos. Papadopoulos had suggested to a representative of that foreign
government that the Trump Campaign had received indications from the Russian government that
it could assist the Campaign through the anonymous release of information damaging to
Democratic presidential candidate Hillary Clinton. That information prompted the FBI on July
31, 2016, to open an investigation into whether individuals associated with the Trump Campaign
were coordinating with the Russian government in its interference activities.
That fall, two federal agencies jointly announced that the Russian government "directed
recent compromises of e-mails from US persons and institutions, including US political
organizations," and, " [t]hese thefts and disclosures are intended to interfere with the US election
process." After the election, in late December 2016, the United States imposed sanctions on Russia
for having interfered in the election. By early 2017, several congressional committees were
examining Russia's interference in the election.
Within the Executive Branch, these investigatory efforts ultimately led to the May 2017
appointment of Special Counsel Robert S. Mueller, III. The order appointing the Special Counsel
authorized him to investigate "the Russian government's efforts to interfere in the 2016
presidential election," including any links or coordination between the Russian government and
individuals associated with the Trump Campaign.
As set forth in detail in this report, the Special Counsel's investigation established that
Russia interfere~ in the 2016 presidential election principally through two operations. First, a
fileConn <- file("mueller-report.txt")
writeLines(report, fileConn)
close(fileConn)
# renders pdf to bitmap array
bitmap <- pdf_render_page("mueller-report.pdf", page = 10)
# save bitmap image
png::writePNG(bitmap, "page.png")
jpeg::writeJPEG(bitmap, "page.jpeg")
mueller_report_csv <- "https://raw.githubusercontent.com/gadenbuie/mueller-report/ab74012b0532ffa34f3a45196d2b28004e11b9c2/mueller_report.csv"
mueller_report <- read_csv(mueller_report_csv)
Rows: 19195 Columns: 3── Column specification ─────────────────────────────────────────────────
Delimiter: ","
chr (1): text
dbl (2): page, line
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
mueller_report
mueller_pages <-
mueller_report %>%
# pad pages with fewer lines than expected
complete(
page,
line = 1:max(mueller_report$line),
fill = list(text = "")
) %>%
# Pre-process for {ggpage}
ggpage_build(
ncol = 30,
bycol = FALSE,
page.col = "page",
wtl = FALSE,
x_space_pages = 10,
y_space_pages = 100
) %>%
mutate(
color = case_when(
str_detect(word, "trump|president") ~ "Trump",
str_detect(word, "russia") ~ "Russia",
str_detect(word, "cohen") ~ "Cohen",
str_detect(word, "co(m|rn)ey") ~ "Comey",
str_detect(word, "flynn") ~ "Flynn",
str_detect(word, "manafort") ~ "Manafort",
str_detect(word, "sessions") ~ "Sessions",
str_detect(word, "mcgahn") ~ "McGahn",
TRUE ~ "normal"
),
color = factor(color, c(
"Trump", "Russia", "Cohen", "Comey",
"Flynn", "Manafort", "Sessions", "McGahn", "normal"
))
)
|=========== | 25% ~6 s remaining
|=========== | 25% ~6 s remaining
|============ | 26% ~6 s remaining
|============ | 27% ~6 s remaining
|============ | 27% ~6 s remaining
|============= | 28% ~6 s remaining
|============= | 28% ~6 s remaining
|============= | 29% ~6 s remaining
|============= | 30% ~6 s remaining
|============== | 30% ~6 s remaining
|============== | 31% ~6 s remaining
|============== | 31% ~6 s remaining
|============== | 32% ~6 s remaining
|=============== | 32% ~6 s remaining
|=============== | 33% ~5 s remaining
|=============== | 33% ~5 s remaining
|================ | 34% ~5 s remaining
|================ | 35% ~5 s remaining
|================ | 35% ~5 s remaining
|================ | 36% ~5 s remaining
|================= | 37% ~5 s remaining
|================= | 37% ~5 s remaining
|================= | 38% ~5 s remaining
|================= | 38% ~5 s remaining
|================== | 39% ~5 s remaining
|================== | 39% ~5 s remaining
|================== | 40% ~5 s remaining
|================== | 40% ~5 s remaining
|=================== | 41% ~5 s remaining
|=================== | 41% ~5 s remaining
|=================== | 42% ~5 s remaining
|=================== | 42% ~5 s remaining
|==================== | 43% ~5 s remaining
|==================== | 43% ~5 s remaining
|==================== | 44% ~5 s remaining
|==================== | 45% ~5 s remaining
|===================== | 45% ~5 s remaining
|===================== | 46% ~5 s remaining
|===================== | 46% ~5 s remaining
|====================== | 47% ~5 s remaining
|====================== | 48% ~5 s remaining
|====================== | 48% ~4 s remaining
|====================== | 49% ~4 s remaining
|======================= | 49% ~4 s remaining
|======================= | 50% ~4 s remaining
|======================= | 50% ~4 s remaining
|======================= | 51% ~4 s remaining
|======================== | 51% ~4 s remaining
|======================== | 52% ~4 s remaining
|======================== | 52% ~4 s remaining
|======================== | 53% ~4 s remaining
|======================== | 53% ~4 s remaining
|========================= | 54% ~4 s remaining
|========================= | 54% ~4 s remaining
|========================= | 55% ~4 s remaining
|========================= | 55% ~4 s remaining
|========================== | 56% ~4 s remaining
|========================== | 56% ~4 s remaining
|========================== | 57% ~4 s remaining
|========================== | 57% ~4 s remaining
|=========================== | 58% ~4 s remaining
|=========================== | 58% ~4 s remaining
|=========================== | 59% ~4 s remaining
|============================ | 60% ~4 s remaining
|============================ | 60% ~3 s remaining
|============================ | 61% ~3 s remaining
|============================ | 61% ~3 s remaining
|============================= | 62% ~3 s remaining
|============================= | 63% ~3 s remaining
|============================= | 63% ~3 s remaining
|============================= | 64% ~3 s remaining
|============================== | 64% ~3 s remaining
|============================== | 65% ~3 s remaining
|============================== | 65% ~3 s remaining
|=============================== | 66% ~3 s remaining
|=============================== | 67% ~3 s remaining
|=============================== | 67% ~3 s remaining
|=============================== | 68% ~3 s remaining
|================================ | 68% ~3 s remaining
|================================ | 69% ~3 s remaining
|================================ | 70% ~3 s remaining
|================================ | 70% ~3 s remaining
|================================= | 71% ~3 s remaining
|================================= | 71% ~3 s remaining
|================================= | 72% ~2 s remaining
|================================== | 73% ~2 s remaining
|================================== | 73% ~2 s remaining
|================================== | 74% ~2 s remaining
|================================== | 74% ~2 s remaining
|=================================== | 75% ~2 s remaining
|=================================== | 75% ~2 s remaining
|=================================== | 76% ~2 s remaining
|=================================== | 77% ~2 s remaining
|==================================== | 77% ~2 s remaining
|==================================== | 78% ~2 s remaining
|==================================== | 78% ~2 s remaining
|==================================== | 79% ~2 s remaining
|===================================== | 79% ~2 s remaining
|===================================== | 80% ~2 s remaining
|===================================== | 81% ~2 s remaining
|====================================== | 81% ~2 s remaining
|====================================== | 82% ~2 s remaining
|====================================== | 82% ~2 s remaining
|====================================== | 83% ~2 s remaining
|======================================= | 83% ~1 s remaining
|======================================= | 84% ~1 s remaining
|======================================= | 84% ~1 s remaining
|======================================= | 85% ~1 s remaining
|======================================== | 85% ~1 s remaining
|======================================== | 86% ~1 s remaining
|======================================== | 86% ~1 s remaining
|======================================== | 87% ~1 s remaining
|======================================== | 87% ~1 s remaining
|========================================= | 88% ~1 s remaining
|========================================= | 88% ~1 s remaining
|========================================= | 89% ~1 s remaining
|========================================== | 89% ~1 s remaining
|========================================== | 90% ~1 s remaining
|========================================== | 90% ~1 s remaining
|========================================== | 91% ~1 s remaining
|========================================== | 91% ~1 s remaining
|=========================================== | 92% ~1 s remaining
|=========================================== | 92% ~1 s remaining
|=========================================== | 93% ~1 s remaining
|=========================================== | 93% ~1 s remaining
|============================================ | 94% ~1 s remaining
|============================================ | 95% ~0 s remaining
|============================================ | 95% ~0 s remaining
|============================================ | 96% ~0 s remaining
|============================================= | 96% ~0 s remaining
|============================================= | 97% ~0 s remaining
|============================================= | 97% ~0 s remaining
|============================================== | 98% ~0 s remaining
|============================================== | 99% ~0 s remaining
|============================================== | 99% ~0 s remaining
|============================================== |100% ~0 s remaining
mueller_pages
# manually assigned colors from ggthemes::pal_calc()
colors <- rep("", length(levels(mueller_pages$color)))
names(colors) <- levels(mueller_pages$color)
colors["Trump"] <- "#FF4023"
colors["Russia"] <- "#004983"
colors["Cohen"] <- "#FF922E"
colors["Comey"] <- "#559B30"
colors["Flynn"] <- "#4D276D"
colors["Manafort"] <- "#7BCAFD"
colors["Sessions"] <- "#7F1327"
colors["McGahn"] <- "#FFD040"
colors["normal"] <- "#d0d0d0"
ggpage_plot(mueller_pages) +
aes(fill = color) +
scale_fill_manual(
values = colors,
breaks = setdiff(names(colors), "normal")
) +
labs(fill = NULL, caption = "@grrrck") +
guides(fill = guide_legend(nrow = 1)) +
theme(legend.position = "bottom")
Check out this nice github
https://github.com/batpigandme/tidymueller
Topic Modeling
https://www.christopheryee.org/blog/text-mining-the-redacted-mueller-report/