Danny Ebanks
Dynamics of Political Polarization

Authors: Danny Ebanks. Bojan Tonguz. R. Michael Alvarez
Medium: at Trustworthy Social Media


The American public increasingly finds itself bitterly divided over political differences. Survey indicators, partisan media, and the public’s voting patterns inform this sense of division in our politics. That said, we use applications of Machine Learning and Natural Language Processing (NLP) methods in a novel way to paint a more nuanced picture of divisions in American political opinions.

It turns out that even very simple NLP methods that rely on simple word frequencies in politicians’ tweets can be extremely predictive when it comes to predicting party affiliation, getting over 80% accuracy without any special tuning. These simple models are very robust: a model trained on the tweets from the House of the Representatives can be equally predictive when tested on the tweets from the US Senators. Furthermore, even though politicians of both major political parties have become increasingly partisan and homogeneous in terms of their ideological commitments, these simple models can produce very credible rankings of the politicians along the left-right spectrum. These rankings very closely replicate similar rankings that have been based on politicians voting records. If these results withstand more rigorous scrutiny, NLP methods on politicians’ tweets could become a very simple measuring tool for ascertaining the degree of partisanship for any politician, even those with very meager or nonexistent voting records. In these settings, even simple assumptions on the structure of the topics yields large gains in understanding political polarization amongst political elites.

Written by

Danny Ebanks

Hi, my name is Danny! I am a Postdoctoral Fellow for IQSS at Harvard after having recently earned my PhD at Caltech in Quantitative Social Sciences! With a research passion for political methodology and American politics, I strive to develop and implement statistical methods, to understand the latest in machine learning and AI, and innovate in these areas in ways small and large to better understand our political world. I am always eager to chat about research and statistics, so feel free to reach out. Outside of research, I'm lifelong runner who hails from New York.