The “Sociology of Digital Public Spaces”: @Grok, what do you think?

Our research explores how the integration of Grok, an AI-powered platform-native chatbot, into X (formerly Twitter) reshapes the dynamics of online deliberation. We began by observing something surprising: a social media account (@grokvsmaga) that publishes compilations of thread interactions where users summon Grok to fact-check claims made by Trump supporters that are later revealed to be false or deeply misleading.

Though @grokvsmaga was primarily conceived as a humorous account––to ridicule contradictions in the Trumpian argumentative repertoire—we noticed something more serious: Grok’s interventions prompted almost visceral attitude reversals in users whose beliefs were contested. This recurrent phenomenon raised a compelling question: if the intervention of a chatbot in a debate could simultaneously ground debates in facts and trigger such strong emotional reactions, how else might AI integration reshape the fundamentally human activity of political deliberation? 

To frame this intuition within existing theory, we turned to Jürgen Habermas’s foundational work on democratic deliberation––both his 1962 Structural Transformation of the Public Sphere and its 2023 digital-focused update, A New Structural Transformation of the Public Sphere and Deliberative Politics.

The course provided us with both the theoretical foundation and methodological tools our research required. The course introduced us to key literature on political polarization and various theories of the digital public sphere, all essential frameworks for understanding Grok’s potential impact.

Just as importantly, the course encouraged us to move beyond traditional literature reviews. Through two in-class workshop sessions, we had a chance to explore how experimental and computational methods might enrich our study, ultimately shaping our decision to combine philosophical, sociological, anthropological, and quantitative analysis.

Given the complex and multifaceted nature of digital interactions, we adopted a triangular mixed-method approach that leveraged (1) a qualitative ‘netnography’, (2) an experimental study, and (3) a quantitative study.

Our ‘netnography’ came first. We immersed ourselves in X threads and timelines observing how users organically summoned Grok into debates, and how Grok answered users' queries. From over 50 exchanges, we created a taxonomy that captures the nuance and diversity of these invocations––everything from fact-checking requests to the rhetorical ‘roasting’ of deliberative opponents.

This taxonomy directly informed our experimental study. Based on our observations, we designed and built four personas with different political views and levels of ‘democratic reciprocity’ (broadly speaking, the willingness to engage in good-faith debates), then scripted interactions where they engaged with Grok and each other in a debate about gay marriage in the United States. For example, a post we scripted for our persona @LiberalOrderly mimicked practices observed in our taxonomy, using Grok to counter others’ arguments: “@Grok, remind this patriot cosplayer that marriage equality isn’t up for renegotiation every time someone misreads the Bible and the Constitution in the same breath.” This allowed us to isolate how Grok responds to different argumentative styles in a controlled setting.

Lastly, our quantitative study systematically examined 1,880 X posts generated by Grok between April and September 2025 in English, covering 11 topics we deemed controversial. We first clusterized messages into groups of connected topics, then used Natural Language Interference (NLI) to assess whether Grok was consistent with the information it provided. Additionally, we mobilized sentiment analysis, emotion analysis, and irony analysis models to evaluate Grok’s tone neutrality at scale. 

Our study revealed a paradox at the heart of AI-mediated deliberation. Grok does function as a new ‘epistemic anchor’, improving the quality and facts of debates through what is known as the ‘Hawthorne effect’––users argue more carefully knowing Grok might fact-check them. Its consistent neutral tone also helps de-escalate tensions in highly polarized threads.

Yet these benefits come with significant costs. Grok’s interventions tend to homogenize discussions, flattening the diverse voices and argumentative styles that characterize genuine democratic deliberation. Perhaps more troublingly, Grok tends to re-direct conversations toward itself and away from peer-to-peer exchanges––through what we call ‘verticalization of discourse’, users end up debating Grok instead of one another. 

Our research points to a view of the digital public space that nuances and goes beyond Habermas’s pessimistic claims on algorithmic mediation. It suggests Grok’s integration creates a new affordance (a possibility) that has been ‘domesticated’ or adopted by X users. It also functions as a disposable form of authority, whereby its influence is inconsistent and often wielded by partial participants to support their own biases. Overall, these results suggest that current AI chatbot integration offers a minimalist, ‘patchwork’ form of moderation for digital deliberation, and that we may continue to see more piecemeal efforts to regulate and reframe online democratic speech. 

First, don't be afraid to mix methods. Our group’s diverse backgrounds—ranging from engineering to political philosophy and sociology—made us naturally curious to experiment with innovative methods. For digital research especially, this combination is essential: computational approaches reveal patterns at scale, while ‘netnographic’ and experimental methods capture the lived experiences that those numbers represent. Neither would be complete without the other, and neither alone could have told the full story.

Second, take everyday digital experiences as serious objects of study. Social media accounts like @grokvsmaga––meme-ish, rather than academic, as they intend to be––can reveal profound insights about how technology shapes public life. As digital natives gradually make their way into social science research, we have an opportunity to attach scholarly rigor to what previous generations have routinely dismissed as trivial. If democratic life happens on these platforms, the social sciences need to be there too.