The Frameworks Tech Forgot, Article 1, exploring AI through the lenses of gender, politics, and justice – because the most important questions about technology were never technical.
In 2015, Joy Buolamwini was a graduate student at the MIT Media Lab, building a project she called the Aspire Mirror – a device that would project an inspirational image onto your face when you looked into it. Imagine looking at yourself and seeing Serena Williams reflected back at you. It was playful, creative, exactly the kind of thing a young computer scientist excited about the future would build.
There was just one problem. The mirror couldn’t see her.
The facial recognition software detected her lighter-skinned colleagues instantly. But Buolamwini, a Black woman, was invisible to the machine. The only way the system could find her face was when she put on a white mask.
Let that image sit with you for a moment. A Black woman at one of the most prestigious research institutions in the world, literally masking her identity to be recognized by the technology she was building. If you’ve taken a Women’s & Gender Studies course, you already have a name for what’s happening here. You’ve studied the male gaze, the way film and visual culture construct a ‘neutral’ perspective that is actually male, actually white, actually heterosexual. Buolamwini calls what she discovered ‘the coded gaze’: the way AI systems embed the same selective vision into code, then deploy it as if it were objective.
The Ghosts in the Data
Buolamwini didn’t just write about her experience. She turned it into a research project. Her 2018 Gender Shades study tested commercial facial recognition systems from IBM, Microsoft, and Face++ on a dataset she deliberately constructed to be more representative than the ones these companies used internally. What she found was damning. The systems had error rates below 1% for light-skinned men, the demographic that dominated the training data. For dark-skinned women, the error rate climbed as high as 34.7%.
Think about what that means in practice. These aren’t experimental systems in a lab. They’re deployed in airports, in policing, in hiring software, in the security systems that decide who gets flagged and who passes through. A system that works almost perfectly for one group and fails catastrophically for another isn’t a neutral tool. It’s a system that encodes a hierarchy and then automates it at scale.
The reason this happens is surprisingly straightforward. AI systems learn by studying massive datasets, and if those datasets overwhelmingly represent light-skinned male faces, the system learns that light-skinned male faces are what faces look like. Everything else becomes a deviation, an edge case, an error. The bias isn’t malicious. It’s structural, and baked into the data before the system even starts learning.
If that sounds familiar to you, it should. It’s the same dynamic feminist scholars have analyzed for decades – systems that present one group’s experience as universal while treating everyone else as an exception.
‘AI, Ain’t I A Woman?’
Buolamwini could have published her findings in a peer-reviewed paper and left it at that. She did publish the paper and it became one of the most cited AI ethics publications in the world. But she also did something else. She made art.
Her spoken word video poem, ‘AI, Ain’t I A Woman?‘, runs photos of iconic Black women such as Oprah Winfrey, Michelle Obama, Serena Williams, and Sojourner Truth herself, through commercial facial recognition systems and documents the failures in real time, set to poetry. The systems label these women as male, or fail to detect them entirely. Google’s system identified an image of Sojourner Truth as ‘a clean shaven old gentleman.’
The title isn’t accidental. Sojourner Truth delivered her famous ‘Ain’t I A Woman?’ speech at the Women’s Rights Convention in Akron, Ohio, in 1851. Truth’s speech challenged white feminists to recognize that their movement’s vision of womanhood excluded Black women entirely. Nearly 170 years later, Buolamwini is making the same argument about a different kind of system – one built by engineers rather than convention delegates, but encoding the same hierarchies.
This is where the frameworks that tech forgot become essential. Computer scientists can identify the fact that the training data was unbalanced as the technical problem. They can propose technical solutions like diversifying the dataset, retraining the model, and testing for accuracy across demographics. These are important steps, and they’ve led to real improvements. After Buolamwini’s research, IBM and Microsoft both took corrective action on their systems.
But feminist analysis goes deeper. Why was the training data unbalanced in the first place? Who decided what a ‘representative’ dataset looked like? Whose faces were considered the default, and whose were treated as edge cases? Why did it take a Black woman researcher to notice what entire teams of engineers missed or didn’t think to look for? These aren’t technical questions. They’re questions about power – about who gets to define what’s normal and who gets erased by that definition.
From Individual Bias to Structural Power
Catherine D’Ignazio and Lauren Klein, in their landmark book Data Feminism (available free as an open-access book from MIT Press…seriously, go read it), argue that the most important principle for understanding AI is to examine power. Not just individual bias. Not just bad data. Power. Who has it, who doesn’t, and how it’s encoded into the systems we’re told are objective.
They point out that the narratives around data science and AI are ‘overwhelmingly white, male, and techno-heroic.’ The popular story is about brilliant engineers solving problems. The missing story is about whose problems get solved and whose get created. When Amazon built an AI recruiting tool trained on a decade of hiring data, the system learned to penalize résumés that included the word ‘women’s,’ as in ‘women’s chess club captain’ or ‘women’s studies.’ Amazon scrapped the tool. But the question Data Feminism forces us to ask is – how many similar systems are still running, in companies that haven’t audited them, making decisions about people’s lives?
This is where political science enters the conversation. AI isn’t just a consumer product, it’s infrastructure. It’s embedded in criminal justice (predictive policing, risk assessment algorithms), in social services (benefits eligibility, fraud detection), in immigration enforcement (facial recognition at borders, surveillance of asylum seekers), and in the information ecosystems that shape democratic participation (recommendation algorithms, content moderation, deepfakes). These aren’t peripheral applications. They’re the systems through which political power operates.
When those systems encode racial and gender bias, as Buolamwini proved they do, they don’t just produce bad search results. They produce unjust outcomes, at scale, with a veneer of objectivity that makes them harder to challenge than human prejudice. An algorithm doesn’t look biased. It looks like math.
What You Bring to This Conversation
Here’s what I keep coming back to. The most powerful critique of AI bias didn’t emerge from a computer science department. It emerged from the intersection of feminist theory, Black intellectual history, and technology. Buolamwini’s genius was in recognizing that Sojourner Truth’s 1851 critique of exclusionary feminism was directly relevant to facial recognition software in 2018.
That’s not a coincidence. It’s evidence that the frameworks developed in Women’s & Gender Studies and Political Science, intersectionality, critical race theory, feminist epistemology, and political economy, aren’t just relevant to AI. They’re essential. They ask the questions that engineers weren’t trained to ask and that tech companies have financial incentives not to explore.
The AI governance conversation is happening right now in Congress, in the EU, in university policy committees, in corporate boardrooms. The people doing most of the talking are technologists and lawyers. What’s missing are the perspectives of people who study power for a living. People who know that fairness and justice aren’t the same thing. People who understand that fixing an algorithm without examining the structure it operates in is like treating symptoms without diagnosing the disease.
Joy Buolamwini founded the Algorithmic Justice League to fight for exactly this kind of structural accountability. She has testified before Congress. She has advised world leaders. And she started by noticing that a mirror couldn’t see her face.
What will you notice — once you start looking?
This is the first article in ‘The Frameworks Tech Forgot,’ a series exploring AI through the lenses of gender, politics, and justice. Next: ‘The Ghost Workers Behind Your ChatGPT,’ on the invisible labor force that makes AI possible.
Joy Buolamwini’s spoken word poem ‘AI, Ain’t I A Woman?‘ is available on YouTube. Her book, Unmasking AI: My Mission to Protect What Is Human in a World of Machines, is available wherever you buy books. Data Feminism by Catherine D’Ignazio and Lauren F. Klein is available free at MIT Press.
Citizen's Quill 
Leave a comment