Gender Bias in Natural Language Processing AI

Over the past few years, Artificial Intelligence (AI) has experienced exponential growth regarding both its scale and utility. Accordingly, it is critical we remain mindful of its potentially detrimental effects, especially in cases when inherent data bias is ignored.

In a recent BBC interview, feminist scholars Lauren Klein and Caroline Perez reasoned that because of unaccounted-for data bias, “women are 47% more likely to be seriously injured and 17% more likely to die than men” in similar car accidents [1]. The issue Klein and Perez are talking about is that the data that goes into crash simulations comes primarily from male crash test dummies. Thus, when artificial intelligence algorithms are employed to learn from these simulations and improve essential safety gear like airbags, seatbelts, and headrests, men are protected disproportionally. Thankfully, following the report of the findings, car makers have increased the number of female crash test dummies used and have taken greater care in accounting for the discrepancies in male and female datasets.

Nonetheless, in a world without AI regulation, where is the guarantee that we see this acknowledgment of bias and its subsequent correction? Furthermore, what’s to stop companies and organizations from ignoring it entirely?

Society is becoming increasingly reliant on the internet, big data, and cutting-edge technologies like AI; it is thus critical that we maintain control over all the continued development.

One subfield of artificial intelligence of particular interest is Natural Language Processing (NLP). NLP aims to help computers understand and respond to human text. For example, Apple’s Siri and Amazon’s Alexa are both excellent applications of the technology.

Vast datasets, commonly referred to as corpora, are central to how artificial intelligence operates in the context of NLP. Using these corpora, the AI models can be trained, basing their actions on the data. Subsequently, machine translation and word embedding are two ways in which this learning can be implemented.

Machine translation concerns itself with using AI to translate human language. Certain phrases in one language and their intent can be converted to a different language. For example, the word “watch” in English has more than one meaning, and since Google Translate’s AI can discern the differences in context, it will return the correct word for that specific scenario. However, it doesn’t always work so effortlessly. The Hungarian language is gender-neutral, meaning that there are no gendered pronouns. Despite this, when converting text to English, it is clear how bias in the data affects what we see as output: “She is beautiful. He is clever. He reads. She washes dishes.”

A recent study by researchers in Carnegie Mellon’s Psychology Department found that psychological gender bias was more prevalent in speakers of more gender-biased languages [2]. Thus, it is critical to prevent the propagation of this bias by being more cognizant of the extent to which machine translation can inadvertently fail.

But why exactly did Google Translate produce these results, and is the algorithm at fault? To answer these questions and others like them, it is important to further explore machine translation, and specifically the tools upon which it is built. One of which is word embedding, a technique that looks at the clustering of words in multi-dimensional vector spaces. In simpler terms, it is a way of converting words into numbers so that a computer can understand and organize them - words that appear together more often are placed closer to each other, while those with different associations are further apart. For example, when looking at different professions, you may see “doctor,” “lawyer,” “engineer,” or “computer programmer,” clustered closer to “man,” while “woman” appears more often with “housekeeper,” “waitress,” “secretary,” and “nurse.” The issue here is not necessarily the dataset or its apparent bias, many of the stereotypes above may in fact be true. And since Google Translate’s algorithm functions off these clusters in the dataset rather than a more nuanced semantic understanding, it is conceivable why and how the algorithm would choose a gender for each particular phrase. Nonetheless, there still remains an important distinction to be drawn. Although, for example, more men may be computer programmers than women, this is very different from saying that men are better computer programmers. In 2015, it was uncovered that Amazon’s resume screening software was discriminating against women for this very misconception. Since women were underrepresented in the training dataset, the algorithm learned to associate men with being more successful software engineers and was thus throwing out female resumes at a disproportionate rate [3].

Given the potential for artificial intelligence to unintentionally perpetuate discrimination, it’s essential to consider how to prevent bias in AI systems. Simply debiasing datasets may not be sufficient, so what regulatory measures work or are even feasible to implement?

Aylin Caliskan, Assistant Professor at the University of Washington Paul G. Allen School of Computer Science & Engineering, believes that a great place to start is increasing diversity on artificial intelligence teams: “Diversifying the pool of AI talent can contribute to value-sensitive design and curating higher quality training sets representative of social groups and their needs” [3]. Simply based on lived experiences, a more diverse team is more likely to uncover when and where bias or discrimination may occur.

An alternate solution may be greater transparency in regard to the AI technology used. Recently, Elon Musk made the Twitter recommendation algorithm open source, meaning that it is now accessible to anyone. This is a great step toward a more equitable future, and will hopefully lead to other companies following suit. Increased transparency makes it easier to catch inadvertent discrimination or bias and will make the field easier to regulate altogether, as third-party auditors and scientific researchers can locate and fix any marginalization that may arise.

In the end, regardless of the path we choose, it is of utmost importance we act sooner rather than later, as the future’s AI is trained on the data of today.

References

1. Niethammer, Carmen. “Ai Bias Could Put Women’s Lives at Risk - a Challenge for Regulators.” Forbes, Forbes Magazine, 27 May 2020, https://www.forbes.com/sites/carmenniethammer/2020/03/02/ai-bias-could-put-womens-lives-at-riska-challenge-for-regulators/

2. Ruiz Costa-Jussà, Marta. “An Analysis of Gender Bias Studies in Natural Language Processing.” Nature Machine Intelligence, Springer Science and Business Media LLC, 31 May 2021, https://upcommons.upc.edu/handle/2117/346419

3. Caliskan, Aylin. “Detecting and Mitigating Bias in Natural Language Processing.” Brookings, Brookings, 9 Mar. 2022, https://www.brookings.edu/research/detecting-and-mitigating-bias-in-natural-language-processing-sciencedirect.com/science/article/pii/S0304389420320604