One of the most significant areas of Data Ethics concern is bias in generative AI models.
Concerns about algorithmic bias are not new. Associative model (i.e. neural network) bias is not new. Even the ceding of consequential decisions to these models is not new. Just a few examples: AI is used to screen resumes and evaluate candidates. It is used in loan origination, underwriting, and risk assessment. It is used in health care diagnoses. Facial recognition is used by law enforcement. Some pretty high-stakes, high-impact applications.
The combination of the increased use of these models in applications that touch people in their everyday lives and the recent reporting of several stories demonstrating their potential for bias has created a critical mass of awareness. And skepticism. And concern. And rightly so.
So many examples, but it was the initial rollout of Google Gemini’s image generation that brought the issue of AI bias to the forefront of public discussion. Predictably, it led to calls for the government to regulate artificial intelligence. Curiously, when I searched for high-profile examples of generative AI bias, Gemini was not listed among the Google search results.
A 2025 Stanford University study found that “users overwhelmingly perceive that some of the most popular [Large Language Models] have a left-leaning political bias … The researchers then show that with just a small tweak, many models can be prompted to take a more neutral stance that more users trust.” That tweak, specifically, is to prompt the LLM that its previous response was biased and request one that is more balanced. My first question when I came across this study was how they assessed the objectivity of those who scored the LLMs’ responses. Evaluating bias in the political realm is very difficult due to a lack of agreement on even the most basic facts. Furthermore, to many, bias is defined as an opinion or worldview that differs from their own. It was reassuring to see that the researchers considered this in their analysis.
So, it seems that the skepticism is well founded.
And that’s a problem because confidence in these models is imperative. Their pervasiveness in our lives will only increase. It seems that every company is throwing something, everything, against the AI wall hoping that something will stick. One way to increase confidence is through transparency, but neural networks are the opposite of transparent. (I’ll talk more about observability in an upcoming article.) We can measure bias in our trained models, but…
By the time you’re evaluating bias in a trained associative model, it’s too late.
My graduate school research focused on two different artificial intelligence training methods: neural networks and genetic algorithms. In both cases, once a model was fully trained to solve a specific problem, it was very difficult and often impossible to change the problem without starting over. Different initial configuration? OK. Slightly different physics (like changing the effects of friction or inertia)? Sometimes OK. Accomplishing a second objective simultaneously? Almost never OK.
The more complex the organism (or model) the more difficult it is to substantively change it. Behaviors can be adapted but its basic nature remains the same. Our cravings for sugars and carbohydrates may have at one point been evolutionarily advantageous, but they work against most of us these days. We can adapt our behavior through diet modification, but our basic nature remains the same. Similarly, we can add controls to our generative model interfaces to try to make it appear to respond more objectively, but its basic nature remains the same. Many organizations have had success training one model with a baseline set of information, and then layering additional domain-specific expertise on top of it. Keep it simple and focused.
Since bias, once baked into the model, is extremely difficult to remove, you have to get it right the first time.
Bias is a function of training content.
An associative system will respond in a way that’s consistent with its training. Regardless of domain. An untrained model is a blank slate (or, more accurately, a random slate). Therefore, if bias exists in the model, it must have been learned.
Just as you are what you eat, your models are what they eat.
It is critical that the content of your training data be fully representative across your domain(s) of interest, and that all of the factors are appropriately weighted (i.e. feature engineering). In order to do that, you need to understand your data. Want to know why corporate AI projects underperform at an astonishing rate? Want to know why models often generate unexpected results? All roads lead back to understanding your data. Evaluate the data used to train your models.
If you want an unbiased model, train it with unbiased data; just like you train with accurate data when you want an accurate model.
And just as Data Quality analysis is an ongoing activity, so, too, is generative model input data analysis.
Don’t get me wrong, it is important to evaluate your trained models for bias as well, but evaluating input data for bias (or objectivity) should become a formality like scanning source code for obvious programming errors. Keep the models simple and focused. And above all else, understand the data that was used to train them.
Featured Image Credit: reginaldlewis, “Twinkies Wallpaper,” Wallpaper Cave.
0 Comments