What is Bias & Why Does It Happen in LLMs?
The 3 Levels of Bias
Bias has something profound to teach us about the human condition, which relates to attention.
We all have a limited amount of attention and give our attention to the things we value.
But how do we know what objects to give our attention to?
How does the evaluation happen?
It happens a priori.
We don't have enough time to scan the environment and evaluate every single object.
Values
Instead, we simply filter the world for what we value.
As a result, we are more familiar with and prefer what we value. These preferences can make us blind to other information and miscalculate its importance and value, which leads to bias.
So, bias first happens a priori; it happens before we give our attention to an object, person, experience, or event.
Now, armed with this knowledge, see if you can count how many times the players in white pass the ball in the video below.
Did you get it right?
Odds are you did at the expense of noticing other important changes.
Even if you were just observing, you might have notices all of the big changes but probably don’t know how many times the ball was passed back and forth in total.
This is an example of how we focus on what we value and miss out on other important facts.
The most interesting part of this experiment is that knowing this is not enough.
Perspective
The second level of bias occurs as a result of perspective.
When we look at an object, we don't see all the information about it.
We see it from one viewpoint at one point in time. Our perspective is even more limited with people as we see them in a given situation, environment, and context. So, no matter how we slice it, we have an incomplete and skewed sample of data.
Mean Making
And lastly, there is mean-making.
We are interpretative beings that give meaning to experiences.
Experiences don't have a specific meaning. For example, there is no universal object or situation called 'anger' that produces the feeling of being angry in every human.
The same situation will make some people very angry, and others might not even notice.
We interpret ‘the situation’ and give it meaning. If the situation does not have any meaning, it won’t even be perceived by us!
Take a look at the experiments below:
The Bias Loop
So now we have the complete Bias Loop. Bias happens on these three fundamental levels:
Attention Level: You automatically pay attention based on your values.
Perspective Level: You see things from a single perspective in time and space, which gives you only a very small snapshot of what something is.
Meaning-Making Level: You interpret the information from your perspective based on your current state and your primary drives.
Human Bias’s Impact on LLMs
And this has a direct impact on LLMs.
First, LLMs work via a mechanism called 'Attention'. (To learn more about how this works, check out the paper 'Attention is all you need".)
Second,'model weights' are a byproduct of a mostly modern western perspective, which has its benefits and costs.
Last but not least, people trained these models and decided what things meant. The meaning of things doesn't come from thin air; it is informed by our cultural values and norms and what we believe to be true about the world.
So, where does this leave us?
Hopefully, this makes us more self-aware.
By understanding how our awareness and attention work within us, we can begin to notice patterns that can lead to bias, favoritism, and prejudice at each of the three levels.
We can see how 'good data' and 'good intentions' can leave out the 'bad data' and 'bad intentions,' which produce an incomplete, biased model.
Knowing a bit about bias, we are finally ready to play around and experiment with knowledge bases.
Join the Journey
It’s almost here. Soon, I will be sharing the details for Project X!
Then, over the next few weeks and months, I will create guides and deep dives into how Project X was created and some of the early results.
Along the way, I will also share the technological and philosophical foundations of this project.
Cheers
Stefan