What are cognitive biases?

In my first post, I described epistemic and instrumental rationality. In this post we’re going to take a look at one of the greatest enemies of epistemic rationality: your own brain.

In all seriousness, a human brain does a lot of things right, but it also does a lot of things wrong. One of the ways your brain can do things wrong is called a cognitive bias. If you’ve spend some time around Less Wrong affiliated people, you’ve probably heard of the word “bias”, but what does it mean? In the following paragraphs I’ll attempt to explain what a cognitive bias is.

You can compare a cognitive biases with optical illusions.

You probably already seen this one, so you know all the lines are straight, but you can’t help but perceive them as crooked. A cognitive bias is similar, but instead of tricking the way you see, it tricks the way you think. Even if all the information you have is accurate, your brain can come to incorrect conclusions.

Here‘s an example of how your brain can be tricked:

It’s important that you note the first answer that comes to mind, you shouldn’t be spending time writing this down and working out the equation.

“A baseball bat and a baseball together cost $1.10. The bat costs one dollar more than the ball. How much does the ball cost?”

Just note the first answer that comes to mind.

There’s a good chance you thought $0.10. This is wrong. Feel free to work out why, or click the link if you want to know the correct answer.

And if you got it wrong: don’t feel bad, more than half of the people answering that question made the same mistake.

This is just one of the ways in which your brain can fool itself and the list of all the ways in which this sort of thing can happen is really, really long.

So really, you can’t trust your brain when looking at the world. This isn’t your fault however. Your brain isn’t equipped for all this. It’s the product of evolution, made so you can successfully hunt, gather and do tribal politics. It wasn’t made for solving ball-and-bat problems or looking at weird circles that seem to move. Those things just didn’t happen in the environment you were made for.

And really, a lot of biases are really just shortcuts your brain takes because a lot of the time there isn’t any problem in taking them. Nevertheless you need to be prepared for the times when taking the shortcuts is dangerous. There are people who know about biases and can and will exploit them to get ahead. There are problems you won’t be able to solve properly unless you overcome certain biases.

Which leads us to the good news. Most biases can be defeated or circumvented. It isn’t always easy, but by reading this far you’ve already taken the first step. Knowing about certain biases can be enough to get to more accurate beliefs for some people.

In the future, we will discuss specific biases and how to overcome them. If there’s a particular one you want to see discussed, feel free to let me know.

What does rationality mean, anyway?

The very first thing we need to talk about is what I actually mean when I say “rational” or “rationality.” This is especially important because the way people use the word in everyday life is a bit different from the way aspiring rationalists use it. I believe that some conflicts between aspiring rationalists and people who don’t identify as such comes from this difference in meaning.

I won’t dwell too much on the everyday meaning of the word, since I’d like you to forget that when reading this blog. People often say “rational” when they want to indicate the opposite of “emotional”, which is a pet-peeve of a lot of rationalists. The few occasions I’ve seen the word used outside of a Less Wrong context, it’s often been used to judge people. Things like: “You need to stop being angry about being misgendered, that just isn’t rational.” (Sure, that’s an extreme example, but it gets the point across.)

Suffice to say, aspiring rationalists such as me try to avoid using the word like that. But how do we use the word?

A lot of concepts used by aspiring rationalist of the Less Wrong variety stem from economics and cognitive psychology and “rational” isn’t much different. The way I’ll use the word on this blog differs slightly from how those disciplines use it, but there’s still significant overlap so if you end up reading a book written by an economist, you should have an idea what they’re talking about.

Rationality, in the sense that I’ll use it, consists of two related but different concepts: Epistemic Rationality and Instrumental Rationality. These can be summarized briefly as “knowing true things” and “reaching your goals.”

Epistemic rationality is making sure that you believe true things about the world. This is harder than it intuitively sounds. The way most human brains work is that they care more about the consistency of their beliefs than about how true those beliefs are. There’s a host of cognitive biases (which we’ll talk more about next week) and other things that make it harder to see reality clearly. Even knowing things about yourself is a lot harder than a lot of people tend to assume.

Part of rationality is learning how to counteract those biases and how to deal with them if you can’t do that. It’s also learning to actually change your mind and making sure false information doesn’t stick in your head.

This is part of what I hope you’ll get out of this blog. That you’ll learn ways of making sure the ideas you have about reality become more accurate.

The other part is instrumental rationality, which is about reaching your goals effectively. I think it goes without saying that reaching goals isn’t always easy. There are many ways to not reach them and only a limited number of methods to actually get there. Instrumental rationality is about finding the paths that lead you to your goals. This can be overcoming personal limitations, but also discovering new methods to reach goals, to avoid things that work in opposition to your goals…

It can also be about deciding on your goals, or refining your goals. It’s perfectly possible that you don’t know what your goals are or that the goals you’ve set yourself won’t lead to things you actually want.

I probably won’t teach you how to reach specific goals (like getting your driver’s license, or joining the army, or making spaghetti…) but there are rationality techniques that can be generalized to multiple goals.

Epistemic and instrumental rationality are also intertwined. If your beliefs about the world are more accurate, you’ll find it easier to shape that world in a direction that leads towards your goals and instrumental rationality can help you get the information or resources you need to come to those correct beliefs.

I hope I’ve managed to explain what I mean with rationality, so that we’re all on the same page going forward. And because it’s the first day, I won’t even assign you homework.

If you have questions, comments, remarks… feel free to contact me with them.