#14 Planning Fallacy: Why the best laid plans always go awry
Do you find that things always take longer than you thought? Are you always running late? What if we told you it was all because you’re an irrational optimist? Chances are, you wouldn’t change a thing. In this episode, Mel and Dan consider the planning fallacy, and how our optimistic view of the world makes us underestimate how long things will take, and how much they'll cost.
Mel: 00:20 Hi and welcome to Bad Decisions.
Dan: 00:22 The podcast that helps us understand why we choose what we choose.
Mel: 00:25 Why we think, what we think.
Dan: 00:26 And how to explore this stuff for fun and commercial gain.
Mel: 00:29 I'm Dr. Mel Weinberg. I'm a performance psychologist.
Dan: 00:31 And I'm Dan Monheit, co-founder of Hardhat, a creative agency built for today. Spin it.
Hey, you know what I was thinking?
Mel: 00:44 What?
Dan: 00:44 Well I was thinking how as you get older, so many of these things that seem like good advice or conventional wisdom are actually just completely and utterly incorrect.
Mel: 00:54 It's like all those quotes on Instagram that people say that just aren't really true.
Dan: 00:57 Yeah, but even even like pre-Instagram things. So the one that actually got me thinking about it last week was the idea that whatever doesn't kill me can only make me stronger.
Mel: 01:07 Yeah. Okay. I don't like that at all.
Dan: 01:07 No, this makes no sense. Like, especially as I'm getting older and lots of things hurt me, but don't kill me, definitely make me weaker.
Mel: 01:13 And from a psychological perspective, people are most vulnerable to psychological illness after traumatic events happen, they don't get stronger from them they actually get weaker.
Dan: 01:21 Yeah, so it's like, hey, I just had half my arm mauled off by a bear, didn't kill me, probably not going to be stronger than before. In fact, I have like a massively open infectious, gaping wound where my arm used to be.
Mel: 01:34 Makes you feel good about yourself though, to say it.
Dan: 01:35 Officially not stronger than before.
Mel: 01:37 Officially, scientifically weaker.
Dan: 01:42 And you know, other things like sleeping like a baby, that's total bullshit. Babies wake up all the time and they shit the bed, like that's no way to sleep. But actually the one that really got me thinking was this idea that if we fail to plan, we plan to fail.
Mel: 01:54 Why?
Dan: 01:55 Well, because I feel like I plan a lot and I usually get it wrong. In fact, it seems like whether you do plan or you don't plan, you're pretty much going to screw it up anyway.
Mel: 02:02 Give me an example.
Dan: 02:03 Well, what actually got me started thinking about this was I was doing some research into different iconic Australian destinations, or locations.
Mel: 02:10 Must have been a thrilling weekends for you.
Dan: 02:13 No, this is what happens in agency land. And I was looking at the Sydney Opera House and I guess for anybody that was alive when this was being built, maybe this is old news, but I almost fell off my chair when I learned that the Sydney Opera House was meant to be completed in 1963 and was actually completed, guess?
Mel: 02:27 1965.
Dan: 02:29 1973, which was a lazy 10 years, whatever. It was simpler times maybe nobody really cared, but the thing that's really crazy is not the extra 10 years it took, it's the fact that it was originally meant to cost $7,000,000, which now is like a two bedroom townhouse in Sydney anyway. So it was meant to cost $7,000,000. Guess how much it ended up costing by the time I'd finished 10 years behind schedule?
Mel: 02:51 Tell me.
Dan: 02:52 $102 million.
Mel: 02:54 That is so much more than 7,000,000.
Dan: 02:54 Yeah, so I don't know what that is, it's like 15 times over budget. And it's like, this is a big project and I should mention that the project that got launched 10 years late and 15 times over budget was actually a scaled down version of what they actually intended to do.
Mel: 03:08 That's ridiculous.
Dan: 03:08 It is ridiculous. So it shows that even on the most high profile projects where there's best people working on them, planning doesn't really seem to help.
Mel: 03:16 So it's interesting you mentioned this and I think it's the same sort of thing that explains why I'm always late to everything. Even though I know there's a possibility of traffic, like if I'm going home I feel like it's going to take me 10 minutes.
Dan: 03:27 Yeah. Why would there be traffic - we're optimists?
Mel: 03:30 Yeah. I can just like scoot over it, flying right over the traffic. It's also the same belief that makes me think that I'm going to get a whole bunch of work done on the weekend, so I'll take a whole bunch of stuff home, and then sit and watch the footy all day.
Dan: 03:44 Yeah.
Mel: 03:44 Yeah, you know what it is?
Dan: 03:44 What is it?
Mel: 03:45 It's called the planning fallacy.
Dan: 03:47 The planning fallacy.
Mel: 03:48 And we're all victim to it.
Dan: 03:49 Every single one of us.
Mel: 03:50 It's something that particularly bothers me and I'm sure it affects you as well in sort of the consulting world where people are planning projects and stuff because people are really bad at planning how long things are going to take and how much things are going to cost.
Dan: 04:03 But you know where we're really good at?
Mel: 04:03 What's that?
Dan: 04:05 Being pressured into saying “yeah, yeah, we can do heaps of stuff in not very much time at all” and clients expect it.
Mel: 04:09 And I think it's the industry that both of us are in that when you're trying to win a contract or when you're trying to compete against other firms or other agencies for whether it's grants or whether it's projects, that you have to actually pitch yourselves as being able to do a whole lot of stuff in minimal time.
Dan: 04:25 Yeah, which optimistically you'll probably be fine to do, but realistically might be tougher than you anticipated.
Mel: 04:30 So this is the planning fallacy and we're all victim to it. It's the tendency to hold a confident belief that one's own project is going to proceed as planned even while knowing that the vast majority of similar projects have run late.
Dan: 04:44 Yeah, because the normal rules of the universe don't apply to me, just everybody else.
Mel: 04:44 Exactly.
Dan: 04:49 I mean, it feels like a silly question to ask, but do you have any research to support this?
Mel: 04:53 As a matter of fact, I do.
Dan: 04:54 Let's get the research music.
Mel: 05:02 So today's research comes from 1994.
Dan: 05:05 You get a certain glow about you when you start talking about the research.
Mel: 05:06 As I talk about the research can you tell I'm passionate?
Dan: 05:09 Which is not apparent in any other components of the podcast. I'm just saying.
Mel: 05:13 So 1994 Buehler, Griffin and Ross were exploring the ways that people have these self serving optimistic biases when it comes to planning things. And these research is particularly close to my heart because it involved honours in psychology students. I completed my honours in psychology, I supervised...
Dan: 05:35 Do you have a qualification? I was not aware.
Mel: 05:35 And I even supervised a number of students to gain their fourth year qualifications in psychology. So this is really close to my heart.
Dan: 05:43 You know if I said I observed a number of students it would be creepy, but when you do it is fine.
Mel: 05:46 Supervised.
Dan: 05:47 Supervised. Okay, all right, go on.
Mel: 05:49 Don't you go observing any students, it's weird. And basically what they did was they had people in their final semester, students in their final semester and they asked them to estimate how long it would take you to finish your thesis? You should be pretty close to the end of it, right? Your finals semester.
Dan: 06:05 And you're probably pretty smart by this part of your education.
Mel: 06:08 Well, it depends which of my students you were but... Okay, okay I'm kidding, I love all my students, they're all good.
Dan: 06:14 You're all above average.
Mel: 06:16 Just remember that. How long will it take you to finish your thesis? You're in your final semester so you're getting through it and most people said that at that time, yeah, I think it's going to take me about it's like 34 days, okay? Now, assuming like most people at this stage, they've got a plan, they're writing it, they've got their data, they're just writing it up, right all they have to do is really write it up.
Hopefully they’ve got a good supervisor who will get them through. They also ask them to say, you know, look if everything goes totally according to plan, how long is it gonna take? And they said “look, if everything going totally according to plan, I'll give myself a week less maybe 27 days”. And also what about if like everything stuffs up, what about if everything that could possibly go wrong goes wrong? And they say “yep, that's 48 days”
Dan: 06:57 So we got 27?
Mel: 06:59 At best.
Dan: 06:59 34?
Mel: 07:01 33, just saying how long, yeah.
Dan: 07:03 And 48?
Mel: 07:04 48 at absolute worst.
Dan: 07:05 Okay, yeah.
Mel: 07:06 And you know how long it took, the average that they took these people to finish?
Dan: 07:09 How long?
Mel: 07:10 55 days.
Dan: 07:11 That's upsetting.
Mel: 07:12 So even when they thought about everything that could possibly go wrong, it was another week on top of that and that average did not include a number of students whose final numbers couldn't even be included because they hadn't even finished by the time the data collection for this research was finished.
Dan: 07:25 Humans. This is a weird thing. Like I just think in most of the heuristics we've talked about, they kind of at least try to set us up for success and this just seems like a weird thing to have had this hardwired thing in our brains that makes us constantly disappoint ourselves and those we love.
Mel: 07:42 It's like you think something's going to happen. You plan for something to happen. And you're wrong.
Dan: 07:46 Yeah, but all the time, constantly. Why would this be like, does the universe want my wife to be constantly angry with me because I'm home later than I said I would be because why would there be traffic?
Mel: 07:55 Look at it does serve a motivational function, which is that at least makes us think that things are doable, things are achievable because if you think too far in advance, if you think that things are just too big and too grand and too expensive, you're not going to do anything. So it serves a highly motivational function. It makes us think that we can actually do something, we can actually get things done. We can get things done on time, we can get places on time and it drives us to actually do it.
Dan: 08:17 So if I understand correctly, what you're saying is if we were actually realistic and rational about how we saw our chances of completing things-
Mel: 08:24 We'd never get anything done.
Dan: 08:25 We wouldn't even bother starting. We'd all be miserable and depressed.
Mel: 08:28 Yeah. Well that's the flip side of things that you're constantly planning for failure and constantly planning for the worst case scenario. And that's not a good way to live.
Dan: 08:33 No, those people suck.
Mel: 08:34 Yeah.
Dan: 08:35 All right. So obviously you know, when human beings try to complete projects on the balance of probability, things are not going to go as planned. And so I would suggest that there are two giant contributors to this, one set is to do with people and the other set of giant contributors is just to do with the nature of projects. So how about we tackle one of these each? Maybe we can flip for it.
Mel: 08:56 Okay. I'll talk about the humans.
Dan: 08:58 Okay, you do humans.
Mel: 09:00 I win humans please, you talk projects.
Dan: 09:02 All right. What do you got?
Mel: 09:03 To explore the human factors that contribute to these, we're actually going to go back to one of the first papers, would you believe one of the first research papers that I ever read when I was studying well-being and human performance. And it was... Yeah, it's nice, it's got a bit of meaning to it this one. It's a study by Taylor and Brown in 1988. Sorry for more research, but it's obviously the thing I like and it's about how people have these positive mis-perceptions like those things you were talking about at the start those sayings that are not true, but help us feel good. There were three main things that we do that are called illusions because that sounds really magical, we call them positive cognitive biases. What Taylor and Brown in 1988 were talking about and what they actually challenged was this conception that a mentally healthy mind was a rational mind. Right?
What they actually showed was that a mentally healthy mind is one that is actually completely irrational. Sounds a little bit contradictory to what we would think, but in order to maintain our well-being, our motivation all these sorts of things, what we need to do is actually have these misguided views of the world and we do that according to three things. The first is that we think that we're better than everybody else, so we have this exaggerated sense of self esteem, which is why when it comes to the planning fallacy, we think, I know everybody else has failed at this, but I'm not going to do that.
Dan: 10:20 Because I am awesome.
Mel: 10:20 I am so much better than everybody else, but yeah, they've all stuffed up, but I'm going to be fine.
Dan: 10:24 Look I know I haven't done stats, probably to the great degree that you have, but it's pretty much impossible for us all to be above average, right?
Mel: 10:30 Correct. You can do that with minimal statistical knowledge.
Dan: 10:36 It breaks the definition of average, but we all think we're above average.
Mel: 10:38 The other thing is that we all think we can control the future and as nice as it sounds, we are very bad at estimating how unpredictable the future is and there are a lot of double negatives in that sentence I just said, but basically we fail to predict how unpredictable the future is.
Dan: 10:54 And it's kind of funny to think that we can't control the future at all, but there are like huge industries set up just around helping us create the illusion, do we need magical music? The illusion that we can control all of the project planning software and the gantt charts and the spreadsheets. We're all just making shit up, let's be honest. Unless you're one of my clients, in which case we know exactly what we're doing and it is going to follow this gantt chart exactly.
Mel: 11:19 And where everything will be done and delivered on time and to budget. The final thing, and we've talked about this before, is that we have an an unrealistic sense of optimism, right. We think that everything's going to go fine and the bad things are going happen to everybody else and not us, and our project is going to go completely according to plan. So all of these factors, the idea that we have, and enhanced sense of self esteem, and enhanced sense of control over the world, and an enhanced sense of optimism all contribute to us making massive errors when it comes to planning.
Dan: 11:47 Right. Yeah. So that's a pretty dangerous triad of afflictions that we all suffer from. It's like you know that that sentiment that like everything happens for a reason?
Mel: 11:57 Mmm it doesn’t ...
Dan: 11:58 Well sometimes the reason is that we're idiots.
Mel: 12:01 Yeah that's true, that's the underlying reason.
Dan: 12:03 That's the reason, right. Okay. So just humans are basically not wired to be very good at planning projects. We all think we're going to smash it and we're better than everyone else and we can control our future but even if that was aside, there's just stuff about the nature of projects that makes them really, really hard to plan for accurately. So I'll see your three and match it with three.
Mel: 12:03 Give me three.
Dan: 12:23 Well, the first one is that there's this natural asymmetry of information you find out as you start a project, so you go into a project knowing that you don't really know what's going to happen in there, and you're going to discover new things, but there's a natural asymmetry between the amount of bad or adversely effective things that you find versus the number or the amount of good things that you find.
So of all the things that you don't know that you're going to run into, more than half of them are going to be bad. You might find a couple of nice little upside, you might go hire somebody halfway through the project who freakishly has done this exact project before, but chances are most new information will be bad information.
Mel: 12:59 That's a scary thought.
Dan: 13:00 Yeah. So that's one. The second one is just issues inherent with scaling. So we tend to underestimate how hard scaling is and the problems and issues are not linear. So if you are building one bridge, or you've built one bridge before and you think you kinda know what you're doing and what sort of issues you're going to overcome, and then you decide that you're going to build two interconnecting bridges, the issues will not be doubled.
Mel: 13:23 Right, so you're talking about up-scaling but not understanding how the errors associated with that work.
Dan: 13:29 Yeah, so it's like you could double the size of a project and you will get more than double the amount of errors in the same way if you think about what it takes to run a 50 person business is not double what it takes to run a 25 person business. It's more because let's say at 25 people, as an owner, you kind of know everyone and can manage everyone and you've got visibility over every project and every client and every person, whereas at 50 you don't. So it's not like running two 25's because you need a whole layer of management, you need systems, and processes, and performance reviews, and career progression plans, and all of these sorts of things. So every project grows in complexity more than it grows in size or disproportionately to how it grows in size.
Mel: 13:29 That makes sense.
Dan: 14:10 Yeah. So there's the natural asymmetry, there's the scalability issues, and then the third one is something I discovered called Brook's Law. Well Brook's Law is an observation that relates to software projects but probably relates to most things in business and in life.
Mel: 14:25 What's Brook's Law?
Dan: 14:26 So Brook's Law is the idea that if a project is already running late, which as we've learned today, most projects are already running late before they even started and you add a new person to the project.
Mel: 14:36 Right, to help it along.
Dan: 14:37 Yeah. To help it along, chances are it's going to be later than if you hadn't added that extra person or extra people. So there's a whole bunch of reasons for this and if you work in in software like you see this happen all the time. So number one, the new person or the new people that come in need ramp up time, so you get loss productivity while everyone's trying to catch these people up. The second thing is around communication overheads, so as you add more and more people to the project, you need to spend more and more time explaining to everybody what's going on and having pre-project catch-ups, and mid project catch-ups, and making sure everybody knows what everybody else is doing so that takes time and further blows things out.
And then the third thing is the non divisibility of work, so sometimes we think that if you have a person doing something that's going to take a day, if you just put two people on it, it's going to take half a day, but a lot of projects actually don't work like that and you can't effectively break up a task evenly into little pieces for more and more people to get involved. Sometimes it's like putting something in a microwave, you can't put something in two microwave for five minutes each at the same time instead of just putting it in one microwave for 10 minutes.
Mel: 15:39 All of that would change my cooking but, yeah. I also wonder whether when somebody comes in late to a project because they know it's already late, it's almost like they have this idea or this view that you know what, it's acceptable in the scope of this project, lateness is acceptable and lateness can be fixed. So there's almost like no pressure or not as much pressure to be done on time because lateness is something that is part of the project.
Dan: 16:00 Or the person's come in and it's already fucked, so like you can't fall off the floor and the thing is most projects, in fact, I would almost say almost every project has a new person added to it somewhere beyond the halfway mark. So between the natural asymmetry of what we've learned about projects as we take them on, the inherent issues in scaling, and the fact that adding new people to an already late project is just going to make it even later it honestly is a wonder that we ever get anything done.
Mel: 16:24 It's amazing.
Dan: 16:25 And sometimes I've just wish I had a psychologist friend that I could turn to and say with all of this stuff stacked against us, with the issues of being human, and the issues of working on projects, whatever can we do?
Mel: 16:36 Well, Dan, let me be that psychologist. I'm going to offer you a couple of things that people can do and then you'll tell me some brand new applications.
Dan: 16:45 Sure.
Mel: 16:45 Okay. We'll do a trade.
Dan: 16:46 That's my job here.
Mel: 16:47 So, the first thing that we can do is when we're planning how long something is going to take and how much something is going to cost, it can really help to have somebody play devil's advocate because we're all vulnerable to these things at an individual level and like I think I'm going to get everything done, but you maybe not so much, right? If I'm overlooking your project, then I might be able to highlight some things that could potentially go wrong with your project that maybe you wouldn't have seen and it could be really hard for you to hear that because obviously you think you're fantastic and everything's going to go according to plan and nobody likes to hear that sort of negative feedback, but it can be really important in terms of planning properly. It's basically conducting a pre-mortem of the project of going what are all the things that could possibly cause this project to fail and work backwards from there.
Dan: 17:31 So it's like ‘bring a hater to work’ day.
Mel: 17:34 Pretty much you can do that.
Dan: 17:35 You just got to invite someone whose going to shit all over your project, it's sounds fun.
Mel: 17:40 And go from there. Yeah, right fun ways to work.
Dan: 17:42 It's so much more enjoyable to just assume everything's going to go great.
Mel: 17:45 Yeah. The next thing that we could do and in terms of a budget perspective and thinking how we tend to underestimate how much things are going to cost, if possible you can put some contingency planning in your budget. So you could have an amount of money that just goes under contingency in the budget. It's like, we're not quite sure what we're going to spend this money on, but we're fairly sure that we're going to need it for something, so let's just have some spare change in the back that in case shit hits the fan, we've got some money to cover it and it's part of our budget.
Dan: 18:11 And when you say in case what you actually made news in the highly likely event that everything goes wrong.
Mel: 18:17 We're planning for the inevitable uncertainty.
Dan: 18:19 At least we'll have money to have a party.
Mel: 18:21 It's like we don't know what we're planning for, but we're planning for it.
Dan: 18:21 And it's going to cost cash.
Mel: 18:24 And when it happens, we're going to say we knew about it. The last thing we can do is that we can break down tasks. Okay, because looking forward into the future, things seem so abstract, so vague in the more abstract something is, the harder it is to conceptualise it. So if we break it down into small tasks, sort of like goal setting, right? If we break it down into things that we can achieve where we have more certainty about it, things that we know we can do, things that we know how long it will take, even if they're like minimal tasks, the more concrete we can make it, the more we're able to accurately estimate how much time it'll take and how much it's going to cost.
Dan: 18:58 For sure. I mean first hand experience, I can attest to this, like in the agency we work on lots of big complicated projects where at the start you really have no idea of even what you're doing. And so we force ourselves as a team to go through and turn this big ambiguous project into a whole bunch of cards, task cards. And a task card is four hours worth of work for somebody to do or less. And if a task is more than four hours, we need to split it into multiple tasks because four hours, like half a day you can kind of accurately estimate what you're going to get done but anything beyond that, we're basically just guessing.
Mel: 19:32 So putting it into much more practical, realistic terms.
Dan: 19:35 Bite size chunks.
Mel: 19:36 Bite size chunks?
Dan: 19:37 Yeah. So I mean look, as agencies and service providers, that's one thing that we can do but some of the other things we can do when we're trying to sell stuff is to really let this bias work in our favour. So really anytime you've got a service offering where you can let people self project, how often they'll use a product or service, chances are they're going to overstate how much they're going to use it. So if you imagine like selling an annual pass to a gym or to a swimming pool or something like that, people are doing the calculations on three or four times a week because of the planning fallacy they assume they're going to be able to organise their life to get that three or four times a week but in reality...
Mel: 20:15 Well, that's the thing and with the gym membership, all of a sudden it becomes, yeah, if I go four times a week, it's only going to cost me like $2 a session.
Dan: 20:21 Yeah, yeah, so that's why we have things like unlimited class passes because that lets the person do the calculations themselves and what do you know, they come up with a really optimistic answer. Similarly, like when you go to Queensland, you can get these three park super passes, and basically anything that's bundled and relies on people assuming they can do a certain number of things in a certain amount of time is going to work in your favour as a brand because they're probably not going to be able to do that. Let's just be honest about it.
Mel: 20:47 Yeah, and people just aren't as fit or as keen as they think they are to go to Dreamworld, Wet and Wild. What else? Movie World.
Dan: 20:54 Yeah, we're just, none of us are as organised as we think we might be.
Mel: 20:58 It sounds like a really good idea and then you get there and you're like, no, I just rather sit by the beach.
Dan: 21:02 Yeah. How good's the bench?
Mel: 21:03 Yeah, humans are lazy.
Dan: 21:04 Well, I think that's what we've learned today.
Mel: 21:07 You know Dan, at the end at the end of the day, you could always just apply Mel's theory.
Dan: 21:10 Mel's Theory?
Mel: 21:11 Yeah Mel's theory of planning.
Dan: 21:12 Should I check Wikipedia or are you just going to tell me what it is?
Mel: 21:13 Not yet, it’s not yet a thing - I'm telling you now, you're the first to hear this.
Dan: 21:18 Okay you heard it first listeners.
Mel: 21:19 Mel's theory when planning, when you have a project and you're thinking about how much time it's going to take you to do, estimate that amount of time and then just chuck on two weeks.
Dan: 21:28 What if it's like a three day project?
Mel: 21:29 I don't have three day projects.
Dan: 21:32 All right great theory, Dr. Mel.
Mel: 21:34 Thank you. I’m still working on it …
Dan: 21:36 So if you guys got thoughts on the planning fallacy or just on the show or just on Mill's theory.
Mel: 21:41 Any negative feedback is welcome @DrMelW.
Dan: 21:43 Yeah. Positive feedback is good for me @Danmonheit. I will see you guys next week, which is going to be here way quicker than any of us realise.
Mel: 21:50 See you next time.