All Predictions are Mistaken; the Role of Intelligence is to Lessen the Gap Between Estimates and Outcomes

[Arik Johnson]

Tom Hawes had a blog post recently about the trustworthiness of predictions which got me thinking it might be a useful exercise here to clear up a major misconception that has been a perennial plague on intelligence people and their value proposition; that is, the real role of intelligence in crafting predictions. Here's the thesis:

All Predictions Are Mistaken; the Role of Intelligence is to Lessen the Gap Between Estimates and Outcomes

Now, I'm certainly not alone in this position - I've heard versions of the idea from many colleagues over the years, yet there is a widespread belief that predictions are something solid decision makers can bank on.

Quite the contrary, as decision makers grow ever more intoxicated by the assumptions which accompany the responsibilities invested in them - many true, others false - intelligence must reveal these differences in a way that makes skepticism palatable and, indeed, valued as their foremost contribution to deliberation.

Let's talk this through - I still think too many, particularly new practitioners, remain convinced they were hired to predict outcomes, when in fact their job is to simply lessen the gap between estimates and outcomes.

Nothing short of the sustainability of our institutions is on the line.

[Eric Garland]

A few additional thoughts:

1. Intelligence sorts are hardly the only ones to make predictions - nearly every manager with P&L responsibility does sales forecasts, every investor relations manager does earnings guidance, etc. Yet intelligence professionals (especially futurists) get saddled with the crystal ball jokes and forced to feel like they are the only ones taking a look at the future that might be inaccurate. EVERYBODY thinks about the future and NOBODY is right.

2. If the intelligence profession has any use, it is because we are experts in not really knowing things. We point out where information is imperfect, and track it down. Because nobody has the correct view of the future, we invite multiple parties to the table to discuss the implications of the admittedly-imperfect information we have taken pains to uncover. In short, everyone talks about the future; we're the only people who will be honest about what we don't know and methodical about getting the most ROI from the information we do have.

This is an opportunity for branding, which leads us to Friday's Intel Collaborative conference call with Jason Voiovich.

[Arik Johnson]

Amen! Preach that good news, brother Eric ;-)

FYI, this Friday the 9th is the Wiki barn-raising discussion, Jason is joining us on Friday the 16th. I'll get a couple events up here today to broadcast and clarify.

[Jason Voiovich]

You guys really have no idea how interesting this discussion is to an "outsider" like me. It's not the content of the discussion, necessarily, but rather that you are having the discussion at all.

Put in context, in the branding/marketing/communications discipline (my field), we tend to be too busy patting each other on the back for our "brilliant" work. Rarely, will we engage in an open forum of criticism unless we are trying to "win" or "steal" an account.

Very simply, it harms the industry's ability to move forward.

I might say the ability to ask questions, with a process to answer them, and generate insights, might be as good a description of the intelligence brand. I can't think of another profession who brings this skill set to bear for its clients in quite the same way.

That seems like a defensible brand position to me, but heck, what do I know...

TTFN!

[Miguel Duarte Ferreira]

Fully agree - Great discussion!!!

Coming to your proposal I tend to understand it more as a description of the process rather than of the brand.

As human beings and not necessarily as Intelligence professionals, we deal with all timeframes dimensions – Past, Present, Future - when supporting decisions and/or making decisions.

It is my view that part of the complexity of trying to develop/create a brand out of this highly complex setting – three time dimensions – is one of the reason why we haven’t yet reached a satisfactory definition of our core benefit(s) and therefore of a strong enough brand.

I would also claim that I am not sure that we need to have a brand but that is another discussion (that I am ready to entail as well).

Yes, we need to ask questions. Yes, we need to have a process. And yes we need to generate insights. But in order to do all that, with a high degree of benefit to our customers, we need to have a reasonable understanding of the past, we need to grasp as much as possible the present and with all these we need to make sense of what can be expected, based on the previous dimension, and from there, hopefully have some positive (?) influence on the decision(s).

Now, I do not want to argue, at this stage if we are futurist or even what is a possible definition of futurist and in what way does it relates to all we are talking in this topic.

I am more interest to reflect on the interest of the above description of the process – I am still talking of a possible view of the process – and in what way can this be presented as something meaningful and of benefit to the ‘other’ (we can assume that it is a decision maker but not necessarily, it can be someone to whom we are trying to explain what we do and what is the benefit of doing it).

I agree with the topic title but I think it is still positioned or articulated in our ‘own eyes’ and not on the ‘eyes/shoes/brain’ of the ‘others’.

Now, how can we presented this ‘Role of Intelligence’ in a way that makes sense to ‘others’ and be perceive as a benefit? This can be a way to answer the quest on how to build a brand.

I do not have an answer, and I am sure that I know very little of this subject and others, but I am sure that we are closer to reach a ‘possible one’ with this kind of discussions.

[Arik Johnson]

"Sell the sizzle not the steak" as they say, don't sell a quarter-inch drill bit, sell a quarter-inch hole, etc.

I think this discussion of what we are and are not is extremely important to developing consensus around a new value proposition and the identity that comes with it. It's a little bit chicken-and-egg however - I think we need to mode-switch between the features and the benefits in order to nudge the message into shape properly.

Which is the reason I used a deliberately provocative thesis statement for this discussion - because we first must sober up from our intoxicated positioning before we can compel customers to find value in what we do for the long term.

[Eric Garland]

Miguel, sorry I'm American - what is this "past" you keep referring to? You mean the early 1990s, before the Internet?

Thanks in advance for clearing this up.

[Miguel Duarte Ferreira]

Eric,

I am sorry for the delay but I am 'on the road' and with limited access to the Web.

Nothing to excuse I am the one who needs to be careful when using the English language. Sorry to not be clear.

I'll try it now, briefly, and I'll come back to you on more detail later.

When referring to'Past' I meant to everything that was before the moment we are taking in consideration to our matter in hand our subject and relevant to our adequate understanding of the present and that will be of help to perveice (on the lack of a better word) what comes ahead (=future?).

[Eric Garland]

Sorry, Miguel, that was a sarcastic joke about American managerial culture. Since large swaths of this country consider buildings from 1920 "really old," there is a somewhat famous ignorance and disdain toward history. Other countries tend not to have this blindspot. I think Americans, particularly today, ought to consider the historical patterns of other nations to understand our future.

Your English is fantastic, don't worry!

[Arik Johnson]

Jason, thanks so much for the thoughtful reply and I fully agree - espousing the core values of the profession - e.g., skepticism, curiosity, etc. - requires that we let our collective guard down and have an honest conversation about what we stand for (and don't).

There are many who would say my thesis is under-promising when in fact we can deliver on that promise quite well. But why in the world are we so desperate to over-promise that we can do such an unbelievable thing as actually forecast what is going to happen in such a way that leaders can bank their capital on it.

We can't - at least not consistently, though anybody can get lucky from time to time.

Furthermore, I'll argue we can't even appreciate the circumstances of leaders who are the risk stewards of their shareholders' capital when we don't have our own capital in the game.

I love the positioning statement on capabilities however - our skepticism (and the humility that must accompany it) really is core to what we have to offer.

Pride always goes before the fall.

[Kristan J. Wheaton]

First, I agree with most of the comments to this post that Intelligence is about reducing uncertainty, not eliminating it. Communicating that effectively to the decisionmakers that intelligence supports is extremely important. Both our research and our experience at Mercyhurst suggests that the form of the estimate, forecast, prediction, etc. is as important than the content.

Specifically, we believe a complete estimate should contain not only the estimative statement but also a statement that indicates how likely it is that the analyst is wrong and, if the estimate turns out to be wrong, what are the likely reasons for failure. Using this form makes it clear to the decisionmaker that the estimate is just that -- an estimate.

But I think you need to take your thesis another crucial step forward. I would add that the role of intelligence is to lessen the gap between estimates and outcomes "regarding those things that are relevant to the success or failure of my organization (company, etc) but are outside of my organization's control." That is, intelligence is focused on those things that are external to the intelligence unit's organization.

I think this is a critical, but often forgotten or assumed away, factor that distinguishes intel from other activities.

Intelligence has always been about "the other guy" -- the competitor, the criminal, the "enemy". Understanding how these external forces are likely to evolve allows decisionmakers to manage their internal resources more effectively.

Highlighting this external focus solves many of the issues raised in these posts. Making this distinction explicit helps make the value proposition, for example. Who in my organization is explicitly tasked with looking at those external forces that are relevant to our success or failure but are outside our control? If we look at our organization's job descriptions, the answer is often "nobody". Such a task is rarely made explicit but to ask the question, "Shouldn't we have someone looking at this stuff?" is to answer it: Of course we should.

Sometimes the answer is "everybody". Obviously BS. The easy response is to say, yes, and everyone cares about Quality and People but that doesn't keep organizations from having dedicated quality control and HR personnel. More convincingly, decisionmakers rarely have the time to focus on the external and even when they do, who is collecting, coordinating and collating and analyzing their information? There is a clear need for an intelligence unit within an organization -- any organization.

In addition, explicitly including an external focus helps define what kind of products the intelligence unit should produce. For example, intel units should not make "recommendation" in my mind. To make a recommendation implies that the intel unit has an intimate understanding of the capabilities and limits of its parent organization. Yet, if it has been doing its job, it has been focused externally and is not in a particularly good position to make recommendations to the VP of operations (for example) about what he or she should or should not do.

Rather, intel should deliver "findings and estimates" regarding how the relevant external forces are likely to change over time. This gives those people in the organization with real expertise regarding the internal forces an opportunity to maximize the use of resources to take advantage of opportunities or mitigate (or eliminate) threats.

Making it clear that intel is about the external forces buffeting an organization (and not about organizing a response to those forces) makes the task of "selling intel" to decisionmakers much easier.

[Arik Johnson]

Thanks Kris and apologies it took me so long to reply. I agree as well that you've got the right point of view going here.

Just out of curiosity, will you be available to join the Intel Collab teleconference Friday morning at 10:30? I'd be very interested in your take on the "Rebranding CI" discussion - here's the event link:

http://competitiveintelligence.ning.com/events/20100416-intelligence