Are we at the limits of quantitative forecasting?

[Eric Garland]

A polemic for us competitive intelligence types...

The financial blog ZeroHedge points out that JP Morgan has now taken out a $3 billion reserve to hedge against the potentially faulty judgments of their quantitative analysts.

For those of us in the world of largely qualitative analysis, this is a fairly unprecedented move, one that cuts across the grain of most schools of modern managerial thought. After all, wouldn't you say that numbers, spreadsheets, ATTRACTIVE PIE CHARTS, no matter how fallacious, are still better to most executives than incisive, effective, insightful analysis? Or, in the words of our dear colleague August Jackson, fake numbers will trump real insight almost every time.

Surely, as intelligence mavens, we're not against hard numbers, but you should be able to analyze the assumptions behind those numbers. Speaking of which, the ZeroHedge article pulls a shocking statistic out of the history of the subprime debacle. Check out what the quantitative model predicted subprime losses to be, as opposed to the actual losses, factors of 100 greater. Holy cats...

Given all the fake numbers in forecasts lately, what do you think of the future between quantitative and qualitative analysis?

[klaus Söilen]

That is fine, Eric, I am old enough to understand a friendly hug!

K

[Nimalan]

Interesting topic as this is something close to my heart!

I am of the belief that quantitative analysis (especially of competitor's financial statements) should be an integral part of CI analysis. I have discovered in my life as a CI analyst, that looking at the balance sheets and cash flow statements can help quickly answer some of the client's questions about the competition. This also helps to corroborate intelligence from primary / secondary sources.

Again, quantitative analysis of the financial statements can help focus our collection efforts by pointing out that "something" is not right and that the competitor should be up to, well.. "something". So instead of searching in the air for anything and everything; we can now focus our efforts and be much more effective as we would now know what to look for. As Sherlock Holmes, in a story picks up a burnt match stick from the crime scene after the Yard has gone through it, famously remarks that he found it because he was specifically looking for it unlike the poor policemen.

But sadly I do not find too many uses of financial analysis in CI. This could be partly due to the fact that financial analysis looks only at tangible numbers and does not take into consideration the so called "intangibles" which is often argued to be the reason for most of the financial analysis in itself going wrong. So if we could figure out a way that measures and considers these "intangibles" (which we as CI analysts are quite capable of) and incorporating that with financial analysis, not only will we have solid intelligence products but can also make a neat buck on Wall Street.

I am currently looking for books / papers on this subject. I have found that Dr. David Rogers does workshops on these lines..if there are any books by him or any others that you know of, please do let me know. SCIP also did not have any work shop / session on this subject last week in DC, which was sad.

Nimalan

[Ellen Naylor]

Mark Johnson, who is a member of this CI Ning is a real whiz on the numbers, Nimlan. Sorry to miss you at SCIP! Sounds like you had a good time!

All the best,

Ellen

[Nimalan]

Thanks a ton Ellen. I will contact Mark Johnson. Sorry to have missed you too at SCIP but then there would be other opportunities..And SCIP was nice.. especially meeting all the people and the impromptu discussions in hallways, over drinks and during the work shops.

Nimalan

[Anna F. Shallenberger]

WARNING -- philosophical treatise follows... :-)

Late to this party, but...What's the old line -- something to the effect of "Figures Don't Lie, But Liars Do Figure" ??? Love the AJ quote BTW. And humans are very visual -- many absorb that data faster/better -- witness the popularity of Tufte in the 90s... And it seems like having some sort of numbers became a sort of "pay to play" in building a business case for anything. From the planning/predictive front end to the analytical/monitoring middle and a metrics back end.

Like others, I see the need for balance between the qualitative and the quantitative. But in some ways, this actually brings more to mind a related issue. That is, the increasing "subjectivity" of underlying assumptions, Qual/Quant aside -- in a world where often people can't even agree on the basics of what is "factual" -- it becomes increasingly challenging to evaluate quant data. Qual too for that matter...

I have progressively found that those I am working with not only want the output, but require [more detail than historically requested] around methodology / inputs etcetera. Which is generally proprietary, beyond the basics -- e.g. sample size, time frame, collection method etc. I would be interested to hear others discuss if they have faced similar concerns. It seems like even most "raw" data often has a degree of [for lack of better term] "modeled contamination" these days...

Sometimes we've tried to reverse engineer #s to understand the model, but generally speaking -- one rarely has enough confirmed details to accomplish that. Even though I majored in Econ -- I can't claim to be a stats guru -- even though, as my advisor [PhD Economist] quipped "Economists are people who aren't interesting enough to be accountants" -- Ouch!

It also seems as if, while one may want to understand the wisdom of crowds -- there is a perception that stats / #s / quant --aid one in avoiding "group-think blind spots" -- which I don't believe provides as much of a shield as some believe... I sometimes struggle with balancing my common sense / BS detector with making sure I'm not too locked into my personal experiential box...

Perhaps I'm being naive. However, and to me, yes, intel requires many subjective assessments/interpretations of a host of data inputs... I'm curious though if y'all feel we currently face more shades of gray, if you will...

Realize this is somewhat stream of consciousness here and veering off the original post -- but it has been on my mind.