What is the relation between CI Pros and Data Scientists? Wouldn't the future of CI PROs to become Data Scientists?

[Sandro Tavares]

IBM: What is a data scientist

http://www-01.ibm.com/software/data/infosphere/data-scientist/

HBR: Data Scientist: The Sexiest Job of the 21st Century

http://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-c...

NETWORKWORLD: Could data scientist be your next job?

http://www.networkworld.com/news/2012/060412-data-scientist-259697....

FORBES: The Data Scientist Will Be Replaced By Tools

http://www.forbes.com/sites/gilpress/2012/08/31/the-data-scientist-...

[Sridhar]

As a technology professional with an interest in big data technologies such as Hadoop and also in CI, I ask if this is really an either/or situation. Instead, I like to see it as a hierarchical structure. At the top you have people who consume insights - call them decision makers. Below them are the people who find the insights, especially the sort that are 'actionable', immense strategic value that provide huge time and opportunity advantage, like CI professionals seek using primary, secondary intelligence tools. Data scientists are the new professionals being touted as tomorrow's make or break people. These are typically experts in modeling languages like R and know all about statistical methods, heuristics, etc. Their recent prominence is due to the emergence of big data or vast amounts of data, like Mr.Tavares pointed out. There was no problem with their tools such as spreadsheets, relational databases until the data volume to be computed exploded, especially the unstructured stuff in social. Below the data scientists are technologists (yours truly) who provide technology to run very large 'batch jobs' or 'real time reporting' systems that data scientists use to compute and sort, and surface most relevant 'insights.' IMO, the data scientists are at a level merely facilitating dealing with large volumes of data in a relevant manner, much like a feeder makes a pass to the shooter. The CI professional is the shooter and puts the ball in the basket supported by data scientists...I hope!

[Sandro Tavares]

Mr. Sridhar,

I agree with you concerning that this is more a hierachical structure between these professions.

My point is that with so much data that will be available in the near future and tools to extract significance from it , less room CI Pros will have! Until (Up to) what point is going this advance of data science profession "inside" the CI profession? Aren't these professions going to "merge"?!

"With enough data, the numbers speak for themselves." Chris Anderson (Wired editor)

 

“If marketing is both an art and a science, then clearly the pendulum has shifted toward the science, thanks, in part, to advances in technology and data analytics.” (CMG Partners)

[monica nixon]

Hello Sandro,

 

First you begin by inquiring about the relationship between two fields- and then you immediately proceed to an unsubstantiated conclusion phrased as a question that presumes a certain answer to your initial question.

 

Let's just back up - I think what you might do to get a better feel is a) Delve into the skills and attributes that CI and for that matter BI Professionals, tend to have and the questions they are expected to be able to answer/problems they are supposed to solve using what data types and b) the skills Data Scientists tend to have and the questions they are expected to answer using what data types first before going down the garden highway and assuming that the two overlap, and/or are the same, etc. If you undertook this exercise, you'd see there are some most distinct differences.

That said, having worked in high tech, INFOSEC, intelligence SW domains I have some SME knowledge here so maybe I can shed some light....    

1) CI Professionals tend to look at EXTERNAL data, FOCUS on COMPETITORS and the goal is to be ANTICIPATORY and the emphasis is NOT highly QUANTITATIVE but far more QUALITATIVE.

2) Historically, BI experts tend to look more at INTERNAL data in CRM/ERP systems or other enterprise repositories, they are highly QUANTITATIVE, and INTERNAL and/or CUSTOMER focused. These guys and gals are looking for customer buying patterns, preferences and how to capitalize on what they uncover. It tends to be about driving more sales, reducing costs, etc.

3) Data Scientists at big companies (retail, finance and CPG companies, pharma, insurance ) take the BI thing to a whole new level in my opinion and there are privacy and PR issues that ensue and they will be an extremely hot button issue on a go forward basis as the public becomes more aware of how data mining is done.

Anyway from what I've seen the DS folks emphasis hasn't been on competitors and being anticipatory on what they might do (hence not falling over into Corp CI domain), rather their focus is on taking CRM data, Social media data, credit card data, loyalty card data, your web data, your location data- tying some of this together, building a profile on YOU as an individual so ahem choke they can capitalize at opportune times for example . See here: http://storageguru.org/archives/2012/02/big-data-how-target-knows-y...

Scared yet? There's the real story....GOT PRIVACY?

 

MN

 

 

[Sandro Tavares]

Hello Monica,

I see that my question was about "the relation between two professions", and if they are going to merge in the near future. I stated a question and immediately dive into one of the possible hypothesis, the one I think more plausible. My idea was to make a question and be provocative...

Considering your points

1) The focus os CI PROs is still on competitors?! Isn't that "Competitor Intelligence", just one of the parts of CI? How about costumers, science and technology, suppliers, new entrants, demographics...?

2) "Historically" yes! In the question that I raised here, the focus is on the future, the near future!

3) "Data Scientists at big companies take the BI thing to a whole new level in my opinion (...)" That's what I'm talking about!

 

My questionn are, with so many data about the costumers (the real target), will be necessary too much energy on competitors? Couldn't the big data cover competitors/suppliers/substitutes... too?

 

Below you can read part of a very well written text by Bonnie Hohfof in the editorial of the latest issue of SCIP Magazine, that made most of the points more clear to me about this subject:

 

Driven by the resources of companies such as IBM, Big Data is moving beyond analyzing internal corporate data and information to look at the outside world. Data analytics focuses on making this information easily accessed and understood. Big Data also focuses on developing an awareness of how early identification of patterns and trends affects efficient decision-making. The conversation about Big Data, accurate or not, is being shifted into how it creates decision-making insights.

The growth of Big Data’s analytical capabilities and its movement into external information has the potential to marginalize the traditional stand-alone competitive intelligence function, incorporate CI’s skills and processes into the Information Technology department, or provide the impetus for CI’s increased value and visibility. Which scenario prevails will be based in large part on how we choose to adapt the software’s capabilities.

Dismissing Big Data’s current visibility and attraction as just another “data warehouse” retread is shortsighted. The high visibility of the need to incorporate external information and the growth of high-speed social information exchange is unlikely to abate.

http://www.scip.org/Publications/CIMArticleDetail.cfm?ItemNumber=17732

Sandro

[Sandro Tavares]

Nice point about Big Data!

"Big data can be great, but not when it’s bad data."

Why Google Flu Is A Failure

http://www.forbes.com/sites/stevensalzberg/2014/03/23/why-google-fl...

[Sandro Tavares]

By the way, nice point in this article of Clear CI:

"Does that mean CI professionals will eventually rely solely on secondary intelligence? No, but using Big Data technology will become a standard practice."

What Will Competitive Intelligence Be Like in the Future?

http://www.clearci.com/blog/bid/195670/What-Will-Competitive-Intell...

[Trip Krant]

I'm doubtful. The CI market has a hard enough time paying for traditional products like clearCI/Digimind/Cipher/Comintelli, let alone actual “big data” augmented intelligence vendors like Quid or RecordedFuture. On the human capital level “big data” requires data science skills, which will be highly in-demand for the foreseeable future - the CI field today can't offer the opportunities that are available to someone with data science skills.

The real value of data science to CI or corporate strategy will come augmented intelligence, which comes from the field of human-computer interaction. Augmented Intelligence and HCI are incredibly complicated fields that I struggle to even comprehend, let alone describe – so I really recommend watching Quid's co-founder and CTO Sean Gourley's TEDx talk.

[Pradhuman Singh]

Hi Sandro

I had been following this discussion for the last two years, and trust me, the answer to this question will have far reaching impact.

Traditionally as intelligence professionals, qualitative information would have been the cornerstone of the decision makers ranging from members of the board to the stakeholders of public listed firms. However, post 2008, market dynamics have undergone a sea change.

A certain degree of Quantitative support for Qualitative information is a momentous necessity for the top management of a company that relies on Real time Competitive Intelligence for steering the organisation through turbulent and favourable times. This is where the need for a Collaborative Team that functions like the eyes and ears of the Top Management a.k.a. CI Pros coupled with Data Scientists will come into play.

As a CI practitioner and a PhD student who heavily relies on Quantitative Analysis using R and SPSS, I have come to learn that eventually a merger of both skill set may actually occur in time irrespective of what technology comes into being. However, there would be a selected few who would be equally gifted both in right and left brain abilities. 

In terms of Darwin, Evolution never ended, it just became more subtle. Similarly, a CI pro would be spending more time in the field, while his Data Scientist team mate would be working on the Intelligence that CI pro has collected in real time. Their boss however, will be someone who has had exhibited proficiency in both aspects and knows in which direction to focus the energies to bring about the best insight.
If you've spent time in the Special Forces, you'll understand  ;)

[Trip Krant]

That is pretty cool that you are doing Data-Sci and have a CI background, Pradhuman.  I am looking forward to hearing more about your activities.

[Pradhuman Singh]

Good to hear that Trip..

You can connect me on linkedin at 

https://in.linkedin.com/pub/pradhuman-singh/a/b00/409