At Hitachi Vantara’s NEXT 2018 event, this week in San Diego, we were fortunate to have Malcolm Gladwell as a keynote speaker. Malcolm Gladwell is a writer for The New Yorker and has published five books that have been on the Times bestselling list: Tipping Point, Blink, Outliers, What the Dog Saw, and David and Goliath. Malcolm’s books and articles often provide unexpected insights which helps us understand the events around us and enable us to make better decisions.
In the general session our CEO Brian Householder did an interview with Malcolm. Some of the ideas that came out were: A model is only as good as the data that goes into it which he illustrated with the use of standardized student testing to measure the quality of teachers. Another was to focus on the core issue and not to panic when the circumstances changes. The example here was the music industry where the business shifted from recordings to and streaming music. The music industry was panicked over the loss of revenue, but today the music industry is making more money than ever due to live performances which are promoted by their music. I could see a similar transition in our industry where cloud was a threat to the IT vendors, but today the revenues are increasing for IT vendors due to software and services which makes it easier for their customers to develop applications and generate information.
Later I had the opportunity to moderate a Q&A session with Malcolm and a group of VIP customers. Here we started with the paradigm of puzzles versus mysteries. While Malcolm is known for his best sellers, there are a lot of ideas that are created in his New Yorker articles that are creating even more interest today. In 2007 he wrote an article “Open Secrets “, in which he raised the paradigm of puzzles versus mysteries.
For example the whereabouts of Osama Bin Laden was a Puzzle. This was eventually solved and taken care of. Puzzles can be solved by gathering more data. There is an answer. On the other hand, what happens to Iraq after the fall of Saddam Hussein was a mystery. Mysteries require judgments and the assessment of uncertainty, and the hard part is not that we have too little information but that we have too much. A mystery is a problem caused by an excess of information--and can only be solved by trying to make sense in a more sophisticated way of what is already known. Today we live in an age of information overload and all professions must manage the transition from puzzle solving to mystery solving.
While the direction of the Q&A did not allow him to go deeper into this topic, he has talked about this in other interviews. In an interview in inc.com. He said that most businesses and industries are built for solving puzzles, not mysteries. With greater access to data comes greater responsibility. Mysteries require a shift in thinking that most industries simply are not organized or prepared to handle. Complex mysteries require a different kind of thinking and analytic skills. You need to decide whether the problem you are solving is a puzzle or a mystery. If you think you are solving a puzzle and collect more and more data, the overload of data might bury the key nuggets that could help you solve the mystery. Data that is used to solve a puzzle must be looked at differently when you are solving a mystery. One of the biggest mistakes that businesses make is treating all data equally, by giving all data equal weight.
“This idea gets back to Nate Silver's The Signal and the Noise concept. Some data will tell you precisely what you need to know and help you solve the mystery you're after, but most data is noise, distracting you from the answers you seek. Today, the trick is not analyzing the data so much as understanding which data represents the signal you pay attention to versus the data that is the noise that you must ignore. We have to start ranking data in terms of its value.”
The transition from puzzles to mysteries should resonate with most CIOs in regards to their data. Data bases were great at solving puzzles, but the complex nature of today’s business is more of a mystery that requires big data and analytics. In order to be competitive, companies are shifting from data generating to data powered organizations and big data systems are becoming the center of gravity in terms of storage access and operations. Data curation is needed to understand the meaning of the data as well as the technologies that are applied to the data so that data engineers can move and transform the essential data that data consumers need. Hitachi Content Intelligence and Pentaho Data Integration are key tools for searching, classifying, curating, enriching, and analyzing the data to understand what you have and how it can be used to solve mysteries.
from Hitachi Vantara Community : All Content - Hu's Place https://ift.tt/2Q7GG2n
via https://ifttt.com/ IFTTT
No comments:
Post a Comment