This week’s Strata Conference was a truly magical event for those immersed in the world of Big Data. Congratulations to Tim O’Reilly, Gina Blaber and the rest of the O’Reilly team for throwing a fantastic event. It was great to be a part of it and I’m looking forward to being deeply involved in the next edition, New York style. It also afforded myself and the IA Ventures team the opportunity of spending quality time with our fellow data geek-masters (and mistresses) such as Hilary Mason, Mike Driscoll, Drew Conway, Bradford Stephens, Flip Kromer and others with whom we consumed many fine (and not so fine) beverages and eats. Such an assemblage of brain power and personality is seldom observed in nature, but Strata had it in spades.
When reflecting back on the conference, the hallway meetings and late-night conversations, one feature of my myriad mind-bending explorations emerged: the importance of interface design and user experience in helping display the power and value of sophisticated Big Data technologies and analytics. This theme also emerged in a discussion I had with Mac Slocum of O’Reilly. I find that I never learn so much about what is going on inside my head as when I write or am interviewed, as being forced to let stream-of-consciousness flow minimizes the effect of preconceived notions and biases.
So much of Strata and, in fact, the dialogue around Big Data in general is focused on hard-core technologies, bleeding-edge analytics, data manipulation and consumption via APIs. The truth is, however, that much of the complexity and depth of Big Data analysis only comes to life and becomes actionable when presented in a clear and intuitive manner. This places a huge premium on start-ups with awesome UI/UX skills. And when I started to reflect on the IA Ventures portfolio - BankSimple, BillGuard, Kinetic Global, Metamarkets, PlaceIQ, Recorded Future, Sulia and TraceVector - almost all of our investments have an essential focus on interface design and user experience to extract value from extremely complex data-driven architectures. In my talk with Mac I used the example of BankSimple as a firm with a core focus on UI/UX - so much so, in fact, that the company really grew out of the question “What do consumers really want and how can we optimize their retail banking experience on mobile devices?” and developed an architecture and set of business processes to deliver on this value proposition. But such thinking isn’t merely the province of B2C; it also applies to those selling to the enterprise. Metamarkets ingests terabytes of publisher data in real-time and performs sophisticated analysis to provide them with powerful, actionable information that impacts inventory pricing decisions. The importance of the design and usefulness of the Metamarkets dashboard can’t be overstated; several large, global publishers are dependent upon this information, and it is Metamarkets job to make it readily consumable, easy to understand and immediately actionable. Without a great interface, the power of massive data and valuable analytics wouldn’t be nearly as profound.
Another interesting feature of these Big Data companies is their mixed DNA: world-class hackers and data scientists together with data visualization and user experience experts. Clearly these worlds overlap; many a great visualization guru is a top-flight data scientist. It’s just that getting these multiple personalities and skill sets to work together in a seamless manner to drive value to the client, whether they be consumer or business, is no mean feat. But these companies have been able to instill the importance of “customer first” within their organizations, forcing the intersection of real-time actionable information with a great user experience in perfect harmony. Now THIS is a Big Data revolution: giving the props not only to the data engineers but to the data depicters. Algos are great, but a picture is worth a thousand words.
This line of thinking has been reinforced in a book I read recently, A Whole New Mind by Daniel Pink. The essence of his thesis: right-brain (conceptual) thinking will become increasingly important to the West where much of the left-brain (analytical) tasks are being commoditized and outsourced to Asia. Most great data scientists I know are a synthesis of right and left-brain attributes: super powerful analytical minds but with a rich creative streak that extends into elegant code, unusual and insightful analytics and highly effective visualizations of complex data sets. And I believe the market has spoken. Great UI/UX people are in high demand, as are the most creative and efficient coders and data hackers. And these people aren’t 2x or 3x better than the merely good: they are 50x or 100x more valuable. Supply and demand are massively out of whack, and I fully expect this to continue unless our educational system moves away from rote memorization into critical thinking and celebration of orthogonal ways of looking at problems. Time will tell, but the role of the US in the Big Data revolution may well depend on it.