John Foley, Oracle
The fact is that all businesses are on the big data adoption curve. Some are just getting started; others have well honed plans. The rewards will go to those that figure out how to manage the growing terabytes of data they collect, glean insights from that data, and capitalize on them. At this point, that’s a minority of companies.
Big data isn’t a “should we or shouldn’t we” proposition, nor is it a question of timing. All businesses are being swept into the world of big data, and it’s happening now.
A better way to think about big data is to understand where your company is on the implementation curve and develop a plan for moving up the curve as quickly as possible. With that in mind, here are eight best practices to get you there.
- Data creation. With our smartphones, social media apps, online shopping, video streaming, and web surfing, we all create what Oracle ORCL +0.12% big data strategist Paul Sonderegger calls “data exhaust.” In other words, your company’s customers and employees have become data output machines, leaving a digital trail wherever they go. These digital trails can be measured in terabytes, and when you multiply that by a thousand consumers or knowledge workers, you reach a petabyte. Voila—you’ve got big data.
- Tiering. Step 2 is all about data accumulation. IT departments invest in tape storage, disk storage, flash storage, desktop storage, and cloud storage as a way of taking it all in. Best practices involve tiered storage, where data is moved to the most cost-effective medium. As terabytes become petabytes, Darwinian principles kick in—you must have a well conceived storage strategy to survive. Even so, capturing data isn’t the same as managing it, and managing it isn’t the same as applying it.
- Optimization. Data management is the bedrock of corporate IT implementation, but the influx of a thousand times more data has upped the game. (A petabyte is a thousand terabytes, and an exabyte is a thousand petabytes.) Exponential growth requires IT teams to rethink what’s worked in the past and bring in new tools to optimize database workloads that are orders of magnitude greater than before. In-memory and other specialized databases, Hadoop, engineered systems, and data-integration middleware are all pieces of the puzzle.
- Analytics. Once your big data house is in order, the next step is to run algorithms against the data in search of insights. Most companies have some experience with business intelligence, but here too new tools and techniques are an absolute necessity. Keep in mind that big data isn’t just more data—it’s new types of data, coming in faster and from new sources. For example, social media buzz is just white noise—and a lot of useless data–without the right tools for the job. Likewise with data from mobile devices and sensors. Your father’s data warehouse won’t cut it.
- Share the wealth. Big data strategists can learn a thing or two from BI efforts of the past, including the incredible value of making this an enterprisewide initiative, not the exclusive realm of a few highly specialized data analysts. Put big data tools and access into the hands of as many employees as possible, and mobilize those capabilities in the same way that you have other enterprise apps. You may want to share big data even more broadly by using APIs to make data sets available outside your organization, as government agencies are doing on Data.gov.
- Run the business. Now that your terabytes of raw data have been transformed into information, customer, sales, and operational data can be used to better run all aspects of your business, making it a case study in big data implementation. You’ve accomplished so much at this point that it would be nice to stop, but don’t because your competitors are moving in the same direction. To propel your company from mere competency to paradigm-busting big data powerhouse, you need data-driven innovation your rivals can’t copy.
- Change the business. This is the step that gives your company a sharper competitive edge. Big data lets companies get lightening fast answers to new questions that only they think to ask, accelerate decision making and actions, predict outcomes, and create new data-driven products and services. Companies that bring their “run the business” and “change the business” efforts into alignment stand to benefit the most, says Oracle big data strategist Sonderegger. (Be sure to read his recent post, “Big Data At Work: The World Is Making A Digital Copy Of Itself.”)
- Disrupt your industry. This is the height of big data execution—and where you want to be. Buyers used to think that wine quality was best judged by the expert palettes of finicky connoisseurs. That was before a Princeton University economist developed predictive models using weather data that worked equally well or better. Swirling, sniffing, and spitting Bordeaux hasn’t been the same since. That’s one example of industry disruption shared by Andrew McAfee, principal research scientist at Massachusetts Institute of Technology’s Center for Digital Business, during Oracle’s recent Big Data at Work event. Big data can outperform executive management in corporate decision making, McAfee says.
The good news is these best practices are within reach for all companies. The market is quickly maturing to a point where the products and services required to accomplish these steps are coming into the mainstream. Oracle, for example, provides a complete line of big data products, services, and resources.
There’s no time to waste. As I pointed out in an earlier column, it’s no longer enough to plan for data volumes that are double or triple what you have today—the new rule of thumb is to multiply by a thousand. Some organizations are already planning for yottabytes of data. The big data curve is headed upwards. Don’t be stuck at the bottom.