Kossi Gavi drives to class on Sunday afternoons to learn retail software, and the reason is simple.
People who wield computers to analyze large amounts of digital information are in high demand, and Gavi is learning a program that chain stores worldwide use to run their businesses. Workers who know the program can earn up to $80,000 per year.
“It’s a very good program if you want more opportunities to make more money,” said the 43-year-old former refugee from Togo, now a few months from earning a bachelor’s degree at Metropolitan State University.
Businesses today control massive and growing streams of information that flow from cash registers, patient records, smartphones, warehouses, the sensors in your Nikes, databases, Facebook and good old-fashioned loyalty cards.
The challenge is finding people who can put it all together and make better strategy. Everyone from the Central Intelligence Agency to Gander Mountain is on the hunt.
“I would challenge you to describe to me an organization of any size in any industry or not-for-profit setting that will not be leveraging this,” said Isaac Cheifetz, a headhunter working to find the Mayo Clinic a head of information management. “Name one. I can’t.”
Businesses have the data to keep sale racks thin, streamline shipping and get more people to click ads. What they need are better analysts. It’s a new kind of job, and it’s coming to your workplace if it’s not already there.
The McKinsey Institute predicted in 2011 that a big-data boom would create up to 190,000 new deep-analytics positions in the United States, and demand for 1.5 million data-savvy managers.
If you can run Hadoop — open-source software used by Google, Yahoo and Facebook to analyze the deluges of information churned out by the Internet — you might get a free flight to the San Francisco Bay Area for a job interview, said Ravi Bapna, director of the University of Minnesota’s Social Media and Business Analytics Collaborative.
“The premium for these sort of people is already very high, and it will only increase over time,” Bapna said. “There is a huge shortage of people who can handle the data, who have the business acumen to be able to ask the right questions, to do the experiments and make the right inferences.”
Big data refers to a series of software and hardware advances, but one of its biggest advances is a new ability to impose structure on vast pools of complex information — like pictures, consumer preferences, geographic locations and video of the ocean.
“Retail’s been big data for decades,” said Mike Webster, general manager of Oracle Retail. “It gets real complicated real fast,” Webster said. “Our approach is to try to simplify that as best we can.”
Traditional data is — to use one of Cheifetz’s analogies — like the zoo. Think Excel spreadsheets. Everything has a label and fits in a format where it can be easily sorted.
But the world is awash in the unstructured information of the Internet, mobile phones, social media. Instead of a zoo, this information spreads out like a nature preserve. It’s moving, wild, and can’t be captured in database cells.
A major achievement of big data has been its ability to sort the unstructured information of the preserve, and to do complex analysis of huge amounts of information in parallel on several machines.
Software like Hadoop makes it possible to analyze, for instance, a photo, attach a digital signature to the photo that describes it, and to compare that signature to the signatures of extremely large numbers of other photos.
Thus unstructured data gets structure, and analyzing huge amounts of information becomes practical. Similar approaches are used to analyze buying behavior, what types of ads people respond to, even fraud.
Oracle sells heavy-duty software that allows companies to track purchasing, supply chain, shipping, inventory and sales in stores and on the Internet. All of the top 20 retailers in the world use Oracle.
Now, companies want to wed that type of data with information on where customers are, what they want, what they’re saying on their social network, and how and when to ship products to them. As more retailers try to harness all that information, Oracle has been doing more business.
Meanwhile, the world keeps churning out data. Worldwide mobile data usage — mostly smartphone traffic — grew 70 percent in 2012. It was 885 petabytes per month, or 12 times more than all of global Internet traffic in 2000, according to Cisco.
The Carlson School has been offering data science electives at UM for eight years, and now wants to start a master’s program in business analytics and data science. The proposal, OK’d by the Carlson faculty, awaits approval from the Board of Regents.
Bapna’s vision is broader than teaching a particular software program.
He thinks the data generated by social media is an unprecedented social research graph, “a global laboratory, where we can ask fundamental questions about human behavior.”
He wants to build a master’s program that sculpts people with the technical training and business savvy to ask the clever questions and write the clever computer code that yields profitable insight.
Universities like Stanford, the Massachusetts Institute of Technology, University of California-Berkeley, Harvard and Carnegie Mellon already run programs that kick out data scientists, and several other universities are shifting resources toward such training. Minnesota’s master’s program could start as early as the fall of 2014.
For now, people tend to fall into big data jobs accidentally, Cheifetz said. Someone gets a Ph.D. in math, and ends up working on algorithms for Wall Street. An above-average IT manager learns the new software and takes ownership.
But that will change. Companies will begin to seek out workers with a strong background in computer science and statistics, and experience running predictive models, Cheifetz said. Also important is the ability to translate the data into a clear narrative. “We’re almost talking about a computer science and statistics undergrad, with a minor in theater so they can talk to people,” Cheifetz said.