With each tweet, status update, playlist, and purchase, we heap new layers of data on top of the existing mountains.
Last year, technology giant IBM estimated that 90 percent of the world’s data had been created in the previous two years.
The words “big data” have become the buzziest of tech buzzwords in recent years, showing up on magazine covers, newspapers and websites galore. Our world certainly isn’t suffering from a shortage of information, and harnessing that data to solve big problems can enable us to market products more effectively, control government spending more precisely, or even build better baseball teams.
Of course, that’s the hard part.
“Each new problem has its own wrinkles,” says Brian Yandell, professor and chair of the Department of Statistics at the University of Wisconsin-Madison.
And ironing out those wrinkles requires a particular set of skills.
A 2011 report from the McKinsey Global Institute and McKinsey’s Business Technology Office projected “a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions” by 2018. And that’s just in the United States.
Companies of all sizes are searching for so-called data scientists, a fresh job title that’s arrived on the careers scene.
UW-Madison’s statistics department is responding to that demand by creating a new data science option within its master’s program, with student recruitment starting next spring and enrollment beginning in fall 2015.
Courses in topics such as data mining, statistical programming and big data analytics design — along with classes that develop the skills needed to communicate data findings to outside audiences — will prepare students for careers as data scientists, data engineers or data analysts, or better equip returning students already working in those fields. Indeed, Yandell envisions non-traditional students coming from companies such as Epic, Johnson Controls, John Deere and GE Healthcare, and the department plans to use its industry connections to produce graduates who won’t require extensive re-training once on the job.
“Data’s on the table. We should be at the corporate table engaged in planning and decision-making,” Yandell says. “We are developing a new generation of leaders to solve big problems using data science.”
But statistics isn’t the only L&S department grappling with big data.
The Department of Computer Sciences, is a national leader in data processing and database technology, going back to the 1970s, when Professor Emeritus David DeWitt established the department’s database systems research group.
Computer sciences faculty, among many others across the campus, are engaged in all sorts of big data-related work, including the actual storage and retrieval of data, the associated security and privacy concerns, automated analysis, and the semantics of data (determining what pieces of data truly represent so they can be properly analyzed or compared).
“You can’t get complacent, you can’t keep doing the same old thing, or you’ll be irrelevant pretty quickly,” says Professor of Computer Sciences Jeff Naughton, whose department has forged industry connections with companies such as Microsoft, Google, Twitter and WalmartLabs.
Researchers in the School of Library and Information Science (SLIS) are examining the information management aspects of data: How is it organized? How is it shared? And how are the algorithms and models used to crunch that data saved and managed?
SLIS also offers a course on digital curation, a class that prepares students to manage data by delving into topics such as data ethics, file formats, and storage.
“One of the problems with big data is you have a lot of it, and it becomes sort of this bear to manage, even to find the right stuff to run your analyses on,” says SLIS Director Kristin Eschenfelder.
Chris Wells, an assistant professor in the School of Journalism and Mass Communication (SJMC), developed a course on concepts, practices and tools for data and data visualization that is being offered for the first time this semester. Wells says the course will build on students’ strong reporting and research experience, improve their technical skills, and help them discover how data visualization can enhance storytelling.
Fellow assistant professor Molly Steenson teaches a course in the SJMC on information landscapes and data cultures, delving into information architecture.
Faculty members in the Department of Mathematics work on ways to produce more efficient algorithms, while students who study geographic information systems (GIS) in the Department of Geography learn all about acquiring, storing, managing, analyzing and creating visualizations of spatial data.
The reality is, as Yandell notes, there are data scientists all across campus.
“Who knows what will emerge 10 years from now? It may be that data science will emerge as its own field or it may be a confederation of fields, where you don’t think of it on its own, you think about it in context of other fields,” Yandell says.
One thing is certain: The amount of data will continue to increase, particularly as more and more devices are connected to the Internet.
“Big data,” says Professor of Computer Sciences AnHai Doan, “is going to be a challenge of our time.”