This story appeared in the Fall 2020 Letters & Science magazine.
Miron Livny has always been a big believer in the power of sharing. Livny, a professor of computer science and the director of UW-Madison’s Center for High Throughput Computing as well as the chief technology officer for the Morgridge Institute for Research and the Wisconsin Institutes for Discovery (WID), laid it out in the conclusion of his doctoral defense, a document he wrote almost 40 years ago. “In the early stages of mankind, individuals came together to create community, because they can do more and do better together,” says Livny, who came to UW-Madison as a faculty member in 1983. “My dissertation was about building a community of computers.”
What he built was more than just a simple community—it became a global phenomenon that transformed scientific research. Livny’s ideas have been embodied in what came to be known as HTCondor, an open-source, high throughput computing software suite that allows thousands of computers to work together to manage complex computing tasks that may involve massive amounts of data. Where once a scientist might have taken weeks and months to accomplish a computational task, high throughput computing shortens that span significantly. In addition to supporting hundreds of research projects around the world, from disease projections to nuclear physics to botanical studies of corn, Livny’s creation was instrumental in two Nobel Prize–winning discoveries: the Higgs boson particle in 2012 and gravitational waves by the Laser Interferometer Gravitational-Wave Observatory (LIGO) lab in 2017.
“The power of sharing and coming together is why distributed computing is so powerful,” explains Livny. “It’s about putting smaller pieces together and working effectively on a shared mission.”
In the early 1980s, Livny was finishing his PhD at the Weizmann Institute of Science in his native Israel when a fellow postdoc who’d studied at UW encouraged him to travel there. After making a few contacts in the computer sciences department, he quickly landed an assistant professorship—“back in those days, you said that you saw a computer and they gave you a job,” Livny jokes—and the charm of the city, the systems-oriented culture of the computer sciences department and the collaborative nature of the campus have kept him here his entire career.
Livny arrived on campus at a time when computers serving campus researchers were hulking, room-sized objects kept behind glass walls, the kind of things students might view from a distance on a field trip. Livny, who had always been fascinated with questions around load balancing—the idea that distributing the work over a series of computers made the process more effective—saw an opportunity to do more.
The first HTCondor setup, back in the ’80s, consisted of 20 workstations. Today, the Ice Cube Neutrino Observatory at the South Pole (WIPAC, the Wisconsin IceCube Particle Astrophysics Center, is located on the UW-Madison campus, encompassing faculty from the physics and astronomy departments) uses HTCondor to harness as many as 51,000 CPUs in 28 separate cloud regions to support their processing and analysis of astrophysical data.
The same idea behind HTCondor also powers Livny’s other major contribution to the field of distributed research computing—the Open Science Grid, a project originally created to marshal resources to process data generated by the Large Hadron Collider, the world’s largest particle accelerator. Since 2005, Livny has been the technical director of this national computing infrastructure, which provides high throughput computational services to researchers around the world.
While the horizon appears limitless, Livny does see challenges coming for his chosen field. As the ability to harness distributed computing has expanded and multiplied, so has the ability of researchers to collect signals from nature, like the radio signals astronomers collect from space. These signals need to be processed by a computer to become data. And the number of available signals to collect is truly mind-boggling.
“We are struggling with this,” he says. “Our science is limited by our capacity to process signals. There may have to be tradeoffs in what we can and cannot do. The question becomes, how much money and energy do we want to invest in something that’s increasing so dramatically?"
Other transformative breakthroughs, like quantum computing, may eventually address some of the shortfalls, says Livny, but it’s too soon to tell. The bigger issue, he says, may not be technological but sociological: In other words, while it’s now possible to share computing resources with anyone around the world, the first step is to help your potential collaborators see the benefits of sharing. Recognizing that issue, he says, is half the battle.
“When you bring resources to be shared, you have control over where and with whom it can be shared,” he says. “It’s not always easy to keep that in mind.