Berkeley astrophysicists harnessed the campus supercomputing cluster, Savio, to make important advances in understanding how black holes behave. Working with Berkeley Research Computing (BRC) staff to tune their software to the Savio environment, Alexander Tchekhovskoy was able to produce findings published in six journal articles even in the face of unanticipated restrictions in his allocation on nationally-run supercomputing infrastructure.
BRC Program Director Patrick Schmitz interviewed Tchekhovskoy to understand his research and how BRC has helped to advance work in the Astrophysics group led by Professor Eliot Quataert.
PS: What is your research?
AT: I have three projects related to understanding the physics of black holes and neutron stars. The first project is a collaboration with Sean Ressler [a UCB graduate student] and Prof. Eliot Quataert, and addresses the radiation produced by black holes. We’re looking at two black holes: Sagittarius A* (the black hole at the center of our Milky Way galaxy), and M87 (a much larger, but much more distant black hole). Around a black hole, there is whirlpool of material, the “accretion disk”, which glows (radiates) as it falls into the black hole. As gas falls in, the tremendous heat strips protons and electrons apart. Most simulations model protons (which get most of the heat), but we had no idea what the electrons are doing, and they are what radiate. The existing models assumed electrons are cooler, but their predictions did not match observations of radiation detected from Sagittarius A*. Monica Moscibrodzka and collaborators found they needed to “paint” the regions above and below black holes with high temperature, high energy electrons. That model matches observations, but it does not explain why electrons are hot. We took a forward modeling approach. We started from simulations, and added our best understanding of the physics of electrons. We used Savio to test and refine the models, ran the simulations for weeks, and computed and analyzed the resulting radiated spectra and images. We got results that matched observations! Electrons are heated more effectively in regions of high magnetic fields, which explains the observed phenomena. This is an important fundamental result for a question that people have been looking at for some time.
The other projects are in collaboration with Omer Bromberg, a postdoc at Hebrew University, and have to do with two fascinating systems. One concerns exploding stars (which leave behind a black hole or a neutron star), polar jets, and gamma ray bursts. The other looks at supermassive black holes, and how they also produce these jets as they accrete material. We were not sure how these jets shine and what physical processes determine their shape. Omer and I simulated them, using Savio, where we also designed, tested, and debugged our simulation code. If you take a spring and squeeze it, it can fly sideways. The jets are giant magnetic springs, and we discovered that the same process -- called the magnetic kink instability -- plays an important role in jets. In the gamma-ray bursts, it causes the jets to emit. In supermassive black hole jets, it determines their shape and also causes their emission.. These two discoveries provide attractive explanations of observations that have puzzled us for the past several decades.
PS: How have Savio and the BRC program helped you?
AT: The work we are doing is computationally intensive. A local resource like Savio is really useful because we need a day-to-day ability to test new ideas. You may not know what physics effect is important, so you want to experiment, to run many models to bracket the science. If we can try something, and get an answer in a few hours, that is great. At a national center, you may wait a day or more for job to even start. Getting fast turnaround is a fantastic capacity that we cannot get elsewhere. Also, when we are developing new software models, we need a local resource to test and debug our software. Getting early quick results with Savio allows us to get more time on national resources, and to use them more effectively, to prove out our models. Without Savio, it is hard to develop models, and hard to demonstrate their viability in order to get larger resources at national centers [allocations on national systems are granted by competitive application].
Another lifesaving capacity is visualization support. The BRC team got our visualization tools to work in the Savio environment. No-one else in other supercomputing centers could get these tools to work. Everyone struggles with these issues at other Universities, but here it just works, which makes a huge difference in my ability to do research. I am able to run all the visualizations locally, which makes responsiveness much better, even in the age of fast internet. Imagine you have a huge data set, from a simulation. You want to see something in 3-D and then manipulate it to see different features, etc. With a local resource, you can interact with the data in ways that would be difficult if not impossible if [the software ran at the national computing center] in Texas. In fact, with our software, the delays would be prohibitively long to explore the data interactively.
One more thing on why we need local resources: National compute resources are unpredictable. When Kraken [a major cluster in the national system] went offline, everyone’s allocations were reduced, and ours went down from 6 million cpu-hours to just 800,000! Faced with this shortage, having a local resource was huge - a life-saver for us. Our third project was made possible by having the low priority queue [a new service for condo participants to glean unused capacity]. With only 800,000 cpu-hrs on the national systems, we had to do many more parameter study runs locally, which ended up being serious production runs. Running tests that guided our physics intuition was awesome, lifesaving. I am not exaggerating.
PS: What’s next for your research?
AT: You may have seen the New York Times article on Black Hole hunters. The BlackHoleCam project and the Event Horizon Telescope (EHT) will try and capture images of an Einstein Ring [the signature of a black hole]. This is the expectation, but with lots of data streaming at us from the EHT, we want to do better. We want to consider what images we can expect to see, and what that tells us. For example, can we determine how fast the black hole spins, model our expectation of how this will warp light around the black hole, and then compare our models to the observations gathered by the EHT.