Each of the 11 Tier-1 centers are connected to CERN via a dedicated 10 gigabit per second link, and the Tier-1 centers are connected to each other by a general purpose research network. Each Tier-1 center receives only certain subsets of information. Brookhaven, for example, is dedicated to ATLAS, one of several large detectors housed at the LHC, while Fermilab handles data from the CMS (Compact Muon Solenoid) detector.
The Tier-1 centers are responsible for reprocessing raw data, which is then kept on local disk and tape storage and distributed to Tier-2 centers, which are located in most parts of the world.
Tier-2 centers are connected to Tier-1 sites and each other by general purpose research networks, such as the US Department of Energy's Energy Sciences Network. Tier-2s are located mainly in universities, where physicists will analyze LHC data. Ultimately, about 7,000 physicists will scrutinize Large Hadron Collider data for information about the origins and makeup of our Universe, according to CERN.
The LHC collisions will produce 10 to 15 petabytes of data a year, says Michael Ernst of Brookhaven National Laboratory, where he directs of the program that will distribute data from the ATLAS detector. Brookhaven, as a Tier-1 site, will be responsible for filtering data so it can be easily readable by scientists located at the more numerous Tier-2 facilities, Ernst says.
Brookhaven has about 1,200 multicore x86 servers dedicated to the LHC, along with disk and tape storage that holds seven petabytes of data. Ernst says Brookhaven will have to scale that storage up significantly by 2012, when he expects to be storing 13 petabytes of Large Hadron Collider data.
Worldwide, the LHC computing grid will be comprised of about 20,000 servers, primarily running the Linux operating system. Scientists at Tier-2 sites can access these servers remotely when running complex experiments based on LHC data, Pordes says. If scientists need a million CPU hours to run an experiment overnight, the distributed nature of the grid allows them to access that computing power from any part of the worldwide network, she says. With the help of Tier-1 sites such as Brookhaven, the goal is to make using the grid just as easy for universities as using their own internal networks, according to Pordes.
Asked if the LHC project is the most complicated thing he's ever worked on, Ernst gave a quick laugh and said, "Yeah, I would say so."
