Executive opinion
Hot topics
Business cases
Experts voice
Quick Poll
At a glance
PDF version

Subscribe to
Bull Direct:


January 2007
Storage: the CEA’s experience or the new supercomputing paradigm

IDEAS, a worldwide recognized industrial analyst specialized in IT, has recently published a new report highlighting “Bull’s systems integration capabilities within a storage environment”. Here is an extract of the report:

In supercomputing centers, data is at the heart of the system architecture. Via the middleware layer, the data will be able to be used in every phase of a scientific or engineering task from initial modeling to computation, and exploitation of the final results. Compared with existing solutions, the main effect will be to add middleware layers that make the use of the various IT resources involved in different phases much more transparent. By adapting much more closely to user needs, this kind of architecture will improve overall system efficiency to an even greater extent. Moreover, for many IT departments, to store, extract, save and archive data, powerful, scalable and robust storage solutions that are fully integrated within the Information System are mandatory.
For CEA, with its Tera-10 supercomputer that produces data equivalent in volume to the one of the French National Library (BNF) in three days, storage is of paramount importance.

Bull Helps CEA Stay on “Tera-Firma”

The French atomic research organization, Commissariat à l’Energie Atomique (CEA), employs roughly 15,000 people and focuses on energy and health technologies, as well as defense and national security. Within CEA there is an agency called the Military Applications Directorate (DAM), which maintains nuclear deterrence credibility and supplies nuclear weapons to the French Army. Within this agency, Hervé Lozach is involved in a very large program called “The Simulation Program,” which concentrates on improving the physical models used in numerical simulation, improving numerical simulation itself (Hervé Lozach’s area), and validating simulations with experimental results. Hervé’s group of about 50 people works with a very large computing facility where Tera systems are hosted.
Tera actually refers to three different computing environments at varying stages of deployment (with four years between the installations of each successive one). The first, Tera-1, is in use since many years. Hervé Lozach currently works on the second, Tera-10 – a system installed by Bull in 2005. Based on the latest TOP500 results, Tera-10 is one of the largest computing system in the world. Tera-100 is the third planned system.

Within the computer center where the Tera installations reside, systems exist for two purposes: “pure computing” and archiving. Tera-10 supports the pure computing environment, and contains about 600 nodes, roughly 10% of which are I/O channels. Fifty-four Data Direct Networks (DDN) I/O controllers are attached directly to the nodes (eliminating the need for a SAN), and the I/O servers are designed to take advantage of the controllers’ full bandwidth. For failover purposes, the I/O controllers are multi-access and shared. Tera-10’s archival system, also designed and integrated by Bull, is built upon 13 Intel® Xeon®-based nodes, 4 petabytes of disk (DDN controllers), and the IBM High Performance Storage Server.
Hence, Hervé’s environment is both large and complex, and for this reason, his view of system integration is quite comprehensive. In his view, system integration involves the design, delivery, and (of equal importance) maintenance of complex solutions such as his, including all of the hardware, software (including open source), and networking components involved.

When seeking an integration provider, CEA generated an RFP for both the pure computing and archiving environments. The pure computing RFP was particularly complex with hundreds of requirements; however, the greatest emphasis was placed on I/O bandwidth (30 terabytes per day requiring 100 gigabytes per second). Internodal connectivity was particularly important as well.
Bull was chosen to be the integration provider for both environments in 2004 and the Tera-10 system was installed at the end of 2005. One of the reasons Bull was selected was its perceived ability to provide servers that
a) offer the best computing power,
b) are based on industry-standard technologies, and
c) support open source operating environments.
Indeed, Tera-10’s 600+ servers are all Bull NovaScale servers running Linux, and contain 9,000+ Intel® Itanium® 2 cores.

Another reason was Bull’s recommendation that CEA use the QSNetll ®clustering solution from Quadrics, a European high-speed networks provider that was able to supply the quality and performance CEA required. CEA’s stringent requirements for high-speed data transfer between servers and storage components were adequately addressed through a combination of Lustre® (an open source file system) and Data Direct Networks disk controllers, providing CEA with transfer speeds exceeding 100 GB/sec. Bull was also able to provide powerful nodes (up to 16 processors and large I/O capabilities) due to its NovaScale architecture. The interior connection (midplane) of these systems allows the interconnection of four QBB (the standard Intel® four-way Itanium® building block) and one or two I/O boxes incorporating a Bull-designed ASIC.

Hervé indicated that Tera-10 will be operational through 2010, when RFPs for Tera-100 will be created. CEA is not assuming Bull will be the vendor of choice for Tera-100, but given Bull’s history of keeping CEA satisfied, it improved its chance of once again winning the day.

Download the full analyst report

Contact  |  Site map  |  Legal  |  Privacy