Data storage

Growing technical possibilities of experimental equipment and tools of common use lead to increasing processed data volume. An extreme example can be the Large Hadron Collider in CERN which requires a pan-European storage and computing network to enable processing the measurement results. However, even more affordable resources generate more and more data. Measurement precision as well as digital microscope or camera resolutions are growing. More and more materials are digitally processed.

All this results in strong demands for data storage capacities. It is not always effective to build them locally, so the CESNET e-infrastructure includes also data storage infrastructure.

Hierarchical storage

Storage capacities are built in a distributed and hierarchical way. The distributed character (the first phase involves Pilsen, Jihlava, and Brno) brings the following main advantages: much higher tolerance to data loss caused by media failure, fire, user or administrator fault; possibility to share the data in a controlled way; ability to deploy the closest storage site, thus enabling the fastest achievable communication.

Storage hierarchical structure means that each storage includes several media types. The aim is to find a suitable compromise between the speed and size of storage with available investments and operations finances. The fastest media tend to be the most expensive ones therefore the fastest storage layer can have only limited capacity. Management software is responsible for moving less often used data automatically to slower layers with larger capacities while hot data remain in the fastest layer so that they are readily available.

For a user, the whole system resembles a black box that stores or releases needed data upon request. The involved storage layer can be identified only indirectly depending on the storage response time.

Services

The storage is available using various means – from specific communication protocols to simple applications for common users such as FileSender. Details are available in the data storage services section.

The storage is linked to authentication mechanisms of CESNET e-infrastructure. This facilitates data sharing within distributed teams where members come from different organizations. At the same time, the data storage can be used directly from the computing environment of the national grid.

Last change: 29.5.2013