Tuesday, February 5, 2013

Lustre: A Scalable, High-Performance File System


by Cluster File Systems, Whitepaper 2002.

Abstract:
Today's network-oriented computing environments require high-performance, network-aware file systems that can satisfy both the data storage requirements of individual systems and the data sharing requirements of workgroups and clusters of cooperative systems. The Lustre File System, an open source, high-performance file system from Cluster File Systems, Inc., is a distributed file system that eliminates the performance, availability, and scalability problems that are present in many traditional distributed file systems. Lustre is a highly modular next generation storage architecture that combines established, open standards, the Linux operating system, and innovative protocols into a reliable, network-neutral data storage and retrieval solution. Lustre provides high I/O throughput in clusters and shared-data environments and also provides independence from the location of data on the physical storage, protection from single points of failure, and fast recovery from cluster reconfiguration and server or network outages.

Link to the full paper:
http://www.cse.buffalo.edu/faculty/tkosar/cse710_spring13/papers/lustre-whitepaper.pdf

7 comments:

  1. In intent based locking scheme, after MDS server granting lock to client on new file, what happens if the client goes down?

    ReplyDelete
  2. 1) The RAID strategy is used to stripe data across several OSTs. Does/can lustre use the same strategy to further increase the performance by stripping the data within disks in the same machine?

    2) How the MDS and the backup MDS synchronize their data? Or they share the storage device?

    ReplyDelete
  3. Paper says that lustre uses existing security systems but haven't specified about it. Can you provide some information about which security system does it use?

    ReplyDelete
  4. How luster can be used to support third party OSTs?

    ReplyDelete
  5. The objects (that hold data associated with the file) allocated on OST's can be striped across several OST's in RAID pattern. Wouldn't this cause drastic overhead due to network operations and server contention?

    ReplyDelete
  6. (1) Is there RAID support even for the Meta Data Server(for faster access and data recovery)? and (2) Can any File System be used as the underlying FS in the MDS?

    ReplyDelete
  7. How is small write problem avoided when object is striped across several OST's in RAID pattern?

    ReplyDelete