SC is the International Conference for
 High Performnance Computing, Networking, Storage and Analysis

SCHEDULE: NOV 13-19, 2010

Scale and Concurrency of GIGA+: File System Directories with Millions of Files

SESSION: Doctoral Research Showcase II (Workflows and Parallel and Distributed IO Optimization)

EVENT TYPE: Doctoral Research Showcase

TIME: 2:24PM - 2:42PM


Presenter(s):Swapnil Patil


Over the last decade file system evolution has favored scaling for large files instead of scaling for large number of files. And two forces are calling to change this: new workloads that generate large number of small I/O accesses at high speeds and the dramatically increasing application-level parallelism. Data-intensive HPC applications have a growing need for POSIX-like file systems for trillions of files and directories with billions of files -- and most scalable file systems are ill-equipped to meet these new requirements. This thesis proposes to understand the tradeoffs in scaling traditional file system directories to store billions of files and sustain hundreds of thousands of concurrent mutations per second. We will also explore the challenges in using such large mutating directories with the existing programming API and production file system implementations. Finally, we explore how scalable directories can provide abstractions to simplify the development of data management systems.

Chair/Presenter Details:

Sadaf R. Alam (Chair) - Swiss National Supercomputing Centre

Swapnil Patil - Carnegie Mellon University

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

   Sponsors    IEEE    ACM