SC is the International Conference for
 High Performnance Computing, Networking, Storage and Analysis

SCHEDULE: NOV 13-19, 2010

Automatic Run-time Parallelization and Transformation of I/O

SESSION: Runtime Resource Allocation and Scheduling


TIME: 4:00PM - 4:30PM


AUTHOR(S):Thorvald Natvig, Anne C. Elster, Jan Christian Meyer


As the size of computational clusters grows, one can expect that I/O will consume an increasing portion of wall-clock time as the problem and node sizes are scaled up, unless parallel I/O is introduced. Unfortunately, using parallel I/O is non-trivial, so few applications developed by individual researchers enjoy its benefits. In this paper, we describe our novel method for analyzing I/O and communication operations at run-time. When nodes perform I/O or communication operations, our technique protects the memory associated with the requests from the application. Subsequent operations are analyzed for overlap between communication and I/O operations. When found, the I/O operation is automatically transformed, by our injected library, from an individual operation to a collective and shared MPI I/O operation. This allows users to benefit from parallel file systems without redesigning or recompiling their applications, and we demonstrate speedup for common usage patterns.

Chair/Author Details:

Joel Saltz (Chair) - Emory University

Thorvald Natvig - Norwegian University of Science & Technology

Anne C. Elster - Norwegian University of Science & Technology

Jan Christian Meyer - Norwegian University of Science & Technology

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

The full paper can be found in the ACM Digital Library and IEEE Computer Society

   Sponsors    IEEE    ACM