Parallel I/O Optimizations through Request Delegation for High-Performance Computing Systems
SESSION: Doctoral Research Showcase II (Workflows and Parallel and Distributed IO Optimization)
EVENT TYPE: Doctoral Research Showcase
TIME: 1:30PM - 1:48PM
SESSION CHAIR: Sadaf R. Alam
ABSTRACT: Massively parallel applications often require periodic data checkpointing for program restart and post-run data analysis. Writing multidimensional array data in the canonical order poses a great challenge for parallel I/O architecture and software designs. Parallel programs generating data on large scale systems often face serious I/O serializations, introduced by lock contention and conflicts at file system layer. Although high performance computing resources provide massive parallelism and computing power to fulfill the crucial requirements of the scientific applications, the I/O tasks of high-end applications do not scale.
We propose an I/O software architecture to bridge the gap between scientific applications and parallel storage systems. A static file domain partitioning method is developed to align the I/O requests and produce a client-server mapping that minimizes the file lock acquisition costs and eliminates lock contention. Performance evaluations using production application I/O kernels demonstrate very high I/O bandwidths which are scalable with problem size.
Sadaf R. Alam (Chair) - Swiss National Supercomputing Centre