SC is the International Conference for
 High Performnance Computing, Networking, Storage and Analysis

• Overview• Registration• Schedule• Keynote Speaker• Papers• Tutorials• Panels• Posters• Workshops• Birds-of-a-Feather• Doctoral Showcase• Awards• Disruptive Technologies• SC10 Challenges• Storage Challenge • Masterworks• Submissions Website




The Storage Challenge Submission Guidelines and Timeline



Important Dates

Submissions Open
March 1, 2010

Submissions Due
July 19, 2010

Finalists Notified
August 9, 2010

Final Report Due
November 8, 2010

Technical Session Presentations
November 16, 2010

Finalist Interviews
November 17, 2010

Awards Sessions
November 18, 2010

 

To make submissions, go to the submission website: http://submissions.supercomputing.org/

Questions: storage-challenge@info.supercomputing.org

OVERVIEW: Participating in the HPC Storage Challenge


Teams participating in the SC09 HPC Storage Challenge will begin work at their earliest convenience and must submit a Project Proposal by July 19, 2010; the Project Proposal is the Submission. The Project Proposals will be used to select a small number of finalist teams who will be notified of their status by August 9, 2010.Finalist teams may continue working on their challenge projects after submitting their Project Proposal until the Final Project Report is due (November 8, 2010). During the conference, finalist teams will deliver a technical session presentation that will be part of the Storage Challenge judging. They will also participate in a private follow up interview with the committee. Titles, abstracts and team member names will be posted in the Conference Program and the winner(s) will be announced at the conference awards session.

HPC Storage Challenge Resources


Teams participating in the SC10 HPC Storage Challenge will work on their challenge projects in advance of the conference with resources they can obtain outside of the conference. The conference can not supply the storage and other equipment resources for use by the teams.

Size of the Storage System is Not the Basis for Success

The size of a storage system is not a basis for judging in the HPC Storage Challenge.As the HPC community matures there a large variance in the size of HPC storage systems is emerging.Often designs for large systems begin with smaller systems used as a proof of concept. Moreover, a great deal of creative solutions viewed as smaller can be as compelling as larger systems.Besides, size is a relative term and difficult to define without being arbitrary. Things that determine the size of a storage system include any number of factors, not just the storage system itself (e.g., the number of nodes, number of processors per node, size of the network, amount of storage, bandwidth potential for the storage, etc.). Therefore, teams using systems of all sizes are encouraged to make submissions.

Finalists and Winners


Finalists will be notified by August 9, 2010 that they have been selected. Their sponsoring institutions may use this information for publicity. Each finalist team will be expected to make a presentation in a technical session during the conference. The schedule for the technical presentation sessions will be posted on the Storage Challenge web site several weeks after the notification date and listed in the final program.The Conference Program will also include a 150 word abstract and team member names along with their sponsoring institutions. The final version of this abstract and team list is due July 19, 2010 along with the project proposal. After the finalist teams have been notified, any changes made to the abstract and team list can not be reflected in the printed program. The winners will be announced in the conference awards session. Both finalists and winners will receive certificates for their accomplishments in this challenge.

Reports

Two principal reports must be submitted as part of the SC10 HPC Storage Challenge. The first one is the project proposal due July 19, 2010. The second one is the Final Project report due November 8, 2010.

Project Proposal

Participating teams are required to submit a Project Proposal by July 19, 2010 via the submission website which will be used by the judges to select finalists. Teams are encouraged to include results from proof of concept tests or works in progress to lend credibility to assertions made in their Project Proposal. A report in the form of a PDF file with at most 12 pages is to be included as part of the submission. A suggested format for this report follows

  • Title
  • Team Information
  • Abstract
  • Problem Statement (including hypothesis)
  • Hardware Configuration
  • Data and Storage Layout
  • Software Used (including application and systems software)
  • Description of the solution to the problem
  • Experiments to be done, and measurements and quantitative results to be collected
    • Include results from proof of concept tests or works in progress
  • Claims (how this project satisfies judging criteria)
    • Overall Performance
    • System and/or Application Scalability
    • Storage Resource Utilization
    • Innovation
    • Effectiveness
  • Acknowledgements
    • List of individuals or agencies not listed in the team information who made significant contributions to, or supported the work being reported

Final Project Report

Finalist teams in the SC10 HPC Storage Challenge are required to submit a Final Project Report by November 8, 2010 via the submission website which will be used by the judges to help select the winner(s). A report in the form of a PDF file with at most 12 pages is to be included as part of the submission. It must also include slides for the challenge finalist presentation in an appendix. Other appendices may also be included, but it is not guaranteed that they will be reviewed by the judges. The appendices are not included in the 12 page upper limit. A suggested format for this report follows

  • Title
  • Team Information
  • Abstract
  • Problem Statement (including hypothesis)
  • Hardware Configuration
  • Data and Storage Layout
  • Software Used (including application and systems software)
  • Description of the solution to the problem
  • Experiments, measurements and quantitative results of solution
    • Describe actual experiments and results
    • Describe how the actual measurements were made
    • Explain how they support the hypothesis
  • Describe conclusions based on the experiments
  • Describe how the judging criteria apply to the project
    • Overall Performance
    • System and/or Application Scalability
    • Storage Resource Utilization
    • Innovation
    • Effectiveness
  • Describe any deviations from the Project Proposal
    • Deviations from the Project Proposal as part of the exploratory process are anticipated, but the work must be in the same spirit as proposed
  • Price/performance information (see explanation below)
  • Appendices
    • Slides for challenge finalist presentation (required)
    • Other optional appendices
    • Acknowledgements: list of individuals or agencies not listed in the team information who made significant contributions to, or supported the work being reported

Finalist Interviews


Finalist teams in the SC10 HPC Storage Challenge will participate in a private interview session with the Challenge committee. The purpose of this interview is to provide the committee with an opportunity to ask questions and seek clarifications that may arise from the finalist presentations and/or the Final Project Report.

The interviews are scheduled for Wednesday, November 17, 2010 starting at 8:00 AM. The exact interview schedule and location in the convention center will be sent to finalist teams via email. The schedule will also be posted on a sign just outside the interview room.

Judging

Project Proposal submissions and Final Project Report submissions to the SC10 HPC Storage Challenge will be evaluated by the committee based on a mixture of quantitative and qualitative criteria. Judging decisions are final.

Judging will be based on 3 broad categories: performance and scalability, effective use of storage and storage related resources, and innovation. The following list presents examples of things to be considered within each of these categories. These lists are neither exclusive nor exhaustive; it will be up to each team to select the appropriate measurements and analyses to make the case for their project.

  1. Performance and Scalability
    1. File size, application I/O request, file system block size
    2. Application measurements (select appropriate measurement)
      1. I/O rate in MB/s (both as an aggregate and harmonic mean)
      2. IOP rate (both as an aggregate and harmonic mean)
      3. File creations per second
      4. Bandwidth per kilometer
    3. System measurements
      1. Bandwidth tools (e.g., iostat, netstat)
      2. Latency tools (e.g., histograms of record size vs. time)
    4. Storage system or vendor tools
    5. Scalability; ability to expand solution to increasing numbers of
      1. Disks and disk controllers
      2. Nodes
    6. Ensure that the magnitude of I/O operations is large enough to actually use disk
  2. Effective use of resources
    1. Measure storage and storage related resource utilization
      1. disk/disk controllers, SAN, LAN
      2. percent of peak BW per disk controller
      3. percent of SAN/LAN utilization
      4. assumes the application is storage I/O intensive
    2. Requires "compelling" baseline measurements
  3. Innovation
    1. Novel ways to solve traditional and/or difficult problems
    2. Effective use of new technologies
    3. Well designed application
    4. Best practices
    5. The "Wow, that's COOL!" factor

Some Practical Advice

This is now the fourth year that the HPC Storage Challenge has been offered. These guidelines provide various suggested outlines and judging criteria. Experience shows that adhering to the outlines but more importantly providing complete information without being excessively wordy and explicitly addressing judging criteria is an effective strategy.

Good luck to all teams and have fun!

To make submissions, go to the submission website: http://submissions.supercomputing.org/

Questions: storage-challenge@info.supercomputing.org

   Sponsors    IEEE    ACM