SEARCH
ISTC-CC NEWSLETTER
RESEARCH HIGHLIGHTS
Ling Liu's SC13 paper "Large Graph Processing Without the Overhead" featured by HPCwire.
ISTC-CC provides a listing of useful benchmarks for cloud computing.
Another list highlighting Open Source Software Releases.
Second GraphLab workshop should be even bigger than the first! GraphLab is a new programming framework for graph-style data analytics.
ISTC-CC Abstract
I/O Containers: Managing the Data Analytics and Visualization Pipelines of High End Codes
Workshop on High Performance Data Intensive Computing (HPDIC'13), with IPDPS 2013, May 2013. Best paper.
Jai Dayal, Karsten Schwan, Jay Lofstead†, Matthew Wolf, Scott Klasky*,
Hasan Abbasi*, Norbert Podhorszki*, Greg Eisenhauer, Fang Zhen
Georgia Institute of Technology
*Oak Ridge National Laboratory
†Sandia National Laboratory
Lack of I/O scalability is known to cause measurable slowdowns for large-scale scientific applications running on high end machines. This is prompting researchers to devise 'I/O staging' methods in which outputs are processed via online analysis and visualization methods to support desired science outcomes. Organized as online workflows and carried out in I/O pipelines, these analysis components run concurrently with science simulations, often using a smaller set of nodes on the high end machine termed 'staging areas'. This paper presents a new approach to dealing with several challenges arising for such online analytics, including: how to efficiently run multiple analytics components on staging area resources providing them with the levels of end-to-end performance they need and how to manage staging resources when analytics actions change due to user or data-dependent behavior. Our approach designs and implements middleware constructs that delineate and manage I/O pipeline resources called 'I/O Containers'. Experimental evaluations of containers with realistic scientific applications demonstrate the feasibility and utility of the approach.
KEYWORDS: Data Staging, Data Analytics, in-Situ, Visualization, Scalable I/O, Runtime Management, resource sharing
FULL PAPER: pdf
