SEARCH
ISTC-CC NEWSLETTER
RESEARCH HIGHLIGHTS
Ling Liu's SC13 paper "Large Graph Processing Without the Overhead" featured by HPCwire.
ISTC-CC provides a listing of useful benchmarks for cloud computing.
Another list highlighting Open Source Software Releases.
Second GraphLab workshop should be even bigger than the first! GraphLab is a new programming framework for graph-style data analytics.
ISTC-CC Abstract
PACMan: Coordinated Memory Caching for Parallel Jobs
NSDI '12, San Jose, CA, April 25-27, 2012.
Ganesh Ananthanarayanan*, Ali Ghodsi*‡, AndrewWang*, Dhruba Borthakur^,
Srikanth Kandula†, Scott Shenker*, Ion Stoica*
* University of California, Berkeley
^ Facebook
† Microsoft Research
‡ KTH/Sweden
Data-intensive analytics on large clusters is important for modern Internet services. As machines in these clusters have large memories, in-memory caching of inputs is an effective way to speed up these analytics jobs. The key challenge, however, is that these jobs run multiple tasks in parallel and a job is sped up only when inputs of all such parallel tasks are cached. Indeed, a single task whose input is not cached can slow down the entire job. To meet this "all-or-nothing" property, we have built PACMan, a caching service that coordinates access to the distributed caches. This coordination is essential to improve job completion times and cluster efficiency. To this end, we have implemented two cache replacement policies on top of PACMan's coordinated infrastructure – LIFE that minimizes average completion time by evicting large incomplete inputs, and LFU-F that maximizes cluster efficiency by evicting less frequently accessed inputs. Evaluations on production workloads from Facebook and Microsoft Bing show that PACMan reduces average completion time of jobs by 53%and 51%(small interactive jobs improve by 77%), and improves efficiency of the cluster by 47% and 54%, respectively.
FULL PAPER: pdf
