Past Details

Self-Tuning Hadoop for Big Data Analytics

Date / Time:
24/02/2014 Download ICS File
Location:
Carnegie Mellon Qatar, room 1202
Category:
Lectures and Activities
QCRI-Logo-new-2016.jpg Dr. Ashraf Aboulnaga is a Principal Scientist in the Qatar Computing Research Institute, where he heads the Distributed Systems group. His current research interests are in the area of data intensive cloud computing, with a current focus on platforms for Big Data, distributed storage systems, and data integration on the web. Dr. Aboulnaga obtained M.S. and Ph.D. degrees from the University of Wisconsin - Madison, and B.S. and M.S. degrees from Alexandria University, Egypt. He holds an Associate Professor position at the University of Waterloo in Canada, where he has worked since 2004.


Hadoop has become one of the most popular platforms for Big Data analytics, enabling flexible and powerful analysis of massive data sets on commodity computing clusters. However, tuning and administering Hadoop clusters is still not a well-studied area. In this talk, I will present two projects that aim to make Hadoop more self-tuning. The first is ReStore, a system for reusing MapReduce results. ReStore focuses on queries that are expressed in high-level query languages such as Pig Latin or Hive. The compilers of these languages translate queries into workflows of MapReduce jobs. Each job in these workflows reads its input from the Hadoop file system (HDFS) and produces output that is stored in HDFS.  The current practice is to delete all intermediate results from HDFS at the end of executing the workflow. ReStore keeps these intermediate results and reuses them for future queries, resulting in significant performance improvement. I will also present PStorM, a system for tuning Hadoop configuration parameters based on the history of previously observed MapReduce jobs. The two projects presented in the talk share the goal of automatically improving the performance of Hadoop based on workload characteristics.

Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

Forbes fake news pic.jpg

Can AI Put An End To Fake News? Don't Be So Sure

07/10/2018

Fake news was the Collin’s word of the year for 2017 with good reason. In a year where politics-as-usual was torn apart at the seams, high-profile scandals rocked our faith in humanity and the ...

Read More

roadtracer.png

MIT/QCRI system uses machine learning to build road maps

22/04/2018

Map apps may have changed our world, but they still haven’t mapped all of it yet. Specifically, mapping roads can be difficult and tedious: even after taking aerial images, companies still have to ...

Read More

Economist story pic.JPG

Improving disaster response efforts through data

08/02/2018

Extreme weather events put the most vulnerable communities at high risk. How can data analytics strengthen early warning systems and and support relief efforts for communities in need? The size and ...

Read More

Events

2019

Dr Farnam Jahanian (2).jpg

“The Future of Higher Education in the Age of Technological Disruption” by CMU President Dr. Farnam Jahanian

Download ICS File 24/03/2019 ,

Dr. Farnam Jahanian, the President of Carnegie Mellon University, will deliver a public lecture, “The Future of Higher Education in the Age of Technological Disruption” at CMU’s Qatar campus on Sunday, March 24.

Read More

QCRI CSAIL Logos.JPG

QCRI - MIT CSAIL 2019 Annual Project Review

Download ICS File 25/03/2019 ,

Executive Overview Sessions Open to publi Date: March 25, 2019 Time: 10:15AM - 5:15PM Venue: Hamad Bin Khalia Reseach Complex Multipurpose Room To view agenda, please click here . To RSVP to this ...

Read More

Torralba.png

"Learning to See" Public talk by Professor Antonio Torralba (MIT-CSAIL)

Download ICS File 25/03/2019 ,

Visit by Antonio Torralba, who teaches machines to automate tasks that a human visual system can accomplish, is part of annual spring research update between QCRI and MIT-CSAIL.

Read More

News

QCRI-iMMAP MOU.jpg

QCRI and iMMAP announce Memorandum of Understanding

03/03/2019

Pact aims to apply data analysis and artificial intelligence techniques to solve humanitarian problems.

Read More

UNDP workshop.JPG

UNDP partners with QCRI to use AI for social good

11/02/2019

Qatar forum on leveraging AI to solve humanitarian problems fills to capacity.

Read More

C. Mohan pic.jpg

Renowned computing expert C. Mohan to bust blockchain myths in Qatar talk

22/01/2019

Well-known inventor of database recovery algorithms to deliver keynote at QCRI's first blockchain workshop.

Read More