Past Details

Self-Tuning Hadoop for Big Data Analytics

Date / Time:
24/02/2014 Download ICS File
Location:
Carnegie Mellon Qatar, room 1202
Category:
Lectures and Activities
QCRI-Logo-new-2016.jpg Dr. Ashraf Aboulnaga is a Principal Scientist in the Qatar Computing Research Institute, where he heads the Distributed Systems group. His current research interests are in the area of data intensive cloud computing, with a current focus on platforms for Big Data, distributed storage systems, and data integration on the web. Dr. Aboulnaga obtained M.S. and Ph.D. degrees from the University of Wisconsin - Madison, and B.S. and M.S. degrees from Alexandria University, Egypt. He holds an Associate Professor position at the University of Waterloo in Canada, where he has worked since 2004.


Hadoop has become one of the most popular platforms for Big Data analytics, enabling flexible and powerful analysis of massive data sets on commodity computing clusters. However, tuning and administering Hadoop clusters is still not a well-studied area. In this talk, I will present two projects that aim to make Hadoop more self-tuning. The first is ReStore, a system for reusing MapReduce results. ReStore focuses on queries that are expressed in high-level query languages such as Pig Latin or Hive. The compilers of these languages translate queries into workflows of MapReduce jobs. Each job in these workflows reads its input from the Hadoop file system (HDFS) and produces output that is stored in HDFS.  The current practice is to delete all intermediate results from HDFS at the end of executing the workflow. ReStore keeps these intermediate results and reuses them for future queries, resulting in significant performance improvement. I will also present PStorM, a system for tuning Hadoop configuration parameters based on the history of previously observed MapReduce jobs. The two projects presented in the talk share the goal of automatically improving the performance of Hadoop based on workload characteristics.

Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

Dacier pic1.jpg

Cyber security experts advise caution after attacks

16/05/2017

Cyber security experts in Qatar and the region have advised caution and not to access any emails or other information from unknown sources, following the recent wave of cyber attacks across the world...

Read More

video game study Haewoon.jpg

People remain calm as the world ends, video game study suggests

28/03/2017

As the world ends, will you lock arms and sing “Kumbayah” or embark on a path of law-breaking, anti-social behavior? A new study, based upon the virtual actions of more than 80,000 players of the ...

Read More

overweight-experiment-pic.jpg

Computer vision research aims to identify overweight people from social media face photos

16/03/2017

Researchers from MIT and the Qatar Computing Research Institute have developed a novel new facility in the current rush of interest towards computer vision – an algorithm that can identify overweight...

Read More

Events

Past Events

2017

CS 1.jpg

QCRI’s Creative Space holds Open House event for kids

Download ICS File 20/05/2017 ,

The Qatar Computing Research Institute’s new Creative Space, which conducts fun activities to teach children computing skills, has successfully held its first Open House event. About 100 children ...

Read More

Slide1.JPG

QCRI-MIT CSAIL Annual Research Project Review 2017

Download ICS File 27/03/2017 ,

The QCRI – MIT CSAIL Annual Research Project Review is open to the public on Monday, March 27, 2017, at the HBKU Research Complex Multipurpose Room. The annual meeting is a highlight of a ...

Read More

MLDAS 2017

(MLDAS 2017) Machine Learning and Data Analytics Symposium

Download ICS File 13/03/2017  - 14/03/2017 , Qatar National Convention Centre

Machine Learning and Data Analytics Symposium - MLDAS 2017 Building on the success of the three previous events , Boeing and QCRI will hold the Fourth Machine Learning and Data Analytics Symposium (...

Read More

News

Majd interns ex-irina (2).jpg

QCRI’s summer internship program kicks off

07/05/2017

The Qatar Computing Research Institute’s 2017 summer internship program is underway. This year’s program involves computer science and computer engineering students from Carnegie Mellon University, ...

Read More

EMan pic for internships (2).jpg

QCRI summer 2017 internship applications open

11/03/2017

The Qatar Computing Research Institute, part of Hamad bin Khalifa University, has announced that applications for its summer internship program have opened. The internships are being offered to ...

Read More

Slide1.JPG

MIT-CSAIL's Patrick Winston: 'AI is not a threat'

05/03/2017

Artificial intelligence expert tells Qatar audience that humans should not consider AI to be a major threat to their existence but should use it to enhance it.

Read More