Past Details

Self-Tuning Hadoop for Big Data Analytics

Date / Time:
24/02/2014 Download ICS File
Location:
Carnegie Mellon Qatar, room 1202
Category:
Lectures and Activities
QCRI-Logo-new-2016.jpg Dr. Ashraf Aboulnaga is a Principal Scientist in the Qatar Computing Research Institute, where he heads the Distributed Systems group. His current research interests are in the area of data intensive cloud computing, with a current focus on platforms for Big Data, distributed storage systems, and data integration on the web. Dr. Aboulnaga obtained M.S. and Ph.D. degrees from the University of Wisconsin - Madison, and B.S. and M.S. degrees from Alexandria University, Egypt. He holds an Associate Professor position at the University of Waterloo in Canada, where he has worked since 2004.


Hadoop has become one of the most popular platforms for Big Data analytics, enabling flexible and powerful analysis of massive data sets on commodity computing clusters. However, tuning and administering Hadoop clusters is still not a well-studied area. In this talk, I will present two projects that aim to make Hadoop more self-tuning. The first is ReStore, a system for reusing MapReduce results. ReStore focuses on queries that are expressed in high-level query languages such as Pig Latin or Hive. The compilers of these languages translate queries into workflows of MapReduce jobs. Each job in these workflows reads its input from the Hadoop file system (HDFS) and produces output that is stored in HDFS.  The current practice is to delete all intermediate results from HDFS at the end of executing the workflow. ReStore keeps these intermediate results and reuses them for future queries, resulting in significant performance improvement. I will also present PStorM, a system for tuning Hadoop configuration parameters based on the history of previously observed MapReduce jobs. The two projects presented in the talk share the goal of automatically improving the performance of Hadoop based on workload characteristics.

Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

video game study Haewoon.jpg

People remain calm as the world ends, video game study suggests

28/03/2017

As the world ends, will you lock arms and sing “Kumbayah” or embark on a path of law-breaking, anti-social behavior? A new study, based upon the virtual actions of more than 80,000 players of the ...

Read More

overweight-experiment-pic.jpg

Computer vision research aims to identify overweight people from social media face photos

16/03/2017

Researchers from MIT and the Qatar Computing Research Institute have developed a novel new facility in the current rush of interest towards computer vision – an algorithm that can identify overweight...

Read More

algorithm bridge.jpg

Algorithm can create a bridge between Clinton and Trump supporters

20/02/2017

A growing number of people have expressed their concern about high levels of polarization in society. For instance, the World Economic Forum's report on global risks lists the increasing societal ...

Read More

Events

Past Events

2017

Slide1.JPG

QCRI-MIT CSAIL Annual Research Project Review 2017

Download ICS File 27/03/2017 ,

The QCRI – MIT CSAIL Annual Research Project Review is open to the public on Monday, March 27, 2017, at the HBKU Research Complex Multipurpose Room. The annual meeting is a highlight of a ...

Read More

MLDAS 2017

(MLDAS 2017) Machine Learning and Data Analytics Symposium

Download ICS File 13/03/2017  - 14/03/2017 , Qatar National Convention Centre

Machine Learning and Data Analytics Symposium - MLDAS 2017 Building on the success of the three previous events , Boeing and QCRI will hold the Fourth Machine Learning and Data Analytics Symposium (...

Read More

ArabWic for web.jpg

Women in Data Science

Download ICS File 03/02/2017 ,

Here's a great chance to learn about the latest data science-related research in multiple domains, as part of a global project. Qatar's WiDS event will be held here at the HBKU Research Complex on ...

Read More

News

EMan pic for internships (2).jpg

QCRI summer 2017 internship applications open

11/03/2017

The Qatar Computing Research Institute, part of Hamad bin Khalifa University, has announced that applications for its summer internship program have opened. The internships are being offered to ...

Read More

Slide1.JPG

MIT-CSAIL's Patrick Winston tells Qatar audience: 'AI is not a threat'

05/03/2017

Artificial intelligence expert says humans should not consider AI to be a major threat to their existence but should use it to enhance it.

Read More

MLDAS 2016.JPG

Boeing Partners with QCRI for fourth annual Machine Learning and Data Analytics Symposium (MLDAS)

09/02/2017

The Boeing Company has announced that it will once again partner with the Qatar Computing Research Institute (QCRI), part of Hamad bin Khalifa University, to host the fourth annual Machine Learning ...

Read More