Hadoop Questions and Answers Hadoop Archives
Hadoop Questions and Answers – Hadoop Archives This set of Hadoop Multiple Choice Questions & Answers (MCQs) focuses on “Hadoop Archives”.

Hadoop Questions and Answers Java Interface
Hadoop Questions and Answers – Java Interface This set of Hadoop Multiple Choice Questions & Answers (MCQs) focuses on “Java Interface”.

Introduction to HDFS
Introduction to HDFS This set of Multiple Choice Questions & Answers (MCQs) focuses on “Hadoop Filesystem – HDFS”.

Module 1 Tutorial Introduction
Introduction Welcome to the Yahoo! Hadoop tutorial! This series of tutorial documents will walk you through many aspects of the Apache Hadoop system. You will be shown how to set up simple and advanced cluster configurations, use the distributed file system, and develop complex Hadoop MapReduce applications. Other related systems are also reviewed. Goals for this Module: Understand the scope of problems applicable to Hadoop Understand how Hadoop addresses these problems differently from other distributed systems. Outline Introduction Goals for this Module Outline Problem Scope Challenges at Large Scale Moore's Law The Hadoop Approach Comparison to Existing Techniques Data Distribution MapReduce: Isolated Processes Flat Scalability The Rest of the Tutorial

Module 2 The Hadoop Distributed File System
Introduction HDFS, the Hadoop Distributed File System, is a distributed file system designed to hold very large amounts of data (terabytes or even petabytes), and provide high-throughput access to this information. Files are stored in a redundant fashion across multiple machines to ensure their durability to failure and high availability to very parallel applications. This module introduces the design of this distributed file system and instructions on how to operate it. Goals for this Module: Understand the basic design of HDFS and how it relates to basic distributed file system concepts Learn how to set up and use HDFS from the command line Learn how to use HDFS in your applications

Module 3: Getting Started With Hadoop
Introduction Hadoop is an open source implementation of the MapReduce platform and distributed file system, written in Java. This module explains the basics of how to begin using Hadoop to experiment and learn from the rest of this tutorial. It covers setting up the platform and connecting other tools to use it. Goals for this Module: Set up a pre-configured Hadoop virtual machine Verify that you can connect to the virtual machine Understand tools available to help you use Hadoop

Module 4: MapReduce
Introduction MapReduce is a programming model designed for processing large volumes of data in parallel by dividing the work into a set of independent tasks. MapReduce programs are written in a particular style influenced by functional programming constructs, specifically idioms for processing lists of data. This module explains the nature of this programming model and how it can be used to write programs which run in the Hadoop environment. Goals for this Module: Understand functional programming as it applies to MapReduce Understand the MapReduce program flow Understand how to write programs for Hadoop MapReduce Learn about additional features of Hadoop designed to aid software development.

Module 5: Advanced MapReduce Features
Introduction In Module 4 you learned the basics of programming with Hadoop MapReduce. That module explains how data moves through a general MapReduce architecture, and what particular methods and classes facilitate the use of the Hadoop for processing. In this module we will look more closely at how to override Hadoop's functionality in various ways. These techniques allow you to customize Hadoop for application-specific purposes. Goals for this Module: Understand advanced Hadoop features Be able to use Hadoop on Amazon EC2 and S3

Module 6: Related Topics
Introduction Hadoop by itself allows you to store and process very large volumes of data. However, building a large-scale distributed system can require functionality not provided by this base. Several other tools and systems have been created to fill the gaps and deliver a more full-featured set of distributed systems engineering tools. Goals for this Module: Understand how distributed consensus systems can be used to bootstrap larger distributed systems. Understand how to write queries in the Pig log-processing language

Module 7: Managing a Hadoop Cluster
Introduction Hadoop can be deployed on a variety of scales. The requirements at each of these will be different. Hadoop has a large number of tunable parameters that can be used to influence its operation. Furthermore, there are a number of other technologies which can be deployed with Hadoop for additional capabilities. This module describes how to configure clusters to meet varying needs in terms of size, processing power, and reliability and availability. Goals for this Module: Understand differences in requirements for different sizes of Hadoop clusters Learn how to configure Hadoop for a variety of deployment scopes

Module 8: PigTutorial
Introduction The Pig tutorial shows you how to run two Pig scripts in Local mode and Hadoop mode. Local Mode: To run the scripts in local mode, no Hadoop or HDFS installation is required. All files are installed and run from your local host and file system. Hadoop Mode: To run the scripts in hadoop (mapreduce) mode, you need access to a Hadoop cluster and HDFS installation available through Hadoop Virtual Machine provided with this tutorial. The Pig tutorial files are installed on the Hadoop Virtual Machine under "/home/hadoop-user/pig" directory. It includes the Pig JAR file (pig.jar) and the tutorial files (tutorial.jar, Pigs scripts, log files). These files work with Hadoop 0.18.0 and provide everything you need to run the Pig scripts. This Pig Tutorial is also available on the apache Pig website.