Hadoop Administration

Understanding Big Data and Hadoop

Learning Objectives: In this module, you will understand what is Big Data and Apache Hadoop. You will also learn how Hadoop solves the Big Data problems, Hadoop Cluster Architecture, its core components & ecosystem, Hadoop data loading & reading mechanism and role of a Hadoop Cluster Administrator.

Introduction to big data

limitations of existing solutions

Hadoop architecture

Hadoop components and ecosystem

data loading & reading from HDFS

replication rules

rack awareness theory

Hadoop cluster administrator: Roles and responsibilities

Hadoop Architecture and Cluster setup

Learning Objectives: In this module, you will understand different Hadoop components; understand the working of HDFS, Hadoop cluster modes, configuration files, and more. You will also understand the Hadoop 2.0 cluster setup and configuration, setting up Hadoop Clients using Hadoop 2.0 and resolve problems simulated from a real-time environment.

Hadoop server roles and their usage

Hadoop installation and initial configuration

Deploying Hadoop in a pseudo-distributed mode

Deploying a multi-node Hadoop cluster

Installing Hadoop Clients

Understanding the working of HDFS and resolving simulated problems

Hadoop Cluster Administration and Understanding MapReduce

Learning Objectives: In this module you will understand the working of the secondary namenode, working with Hadoop distributed cluster, enabling rack awareness, maintenance mode of Hadoop cluster, adding or removing nodes to your cluster in an ad-hoc and recommended way, understand the MapReduce programming model in the context of Hadoop administrator and schedules.

Understanding secondary namenode

Working with Hadoop distributed cluster

Decommissioning or commissioning of nodes

Understanding MapReduce

Understanding schedulers and enabling them

Backup, Recovery and Maintenance

Learning Objectives: In this module, you will understand the day to day cluster administration tasks, balancing data in a cluster, protecting data by enabling trash, attempting a manual failover, creating backup within or across clusters, safeguarding your metadata and doing metadata recovery or manual failover of NameNode recovery, learn how to restrict the usage of HDFS in terms of count and volume of data, and more.

Key Hadoop Admin Commands

Trash

Import Check Point

Distcp, data backup, and recovery

Enabling trash

Namespace count quota or space quota

Manual failover or metadata recovery

Cluster planning and management

Learning Objectives: In this module, you will gather insights around cluster planning and management; learn about the various aspects one needs to remember while planning a setup of a new cluster, capacity sizing, understanding recommendations and comparing different distributions of Hadoop, understanding workload and usage patterns and some examples from the world of big data.

Planning a Hadoop 2.0 cluster

Cluster sizing, hardware

Network and software considerations

Popular Hadoop distributions

Workload and usage patterns

Industry recommendations

Hadoop 2.0 and features

Learning Objectives: In this module, you will learn more about the new features of Hadoop 2.0, HDFS High Availability, YARN framework and job execution flow, MRv2, federation, limitations of Hadoop 1.x and setting up Hadoop 2.0 Cluster setup in pseudo-distributed and distributed mode.

Limitations of Hadoop 1.x

Features of Hadoop 2.0

YARN framework

MRv2

Hadoop high availability and federation

YARN ecosystem and Hadoop 2.0 Cluster setup

Setting up Hadoop 2.X with highly availability and upgrading Hadoop

Learning Objectives: In this module, you will learn to setup Hadoop 2 with high availability, upgrading from v1 to v2, importing data from RDBMS into HDFS, understand why Oozie, Hive, and HBase are used and working on the components.

Configuring Hadoop 2 with high availability

upgrading to Hadoop 2

working with Sqoop

understanding Oozie

working with Hive

working with HBase

Project: Cloudera manager and Cluster setup, Overview on Kerberos

Learning Objectives: In this module, you will learn about Cloudera manager to setup Cluster, optimizations of Hadoop/Hbase/Hive performance parameters and understand the basics on Kerberos. You will learn to setup Pig to use in local/distributed mode to perform data analytics.

Cloudera manager and cluster setup

Hive administration

HBase architecture

HBase setup, Hadoop/Hive/HBase performance optimization

Pig setup and working with a grunt, why Kerberos and how it helps