IBM DW606G - IBM Open Platform with Apache Hadoop - Updated

Key Topics:- Unit 1: IBM Open Platform with Apache HadoopExercise 1: Exploring the HDFS- Unit 2: Apache AmbariExercise 2: Managing Hadoop clusters with Apache Ambari- Unit 3: Hadoop Distributed File SystemExercise 3: File access and basic commands with HDFS- Unit 4: MapReduce and YarnTopic 1: Introduction to MapReduce based on MR1Topic 2: Limitations of MR1Topic 3: YARN and MR2Exercise 4: Creating and coding a simple MapReduce jobPossibly a more complex second Exercise- Unit 5: Apache SparkExercise 5: Working with Spark’s RDD to a Spark job- Unit 6: Coordination, management, and governanceExercise 6: Apache ZooKeeper, Apache Slider, Apache Knox- Unit 7: Data MovementExercise 7: Moving data into Hadoop with Flume and Sqoop- Unit 8: Storing and Accessing DataTopic 1: Representing Data: CSV, XML, JSON, and YAMLTopic 2: Open Source Programming Languages: Pig, Hive, and Other [R, Python, etc]Topic 3: NoSQL ConceptsTopic 4: Accessing Hadoop data using HiveExercise 8: Performing CRUD operations using the HBase shellTopic 5: Querying Hadoop data using HiveExercise 9: Using Hive to Access Hadoop / HBase Data- Unit 9: Advanced TopicsTopic 1: Controlling job workflows with OozieTopic 2: Search using Apache SolrNo lab exercisesObjectives:- List and describe the major components of the open-source Apache Hadoop stack and the approach taken by the Open Data Foundation.- Manage and monitor Hadoop clusters with Apache Ambari and related components- Explore the Hadoop Distributed File System (HDFS) by running Hadoop commands.- Understand the differences between Hadoop 1 (with MapReduce 1) and Hadoop 2 (with YARN and MapReduce 2).- Create and run basic MapReduce jobs using command line.- Explain how Spark integrates into the Hadoop ecosystem.- Execute iterative algorithms using Spark’s RDD.- Explain the role of coordination, management, and governance in the Hadoop ecosystem using Apache Zookeeper, Apache Slider, and Apache Knox.- Explore common methods for performing data movement (Configure Flume for data loading of log files, Move data into the HDFS from relational databases using Sqoop)- Understand when to use various data storage formats (flat files, CSV/delimited, Avro/Sequence files, Parquet, etc.).- Review the differences between the available open-source programming languages typically used with Hadoop (Pig, Hive) and for Data Science (Python, R)- Query data from Hive.- Perform random access on data stored in HBase.- Explore advanced concepts, including Oozie and Solr