February 16, 2017
Hadoop Starter Kit – Tutorial In this Hadoop Tutorial a.k.a Hadoop Starter Kit you will learn about the core concepts of Hadoop like HDFS, MapReduce and […]
February 13, 2017
With the Working with HDFS post, we now know how to work with HDFS. It is now time to look at the important components and processes […]
February 9, 2017
In the HDFS – Why another filesytem post, we got ourselves introduced about HDFS its time to try some HDFS commands. You are probably thinking why […]
February 6, 2017
In Understanding Big Data Problem post we saw that HDFS or Hadoop Distributed filesystem takes care of all the storage related complexities in Hadoop. In this […]
February 2, 2017
Finding the MAX tuple with Pig Here is a sample dataset. Our goal is to find the record with maximum record_value which is DEF, 300 record_key, […]
January 30, 2017
How to find directories in HDFS which are older than N days? Cleaning up older or obsolete files in HDFS is important. Even if you have […]
January 26, 2017
How to use multi character delimiter in a Hive table? Sometimes your data is slightly complex to delimit the individual columns with a single character like […]
January 23, 2017
Change field termination value in Hive This blog post describes how to change the field termination value in Hive. Assume when you created the Hive table, […]
January 19, 2017
DataNode process killed due to Incompatible clusterIDs error This blog post will describe how to address Incompatible clusterIDs with DataNodes. 2013-04-11 16:26:15,720 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed […]
January 16, 2017
FSNamesystem initialization failed FSNamesystem initialization failed is a common error Hadoop users gets especially if there are trying to set up of a Hadoop cluster for […]
January 12, 2017
Hadoop safemode recovery – taking too long! Any time NameNode is restarted or started, NameNode first goes into maintenance state called Safe Mode. When NameNode is […]
January 9, 2017
There are 0 datanode(s) running and no node(s) You are trying to write a file to HDFS and this is what you see in your datanode […]
October 9, 2016
Hadoop Administrator In Real World – Course Coverage We launched Hadoop Developer In Real World course on Nov 2015 and we got excellent response from the […]
October 1, 2016
What employers expect from Hadoop Administrators? In this post we will discuss what employers expect from Hadoop Administrators. We also have a video version of this post, […]
September 24, 2016
Is Hadoop Administration right for me? When we first announced that we are working on a new Hadoop Administration course we had several students and members […]
July 10, 2016
Changing The Output File Prefix Of Hadoop MapReduce Job Your Hadoop job can have multiple reducers and each reducer will create a file by default with […]
June 23, 2016
Missing Artifact JDK Tools Jar Some versions of Maven / eclipse will give you the below error in your pom.xml Missing artifact jdk.tools:jdk.tools:jar:1.7.0_79 It is very […]
June 22, 2016
Hadoop Mapper and Reducer Output Mismatch Can you have different output Key Value pair types for Mapper and Reducer in a MapReduce program? Short answer – […]
December 31, 2015
Apache Pig Tutorial – Map Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. All posts […]
December 31, 2015
Apache Pig Tutorial – Tuple & Bag Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. […]
