Apache Hive

Both partitioning and bucketing are techniques in Hive to organize the data efficiently so subsequent executions on the data works with optimal performance. Partitioning Let’s take […]

Hive provides 3 options to order or sort the result of records – order by, sort by, cluster by and distribute by. Which option you choose […]

LATERAL VIEW is a very powerful concept in Hive. It is used when we have to work with data with complex types. Let’s see this with […]

You are trying to calculate median and you couldn’t find a function to calculate median in Hive. No worries you have come to the right place. […]

It is a pretty common use case to export the contents of a Hive table into a CSV file. It’s pretty simple if you are using […]

Apache Pig was created by Yahoo. Apache Hive was created by Facebook. Both tools aimed at hiding the complexities of writing MapReduce jobs. Pig is similar […]

Both explode and posexplode are User Defined Table generating Functions. UDTFs operate on single rows and produce multiple rows as output. explode() There are 2 flavors […]

HiveServer (or HiveSerer1) was introduced when Hive first came out and it had several limitations. HiveServer2 was later introduced with Hive 0.11 and aimed to solve […]

Hive has 3 different types of functions – User Defined Function (UDF), User Defined Aggregate Function (UDAF) and User Defined Table generating Function (UDTF). User Defined […]

Let’s say you created a table with one of the columns as STRING and after a couple of days you realized the column should actually be […]

If you have read about MapReduce you know what a word count problem is. Word count is simply counting the number of words in a dataset. […]

There are several different variations and ways when it comes to inserting or loading data into Hive tables. This post will cover 3 broad ways to […]

Both INNER JOIN and LEFT SEMI JOIN return matching records between both tables with a subtle difference. Let’s consider 2 tables – employee and employee_department_mapping with […]

This is a common problem because most of the data files that come from the legacy system will contain a header in the first row. This […]

Hive stores metadata information about the tables created in Hive in a relational database like Derby, MySQL etc. The metadata information includes table name, structure of […]

Converting a String column to an Integer column or converting a column from one type to another is quite simple in Hive. Simply use the cast […]

So you were installing Hive and ran into the below issue when Hive was trying to set up the metastore database.  Exception in thread "main" java.lang.RuntimeException: […]

It is a common use case in your production jobs or Hive scripts to update or drop a Hive partition from your table. Let’s see how […]

Hive scripts which are scheduled to run in production always take in variables. These variables are set in dynamically and you would need to pass the […]