Apache Pig Tutorial - Executing Script with Parameters - Big Data In Real World

Apache Pig Tutorial – Executing Script with Parameters

Apache Pig Tutorial – Executing as a Script
December 20, 2015
Apache Pig Tutorial – Tuple & Bag
December 31, 2015
Apache Pig Tutorial – Executing as a Script
December 20, 2015
Apache Pig Tutorial – Tuple & Bag
December 31, 2015

Apache Pig Tutorial – Executing Script with Parameters

Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. All posts will be short and sweet. Most posts will have (very short) “see it in action” video.

In the previous post, we saw how to run Pig Latin instructions as a script. But there is one problem with that; we hard coded the Input and the Output location and it is not ideal. In this lesson we will see how we can pass parameters to the script.

Parameter Placeholders

Take a look at the below sample. ‘$input’ is a parameter which defines the input location and ‘$output’ is a parameter which defines the output location.

prices = LOAD '$input' USING PigStorage(',') as (exchange:chararray, symbol:chararray, date:datetime, open:float, high:float, low:float, close:float,volume:int, adj_close:float);
STORE top10 INTO '$output' USING PigStorage(',');

Passing Parameters

Now we know how to set the parameters, let’s see how to pass parameters from the command line. Use -param  and the parameter name to set and pass the value for the paramaters. In the below sample, at run time the value of the $input  in the script will be substituted as /user/hirw/input/stocks

pig -param input=/user/hirw/input/stocks -param output=output/pig/avg-volume-params /hirw-workshop/pig/scripts/average-volume-parameters.pig

 See It In Action

Previous Lesson : Running as a Script

 Next Lesson : Tuple & Bag

Big Data In Real World
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.

2 Comments

  1. […] Next Lesson : Executing Script with Parameters […]

  2. […] Previous Lesson : Execute Script with Parameters […]

Apache Pig Tutorial – Executing Script with Parameters
This website uses cookies to improve your experience. By using this website you agree to our Data Protection Policy.

Hadoop In Real World is now Big Data In Real World!

X