Installing and running PIG 0.11.1 with Hadoop 1.2.1

1. Download pig from pig.apache.org
2. copy it to the hadoop user /home/hduser/
3. extract , make sure the extracted directory has the same permission (hduser:hadoop), insert path to $PATH
4. Start the hadoop server (./start-all.sh), check with jps
5. Copy the data to HDFS,
>hadoop dfs -copyFromLocal /user/hduser/<file-name>
6. run script using pig –
>pig <script-name>
7. check if output is generated
>hadoop dfs -ls /user/hduser/<file-name>/ [Note that , the output directory is automatically created and has the same name as the input file]
8. check output file
> hadoop dfs -cat /user/hduser/<output-dir>/part-r-00000|head -5

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s