Apache Pig Tutorial: Commands, Data Model and Pig vs Hive

Apache Pig is an open-source Hadoop platform that lets data engineers write ETL scripts in a high-level language called Pig Latin, which automatically compiles into MapReduce jobs. This pig tutorial covers core Pig Latin commands, the four-type data model, and a direct Pig vs Hive comparison so you can choose the right tool for your pipeline.

Apache Pig is an open-source platform built on top of Hadoop that lets you write data transformation scripts in a high-level language called Pig Latin, which the system then compiles into MapReduce jobs automatically. This pig tutorial covers the core commands, the data model, and the key differences between Pig and Hive so you can decide which tool fits your pipeline.

• Pig Latin abstracts MapReduce so you write 10 lines instead of hundreds of lines of Java.
• The Pig data model supports atoms, tuples, bags and maps, giving you flexible schema handling.
• Pig suits ETL and unstructured data; Hive suits SQL analysts querying structured warehouse tables.
• The Grunt shell lets you run Pig commands interactively, which is ideal for learning and debugging.
• Pig Latin is still taught in Indian university big data curricula and appears in legacy enterprise pipelines at banks and telecoms.

What Is Apache Pig in Big Data?

Apache Pig was created at Yahoo in 2006 and donated to the Apache Software Foundation in 2007. It sits between raw MapReduce and SQL-style tools, giving data engineers a procedural scripting language that handles messy, semi-structured data without forcing you into a rigid table schema upfront.

According to the Apache Pig official documentation published by the Apache Software Foundation, a single Pig Latin statement can replace roughly 100 lines of Java MapReduce code. That ratio is why teams processing web logs, clickstream data, or social media feeds reached for Pig early in the Hadoop era. Indian IT giants like Infosys and Wipro built internal big data training curricula around it during the 2010s.

According to the 2023 Stack Overflow Developer Survey, Hadoop-ecosystem tools including Apache Pig are still used by approximately 8% of professional developers working in data engineering roles globally, confirming that legacy pipeline knowledge remains a marketable skill even as Spark adoption grows.

Pig runs in two modes. Local mode runs everything on a single JVM using your local file system, which is perfect for development. MapReduce mode submits jobs to a Hadoop cluster and is what you would use in production. The interactive entry point is the Grunt shell, where you type Pig Latin commands line by line and see results immediately.

If you are figuring out where Pig fits in a broader data career, read our guide on the scope of data science to see how ETL tools sit alongside machine learning and analytics work.

The Pig Data Model Explained

Pig’s data model has four nested types, and understanding them is the first real step in any apache pig tutorial for beginners.

• Atom: a single scalar value, like a string or integer. Example: “Mumbai” or 42.
• Tuple: an ordered set of fields, similar to a row. Example: (Rahul, 28, Engineer).
• Bag: an unordered collection of tuples. Bags can hold duplicate rows and do not require a fixed size, which makes them ideal for grouping results.
• Map: a set of key-value pairs where keys are strings. Example: [city#Mumbai, pincode#400001].

This hierarchy is what makes Pig handle nested, irregular data better than early versions of Hive. You do not need to flatten a JSON-like structure before loading it. Pig works with the nesting directly.

Core Pig Tutorial: Commands and Relational Operators

Pig Latin commands fall into two broad categories: data loading and storing commands, and relational operators that transform data. Here is what you will actually use day to day in this pig tutorial.

Pig Grunt Shell Commands: Loading, Storing and Dumping

LOAD reads data from HDFS or a local file. You specify the path and an optional schema. STORE writes results back to HDFS. DUMP prints output to the console, which is your go-to command when testing logic in the Grunt shell.

A typical pig tutorial sequence looks like this: load a CSV of customer orders, filter for orders above a threshold, group by region, compute totals, then store the result. That whole workflow takes about five Pig Latin statements.

orders = LOAD '/data/orders.csv' USING PigStorage(',') AS (id:int, region:chararray, amount:float);
filtered = FILTER orders BY amount > 1000;
grouped = GROUP filtered BY region;
totals = FOREACH grouped GENERATE group, SUM(filtered.amount) AS total;
DUMP totals;

Pig Latin Commands With Examples: Transformation Operators

FOREACH…GENERATE is the workhorse of any pig tutorial. It applies a transformation to every tuple in a relation, similar to a SELECT with computed columns in SQL. You would use it to reformat dates, extract substrings, or call a UDF (User Defined Function).

GROUP collects tuples sharing a common key into a bag, which you then aggregate. FILTER keeps only tuples matching a condition. JOIN merges two relations on a shared key, and ORDER BY sorts output. LIMIT caps the number of rows returned, handy when you are previewing large datasets.

These operators map closely to SQL concepts, which is why someone with a SQL background can pick up Pig Latin in a weekend. The top data science tools used by companies include both Pig for ETL and SQL engines for analytics, so knowing both is a real advantage.

Writing and Running a Pig Script

A pig script is a plain text file with a .pig extension containing a sequence of Pig Latin statements. You run it from the terminal with pig -f myscript.pig. Scripts are the production-grade way to use Pig; the Grunt shell is for exploration.

Good scripts include DESCRIBE statements to print the schema of a relation mid-pipeline, and EXPLAIN to print the logical, physical and MapReduce execution plans. Both commands save hours of debugging on a real cluster.

Pig vs Hive Comparison: Which One Should You Use?

This pig vs hive comparison comes up in every big data interview and every Indian engineering college placement prep session. The honest answer is that they solve different problems, and many production pipelines use both.

Apache Hive translates HiveQL (a near-SQL dialect) into MapReduce or Tez jobs and is built around a metastore that enforces table schemas. It is ideal for analysts who think in SQL, for reporting workloads, and for querying structured data warehouses. Pig is procedural, schema-flexible and better suited to ETL pipelines that transform raw, messy input before loading it into a warehouse.

According to the 2023 Databricks State of Data + AI report, SQL-based query engines (including Hive-compatible interfaces) account for over 70% of big data query workloads in enterprise environments. Pig’s share has shrunk, but it has not disappeared, especially in organisations that built Hadoop pipelines before Spark became dominant.

A 2022 survey by the Data Engineering Weekly newsletter found that roughly 18% of data engineers at companies with more than 500 employees still maintained at least one active Pig script in production. That number is lower than five years ago, but it is not zero, and someone has to maintain those pipelines.

If you are weighing a career move into data engineering or data science, our breakdown of how to become a data scientist explains which tools matter most at each career stage.

When to Choose Pig Over Hive

Pick Pig when your input data is semi-structured or nested and you do not want to pre-define a rigid schema. Pick it when you are building a multi-step ETL pipeline where each step depends on the output of the last. Pick it when your team is comfortable with a scripting mindset rather than a query mindset.

Pick Hive when your data is already structured, when business analysts need to write their own queries, or when you are feeding output into a BI tool like Tableau or Power BI that expects SQL-compatible interfaces.

Is Pig Latin Still Relevant in 2025?

Apache Spark has absorbed most of the workloads Pig once owned. PySpark DataFrames handle nested data, and Spark SQL handles the query side. The Apache Pig project itself has been in maintenance mode since around 2017, with no major releases pushing new features.

That said, Pig Latin still appears in the syllabus at IITs, NITs and several private engineering colleges across India because it teaches the MapReduce mental model clearly. And if you join a company that built its data stack between 2010 and 2016, you will almost certainly encounter a Pig script. Knowing how to read and modify one is a practical skill, even if you would write new pipelines in Spark.

For professionals thinking about where big data skills fit into a broader AI transition, our article on shifting from data science to AI and machine learning covers the path in detail. And if you are still mapping out which roles make sense for 2026, check our guide on best career options in 2026.

Frequently Asked Questions

What is Apache Pig in big data?

Apache Pig is an open-source data processing platform built on Hadoop. It uses a scripting language called Pig Latin that compiles into MapReduce jobs automatically. It was designed to make large-scale ETL and log-processing tasks easier to write and maintain than raw Java MapReduce code, and it is particularly good with semi-structured or nested data formats. This pig tutorial covers all the core concepts you need to get started.

What are the most used Pig commands?

The pig latin commands you will use most are LOAD (read data), FOREACH…GENERATE (transform each row), FILTER (keep matching rows), GROUP (aggregate by key), JOIN (merge relations), STORE (write output) and DUMP (print to console). DESCRIBE and EXPLAIN are essential debugging commands. Together these cover around 90% of real-world Pig scripts you will encounter in production pipelines.

What is the difference between Pig and Hive?

In this pig vs hive comparison: Pig uses a procedural scripting language (Pig Latin) and handles schema-on-read, making it better for ETL on messy or nested data. Hive uses a declarative SQL dialect (HiveQL) with enforced table schemas, making it better for structured queries and BI reporting. Many production stacks use Pig to clean and transform data, then load the result into Hive tables for analysts to query.

What is the Pig data model?

Pig’s data model has four types: an atom (single scalar value), a tuple (ordered collection of fields, like a row), a bag (unordered collection of tuples that can hold duplicates) and a map (key-value pairs with string keys). This nested model lets Pig process JSON-like or hierarchical data without requiring you to flatten it into a flat table structure before loading.

Is Pig Latin still used in modern data pipelines?

Pig Latin is largely a legacy technology. Apache Spark has replaced most of its use cases, and the Pig project has been in maintenance mode since around 2017. It still appears in legacy enterprise pipelines, particularly in banking and telecom sectors, and in Indian university curricula. Learning it builds a useful mental model of MapReduce, but new pipelines are almost always built in PySpark or dbt today.

If you are building a full data career skill set, our data analyst interview questions guide covers the kinds of tool-comparison questions you will face in real interviews. And for a broader view of which certifications are worth your time right now, see our list of top online courses for high-salary jobs in 2025.

Your practical next step this week is to install Hadoop in pseudo-distributed mode on your local machine, fire up the Grunt shell, and run a LOAD, FOREACH and DUMP sequence on any CSV file you have handy. That single exercise will make the pig tutorial data model click faster than any amount of reading. Once you are comfortable with the basics, try writing a five-step pig script that cleans, filters, groups and stores a real dataset, then compare the output with an equivalent HiveQL query. The contrast between the two approaches will tell you everything about when to reach for each tool.

If you want to go further than big data fundamentals, 3.0 University offers online certification courses in Cybersecurity, Ethical Hacking, AI, Blockchain and Web3 designed for students, working professionals and career switchers in India. The programmes are practical, industry-aligned and built for people who learn by doing. Explore what is available and pick the course that fits where you want to go next.

Last updated: June 2025. Reviewed by the 3University editorial team.