Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change 
notification.

The following page has been changed by CorinneC:
http://wiki.apache.org/pig/FrontPage

------------------------------------------------------------------------------
  
  = Pig Wiki =
  
- [http://incubator.apache.org/pig/ Pig] is a dataflow programming environment 
for processing very large files. Pig's language is called Pig Latin. A Pig 
Latin program consists of a directed acyclic graph where each node represents 
an operation that transforms data. Operations are of two flavors: (1) 
relational-algebra style operations such as join, filter, project; (2) 
functional-programming style operators such as map, reduce. 
+ [http://incubator.apache.org/pig/ Pig] is a platform for analyzing large data 
sets. Pig's language, Pig Latin, lets you specify a sequence of data 
transformations such as merging data sets, filtering them, and applying 
functions to records or groups of records. You can also create your own 
functions to do special-purpose processing 
  
- Pig compiles these dataflow programs into (sequences of) map-reduce jobs and 
executes them using Hadoop. It is also possible to execute Pig Latin programs 
in a "local" mode (without Hadoop cluster), in which case all processing takes 
place in a single local JVM.
+ Pig Latin programs run in a distributed fashion on a cluster (programs are 
complied into Map/Reduce jobs and executed using Hadoop). For quick 
prototyping, Pig Latin programs can also run in "local mode" without a cluster 
(all processing takes place in a single local JVM).
  
- '''[http://www.cs.cmu.edu/~olston/publications/sigmod08.pdf Why Pig Latin 
instead of SQL?]'''
+ '''Do you Pig?''' Most of Yahoo! does: 30% of Hadoop jobs are run with Pig.
+ 
  
  == News ==
  
+ '''Why Pig Latin instead of SQL?'''  
[http://www.cs.cmu.edu/~olston/publications/sigmod08.pdf Pig Latin: A 
Not-So-Foreign Language ...]
+ 
+ '''Pig Has Grown Up!'''. On 10/22/08 Pig graduated from the 
[http://incubator.apache.org/ Incubator] and joined [http://hadoop.apache.org/ 
Hadoop] as a subproject.
+ 
  '''Pig is Getting Faster!'''  2-6 times faster, for many queries.  We've 
created a set of benchmarks and run them against the pig 0.1.0 release 
(modified to run on hadoop
- 0.18) and against the code on the types branch (which will soon be merged 
into trunk and released as version 0.2.0).  Joins and order
+ 0.18) and against the code on the types branch (which will soon be merged 
into the trunk and released as version 0.2.0).  Joins and order
- bys in particular made large performance gains.  See PigMix for complete 
details.
+ bys in particular made large performance gains. For complete details see 
PigMix.
  
+ '''Interested in Pig Guts?''' We are completely redesigning the Pig execution 
and optimization framework. For design details see the PigOptimizationWishList 
and PigExecutionModel. 
- '''Pig Has Grown Up!'''. As of today (10/22/08) Pig has graduated from the 
[http://incubator.apache.org/ Incubator] and joined [http://hadoop.apache.org/ 
Hadoop] as a subproject. The actual transition will occur in the next couple of 
weeks.
- 
- '''Do you want your Pig to Fly?''' Check PigUserCookbook for tips on how to 
write efficient Pig scripts.
- 
- '''Do you Pig?''' Most of Yahoo does!
-  * 100s of uses!
-  * 1000s of jobs!
-  * 30% of Hadoop jobs are run via Pig
- 
- '''New to Pig?''' First BuildPig or download it, then learn how to configure 
and RunPig, and get a PigOverview.  Finally, try our fresh off the press 
PigTutorial!
- 
- '''New to Pig Latin?''' Take a look at our Pig Latin Reference Manual: 
attachment:PL_Manual (for use with Pig code on the types branch)
- 
- '''Need Pig functions?''' Take a look at our brand new PiggyBank! '''Want to 
know more?''' Check [http://wiki.apache.org/pig/UDFManual UDF Manual!]
- 
- '''Interested in Pig Guts?''' We are completely redesigning the Pig execution 
and optimization framework. This work includes (1) creating new operator 
representations at the various layers (logical, physical, map-reduce) to 
facilitate optimization and (2) streamlining the execution pipeline. See the 
PigOptimizationWishList and PigExecutionModel for the design details. 
Implementation is already underway ...
  
  == General Information ==
  
   * [http://hadoop.apache.org/pig/ Official Apache Pig Website]
-  * PigOverview - An overview of Pig's capabilities
   * PigTalksPapers - Pig talks, papers, interviews 
   
  == User Documentation ==
  
+ Documentation is now available for Pig code on the Trunk branch and Types 
branch.
-  * Getting Started
-   * BuildPig - How to build Pig
-   * RunPig - How to configure and run Pig (also see the PigOverview)
-   * PigTutorial - Tackle a real task with pig, start to finish.
-   * PigUserCookbook - Tips and Tricks.
  
-  * Pig System
-   * ["Grunt"] - The shell manual (for use with Pig code on the trunk branch)
-   * PigLatin - The language manual (for use with Pig code on the trunk branch)
-   * PigFunctions - Built-ins, Piggy Bank, write your own (for use with Pig 
code on the trunk branch)
+  * New to Pig? Getting Started
+   1. PigOverview - An overview of Pig's capabilities
+   1. BuildPig - How to build Pig
+   1. RunPig - How to configure and run Pig
+   1. PigTutorial - Tackle a real task with pig, start to finish (''Trunk 
branch'')
+ 
+  * Pig Languages
+   * ["Grunt"] Shell and PigLatin Manual (''Trunk branch'')
+   * [attachment:PL_Manual Pig Latin Reference Manual] (''Types branch'')
+ 
+  * Pig Functions
+   * PigBuiltins - Pig built-in functions
+   * PiggyBank - User-defined functions (UDFs) contributed by Pig users!
+   * WriteFunctions - Write your own UDFs (''Trunk branch'')
+   * [http://wiki.apache.org/pig/UDFManual UDF Manual] - Write your own UDFs 
(''Types branch'')
+ 
+  * More Pig
+   * PigUserCookbook - Want Pig to fly? Tips and tricks on how to write 
efficient Pig scripts.
    * [http://hadoop.apache.org/pig/javadoc/docs/api/ Javadocs] - Refer to the 
Javadocs for embedded Pig and UDFs
    * PigFaq - The answer to your question may be here 
  

Reply via email to