Author: olga Date: Tue Mar 17 01:20:31 2009 New Revision: 755066 URL: http://svn.apache.org/viewvc?rev=755066&view=rev Log: PIG-720: further doc cleanup (gkesavan via olgan)
Added: hadoop/pig/trunk/src/overview.html Removed: hadoop/pig/trunk/docs/ Modified: hadoop/pig/trunk/CHANGES.txt hadoop/pig/trunk/build.xml Modified: hadoop/pig/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=755066&r1=755065&r2=755066&view=diff ============================================================================== --- hadoop/pig/trunk/CHANGES.txt (original) +++ hadoop/pig/trunk/CHANGES.txt Tue Mar 17 01:20:31 2009 @@ -390,6 +390,8 @@ PIG-718: To add standard ant targets to build.xml file (gkesavan via olgan) + PIG-720: further doc cleanup (gkesavan via olgan) + Release 0.1.1 - 2008-12-04 INCOMPATIBLE CHANGES Modified: hadoop/pig/trunk/build.xml URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/build.xml?rev=755066&r1=755065&r2=755066&view=diff ============================================================================== --- hadoop/pig/trunk/build.xml (original) +++ hadoop/pig/trunk/build.xml Tue Mar 17 01:20:31 2009 @@ -36,7 +36,6 @@ <property name="src.dir" value="${basedir}/src/" /> <property name="src.lib.dir" value="${basedir}/lib-src/" /> <property name="src.gen.dir" value="${basedir}/src-gen/" /> - <property name="src.docs.dir" value="${basedir}/docs/" /> <property name="docs.dir" value="${basedir}/src/docs/" /> @@ -263,7 +262,7 @@ <!-- ================================================================== --> <target name="javadoc" depends="jar" description="Create documentation"> <mkdir dir="${build.javadoc}" /> - <javadoc overview="${src.docs.dir}/overview.html" packagenames="org.apache.pig.*" destdir="${build.javadoc}" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation"> + <javadoc overview="${src.dir}/overview.html" packagenames="org.apache.pig.*" destdir="${build.javadoc}" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation"> <packageset dir="${src.dir}" /> <link href="${javadoc.link.java}" /> <classpath> @@ -434,7 +433,7 @@ <!-- ================================================================== --> <!-- D I S T R I B U T I O N --> <!-- ================================================================== --> - <target name="package" depends="jar, javadoc" description="Create a Pig release"> + <target name="package" depends="docs" description="Create a Pig release"> <mkdir dir="${dist.dir}" /> <mkdir dir="${dist.dir}/lib" /> <mkdir dir="${dist.dir}/scripts" /> Added: hadoop/pig/trunk/src/overview.html URL: http://svn.apache.org/viewvc/hadoop/pig/trunk/src/overview.html?rev=755066&view=auto ============================================================================== --- hadoop/pig/trunk/src/overview.html (added) +++ hadoop/pig/trunk/src/overview.html Tue Mar 17 01:20:31 2009 @@ -0,0 +1,45 @@ +<HTML> +<BODY> +Pig is a platform for a data flow programming on large data sets in a parallel +environment. It consists of a language to specify these programs, +<a href="http://wiki.apache.org/pig/PigLatin">Pig Latin</a>, +a compiler for this language, and an execution engine to execute the programs. +<p> +Pig currently runs on the <a href="http://hadoop.apache.org/core/">hadoop</a> +platform, reading data from and writing data to hdfs, and doing processing via +one or more map-reduce jobs. + +<h2> Design </h2> +This section gives a very high overview of the design of the Pig system. +Throughout the documents you can see design for that package or class by +looking for the Design heading in the documentation. + +<h3> Overview </h3> +<p> +Pig's design is guided by our <a href="http://incubator.apache.org/pig/philosophy.html"> +pig philosophy</a> and by our experience with similar data processing +systems. +<p> +Pig shares many similarities with a traditional RDBMS design. It has a parser, +type checker, optimizer, and operators that perform the data processing. However, +there are some +significant differences. Pig does not have a data catalog, there are no +transactions, pig does not directly manage data storage, nor does it implement the +execution framework. +<p> +<h3> High Level Architecture </h3> +Pig is split between the front and back ends of the engine. The front end handles +parsing, checking, and doing initial optimization on a Pig Latin script. The +result is a {...@link org.apache.pig.impl.logicalLayer.LogicalPlan} that defines how +the script will be executed. +<p> +Once a LogicalPlan has been generated, the backend of Pig handles executing the +script. Pig supports multiple different +backend implementations, in order to allow Pig to run on different systems. +Currently pig comes with two backends, Map-Reduce and local. For a given run, +pig selects the backend to use via configuration. + + +</BODY> +</HTML> +