Author: gates
Date: Tue Aug 24 17:14:12 2010
New Revision: 988628
URL: http://svn.apache.org/viewvc?rev=988628&view=rev
Log:
PIG-1559 Updates to Pig philosophy.
Modified:
hadoop/pig/site/author/src/documentation/content/xdocs/philosophy.xml
hadoop/pig/site/publish/philosophy.html
hadoop/pig/site/publish/philosophy.pdf
Modified: hadoop/pig/site/author/src/documentation/content/xdocs/philosophy.xml
URL:
http://svn.apache.org/viewvc/hadoop/pig/site/author/src/documentation/content/xdocs/philosophy.xml?rev=988628&r1=988627&r2=988628&view=diff
==============================================================================
--- hadoop/pig/site/author/src/documentation/content/xdocs/philosophy.xml
(original)
+++ hadoop/pig/site/author/src/documentation/content/xdocs/philosophy.xml Tue
Aug 24 17:14:12 2010
@@ -21,10 +21,8 @@
<section>
<title>Pigs Eat Anything</title>
<p>
- Pig can operate on data whether it has metadata or not.
- </p>
- <p>
- It can operate on data that is relational, nested, or unstructured.
+ Pig can operate on data whether it has metadata or not. It can
operate on data that is relational, nested, or unstructured. And it can easily
be
+ extended to operate on data beyond files, including key/value stores,
databases, etc.
</p>
</section>
@@ -32,7 +30,7 @@
<title>Pigs Live Anywhere</title>
<p>
Pig is intended to be a language for parallel data processing. It is
not tied to one particular parallel framework. It has been implemented first
- on hadoop, but we do not intend that to be only on hadoop.
+ on Hadoop, but we do not intend that to be only on Hadoop.
</p>
</section>
@@ -44,12 +42,14 @@
<p>
Pig allows integration of user code where ever possible, so it
currently supports user defined field transformation functions, user defined
- aggregates, user defined grouping functions, and user defined
conditionals. In the future we want to support all the above in non-java
languages,
- as well as streaming, user defined types, and user defined splits.
+ aggregates, and user defined conditionals. These functions can be
written in Java or scripting languages that can compile down to Java (e.g.
Jython).
+ Pig supports user provided load and store functions. It supports
external executables via its stream command and Map Reduce jars via its
mapreduce
+ command. It allows users to provide a custom partitioner for their jobs
in some circumstances and to set the level of reduce parallelism for their jobs.
+ command. It allows users to set the level of reduce parallelism for
their jobs and in some circumstances to provide a custom partitioner.
</p>
<p>
- Currently pig has no optimizer, so it does not do any operation
rearranging. When we add that in the future, it will always be possible for
users to
- turn code rearranging off, so that pig does exactly what they say in
the order they say it.
+ Pig has an optimizer that rearranges some operations in Pig Latin
scripts to give better performance, combines Map Reduce jobs together, etc.
However, users
+ can easily turn this optimizer off to prevent it from making changes
that do not make sense in their situation.
</p>
</section>
Modified: hadoop/pig/site/publish/philosophy.html
URL:
http://svn.apache.org/viewvc/hadoop/pig/site/publish/philosophy.html?rev=988628&r1=988627&r2=988628&view=diff
==============================================================================
--- hadoop/pig/site/publish/philosophy.html (original)
+++ hadoop/pig/site/publish/philosophy.html Tue Aug 24 17:14:12 2010
@@ -199,25 +199,23 @@ document.write("Last Published: " + docu
<h2 class="h3">Pigs Eat Anything</h2>
<div class="section">
<p>
- Pig can operate on data whether it has metadata or not.
- </p>
-<p>
- It can operate on data that is relational, nested, or unstructured.
+ Pig can operate on data whether it has metadata or not. It can
operate on data that is relational, nested, or unstructured. And it can easily
be
+ extended to operate on data beyond files, including key/value stores,
databases, etc.
</p>
</div>
-<a name="N10023"></a><a name="Pigs+Live+Anywhere"></a>
+<a name="N10020"></a><a name="Pigs+Live+Anywhere"></a>
<h2 class="h3">Pigs Live Anywhere</h2>
<div class="section">
<p>
Pig is intended to be a language for parallel data processing. It is
not tied to one particular parallel framework. It has been implemented first
- on hadoop, but we do not intend that to be only on hadoop.
+ on Hadoop, but we do not intend that to be only on Hadoop.
</p>
</div>
-<a name="N1002D"></a><a name="Pigs+Are+Domestic+Animals"></a>
+<a name="N1002A"></a><a name="Pigs+Are+Domestic+Animals"></a>
<h2 class="h3">Pigs Are Domestic Animals</h2>
<div class="section">
<p>
@@ -225,17 +223,19 @@ document.write("Last Published: " + docu
</p>
<p>
Pig allows integration of user code where ever possible, so it
currently supports user defined field transformation functions, user defined
- aggregates, user defined grouping functions, and user defined
conditionals. In the future we want to support all the above in non-java
languages,
- as well as streaming, user defined types, and user defined splits.
+ aggregates, and user defined conditionals. These functions can be
written in Java or scripting languages that can compile down to Java (e.g.
Jython).
+ Pig supports user provided load and store functions. It supports
external executables via its stream command and Map Reduce jars via its
mapreduce
+ command. It allows users to provide a custom partitioner for their jobs
in some circumstances and to set the level of reduce parallelism for their jobs.
+ command. It allows users to set the level of reduce parallelism for
their jobs and in some circumstances to provide a custom partitioner.
</p>
<p>
- Currently pig has no optimizer, so it does not do any operation
rearranging. When we add that in the future, it will always be possible for
users to
- turn code rearranging off, so that pig does exactly what they say in
the order they say it.
+ Pig has an optimizer that rearranges some operations in Pig Latin
scripts to give better performance, combines Map Reduce jobs together, etc.
However, users
+ can easily turn this optimizer off to prevent it from making changes
that do not make sense in their situation.
</p>
</div>
-<a name="N1003D"></a><a name="Pigs+Fly"></a>
+<a name="N1003A"></a><a name="Pigs+Fly"></a>
<h2 class="h3">Pigs Fly</h2>
<div class="section">
<p>
Modified: hadoop/pig/site/publish/philosophy.pdf
URL:
http://svn.apache.org/viewvc/hadoop/pig/site/publish/philosophy.pdf?rev=988628&r1=988627&r2=988628&view=diff
==============================================================================
--- hadoop/pig/site/publish/philosophy.pdf (original)
+++ hadoop/pig/site/publish/philosophy.pdf Tue Aug 24 17:14:12 2010
@@ -80,10 +80,10 @@ endobj
>>
endobj
18 0 obj
-<< /Length 1636 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 1995 /Filter [ /ASCII85Decode /FlateDecode ]
>>
stream
-GatU49on$e&a...@sbkgfmh<@-qR4r'PF>GUI+71J>)p-;98`'bdaP$6E3oc6Vi4nOWD%Z7renku!'oA5t1hsE?e5%\+<qgl402h>%I68*l(d+\`n/<(l=N)f...@nu7ro:WH8]Kq351S.f<eY)\ZJt);Q;MX))0...@$f43o!.6*:N*pr]_uc<p\W3:#\OS1f?G:1'`6WAp#>4cmpQrXMKt!%+.IIa-A`P%3>4I^')HT+-aBkCFVn:1q3g=mp6q_OZu".0;YHjC_V37'cf...@4[lteozqi$hhlwfek>nZ^_iqaBB)Kc0-*pe:t...@yd-s]'X\V]KlWX!VES4>@r'L4t'RDo\)(P?ojs[Ho<Z"0B1_[mO>&(JMN.20H1X6=daYH'ia;cj!!IWrC3oE.jog'Ql4]TA8R3^Va;+)2F`8ii%,Y@"a(?U+b`QBp1a.cF`dZs%>Jf;sotk7+zkjiakx...@hr7amv"*&oSU_]c-/oHtrs*Ah(K3T(L)MZJD=15*$trbuXU!YiD,d-s?ErL4*X(shh9MEV:*0>YT9r%M5Zf`g&`R*UB=mQ!#f*iPL_+KU0afE^,%*gK0ME/hl0APWCT+1X92-_PXJ:Y2%;T5A!Q3'Jej+Zed0E1`J2LHdY"5Bs<WIm;I7)O'hdR-4=b(c=CT2!j#-#<rK!Q;D?jtGE[pU"(\-e...@u@2."5/>AVZXskg=ki.&0*i&=MkIir4'P=27@)5?P^_U,fq%l)OlA8HPIucYn`4!dINi[9oRmBdacu"E/"Lh1>ngOBK4%+sac>'bNi_]#`@"Lk9>IM-C\1W9+r$>ri5oyr0]n\2^tag\l...@9osn-yq-fk&+>;:hjO!0QA^V<plLL52d4g5"VUESE[&dQ(C0cZTr^HY=7jY*M$\o[[=t:^"d0pK]/sFged]-(bOKP5M97-$b"R%gGL#TtuQ=X,P5.jPpo%pZd<B,caPB$JNaKm")^Cn#XoOmGNg3.s[+
0>k34&+J;lRk:WX:8)\_.O'-$YG-?$q)5qN,`MlBD],V(&K>\4YUHjJHpRg[`?-O6VRdH:0dS_Wt1_(EGD[j3G1"l]u\!P5W48,a(2t#4%"%$9p-B.8'J,/n#n#b!jC*Ne(Z4<a6rf*l...@h5xn#cp"r5eAVa8I'#queg4i+`b5l...@fie7s]!4it:P4]#^_u0GO+K&ga[V+d%f-<T,Wr',+75.S`arSMTti[KF9]bdSUlDjf-t[5pA#-SK5qW+8_De(1...@ezrf_k:-p?DeY>g3u5C29'k1/lpdb$dDl%uaU5E6%0)E.\`Mk][R"V^B^h(hrW]Xmp!K-\I`Si"4U4TEhL0r](h5hn87...@p(WJOJ3?9M^l8Ko"/it1t\\\e8=gb...@slobnpl4]j>/P#<(rR`iIhP3JAI">FZ_'MB?dc'7M<@&.^%H.hloqMXf!8_T7KubUO7\MK`2qJ0?dgUm>;';.KOOAca1c1-\E,Zj;&BN3"ED<eT'p-?)qkNP&Q`DmSQDdr"/n,XET/_9Apphi]Nccu2A3GZdcT&!...@2igvx0i*>MsA6',qR#lZ/BGZg.Oa=aOBmkXLoTqaVSs!^Ss:kbK'N^(<Qp?SJjZd$aQl"l...@1%*\&%Er-"+...@l!lrs~>
+Gatm<?*g"Y&:O;Vi2t*p6Hke$T/oQ51sMgP\#j<>d.BU.!QkR6,[o='oj,A#?&Wgm9fncb+FJldc^m;3X#Kq+X*:i/L$m[JB</G(Y<6G,s#ar'l...@e4oz>lmq8FB:f"UH0IiTqe(BmbY5':E]pN-qPE2QSlS<UcNcWsZg"_,q=i4+^RY7Xe_C&f$^fd...@p/-OAPqB&eRChnFbDkU>j=<kkfoFIkm\k7=.\07;(Tj2$S#_9$,!U#]jX+PR9YSC4]0)`-0uWMmpV;(&(e0bB(i<:I2u*1cXF04i%d`pXP&U*C%k_I#dIkd*1[\q!6j?Z06Mij^UO>GflP>i?49,?O+#U>dTRqa)FC)O=k=("^)v...@j&hram7*IW"DA:.ZB'g-&qt+1koe...@t6^db2egby](rCm+``Q8'Zd,`+(b?7?P2$R=C!EQ!d(*6":,0/('D,1Pj_^Jj\0%b/20+MAo%Z9!k^d#!qjeq<sVHSp%BQV,A$(DEQpPGfbo+2(N#V]/\.kIBdU1>AL"f`3bE"/rj8ln!f?v1...@z?5tn%bi]j5(c;d#p:5_=a&@I\F/h...@bqqbn^guloolo>.'\r<OX!9-!S>&J&ZE64c^bg>W:%D\LKY&fai-)=nybok...@tu1qycpcq=);J$dro2+L!S=+AFqOs='^7/aM!=is"olf\...@6l09x1nav(J(-jU'NW_I?(OC`ju;Q9HgcC/,0k,&']IQ5C9Q+Eet`\$fn+<W",m]u;rT"p`EcB12tr>(p...@.l3m+_1k_2mo_,OK1L0!d9iK,8#`j<1;qM9(@tW\J,qlP7keJ5'D^5-LD.I16Uqb!?ONtBn6.!eIY?FW+^?uO7)_c8:C,:!1^h?Wn@&>V3fO-f1E=lmRLq"#,-N)P&Td&,A8RP62PIem#H*`aa9[J1BY2'&]%,/G\4[DA]"DAA!f/C'ZeWi+?OK\01O9<b+*TOX9tL0n>*)3?qAr7hW2h6
<"F[[X$8M\CUt_c/c...@3[65uq=-/3)PKmcZbuM0gSE=KOlFWgUn1>:85GXQGfNY=Rfdi]5L8]SD12JI__q;DAko9mS\"(Xk)%ddZPRBX">m...@leuo0sx6phn(^?Q8V/Z8i28:`_k6f/MaRSO];M&(+[nv1.tf...@qd^.dkn.bfcq#a>)grTQ>mON2DJC<TP:\lMF8bKIU#=<NR!!o0C'o...@gnsj6$keb)A3>R#8_b#`G!oZLG!893R.Y;t8/16?_(`Hh3j(Xu*V`ln!^Y"UK(^pjqlfD(R)KXHnQ&@D+_1.R9X,(%j[Ff4>/Hc;%d$OGY3&oQdqg^S2T]Kg3+HhNq$P/5R:e=l*_s.B&!!*qC>'!'V6q$A'74PgV!;KBKY!P#OHu)E.Xrd&-q`b;OS"(Ij7rK05hM.6:I?Tk\^cXWYfZA'-hkGl$RpaYu,$:xxu...@o\prji=k#n#ufiplb_mvli$eu$k*fsj<ORKL9%K\(2._DW>mm^R`1T(qh(r>JMDQf#/jC3"AU#27\YOL/ls=N7csYoIP[%EX7u27FXXR'D_%LP`XN,Z>e>mT+kmN(j0]FUR:S:.JSd8^g<ju=0Y])\:Ps0jbQ;]p=*rCP)%...@ageidu^@W#U=d!8uR'pYUK<;l&hrjoo4...@mlh[b?+^cl(^*XJ/bD:pG2n6q2nL7"VFh74*BV$C,.TAQpmr0Y;QR;8:=.0uU!9ZqKinq.",ljsm>4O0+hAtc(OL$^qK/8O&Oe\FY%B6,I:?h>5>GJE"Yu3FCQaFsg:joi4)@NMhh;r^"7m4e:^FlD_jqHfJa<+n&gWiJG4EQWL(pAH=uHPX7X)C+J?b-j34dIGmJ9]#MJQgW2a9'LUOm@@m%CZ&j,','M<g9f1>,kBXFboUO2D7ZWI*Hti[/]]$.&@#:]2E*ZN:P^#i]\+_'_J<VNno&a4'<oIm#>h2_;f2hdpCp9VFE6\r5i.h6,MTEa&*JCUAil5oo
i,\e2Q(!K[61*W~>
endstream
endobj
19 0 obj
@@ -199,19 +199,19 @@ endobj
13 0 obj
<<
/S /GoTo
-/D [19 0 R /XYZ 85.0 519.932 null]
+/D [19 0 R /XYZ 85.0 514.732 null]
>>
endobj
15 0 obj
<<
/S /GoTo
-/D [19 0 R /XYZ 85.0 441.198 null]
+/D [19 0 R /XYZ 85.0 435.998 null]
>>
endobj
17 0 obj
<<
/S /GoTo
-/D [19 0 R /XYZ 85.0 280.464 null]
+/D [19 0 R /XYZ 85.0 222.464 null]
>>
endobj
20 0 obj
@@ -222,36 +222,36 @@ endobj
xref
0 31
0000000000 65535 f
-0000005050 00000 n
-0000005115 00000 n
-0000005207 00000 n
+0000005409 00000 n
+0000005474 00000 n
+0000005566 00000 n
0000000015 00000 n
0000000071 00000 n
0000000728 00000 n
0000000848 00000 n
0000000901 00000 n
-0000005330 00000 n
+0000005689 00000 n
0000001036 00000 n
-0000005393 00000 n
+0000005752 00000 n
0000001173 00000 n
-0000005459 00000 n
+0000005818 00000 n
0000001310 00000 n
-0000005525 00000 n
+0000005884 00000 n
0000001447 00000 n
-0000005591 00000 n
+0000005950 00000 n
0000001584 00000 n
-0000003313 00000 n
-0000005657 00000 n
-0000003421 00000 n
-0000003684 00000 n
-0000003890 00000 n
-0000004102 00000 n
-0000004355 00000 n
-0000004494 00000 n
-0000004602 00000 n
-0000004712 00000 n
-0000004825 00000 n
-0000004941 00000 n
+0000003672 00000 n
+0000006016 00000 n
+0000003780 00000 n
+0000004043 00000 n
+0000004249 00000 n
+0000004461 00000 n
+0000004714 00000 n
+0000004853 00000 n
+0000004961 00000 n
+0000005071 00000 n
+0000005184 00000 n
+0000005300 00000 n
trailer
<<
/Size 31
@@ -259,5 +259,5 @@ trailer
/Info 4 0 R
>>
startxref
-5708
+6067
%%EOF
