svn commit: r711158 - in /hadoop/pig/branches/types/src/org/apache/pig: backend/hadoop/executionengine/mapReduceLayer/ backend/hadoop/executionengine/physicalLayer/ backend/hadoop/executionengine/phys

2008-11-03 Thread gates
Author: gates Date: Mon Nov 3 17:45:47 2008 New Revision: 711158 URL: http://svn.apache.org/viewvc?rev=711158view=rev Log: PIG-350 Patch that redoes join to stream one side of the join instead of materialize keys from all inputs into memory. Patch by Pradeep that built on earlier work done

svn commit: r709206 - in /hadoop/pig/trunk: ./ contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/apachelog/ contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/stora

2008-10-30 Thread gates
Author: gates Date: Thu Oct 30 09:56:48 2008 New Revision: 709206 URL: http://svn.apache.org/viewvc?rev=709206view=rev Log: PIG-509: Added CombinedLogLoader, loads logs that were created using Apache's combined log format. Added: hadoop/pig/trunk/contrib/piggybank/java/src/main/java/org

svn commit: r709222 - in /hadoop/pig/branches/types: src/org/apache/pig/builtin/DIFF.java test/org/apache/pig/test/TestBuiltin.java

2008-10-30 Thread gates
Author: gates Date: Thu Oct 30 11:33:43 2008 New Revision: 709222 URL: http://svn.apache.org/viewvc?rev=709222view=rev Log: PIG-511 Fixed flaws in builtin UDF diff pointed out by Crisitan Ivascu. Modified: hadoop/pig/branches/types/src/org/apache/pig/builtin/DIFF.java hadoop/pig

svn commit: r707420 - in /incubator/pig/trunk: ./ contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/util/apachelogparser/ contrib/piggybank/java/src/test/java/org/apache/pig/pig

2008-10-23 Thread gates
Author: gates Date: Thu Oct 23 10:31:19 2008 New Revision: 707420 URL: http://svn.apache.org/viewvc?rev=707420view=rev Log: PIG-503: Changed default date format for DateExtractor. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/contrib/piggybank/java/src/main/java/org

svn commit: r705004 - /incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/LOLoad.java

2008-10-15 Thread gates
Author: gates Date: Wed Oct 15 12:36:58 2008 New Revision: 705004 URL: http://svn.apache.org/viewvc?rev=705004view=rev Log: Removed bogus import from LOLoad. Modified: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/LOLoad.java Modified: incubator/pig/branches/types/src

svn commit: r704589 - in /incubator/pig/trunk: ./ contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/util/apachelogparser/ contrib/piggybank/java/src/test/java/org/apache/pig/pig

2008-10-14 Thread gates
Author: gates Date: Tue Oct 14 10:28:16 2008 New Revision: 704589 URL: http://svn.apache.org/viewvc?rev=704589view=rev Log: PIG-476: Added DateExtractor, a piggybank eval func that extracts a date from a string. Added: incubator/pig/trunk/contrib/piggybank/java/src/main/java/org/apache

svn commit: r704149 - in /incubator/pig/trunk: ./ contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/util/apachelogparser/ contrib/piggybank/java/src/test/java/org/apache/pig/pig

2008-10-13 Thread gates
Author: gates Date: Mon Oct 13 09:18:01 2008 New Revision: 704149 URL: http://svn.apache.org/viewvc?rev=704149view=rev Log: PIG-486: Added SearchEngineExtractor, a piggybank eval func that recognizes a set of the most common search engines in a URL and extracts the name of the search engine

svn commit: r704151 - in /incubator/pig/trunk: ./ contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/util/apachelogparser/ contrib/piggybank/java/src/test/java/org/apache/pig/pig

2008-10-13 Thread gates
Author: gates Date: Mon Oct 13 09:20:49 2008 New Revision: 704151 URL: http://svn.apache.org/viewvc?rev=704151view=rev Log: PIG-487: Added HostExtractor, a piggybank eval func that, given a URL, determines the host. Added: incubator/pig/trunk/contrib/piggybank/java/src/main/java/org

svn commit: r699534 - in /incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer: Launcher.java LocalLauncher.java MapReduceLauncher.java

2008-09-26 Thread gates
Author: gates Date: Fri Sep 26 17:00:03 2008 New Revision: 699534 URL: http://svn.apache.org/viewvc?rev=699534view=rev Log: PIG-457 This just addresses the issue of getting false error messages on success. Launchers changed to send any error messages from hadoop to debug if the job succeeded

svn commit: r699170 - in /incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer: JobControlCompiler.java MRCompiler.java MapReduceOper.java

2008-09-25 Thread gates
Author: gates Date: Thu Sep 25 21:17:28 2008 New Revision: 699170 URL: http://svn.apache.org/viewvc?rev=699170view=rev Log: PIG-461 Changed additional last limit to use sort comparator when the limit follows an order by. Modified: incubator/pig/branches/types/src/org/apache/pig/backend

svn commit: r697229 [2/2] - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/backend/hadoop/ src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/ src/org/apache/pi

2008-09-19 Thread gates
Modified: incubator/pig/branches/types/src/org/apache/pig/impl/io/NullableBooleanWritable.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/src/org/apache/pig/impl/io/NullableBooleanWritable.java?rev=697229r1=697228r2=697229view=diff

svn commit: r695659 - in /incubator/pig/branches/types/src/org/apache/pig: backend/hadoop/executionengine/mapReduceLayer/ backend/hadoop/executionengine/physicalLayer/ backend/hadoop/executionengine/p

2008-09-15 Thread gates
Author: gates Date: Mon Sep 15 15:10:46 2008 New Revision: 695659 URL: http://svn.apache.org/viewvc?rev=695659view=rev Log: Fixing documentation issues. Modified: incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigOutputFormat.java

svn commit: r694112 - /incubator/pig/branches/types/test/org/apache/pig/test/data/GoldenFiles/MRC17.gld

2008-09-10 Thread gates
Author: gates Date: Wed Sep 10 21:43:39 2008 New Revision: 694112 URL: http://svn.apache.org/viewvc?rev=694112view=rev Log: Fixed expected results for MRCompiler testLimit. Modified: incubator/pig/branches/types/test/org/apache/pig/test/data/GoldenFiles/MRC17.gld Modified: incubator/pig

svn commit: r693303 - in /incubator/pig/branches/types: src/org/apache/pig/impl/builtin/ShellBagEvalFunc.java test/org/apache/pig/test/TestBuiltin.java

2008-09-08 Thread gates
Author: gates Date: Mon Sep 8 15:30:40 2008 New Revision: 693303 URL: http://svn.apache.org/viewvc?rev=693303view=rev Log: PIG-327 Removed ShellBagEvalFunc, as it's functionality is replaced by streaming. Removed: incubator/pig/branches/types/src/org/apache/pig/impl/builtin

svn commit: r692581 - /incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/optimizer/ImplicitSplitInserter.java

2008-09-05 Thread gates
Author: gates Date: Fri Sep 5 16:36:33 2008 New Revision: 692581 URL: http://svn.apache.org/viewvc?rev=692581view=rev Log: PIG-401 ImplicitSplitInserter class that should have been in the last checkin. Added: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/optimizer

svn commit: r689983 - /incubator/pig/branches/types/src/org/apache/pig/backend/executionengine/PigSlicer.java

2008-08-28 Thread gates
Author: gates Date: Thu Aug 28 14:07:09 2008 New Revision: 689983 URL: http://svn.apache.org/viewvc?rev=689983view=rev Log: PIG-403 Fix wrong default in PigSlicer that caused all pig jobs to use a single map. Modified: incubator/pig/branches/types/src/org/apache/pig/backend

svn commit: r690039 - in /incubator/pig/branches/types: src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/ src/org/apache/pig/backend/hadoop/executionengine/physicalL

2008-08-28 Thread gates
Author: gates Date: Thu Aug 28 15:59:06 2008 New Revision: 690039 URL: http://svn.apache.org/viewvc?rev=690039view=rev Log: PIG-359 (yet again). Address an issue Santhosh identified with the earlier patch where POUserFunc and POForEach were assuming that project star was a tuple, when in some

svn commit: r689631 - in /incubator/pig/branches/types: src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/ src/org/apache/pig/backend/hadoop/executionengine/physicalL

2008-08-27 Thread gates
Author: gates Date: Wed Aug 27 14:42:11 2008 New Revision: 689631 URL: http://svn.apache.org/viewvc?rev=689631view=rev Log: PIG-359. Shravan's patch to fix * semantics in generate and when passed as an argument to a UDF. Modified: incubator/pig/branches/types/src/org/apache/pig/backend

svn commit: r689673 - in /incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer: LOLimit.java PlanSetter.java optimizer/SchemaCalculator.java optimizer/SchemaRemover.java package.html pars

2008-08-27 Thread gates
Author: gates Date: Wed Aug 27 17:09:04 2008 New Revision: 689673 URL: http://svn.apache.org/viewvc?rev=689673view=rev Log: PIG-399 Added LOLimit to visitors in the optimizer so that when the plan is rearranged limit is handled correctly. Modified: incubator/pig/branches/types/src/org

svn commit: r689289 - /incubator/pig/branches/types/src/org/apache/pig/PigServer.java

2008-08-26 Thread gates
Author: gates Date: Tue Aug 26 17:00:29 2008 New Revision: 689289 URL: http://svn.apache.org/viewvc?rev=689289view=rev Log: PIG-379 Changed describe to not call the optimizer. Modified: incubator/pig/branches/types/src/org/apache/pig/PigServer.java Modified: incubator/pig/branches/types

svn commit: r689292 - /incubator/pig/branches/types/src/org/apache/pig/PigServer.java

2008-08-26 Thread gates
Author: gates Date: Tue Aug 26 17:10:08 2008 New Revision: 689292 URL: http://svn.apache.org/viewvc?rev=689292view=rev Log: PIG-372 Fix NPE in dump schema when there is no known schema. Modified: incubator/pig/branches/types/src/org/apache/pig/PigServer.java Modified: incubator/pig

svn commit: r689293 - /incubator/pig/branches/types/src/org/apache/pig/PigServer.java

2008-08-26 Thread gates
Author: gates Date: Tue Aug 26 17:15:45 2008 New Revision: 689293 URL: http://svn.apache.org/viewvc?rev=689293view=rev Log: PIG-383 Added relation name to describe output. Modified: incubator/pig/branches/types/src/org/apache/pig/PigServer.java Modified: incubator/pig/branches/types/src

svn commit: r684940 - /incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java

2008-08-11 Thread gates
Author: gates Date: Mon Aug 11 14:25:37 2008 New Revision: 684940 URL: http://svn.apache.org/viewvc?rev=684940view=rev Log: PIG-337 Daniel's fix for limit k where k total lines returning too few results. Modified: incubator/pig/branches/types/src/org/apache/pig/backend/hadoop

svn commit: r685032 - in /incubator/pig/branches/types: src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/ test/org/apache/pig/test/ test/org/apache/pig/test/data/GoldenFiles/

2008-08-11 Thread gates
Author: gates Date: Mon Aug 11 21:35:02 2008 New Revision: 685032 URL: http://svn.apache.org/viewvc?rev=685032view=rev Log: PIG-343 Shravan's patch to fix split. Modified: incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer

svn commit: r680890 - /incubator/pig/branches/types/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java

2008-07-29 Thread gates
Author: gates Date: Tue Jul 29 17:43:52 2008 New Revision: 680890 URL: http://svn.apache.org/viewvc?rev=680890view=rev Log: PIG-285 This patch makes it so the the script doesn't fail with error messages about not being able to instantiate the user provided comparator. Modified: incubator

svn commit: r677947 - in /incubator/pig/branches/types: src/org/apache/pig/impl/logicalLayer/parser/QueryParser.jjt test/org/apache/pig/test/TestBuiltin.java

2008-07-18 Thread gates
Author: gates Date: Fri Jul 18 08:54:17 2008 New Revision: 677947 URL: http://svn.apache.org/viewvc?rev=677947view=rev Log: PIG-308 Fixed join rewrites in the parser, with Santhosh's help. Also uncommented two tests in TestBuiltin that should have been uncommented a while ago. Modified

svn commit: r678038 - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/parser/ src/org/apache/pig/tools/pigscript/parser

2008-07-18 Thread gates
Author: gates Date: Fri Jul 18 14:07:09 2008 New Revision: 678038 URL: http://svn.apache.org/viewvc?rev=678038view=rev Log: PIG-323 Checking in Santhosh's patch to back out define changes. Modified: incubator/pig/branches/types/src/org/apache/pig/PigServer.java incubator/pig/branches

svn commit: r677638 - in /incubator/pig/branches/types: src/org/apache/pig/builtin/Utf8StorageConverter.java src/org/apache/pig/impl/mapReduceLayer/PigMapReduce.java test/org/apache/pig/test/TestConve

2008-07-17 Thread gates
Author: gates Date: Thu Jul 17 10:05:06 2008 New Revision: 677638 URL: http://svn.apache.org/viewvc?rev=677638view=rev Log: PIG-312 When casting to an int or long, if the cast fails, try to cast to a double and then to int or long. Modified: incubator/pig/branches/types/src/org/apache/pig

svn commit: r675663 - in /incubator/pig/branches/types: src/org/apache/pig/impl/mapReduceLayer/ src/org/apache/pig/impl/physicalLayer/expressionOperators/ test/org/apache/pig/test/data/GoldenFiles/ te

2008-07-10 Thread gates
Author: gates Date: Thu Jul 10 11:09:36 2008 New Revision: 675663 URL: http://svn.apache.org/viewvc?rev=675663view=rev Log: PIG-293 Previous patches solved infinite loop. This patch makes order by in map reduce mode actually work. Modified: incubator/pig/branches/types/src/org/apache

svn commit: r675273 - in /incubator/pig/branches/types: src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/mapReduceLayer/ src/org/apache/pig/impl/mapReduceLayer/plans/ src/org/apache/pig/i

2008-07-09 Thread gates
Author: gates Date: Wed Jul 9 10:58:42 2008 New Revision: 675273 URL: http://svn.apache.org/viewvc?rev=675273view=rev Log: PIG-285 Shravan's sortfin patch. This addresses some, though not all, order by issues. Modified: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer

svn commit: r674908 - /incubator/pig/branches/types/test/org/apache/pig/test/TestEvalPipeline.java

2008-07-08 Thread gates
Author: gates Date: Tue Jul 8 11:26:18 2008 New Revision: 674908 URL: http://svn.apache.org/viewvc?rev=674908view=rev Log: PIG-287 Modified test to confirm bug is fixed. Thanks Santhosh. Modified: incubator/pig/branches/types/test/org/apache/pig/test/TestEvalPipeline.java Modified

svn commit: r673198 - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/parser/ src/org/apache/pig/impl/logicalLayer/vali

2008-07-01 Thread gates
Author: gates Date: Tue Jul 1 13:33:41 2008 New Revision: 673198 URL: http://svn.apache.org/viewvc?rev=673198view=rev Log: Fixed bugs in equal and notequal physical operators that were returning wrong results for numeric types. Added PlanSetter visitor that walks logical plan and sets all

svn commit: r672801 [2/2] - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logic

2008-06-30 Thread gates
Modified: incubator/pig/branches/types/test/org/apache/pig/test/utils/LogicalPlanTester.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/utils/LogicalPlanTester.java?rev=672801r1=672800r2=672801view=diff

svn commit: r659695 - in /incubator/pig/branches/types: src/org/apache/pig/FilterFunc.java src/org/apache/pig/builtin/IsEmpty.java src/org/apache/pig/impl/logicalLayer/parser/QueryParser.jjt test/org/

2008-05-23 Thread gates
Author: gates Date: Fri May 23 15:31:28 2008 New Revision: 659695 URL: http://svn.apache.org/viewvc?rev=659695view=rev Log: PIG-159 Santhosh's fix to bug that prevented instantiation of UDFs. Modified: incubator/pig/branches/types/src/org/apache/pig/FilterFunc.java incubator/pig

svn commit: r659721 - in /incubator/pig/branches/types: build.xml src/org/apache/pig/PigServer.java

2008-05-23 Thread gates
Author: gates Date: Fri May 23 16:58:27 2008 New Revision: 659721 URL: http://svn.apache.org/viewvc?rev=659721view=rev Log: Fixes in PigServer to deal with change of alias from map string-logicalplan to logicaloperator-logicalplan. Added TestMapReduce to build.xml to begin end to end testing

svn commit: r659161 [4/4] - in /incubator/pig/branches/types: ./ src/org/apache/pig/data/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/schema/ src/org/apache/pig/impl/log

2008-05-22 Thread gates
Added: incubator/pig/branches/types/test/org/apache/pig/test/TestTypeCheckingValidatorNoSchema.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/TestTypeCheckingValidatorNoSchema.java?rev=659161view=auto

svn commit: r659161 [1/4] - in /incubator/pig/branches/types: ./ src/org/apache/pig/data/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/schema/ src/org/apache/pig/impl/log

2008-05-22 Thread gates
Author: gates Date: Thu May 22 10:12:56 2008 New Revision: 659161 URL: http://svn.apache.org/viewvc?rev=659161view=rev Log: PIG-143 Pi's latest type checker checkin. Added: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/schema/SchemaMergeException.java incubator/pig

svn commit: r659206 - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/parser/ test/org/apache/pig/test/

2008-05-22 Thread gates
Author: gates Date: Thu May 22 12:04:25 2008 New Revision: 659206 URL: http://svn.apache.org/viewvc?rev=659206view=rev Log: PIG-159 Santhosh's work to chain together logical plans. Modified: incubator/pig/branches/types/src/org/apache/pig/PigServer.java incubator/pig/branches/types/src

svn commit: r659250 - in /incubator/pig/branches/types/src/org/apache/pig: PigServer.java backend/executionengine/ExecutionEngine.java backend/hadoop/executionengine/HExecutionEngine.java backend/loca

2008-05-22 Thread gates
Author: gates Date: Thu May 22 13:53:05 2008 New Revision: 659250 URL: http://svn.apache.org/viewvc?rev=659250view=rev Log: Changed ExecutionEngine.compile implementations and PigServer to work with the new way logical plans are stitched together. Modified: incubator/pig/branches/types/src

svn commit: r658728 [2/2] - in /incubator/pig/branches/types: src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/validators/ src/org/apache/pig/impl/physicalLayer/plans/ src/or

2008-05-21 Thread gates
Added: incubator/pig/branches/types/test/org/apache/pig/test/TestLogToPhyCompiler.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/TestLogToPhyCompiler.java?rev=658728view=auto

svn commit: r657836 - in /incubator/pig/branches/types: src/org/apache/pig/impl/logicalLayer/parser/QueryParser.jjt test/org/apache/pig/test/TestLogicalPlanBuilder.java

2008-05-19 Thread gates
Author: gates Date: Mon May 19 07:30:30 2008 New Revision: 657836 URL: http://svn.apache.org/viewvc?rev=657836view=rev Log: PIG-159 Santhosh's addition of explicit casts to the grammar. Modified: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/parser/QueryParser.jjt

svn commit: r657848 - in /incubator/pig/branches/types: ./ src/org/apache/pig/ src/org/apache/pig/builtin/ src/org/apache/pig/data/ src/org/apache/pig/impl/logicalLayer/

2008-05-19 Thread gates
Author: gates Date: Mon May 19 08:15:19 2008 New Revision: 657848 URL: http://svn.apache.org/viewvc?rev=657848view=rev Log: Fixed a few TODO FIXs, added DOUBLE to isAtomic in DataType. Modified: incubator/pig/branches/types/build.xml incubator/pig/branches/types/src/org/apache/pig

svn commit: r657939 - in /incubator/pig/branches/types: ./ src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/ src/o

2008-05-19 Thread gates
Author: gates Date: Mon May 19 12:45:33 2008 New Revision: 657939 URL: http://svn.apache.org/viewvc?rev=657939view=rev Log: Make it all compile. With this patch we can now return the build.xml compile-sources rule to its previous state of building all java files below src. There are a number

svn commit: r657224 - in /incubator/pig/branches/types: src/org/apache/pig/ src/org/apache/pig/impl/mapReduceLayer/ src/org/apache/pig/impl/physicalLayer/ src/org/apache/pig/impl/physicalLayer/topLeve

2008-05-16 Thread gates
Author: gates Date: Fri May 16 14:38:40 2008 New Revision: 657224 URL: http://svn.apache.org/viewvc?rev=657224view=rev Log: PIG-162 Shravan's addition of PigProgressable and distinct for MR. Added: incubator/pig/branches/types/src/org/apache/pig/impl/mapReduceLayer

svn commit: r656811 - in /incubator/pig/branches/types: ./ src/org/apache/pig/impl/physicalLayer/plans/ src/org/apache/pig/impl/physicalLayer/topLevelOperators/expressionOperators/ test/org/apache/pig

2008-05-15 Thread gates
Author: gates Date: Thu May 15 12:49:36 2008 New Revision: 656811 URL: http://svn.apache.org/viewvc?rev=656811view=rev Log: PIG-161 Checking in POMap parts of Shubham's CastAndMapLookupPatch. See PIG-161 for comments on cast part. Added: incubator/pig/branches/types/src/org/apache/pig

svn commit: r656913 [3/3] - in /incubator/pig/branches/types: ./ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/ src/org/apache/pig/

2008-05-15 Thread gates
Modified: incubator/pig/branches/types/test/org/apache/pig/test/TestPOUserFunc.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/TestPOUserFunc.java?rev=656913r1=656912r2=656913view=diff

svn commit: r655940 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/Main.java src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java src/org/apache/pig/impl/util/PropertiesU

2008-05-13 Thread gates
Author: gates Date: Tue May 13 09:35:44 2008 New Revision: 655940 URL: http://svn.apache.org/viewvc?rev=655940view=rev Log: PIG-236: Fix properties so that values specified via the command line (-D) are not ignored. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org

svn commit: r655950 - in /incubator/pig/branches/types: src/org/apache/pig/impl/physicalLayer/plans/ src/org/apache/pig/impl/physicalLayer/topLevelOperators/expressionOperators/ src/org/apache/pig/imp

2008-05-13 Thread gates
Author: gates Date: Tue May 13 10:11:43 2008 New Revision: 655950 URL: http://svn.apache.org/viewvc?rev=655950view=rev Log: Shubham's bincond and unary operators patch. Added: incubator/pig/branches/types/src/org/apache/pig/impl/physicalLayer/topLevelOperators/expressionOperators

svn commit: r654629 [2/4] - in /incubator/pig/branches/types: ./ src/org/apache/pig/impl/eval/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/parser/ src/org/apache/pig/imp

2008-05-08 Thread gates
Added: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/LORegexp.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/LORegexp.java?rev=654629view=auto

svn commit: r654629 [4/4] - in /incubator/pig/branches/types: ./ src/org/apache/pig/impl/eval/ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/logicalLayer/parser/ src/org/apache/pig/imp

2008-05-08 Thread gates
Modified: incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/validators/TypeCheckingVisitor.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/src/org/apache/pig/impl/logicalLayer/validators/TypeCheckingVisitor.java?rev=654629r1=654628r2=654629view=diff

svn commit: r654172 - in /incubator/pig/trunk: ./ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/local/executionengine/ src/org/apache/pig/impl/builtin/ src/org/apache/p

2008-05-07 Thread gates
Author: gates Date: Wed May 7 10:12:11 2008 New Revision: 654172 URL: http://svn.apache.org/viewvc?rev=654172view=rev Log: PIG-202: Fix Order by so that user provided comparator func is used for quantile determination. Added: incubator/pig/trunk/src/org/apache/pig/backend/local

svn commit: r654319 - /incubator/pig/trunk/test/org/apache/pig/test/TestBinaryStorage.java

2008-05-07 Thread gates
Author: gates Date: Wed May 7 16:18:06 2008 New Revision: 654319 URL: http://svn.apache.org/viewvc?rev=654319view=rev Log: Changed TestBinaryStorage to create it's test file in the local directory instead of /pig. This fixes the test in local mode on linux machines. Modified: incubator

svn commit: r654320 - in /incubator/pig/branches/types: build.xml src/org/apache/pig/data/DataType.java src/org/apache/pig/impl/logicalLayer/schema/Schema.java test/org/apache/pig/test/TestSchema.java

2008-05-07 Thread gates
Author: gates Date: Wed May 7 16:18:45 2008 New Revision: 654320 URL: http://svn.apache.org/viewvc?rev=654320view=rev Log: Added equals and merge to schema class. Work done by Pi. Added: incubator/pig/branches/types/test/org/apache/pig/test/TestSchema.java Modified: incubator/pig

svn commit: r652896 [2/2] - in /incubator/pig/branches/types: ./ src/org/apache/pig/ src/org/apache/pig/data/ src/org/apache/pig/impl/ src/org/apache/pig/impl/io/ src/org/apache/pig/impl/logicalLayer/

2008-05-02 Thread gates
Added: incubator/pig/branches/types/test/org/apache/pig/test/TestTypeCheckingValidator.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/TestTypeCheckingValidator.java?rev=652896view=auto

svn commit: r652906 - in /incubator/pig/trunk: ./ conf/ src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/ src/org/apache/pig/impl/util/

2008-05-02 Thread gates
Author: gates Date: Fri May 2 13:57:56 2008 New Revision: 652906 URL: http://svn.apache.org/viewvc?rev=652906view=rev Log: PIG-176: Change bag spilling so that bags below a certain threshold are not spilled, thus avoiding proliferation of small files. Added: incubator/pig/trunk/src/org

svn commit: r652935 - in /incubator/pig/trunk: ./ conf/ src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/impl/ src/org/apache/pig/impl/util/

2008-05-02 Thread gates
Author: gates Date: Fri May 2 15:41:58 2008 New Revision: 652935 URL: http://svn.apache.org/viewvc?rev=652935view=rev Log: PIG-215: Cleanup a few dangling ends left by PIG-111. Added: incubator/pig/trunk/conf/log4j.properties Modified: incubator/pig/trunk/CHANGES.txt incubator/pig

svn commit: r650606 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/impl/streaming/ExecutableManager.java test/org/apache/pig/test/TestStreaming.java

2008-04-22 Thread gates
Author: gates Date: Tue Apr 22 11:41:49 2008 New Revision: 650606 URL: http://svn.apache.org/viewvc?rev=650606view=rev Log: PIG-216 Fix streaming to work with commands that use unix pipes. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache/pig/impl/streaming

svn commit: r650688 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/pen/DisplayExamples.java src/org/apache/pig/pen/ExGen.java test/org/apache/pig/test/TestExGenCogroup.java test/org/apache/

2008-04-22 Thread gates
Author: gates Date: Tue Apr 22 15:58:28 2008 New Revision: 650688 URL: http://svn.apache.org/viewvc?rev=650688view=rev Log: PIG-207 Fix illustrate command to work in mapreduce mode. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache/pig/pen/DisplayExamples.java

svn commit: r649710 - in /incubator/pig/trunk: ./ src/org/apache/pig/data/

2008-04-18 Thread gates
Author: gates Date: Fri Apr 18 15:11:59 2008 New Revision: 649710 URL: http://svn.apache.org/viewvc?rev=649710view=rev Log: PIG-213: Remove non-static references to logger from data bags and tuples, as it causes significant overhead. Modified: incubator/pig/trunk/CHANGES.txt incubator

svn commit: r649715 [2/2] - in /incubator/pig/branches/types: ./ src/org/apache/pig/impl/logicalLayer/ src/org/apache/pig/impl/physicalLayer/plans/ src/org/apache/pig/impl/physicalLayer/topLevelOperat

2008-04-18 Thread gates
Modified: incubator/pig/branches/types/src/org/apache/pig/impl/physicalLayer/topLevelOperators/expressionOperators/binaryExprOps/comparators/NotEqualToExpr.java URL:

svn commit: r649154 - in /incubator/pig/trunk: ./ src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/local/executionengine/ src/org/apache/pig/builtin/ t

2008-04-17 Thread gates
Author: gates Date: Thu Apr 17 08:47:08 2008 New Revision: 649154 URL: http://svn.apache.org/viewvc?rev=649154view=rev Log: PIG-114: store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc. Added: incubator/pig/trunk/src/org/apache/pig/ReversibleLoadStoreFunc.java

svn commit: r648768 - in /incubator/pig/trunk: CHANGES.txt build.xml docs/ docs/overview.html

2008-04-16 Thread gates
Author: gates Date: Wed Apr 16 10:46:15 2008 New Revision: 648768 URL: http://svn.apache.org/viewvc?rev=648768view=rev Log: PIG-149, PIG-150: Fix doc target so that ant can generate docs. Added: incubator/pig/trunk/docs/ incubator/pig/trunk/docs/overview.html Modified: incubator

svn commit: r648789 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/impl/PigContext.java

2008-04-16 Thread gates
Author: gates Date: Wed Apr 16 11:32:30 2008 New Revision: 648789 URL: http://svn.apache.org/viewvc?rev=648789view=rev Log: PIG-183: Catch when a UDF has been compiled with the wrong version of java and give a RuntimeException. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig

svn commit: r647997 - in /incubator/pig/trunk: ./ src/org/apache/pig/ src/org/apache/pig/backend/executionengine/ src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/ src/org/apache/pig/ba

2008-04-14 Thread gates
Author: gates Date: Mon Apr 14 14:04:05 2008 New Revision: 647997 URL: http://svn.apache.org/viewvc?rev=647997view=rev Log: PIG-188: Fix mismatches between pig slicer changes and new streaming feature. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache/pig

svn commit: r647208 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/impl/logicalLayer/LOSplitOutput.java test/org/apache/pig/test/TestPigSplit.java

2008-04-11 Thread gates
Author: gates Date: Fri Apr 11 09:14:15 2008 New Revision: 647208 URL: http://svn.apache.org/viewvc?rev=647208view=rev Log: PIG-178: Use of schema on secondary output of SPLIT throws IndexOutOfBoundsException. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache

svn commit: r647253 [2/2] - in /incubator/pig/trunk: ./ src/org/apache/pig/ src/org/apache/pig/backend/local/executionengine/ src/org/apache/pig/data/ src/org/apache/pig/impl/eval/ src/org/apache/pig/

2008-04-11 Thread gates
Added: incubator/pig/trunk/src/org/apache/pig/pen/ExGen.java URL: http://svn.apache.org/viewvc/incubator/pig/trunk/src/org/apache/pig/pen/ExGen.java?rev=647253view=auto == ---

svn commit: r647335 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/Main.java

2008-04-11 Thread gates
Author: gates Date: Fri Apr 11 15:22:09 2008 New Revision: 647335 URL: http://svn.apache.org/viewvc?rev=647335view=rev Log: PIG-203: Fix bug in parameter substitution code where any pig script over 1k caused pig to freeze. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk

svn commit: r647356 - in /incubator/pig/trunk: ./ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/ src/org/apache/pig/impl/eval/

2008-04-11 Thread gates
Author: gates Date: Fri Apr 11 17:33:09 2008 New Revision: 647356 URL: http://svn.apache.org/viewvc?rev=647356view=rev Log: PIG-204: Repair broken input splits. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine

svn commit: r646993 - in /incubator/pig/trunk/conf: ./ pig.properties

2008-04-10 Thread gates
Author: gates Date: Thu Apr 10 15:26:24 2008 New Revision: 646993 URL: http://svn.apache.org/viewvc?rev=646993view=rev Log: PIG-111 Template for properties file that should have been checked in with revision 646988. Added: incubator/pig/trunk/conf/ incubator/pig/trunk/conf

svn commit: r646546 - /incubator/pig/trunk/src/org/apache/pig/tools/parameters/ParameterSubstitutionPreprocessor.java

2008-04-09 Thread gates
Author: gates Date: Wed Apr 9 14:23:38 2008 New Revision: 646546 URL: http://svn.apache.org/viewvc?rev=646546view=rev Log: Removed unnecessary import statement that caused builds to not work on 1.5 JVM. Modified: incubator/pig/trunk/src/org/apache/pig/tools/parameters

svn commit: r646189 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java test/org/apache/pig/test/TestMapReduceResultRecycling.java

2008-04-08 Thread gates
Author: gates Date: Tue Apr 8 21:38:56 2008 New Revision: 646189 URL: http://svn.apache.org/viewvc?rev=646189view=rev Log: PIG-153: Incorrect result caused by dump in between statements. Added: incubator/pig/trunk/test/org/apache/pig/test/TestMapReduceResultRecycling.java Modified

svn commit: r642504 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/PigCombine.java

2008-03-28 Thread gates
Author: gates Date: Fri Mar 28 21:37:30 2008 New Revision: 642504 URL: http://svn.apache.org/viewvc?rev=642504view=rev Log: Fixed PigCombine to not do initialization on every call to reduce, but instead only do it once in the call to configure. Modified: incubator/pig/trunk/CHANGES.txt

svn commit: r642060 - in /incubator/pig/branches/types: ./ src/org/apache/pig/ src/org/apache/pig/builtin/ test/org/apache/pig/test/

2008-03-27 Thread gates
Author: gates Date: Thu Mar 27 19:02:43 2008 New Revision: 642060 URL: http://svn.apache.org/viewvc?rev=642060view=rev Log: Took Shravan's changes to PigStorage and BinStorage so that they'd compile. Added changes to other builtin functions so whole directory now compiles. Changed ant test

svn commit: r639409 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/impl/util/SpillableMemoryManager.java

2008-03-20 Thread gates
Author: gates Date: Thu Mar 20 12:02:00 2008 New Revision: 639409 URL: http://svn.apache.org/viewvc?rev=639409view=rev Log: PIG-164: Fix memory issue in SpillableMemoryManager to partially clean the list of bags each time a new bag is added rather than waiting until the garbage collector

svn commit: r638706 - in /incubator/pig/branches/types: ./ lib/ src/org/apache/pig/impl/logicalLayer/schema/ src/org/apache/pig/impl/plan/ test/org/apache/pig/test/

2008-03-18 Thread gates
Author: gates Date: Tue Mar 18 21:45:46 2008 New Revision: 638706 URL: http://svn.apache.org/viewvc?rev=638706view=rev Log: Moved OperatorPlan off of Apache Commons MultiValueMap to local implementation of MultiMap. This was necessary so that OperatorPlan could implement Serializable. Added

svn commit: r637987 - in /incubator/pig/trunk: CHANGES.txt bin/ bin/pig bin/startHOD.expect scripts/pig.pl scripts/startHOD.expect

2008-03-17 Thread gates
Author: gates Date: Mon Mar 17 10:40:59 2008 New Revision: 637987 URL: http://svn.apache.org/viewvc?rev=637987view=rev Log: Removed Yahoo specific scripts/pig.pl, replaced with generic bash script bin/pig. Moved startHOD.expect to bin. Added: incubator/pig/trunk/bin/ incubator/pig

svn commit: r633652 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java

2008-03-04 Thread gates
Author: gates Date: Tue Mar 4 13:20:52 2008 New Revision: 633652 URL: http://svn.apache.org/viewvc?rev=633652view=rev Log: PIG-120: Support map reduce in local mode. To do this user needs to specify execution type as mapreduce and cluster name as local. Modified: incubator/pig/trunk

svn commit: r630997 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/backend/hadoop/executionengine/MapreducePlanCompiler.java test/org/apache/pig/test/TestCombiner.java

2008-02-25 Thread gates
Author: gates Date: Mon Feb 25 13:15:25 2008 New Revision: 630997 URL: http://svn.apache.org/viewvc?rev=630997view=rev Log: PIG-110: Replaced code accidently merged out in PIG-32 fix that handled flattening the combiner case. Added: incubator/pig/trunk/test/org/apache/pig/test

svn commit: r631029 - /incubator/pig/trunk/build.xml

2008-02-25 Thread gates
Author: gates Date: Mon Feb 25 14:59:57 2008 New Revision: 631029 URL: http://svn.apache.org/viewvc?rev=631029view=rev Log: Added back copy of pig-X.jar from build directory to pig.jar in top directory for backward compatibility. Modified: incubator/pig/trunk/build.xml Modified: incubator

svn commit: r631045 - in /incubator/pig/branches/types/src/org/apache/pig/data: DataByteArray.java DataReaderWriter.java DefaultTuple.java

2008-02-25 Thread gates
Author: gates Date: Mon Feb 25 15:56:02 2008 New Revision: 631045 URL: http://svn.apache.org/viewvc?rev=631045view=rev Log: Files that should have been added in previous checkins. Added: incubator/pig/branches/types/src/org/apache/pig/data/DataByteArray.java incubator/pig/branches/types

svn commit: r630357 [2/2] - in /incubator/pig/branches/types: ./ src/org/apache/pig/builtin/ src/org/apache/pig/data/ src/org/apache/pig/impl/builtin/ src/org/apache/pig/impl/eval/ src/org/apache/pig/

2008-02-22 Thread gates
Modified: incubator/pig/branches/types/test/org/apache/pig/test/TestEvalPipeline.java URL: http://svn.apache.org/viewvc/incubator/pig/branches/types/test/org/apache/pig/test/TestEvalPipeline.java?rev=630357r1=630356r2=630357view=diff

svn commit: r629654 - in /incubator/pig/trunk: CHANGES.txt build.xml

2008-02-20 Thread gates
Author: gates Date: Wed Feb 20 15:39:58 2008 New Revision: 629654 URL: http://svn.apache.org/viewvc?rev=629654view=rev Log: PIG-99: Fix to make unit tests not run out of memory. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/build.xml Modified: incubator/pig/trunk

svn commit: r627897 - in /incubator/pig/trunk: ./ lib-src/shock/org/apache/pig/shock/ src/org/apache/pig/ src/org/apache/pig/backend/hadoop/executionengine/ src/org/apache/pig/backend/hadoop/execution

2008-02-14 Thread gates
Author: gates Date: Thu Feb 14 14:24:04 2008 New Revision: 627897 URL: http://svn.apache.org/viewvc?rev=627897view=rev Log: PIG-84: Converted printStackTrace calls to calls to the logger. Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/lib-src/shock/org/apache/pig/shock

svn commit: r612144 - in /incubator/pig/trunk: CHANGES.txt src/org/apache/pig/data/DataBag.java

2008-01-15 Thread gates
Author: gates Date: Tue Jan 15 08:06:56 2008 New Revision: 612144 URL: http://svn.apache.org/viewvc?rev=612144view=rev Log: PIG-56: Made DataBag implement Iterable. (groves via gates) Modified: incubator/pig/trunk/CHANGES.txt incubator/pig/trunk/src/org/apache/pig/data/DataBag.java

svn commit: r610047 - in /incubator/pig/trunk/src/org/apache/pig/data: DataBag.java DefaultDataBag.java DistinctDataBag.java SortedDataBag.java

2008-01-08 Thread gates
Author: gates Date: Tue Jan 8 08:20:45 2008 New Revision: 610047 URL: http://svn.apache.org/viewvc?rev=610047view=rev Log: PIG-30 Added hashCode() implementation to DataBag and added additional comments throughout DataBag and extenders implementation. Modified: incubator/pig/trunk/src/org