[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- || '''JIRA''' || '''Comments''' || '''Done'''|| || [http://issues.apache.org/jira/browse/PIG-246 PIG-246] || UDF repository creation - needs finalizing the EvalFunc.exec interface and changing DataAtom to String || No || || [http://issues.apache.org/jira/browse/PIG-156 PIG-156] || unit tests broken under windows can be merged now - depends on streaming and other test cases being merged first|| No || - || [http://issues.apache.org/jira/browse/PIG-85 PIG-85] || making PigStorage work with control characters || The necessary changes are already present in PigStorage in Types branch || - || [http://issues.apache.org/jira/browse/PIG-198 PIG-198] || integration with hadoop 0.17; Files not patched: * src/org/apache/pig/backend/executionengine/PigSlicer.java - not present yet in types * src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java - changes depend on Config properties change * src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/PigMapReduce.java - this is present in ../impl/mapReduceLayer now and is different in content - depends on some other change going in first which will cause PigMapReduce to implement MapRunnable and Reducer (in the types branch it currently does not implement any interface) * src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/PigCombine.java - not present in types || Yes || - || [http://issues.apache.org/jira/browse/PIG-176 PIG-176] || memory management; depends on PIG-111; || Yes || || [http://issues.apache.org/jira/browse/PIG-151 PIG-151] || problems with bzip files; '''depends on slicer''' || No || || [http://issues.apache.org/jira/browse/PIG-207 PIG-207] || New illustrate command does not work in mapreduce mode. || No || || [http://issues.apache.org/jira/browse/PIG-59 PIG-59] || Illustrate command; I think should be easy to integrate || No || - || [http://issues.apache.org/jira/browse/PIG-170 PIG-170] || memory management || Yes || || [http://issues.apache.org/jira/browse/PIG-120 PIG-120] || support hadoop map reduce in loal mode || No || - || [http://issues.apache.org/jira/browse/PIG-235 PIG-235] || Performance issues with memory spills || Yes || || [http://issues.apache.org/jira/browse/PIG-245 PIG-245] || Need wrapper UDFs for all java.lang.Math functions || No || || [https://issues.apache.org/jira/browse/PIG-271 PIG-271] || Add tutorial files and builds to Pig SVN || No || + || [https://issues.apache.org/jira/browse/PIG-342 PIG-342] || Bug in how distinct data bag counts|| no || + || [http://issues.apache.org/jira/browse/PIG-198 PIG-198] || integration with hadoop 0.17; Files not patched: * src/org/apache/pig/backend/executionengine/PigSlicer.java - not present yet in types * src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java - changes depend on Config properties change * src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/PigMapReduce.java - this is present in ../impl/mapReduceLayer now and is different in content - depends on some other change going in first which will cause PigMapReduce to implement MapRunnable and Reducer (in the types branch it currently does not implement any interface) * src/org/apache/pig/backend/hadoop/executionengine/mapreduceExec/PigCombine.java - not present in types || Yes || + || [http://issues.apache.org/jira/browse/PIG-85 PIG-85] || making PigStorage work with control characters || The necessary changes are already present in PigStorage in Types branch || + || [http://issues.apache.org/jira/browse/PIG-176 PIG-176] || memory management; depends on PIG-111; || Yes || + || [http://issues.apache.org/jira/browse/PIG-170 PIG-170] || memory management || Yes || + || [http://issues.apache.org/jira/browse/PIG-235 PIG-235] || Performance issues with memory spills || Yes || || [http://issues.apache.org/jira/browse/PIG-18 PIG-18] || making Pig work with HOD 0.4; should be doable now - involves backend - HExecutionEngine.java - a little more involved and needs careful merging|| Yes || || [http://issues.apache.org/jira/browse/PIG-106 PIG-106] || Optimize Pig by replacing String '+' and StringBuffer with StringBuilder; - there are changes in backend/Operators which have changed in structure now || Yes || || [http://issues.apache.org/jira/browse/PIG-250 PIG-250] || speculative execution is broken; || Yes || @@ -112, +113 @@ || [http://issues.apache.org/jira/browse/PIG-129 PIG-129] || need to create temp files in the task's working directory; should be easy to integrate || Yes || || [http://issues.apache.org/jira/browse/PIG-118 PIG-118] || UNION/CROSS/JOIN operations should not allow 1 operand;
[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- === Issues that we believe will be fixed in rework and just need tested === || '''JIRA''' || '''Comments''' || '''Done'''|| - || [http://issues.apache.org/jira/browse/PIG-183 PIG-183] || pig could not resolve UDF class; maybe already fixed; verify || No || + || [http://issues.apache.org/jira/browse/PIG-183 PIG-183] || pig could not resolve UDF class; maybe already fixed; verify || Yes || - || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || No || + || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || Yes || || [http://issues.apache.org/jira/browse/PIG-202 PIG-202] || comparator function not used in local mode; is this already fixed? verify || Fixed in rework || || [http://issues.apache.org/jira/browse/PIG-153 PIG-153] || Incorrect results when there is a dump in between statements; might be fixed || Already fixed in rework || || [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might already have it; need to verify || Fixed in rework which has error message about cast failures and out of bounds access ||
[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- || [http://issues.apache.org/jira/browse/PIG-183 PIG-183] || pig could not resolve UDF class; maybe already fixed; verify || No || || [http://issues.apache.org/jira/browse/PIG-153 PIG-153] || Incorrect results when there is a dump in between statements; might be fixed || Already fixed in rework || || [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might already have it; need to verify || No || - || [http://issues.apache.org/jira/browse/PIG-179 PIG-179] || On hadoop 0.16, some jobs using combiner fail with an NPE || No || + || [http://issues.apache.org/jira/browse/PIG-179 PIG-179] || On hadoop 0.16, some jobs using combiner fail with an NPE || Already fixed in rework (the code using RecordReader in types branch is same as in trunk as of now. For this code, the patch is not applicable) || || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || No || || [http://issues.apache.org/jira/browse/PIG-324 PIG-324] || Combiner error || No || || [http://issues.apache.org/jira/browse/PIG-113 PIG-113] || Make Grunt's explain output more understandable; Explain is already in the right format || Yes ||
[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- || [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might already have it; need to verify || No || || [http://issues.apache.org/jira/browse/PIG-179 PIG-179] || On hadoop 0.16, some jobs using combiner fail with an NPE || Already fixed in rework (the code using RecordReader in types branch is same as in trunk as of now. For this code, the patch is not applicable) || || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || No || - || [http://issues.apache.org/jira/browse/PIG-324 PIG-324] || Combiner error || No || + || [http://issues.apache.org/jira/browse/PIG-324 PIG-324] || Combiner error || This is currently an open issue in trunk (as of 09/15/2008) - however the script in the issue runs fine in Types branch with small data || || [http://issues.apache.org/jira/browse/PIG-113 PIG-113] || Make Grunt's explain output more understandable; Explain is already in the right format || Yes || || [http://issues.apache.org/jira/browse/PIG-237 PIG-237] || pig allows to overwrite existing files addressed in the type branch according to Pi need to verify. (Per Pi: PIG-237 code was copied from type branch so don't need to merge) || Yes ||
[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- === Issues that we believe will be fixed in rework and just need tested === || '''JIRA''' || '''Comments''' || '''Done'''|| - || [http://issues.apache.org/jira/browse/PIG-202 PIG-202] || comparator function not used in local mode; is this already fixed? verify || No || || [http://issues.apache.org/jira/browse/PIG-183 PIG-183] || pig could not resolve UDF class; maybe already fixed; verify || No || + || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || No || + || [http://issues.apache.org/jira/browse/PIG-202 PIG-202] || comparator function not used in local mode; is this already fixed? verify || Fixed in rework || || [http://issues.apache.org/jira/browse/PIG-153 PIG-153] || Incorrect results when there is a dump in between statements; might be fixed || Already fixed in rework || - || [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might already have it; need to verify || No || + || [http://issues.apache.org/jira/browse/PIG-125 PIG-125] || better error handling; might already have it; need to verify || Fixed in rework which has error message about cast failures and out of bounds access || || [http://issues.apache.org/jira/browse/PIG-179 PIG-179] || On hadoop 0.16, some jobs using combiner fail with an NPE || Already fixed in rework (the code using RecordReader in types branch is same as in trunk as of now. For this code, the patch is not applicable) || - || [http://issues.apache.org/jira/browse/PIG-178 PIG-178] || Use of schema on a secondary output of SPLIT throws IndexOutOfBoundsException || No || - || [http://issues.apache.org/jira/browse/PIG-324 PIG-324] || Combiner error || This is currently an open issue in trunk (as of 09/15/2008) - however the script in the issue runs fine in Types branch with small data || + || [http://issues.apache.org/jira/browse/PIG-324 PIG-324] || Combiner error || This is currently an open issue in trunk (as of 09/15/2008) - script in the issue does run fine in Types branch with small data || || [http://issues.apache.org/jira/browse/PIG-113 PIG-113] || Make Grunt's explain output more understandable; Explain is already in the right format || Yes || || [http://issues.apache.org/jira/browse/PIG-237 PIG-237] || pig allows to overwrite existing files addressed in the type branch according to Pi need to verify. (Per Pi: PIG-237 code was copied from type branch so don't need to merge) || Yes || @@ -55, +55 @@ || '''JIRA''' || '''Comments''' || '''Done'''|| || [http://issues.apache.org/jira/browse/PIG-243 PIG-243] || Make pig work on Windows - includes streaming fixes || No || - || [http://issues.apache.org/jira/browse/PIG-226 PIG-226] || problem with streaming optimization || Patch Submitted || + || [http://issues.apache.org/jira/browse/PIG-226 PIG-226] || problem with streaming optimization || Yes || - || [http://issues.apache.org/jira/browse/PIG-182 PIG-182] || Broken pipe if excuting the streaming script via the stream command directory || Patch Submitted || + || [http://issues.apache.org/jira/browse/PIG-182 PIG-182] || Broken pipe if excuting the streaming script via the stream command directory || Yes || - || [http://issues.apache.org/jira/browse/PIG-94 PIG-94] || Streaming implementation || Patch Submitted || + || [http://issues.apache.org/jira/browse/PIG-94 PIG-94] || Streaming implementation || Yes || - || [http://issues.apache.org/jira/browse/PIG-180 PIG-180] || Bug in streaming || Patch Submitted || + || [http://issues.apache.org/jira/browse/PIG-180 PIG-180] || Bug in streaming || Yes || - || [http://issues.apache.org/jira/browse/PIG-272 PIG-272] || Failure running complex script with streaming || Patch Submitted || + || [http://issues.apache.org/jira/browse/PIG-272 PIG-272] || Failure running complex script with streaming || Yes || || [http://issues.apache.org/jira/browse/PIG-230 PIG-230] || auto ship broken is presence of define for another streaming operator || Yes || || [http://issues.apache.org/jira/browse/PIG-232 PIG-232] || Number of input/output rows in the logs is invalid with BinaryStorage || Yes || || [http://issues.apache.org/jira/browse/PIG-231 PIG-231] || validation of files in ship/cache specs || Yes ||
[Pig Wiki] Update of PigTypesBranchIntegration by PradeepKamath
Dear Wiki user, You have subscribed to a wiki page or wiki category on Pig Wiki for change notification. The following page has been changed by PradeepKamath: http://wiki.apache.org/pig/PigTypesBranchIntegration -- === Issues that do not touch rewritten code and should be easy === || '''JIRA''' || '''Comments''' || '''Done'''|| - || [http://issues.apache.org/jira/browse/PIG-149 PIG-149] || fix for doc target can be committed now || No || - || [http://issues.apache.org/jira/browse/PIG-150 PIG-150] || similar to 149; can be committed now || No || || [http://issues.apache.org/jira/browse/PIG-68 PIG-68] || build.xml improvements; might already be in; otherwise trivial to fix || No || || [http://issues.apache.org/jira/browse/PIG-124 PIG-124] || running 1 test at a time; might already be there; trivial otherwise || No || || [http://issues.apache.org/jira/browse/PIG-127 PIG-127] || target descriptions; might already be in; otherwise a trivial change that can be done now || No || + || [http://issues.apache.org/jira/browse/PIG-101 PIG-101] || this was on 2/27 so it might already be there; if not it is just test changes that can be applied now || No || || [http://issues.apache.org/jira/browse/PIG-13 PIG-13] || implement version; trivial change; can be applied now || No || - || [http://issues.apache.org/jira/browse/PIG-101 PIG-101] || this was on 2/27 so it might already be there; if not it is just test changes that can be applied now || No || + || [http://issues.apache.org/jira/browse/PIG-123 PIG-123] || escapes in the parse; can be done now || No || || [http://issues.apache.org/jira/browse/PIG-106 PIG-106] || Optimize Pig by replacing String '+' and StringBuffer with StringBuilder; should be trivial to integrate || No || + || [http://issues.apache.org/jira/browse/PIG-18 PIG-18] || making Pig work with HOD 0.4; should be doable now || No || || [http://issues.apache.org/jira/browse/PIG-122 PIG-122] || remove TokenMgrError and Co from svn properties in src/org/apache/pig/tools/pigscript/parser; || No || - || [http://issues.apache.org/jira/browse/PIG-123 PIG-123] || escapes in the parse; can be done now || No || || [http://issues.apache.org/jira/browse/PIG-58 PIG-58] || parameter substitution; can be moved now || No || + || [http://issues.apache.org/jira/browse/PIG-203 PIG-203] || pig parser hangs on input script bigger ~1kb; can be committed right now after (58) || No || + || [http://issues.apache.org/jira/browse/PIG-150 PIG-150] || similar to 149; can be committed now || No || + || [http://issues.apache.org/jira/browse/PIG-149 PIG-149] || fix for doc target can be committed now || No || + || [http://issues.apache.org/jira/browse/PIG-218 PIG-218] || param sub broken with some commands; can be committed now right after (58) || No || + || [http://issues.apache.org/jira/browse/PIG-220 PIG-220] || incorrect variable definition in parameter substitution; can be committed now after (58) || No || || [http://issues.apache.org/jira/browse/PIG-222 PIG-222] || param subst tests hardwire perl location; can be committed now right after (58) || No || - || [http://issues.apache.org/jira/browse/PIG-220 PIG-220] || incorrect variable definition in parameter substitution; can be committed now after (58) || No || - || [http://issues.apache.org/jira/browse/PIG-218 PIG-218] || param sub broken with some commands; can be committed now right after (58) || No || - || [http://issues.apache.org/jira/browse/PIG-203 PIG-203] || pig parser hangs on input script bigger ~1kb; can be committed right now after (58) || No || || [http://issues.apache.org/jira/browse/PIG-219 PIG-219] || running all test cases in both local amd MR mode; can be merged now || No || || [http://issues.apache.org/jira/browse/PIG-156 PIG-156] || unit tests broken under windows can be merged now || No || - || [http://issues.apache.org/jira/browse/PIG-18 PIG-18] || making Pig work with HOD 0.4; should be doable now || No || === Issues that do touch rewritten code and will be difficult ===
