Use of Minicluster in unit tests

2008-12-30 Thread Pradeep Kamath
Hi, MiniCluster is used to create a Hadoop cluster on the machine running the unit tests to test scripts in an end-to-end manner. Currently the unit tests which use MiniCluster create a temporary file on the local file system instead of the DFS in the miniCluster and supply it to the load

RE: [jira] Commented: (PIG-6) Addition of Hbase Storage Option In Load/Store Statement

2009-01-05 Thread Pradeep Kamath
Did you try the types branch in svn? - (svn co http://svn.apache.org/repos/asf/hadoop/pig/branches/types) -Original Message- From: Oliver Po (JIRA) [mailto:j...@apache.org] Sent: Monday, January 05, 2009 2:58 PM To: pig-dev@hadoop.apache.org Subject: [jira] Commented: (PIG-6) Addition

RE: UDF with parameters?

2009-05-26 Thread Pradeep Kamath
You should be able to send the percentile rank that you want to calculate as a udf argument like the way you stated - generate Percentile(90, duration) - here 90 will be an integer constant sent as the first argument to your udf. -Original Message- From: Brian Long

RE: [VOTE] Release Pig 0.3.0 (candidate 0)

2009-06-22 Thread Pradeep Kamath
+1 for release. -Pradeep -Original Message- From: Alan Gates [mailto:ga...@yahoo-inc.com] Sent: Monday, June 22, 2009 9:30 AM To: priv...@hadoop.apache.org Cc: pig-dev@hadoop.apache.org; gene...@hadoop.apache.org Subject: Re: [VOTE] Release Pig 0.3.0 (candidate 0) Downloaded, ran, ran

RE: pig trunk build

2009-08-26 Thread Pradeep Kamath
What is the URL for the Hudson UI? I tried http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.ne t but that did not work. Pradeep -Original Message- From: Giridharan Kesavan [mailto:gkesa...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 7:41 AM To:

RE: [VOTE] Release Pig 0.4.0 (candidate 0)

2009-09-14 Thread Pradeep Kamath
+1 for release. -Original Message- From: Olga Natkovich [mailto:ol...@yahoo-inc.com] Sent: Monday, September 14, 2009 2:06 PM To: pig-dev@hadoop.apache.org; priv...@hadoop.apache.org Subject: [VOTE] Release Pig 0.4.0 (candidate 0) Hi, I created a candidate build for Pig 0.4.0

RE: two-level access problem?

2009-11-03 Thread Pradeep Kamath
that same contagion will afflict us here? -D On Tue, Nov 3, 2009 at 2:10 PM, Pradeep Kamath prade...@yahoo-inc.com wrote: From comments in Schema.java:    // In bags which have a schema with a tuple which contains    // the fields present in it, if we access the second field (say)    // we

RE: two-level access problem?

2009-11-03 Thread Pradeep Kamath
From comments in Schema.java: // In bags which have a schema with a tuple which contains // the fields present in it, if we access the second field (say) // we are actually trying to access the second field in the // tuple in the bag. This is currently true for two cases: // 1)

RE: [VOTE] Branch for Pig 0.6.0 release

2009-11-10 Thread Pradeep Kamath
+1 -Original Message- From: Alan Gates [mailto:ga...@yahoo-inc.com] Sent: Monday, November 09, 2009 12:43 PM To: pig-dev@hadoop.apache.org Subject: Re: [VOTE] Branch for Pig 0.6.0 release +1. In addition to the new features we've added, our change to use Hadoop's LineRecordReader

Locking trunk for commits to merge on load-store-redesign branch

2009-12-21 Thread Pradeep Kamath
Hi, PIG-1143 and PIG-1149 need special handling on the load-store-redesign branch. PIG-1143 should not be applied to the branch since the code is not applicable and for PIG-1149 there is a separate patch. I am beginning a merge of load-store-redesign branch with current head of trunk. These two

DONE - trunk open for commits RE: Locking trunk for commits to merge on load-store-redesign branch

2009-12-21 Thread Pradeep Kamath
Hi, The process outlined below is now completed and the trunk is open for commits which do not conflict with load-store-redesign branch. Thanks, Pradeep -Original Message- From: Pradeep Kamath [mailto:prade...@yahoo-inc.com] Sent: Monday, December 21, 2009 10:50 AM To: pig-dev

RE: Private variables are not eco-friendly

2010-02-02 Thread Pradeep Kamath
Would it be better to make them protected when a use case for inheritance arises rather than begin as protected? -Original Message- From: Dmitriy Ryaboy [mailto:dvrya...@gmail.com] Sent: Tuesday, February 02, 2010 7:35 PM To: pig-dev@hadoop.apache.org Subject: Private variables are not

Plan to merge load-store-redesign branch to trunk

2010-02-16 Thread Pradeep Kamath
Hi, We would like to merge the load-store-redesign branch to trunk tentatively on Thursday. To do this, I would like to request all committers to not commit anything to load-store-redesign branch or trunk during the period of the merge. I will send out a mail to indicate begin and end of this

BEGINNING merge of load-store-redesign branch to trunk - hold off commits!

2010-02-18 Thread Pradeep Kamath
From: Pradeep Kamath Sent: Tuesday, February 16, 2010 11:34 AM To: 'pig-dev@hadoop.apache.org'; 'pig-u...@hadoop.apache.org' Subject: Plan to merge load-store-redesign branch to trunk Hi, We would like to merge the load-store-redesign branch to trunk tentatively on Thursday. To do this, I

COMPLETED merge of load-store-redesign branch to trunk

2010-02-19 Thread Pradeep Kamath
The merge from load-store-redesign branch to trunk is now completed. New commits can now proceed on trunk. The load-store-redesign branch is deprecated with this merge and no more commits should be done on that branch. Pradeep From: Pradeep Kamath Sent

RE: Begin a discussion about Pig as a top level project

2010-04-05 Thread Pradeep Kamath
I agree with Ashutosh and Santhosh. Just based on the current direction of the project I think we are more closely tied with Hadoop now (with Pig 0.7, our load/store interfaces are very closely tied with Hadoop) - hence for now my vote would be a -1 to be a TLP - if there is change in that

RE: [VOTE] Pig to become a top level Apache project

2010-08-18 Thread Pradeep Kamath
+1 -Original Message- From: Alan Gates [mailto:ga...@yahoo-inc.com] Sent: Wednesday, August 18, 2010 10:34 AM To: pig-dev@hadoop.apache.org Subject: [VOTE] Pig to become a top level Apache project Earlier this week I began a discussion on Pig becoming a TLP (http://bit.ly/byD7L8 ).

[jira] Updated: (PIG-537) Failure in Hadoop map collect stage due to type mismatch in the keys used in cogroup

2008-11-20 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-537: --- Status: Patch Available (was: Open) The issue was in Implicit Split inserter. In this query, the same

[jira] Updated: (PIG-563) PERFORMANCE: enable combiner to be called 0 or more times whenver the combiner is used for a pig query

2008-12-17 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-563: --- Attachment: PIG-563.patch PERFORMANCE: enable combiner to be called 0 or more times whenver

[jira] Updated: (PIG-573) Changes to make Pig run with Hadoop 19

2008-12-19 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-573: --- Attachment: hadoop19.jar PIG-573-combinerflag.patch PIG-573.patch PIG

[jira] Commented: (PIG-573) Changes to make Pig run with Hadoop 19

2008-12-22 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12658662#action_12658662 ] Pradeep Kamath commented on PIG-573: Hbase code in pig doesn't work with hadoop 19

[jira] Updated: (PIG-563) PERFORMANCE: enable combiner to be called 0 or more times whenver the combiner is used for a pig query

2008-12-22 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-563: --- Attachment: PIG-563-v3.patch COUNT.Initial was implemented that way so that in case it is called

[jira] Updated: (PIG-580) PERFORMANCE: Combiner should also be used when there are distinct aggregates in a foreach following a group provided there are no non-algebraics in the foreach

2008-12-31 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-580: --- Attachment: PIG-580-v2.patch Attaching a new version (PIG-580-v2.patch) - the only difference from

[jira] Updated: (PIG-581) Pig should enable an option to disable the use of combiner optimizer

2008-12-31 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-581: --- Summary: Pig should enable an option to disable the use of combiner optimizer (was: Pig should enable

[jira] Updated: (PIG-554) Fragment Replicate Join

2009-01-07 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-554: --- Attachment: PIG-554-v4.patch Changes in new patch (attached): 1) The HashMap now has (tuple, ListTuple

[jira] Created: (PIG-628) PERFORMANCE: Misc. optimizations including optimization in Tuple serialization, set up of PigMapReduce PigCombiner, accessing index in POLocalRearrange

2009-01-20 Thread Pradeep Kamath (JIRA)
- Key: PIG-628 URL: https://issues.apache.org/jira/browse/PIG-628 Project: Pig Issue Type: Improvement Affects Versions: types_branch Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix

[jira] Updated: (PIG-628) PERFORMANCE: Misc. optimizations including optimization in Tuple serialization, set up of PigMapReduce PigCombiner, accessing index in POLocalRearrange

2009-01-20 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-628: --- Attachment: PIG-628.patch Attached patch which implements the changes described in the issue

[jira] Updated: (PIG-634) When POUnion is one of the roots of a map plan, POUnion.getNext() gives a null pointer exception

2009-01-26 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-634: --- Resolution: Fixed Status: Resolved (was: Patch Available) When POUnion is one of the roots

[jira] Commented: (PIG-634) When POUnion is one of the roots of a map plan, POUnion.getNext() gives a null pointer exception

2009-01-26 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12667439#action_12667439 ] Pradeep Kamath commented on PIG-634: Patch committed When POUnion is one of the roots

[jira] Created: (PIG-636) PERFORMANCE: Use lightweight bag implementations which do not register with SpillableMemoryManager with Combiner

2009-01-26 Thread Pradeep Kamath (JIRA)
://issues.apache.org/jira/browse/PIG-636 Project: Pig Issue Type: Improvement Affects Versions: types_branch Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: types_branch Currently whenever Combiner is used in pig, in the map

[jira] Updated: (PIG-645) Streaming is broken with the latest trunk

2009-01-29 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-645: --- Attachment: PIG-645.patch Streaming is broken with the latest trunk

[jira] Updated: (PIG-645) Streaming is broken with the latest trunk

2009-01-29 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-645: --- Fix Version/s: types_branch Affects Version/s: types_branch Status: Patch

[jira] Created: (PIG-649) RandomSampleLoader does not handle skipping correctly in getNext()

2009-01-30 Thread Pradeep Kamath (JIRA)
Versions: types_branch Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: types_branch Currently RandomSampleLoader calls skip() on the underlying input stream (BufferedPositionedInputStream) in its getNext(). The input stream may not actually skip

[jira] Updated: (PIG-651) PERFORMANCE: Use specialized POForEachNoFlatten for cases where the foreach has no flattens

2009-01-30 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-651: --- Status: Patch Available (was: Open) Attached patch implements a simpler POForEachNoFlatten whenever

[jira] Updated: (PIG-649) RandomSampleLoader does not handle skipping correctly in getNext()

2009-01-30 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-649: --- Status: Patch Available (was: Open) The attached patch fixes the issue by keep track of the return

[jira] Updated: (PIG-649) RandomSampleLoader does not handle skipping correctly in getNext()

2009-01-30 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-649: --- Attachment: PIG-649.patch RandomSampleLoader does not handle skipping correctly in getNext

[jira] Updated: (PIG-648) BinStorage fails when it finds markers unexpectedly in the data

2009-01-30 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-648: --- Status: Patch Available (was: Open) Attached patch which now uses ctrl-A, Ctrl-B, Ctrl-C

[jira] Updated: (PIG-649) RandomSampleLoader does not handle skipping correctly in getNext()

2009-01-30 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-649: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available

[jira] Commented: (PIG-428) TypeCastInserter does not replace projects in inner plans correctly

2009-02-02 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12669687#action_12669687 ] Pradeep Kamath commented on PIG-428: Have you tried your query with top of trunk

[jira] Updated: (PIG-653) Make fieldsToRead work in loader

2009-02-09 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-653: --- Attachment: PIG-653-2.comment A new proposal has been attached as a revision of the proposal in comment

[jira] Created: (PIG-665) Map key type not correctly set (for use when key is null) when map plan does not have localrearrange

2009-02-11 Thread Pradeep Kamath (JIRA)
-665 Project: Pig Issue Type: Bug Affects Versions: types_branch Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: types_branch KeyTypeDiscoveryVisitor visits the map plan to figure out the datatype of the map key

[jira] Updated: (PIG-665) Map key type not correctly set (for use when key is null) when map plan does not have localrearrange

2009-02-12 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-665: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available

[jira] Updated: (PIG-545) PERFORMANCE: Sampler for order bys does not produce a good distribution

2009-02-18 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-545: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available

[jira] Updated: (PIG-652) Need to give user control of OutputFormat

2009-02-19 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-652: --- Fix Version/s: types_branch Assignee: Pradeep Kamath (was: Alan Gates) Affects

[jira] Updated: (PIG-652) Need to give user control of OutputFormat

2009-02-19 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-652: --- Attachment: PIG-652.patch Need to give user control of OutputFormat

[jira] Updated: (PIG-652) Need to give user control of OutputFormat

2009-02-19 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-652: --- Attachment: PIG-652-v2.patch Attached new version which addresses the comment regarding having

[jira] Updated: (PIG-652) Need to give user control of OutputFormat

2009-02-23 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-652: --- Attachment: PIG-652-v3.patch Attached a new version of the patch. Changes include: 1) Included

[jira] Commented: (PIG-591) Error handling phase four

2009-02-24 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12676399#action_12676399 ] Pradeep Kamath commented on PIG-591: Code review comments The patch looks good to go

[jira] Resolved: (PIG-591) Error handling phase four

2009-02-25 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath resolved PIG-591. Resolution: Fixed Hadoop Flags: [Reviewed] Santhosh, thanks for the feature contribution. Patch

[jira] Commented: (PIG-682) Fix the ssh tunneling code

2009-02-25 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12676810#action_12676810 ] Pradeep Kamath commented on PIG-682: As noted in https://issues.apache.org/jira/browse

[jira] Updated: (PIG-652) Need to give user control of OutputFormat

2009-02-27 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-652: --- Attachment: PIG-652-v4.patch Attaching new patch - the only difference is: (old code is the first input

[jira] Issue Comment Edited: (PIG-652) Need to give user control of OutputFormat

2009-02-27 Thread Pradeep Kamath (JIRA)
Issue Type: New Feature Components: impl Affects Versions: types_branch Reporter: Alan Gates Assignee: Pradeep Kamath Fix For: types_branch Attachments: PIG-652-v2.patch, PIG-652-v3.patch, PIG-652-v4.patch, PIG-652.patch Pig currently

[jira] Updated: (PIG-691) BinStorage skips tuples when ^A is present in data

2009-03-02 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-691: --- Fix Version/s: types_branch Affects Version/s: types_branch Status: Patch

[jira] Updated: (PIG-691) BinStorage skips tuples when ^A is present in data

2009-03-02 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-691: --- Attachment: PIG-691.patch BinStorage skips tuples when ^A is present in data

[jira] Updated: (PIG-690) UNION doesn't work in the latest code

2009-03-02 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-690: --- Attachment: PIG-690.patch UNION doesn't work in the latest code

[jira] Commented: (PIG-655) Comparison of schemas of bincond operands is flawed

2009-03-02 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12678210#action_12678210 ] Pradeep Kamath commented on PIG-655: I will be reviewing this patch Comparison

[jira] Commented: (PIG-692) when running script file, automatically set up job name based on the file name

2009-03-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12678458#action_12678458 ] Pradeep Kamath commented on PIG-692: +1 for the change when running script file

[jira] Resolved: (PIG-577) outer join query looses name information

2009-03-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath resolved PIG-577. Resolution: Fixed outer join query looses name information

[jira] Updated: (PIG-577) outer join query looses name information

2009-03-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-577: --- Hadoop Flags: [Reviewed] +1, Patch committed - thanks for the fix Santhosh. outer join query looses

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-04 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12679021#action_12679021 ] Pradeep Kamath commented on PIG-627: I committed multi-store-0304.patch into the multi

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-10 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12680724#action_12680724 ] Pradeep Kamath commented on PIG-627: multiquery_0306.patch seems to have a lot of code

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-11 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12680997#action_12680997 ] Pradeep Kamath commented on PIG-627: Sorry about the misunderstanding, I think I looked

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-11 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12681085#action_12681085 ] Pradeep Kamath commented on PIG-627: Committed patch per previous comment that the review

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-23 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12688339#action_12688339 ] Pradeep Kamath commented on PIG-627: Comments for Richard's patch - multiquery-phase2_0313

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-23 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12688356#action_12688356 ] Pradeep Kamath commented on PIG-627: +1 on Gunther's patch - multiquery_explain_fix.patch

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-23 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12688461#action_12688461 ] Pradeep Kamath commented on PIG-627: +1 on Richard's patch - multiquery-phase2_0323.patch

[jira] Commented: (PIG-729) Use of default parallelism

2009-03-24 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12688755#action_12688755 ] Pradeep Kamath commented on PIG-729: Another option maybe to detect mapreduce boundaries

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-03-24 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12688957#action_12688957 ] Pradeep Kamath commented on PIG-627: +1 - committed patch by Gunther to merge changes

[jira] Created: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-03-25 Thread Pradeep Kamath (JIRA)
/PIG-733 Project: Pig Issue Type: Bug Reporter: Pradeep Kamath Assignee: Pradeep Kamath Order by has a sampling job which samples the input and creates a sorted list of sample items. CUrrently the number of items sampled is 100 per map task. So

[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-04-01 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-733: --- Fix Version/s: 0.3.0 Affects Version/s: 0.2.0 Status: Patch Available (was: Open

[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-04-01 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-733: --- Attachment: PIG-733.patch Order by sampling dumps entire sample to hdfs which causes dfs FileSystem

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-04-01 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694859#action_12694859 ] Pradeep Kamath commented on PIG-627: +1, patch committed - thanks for the contribution

[jira] Commented: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-04-06 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12696244#action_12696244 ] Pradeep Kamath commented on PIG-733: Tests are not included in this patch since

[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-04-06 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-733: --- Attachment: PIG-733-v2.patch Order by sampling dumps entire sample to hdfs which causes dfs FileSystem

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-04-06 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12696350#action_12696350 ] Pradeep Kamath commented on PIG-627: +1, patch committed. Thanks for the contribution

[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input

2009-04-09 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-733: --- Resolution: Fixed Status: Resolved (was: Patch Available) patch committed Order by sampling

[jira] Resolved: (PIG-739) Filter in foreach seems to drop records resulting in decreased count of records

2009-04-16 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath resolved PIG-739. Resolution: Duplicate Assignee: Pradeep Kamath This issue has the same root cause of PIG-514

[jira] Commented: (PIG-514) COUNT returns no results as a result of two filter statements in FOREACH

2009-04-16 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12699801#action_12699801 ] Pradeep Kamath commented on PIG-514: I am currently working on implementing the above

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-04-20 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12700925#action_12700925 ] Pradeep Kamath commented on PIG-627: reviewed error_handling_0416.patch for additional

[jira] Created: (PIG-773) Empty complex constants (empty bag, empty tuple and empty map) should be supported

2009-04-21 Thread Pradeep Kamath (JIRA)
Issue Type: Bug Affects Versions: 0.2.0 Reporter: Pradeep Kamath Priority: Minor We should be able to create empty bag constant using {}, empty tuple constant using (), empty map constant using [] within a pig script -- This message is automatically generated

[jira] Resolved: (PIG-514) COUNT returns no results as a result of two filter statements in FOREACH

2009-04-22 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath resolved PIG-514. Resolution: Fixed Fix Version/s: 0.3.0 Hadoop Flags: [Reviewed] Patch committed

[jira] Commented: (PIG-627) PERFORMANCE: multi-query optimization

2009-04-23 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12702005#action_12702005 ] Pradeep Kamath commented on PIG-627: All the work till now (phase 1 and phase2) has now

[jira] Resolved: (PIG-775) PORelationToExprProject should create a NonSpillableDataBag to create empty bags

2009-04-24 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath resolved PIG-775. Resolution: Fixed Patch committed. PORelationToExprProject should create a NonSpillableDataBag

[jira] Commented: (PIG-802) PERFORMANCE: not creating bags for ORDER BY

2009-05-07 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12707064#action_12707064 ] Pradeep Kamath commented on PIG-802: PIG-744 is a duplicate - will be marking that one

[jira] Created: (PIG-807) PERFORMANCE: Provide a way for UDFs to use read-once bags (backed by the Hadoop values iterator)

2009-05-12 Thread Pradeep Kamath (JIRA)
Project: Pig Issue Type: Improvement Affects Versions: 0.2.1 Reporter: Pradeep Kamath Fix For: 0.3.0 Currently all bags resulting from a group or cogroup are materialized as bags containing all of the contents. The issue

[jira] Commented: (PIG-802) PERFORMANCE: not creating bags for ORDER BY

2009-05-12 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12708551#action_12708551 ] Pradeep Kamath commented on PIG-802: Adding some more details: A new kind of bag

[jira] Updated: (PIG-804) problem with lineage with double map redirection

2009-05-13 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-804: --- Fix Version/s: 0.3.0 Affects Version/s: 0.2.1 Status: Patch Available (was: Open

[jira] Updated: (PIG-804) problem with lineage with double map redirection

2009-05-13 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-804: --- Attachment: PIG-804.patch The root cause was in the parsers, in CastExp(), a getFieldSchema() was being

[jira] Created: (PIG-808) getFieldSchema() in ExpressionOperators also sets up lineage information - this can cause issues if getFieldSchema() is called too early

2009-05-13 Thread Pradeep Kamath (JIRA)
: PIG-808 URL: https://issues.apache.org/jira/browse/PIG-808 Project: Pig Issue Type: Bug Affects Versions: 0.2.1 Reporter: Pradeep Kamath Fix For: 0.3.0 See PIG-804 for a use case which exposes this bug. We should probably

[jira] Commented: (PIG-802) PERFORMANCE: not creating bags for ORDER BY

2009-05-21 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711769#action_12711769 ] Pradeep Kamath commented on PIG-802: Review comments: In MRCompiler, does POPackageLite

[jira] Commented: (PIG-802) PERFORMANCE: not creating bags for ORDER BY

2009-05-21 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12711811#action_12711811 ] Pradeep Kamath commented on PIG-802: I think even in the future if ReadOnceBags are used

[jira] Created: (PIG-814) Make Binstorage more robust when data contains record markers

2009-05-21 Thread Pradeep Kamath (JIRA)
: 0.2.1 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.3.0 When the inputstream for BinStorage is at a position where the data has the record marker sequence, the code incorrectly assumes that it is at the beginning of a record (tuple) and calls

[jira] Updated: (PIG-804) problem with lineage with double map redirection

2009-05-26 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-804: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available

[jira] Commented: (PIG-802) PERFORMANCE: not creating bags for ORDER BY

2009-05-26 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12713181#action_12713181 ] Pradeep Kamath commented on PIG-802: Changes look good - still have a comment about

[jira] Updated: (PIG-816) PigStorage() does not accept Unicode characters in its contructor

2009-05-29 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-816: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available

[jira] Commented: (PIG-796) support conversion from numeric types to chararray

2009-06-01 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12715325#action_12715325 ] Pradeep Kamath commented on PIG-796: A few comments: - In TestPOCast.java the variables

[jira] Updated: (PIG-796) support conversion from numeric types to chararray

2009-06-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-796: --- Status: Patch Available (was: Open) support conversion from numeric types to chararray

[jira] Updated: (PIG-796) support conversion from numeric types to chararray

2009-06-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-796: --- Status: Open (was: Patch Available) support conversion from numeric types to chararray

[jira] Updated: (PIG-796) support conversion from numeric types to chararray

2009-06-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-796: --- Status: Patch Available (was: Open) support conversion from numeric types to chararray

[jira] Updated: (PIG-796) support conversion from numeric types to chararray

2009-06-03 Thread Pradeep Kamath (JIRA)
[ https://issues.apache.org/jira/browse/PIG-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Kamath updated PIG-796: --- Status: Open (was: Patch Available) support conversion from numeric types to chararray

  1   2   3   4   5   6   >