RE: [jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-25 Thread Yan Zhou
Reporter: Olga Natkovich >Assignee: Yan Zhou > Fix For: 0.8.0 > > Attachments: compress_perf_data.txt, compress_perf_data_2.txt, > PIG-1501.patch, PIG-1501.patch, PIG-1501.patch > > > We would like to understand how compressing map results as well as wel

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-06 Thread Yan Zhou
Swati, I happen to be working on the logical expression simplification effort (https://issues.apache.org/jira/browse/PIG-1399), but not on the filter split front. So I guess our interests will have some overlaps. I think the filter logic split problem can be divided into 2 parts: 1) the filtering

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
7;ll add my algorithm details soon to my jira. Please take a look and comment as you see appropriate. Thanks, Yan From: Swati Jain [mailto:swat...@aggiemail.usu.edu] Sent: Friday, July 09, 2010 11:00 PM To: Yan Zhou Cc: pig-dev@hadoop.apache.org Subj

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
or the original form should be evaluated. Please let me know if I miss any of your points. Thanks, Yan From: Swati Jain [mailto:swat...@aggiemail.usu.edu] Sent: Monday, July 12, 2010 11:52 AM To: Yan Zhou Cc: pig-dev@hadoop.apache.org Subject: Re: PIG Logic

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
ks, Swati On Mon, Jul 12, 2010 at 3:14 PM, Yan Zhou wrote: > Hopefully by this week. I'm still in the debugging phase of the work. > While you are welcome to reuse some of my algorithms, I doubt you can reuse > the code as much as you want. It's basically for my

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
source C) while ( (a3+b3 > 10) OR (c2 ==5) ) stays put below the JOIN. Please let me know if there is a way of doing the above optimization without converting the original expression to CNF. Thanks, Swati On Mon, Jul 12, 2010 at 4:26 PM, Yan Zhou wrote: > I see. There looks like so

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-13 Thread Yan Zhou
ng the above optimization without converting the original expression to CNF. Thanks, Swati On Mon, Jul 12, 2010 at 4:26 PM, Yan Zhou wrote: > I see. There looks like some disconnect about "Scenario 1". To me, all > filtering logics that can be pushed above JOIN can be figured ou

RE: Add "deepCopy" in LogicalExpression

2010-07-13 Thread Yan Zhou
What I used in logic expression simplification, where a DNF plan is used, is a "logical expression proxy" that wraps up the original expression in the original plan. But it is not translated to physical plans, where I believe extra care would be needed. Yan -Original Message- From: Swati

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-09 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: compress_perf_data_2.txt The data set in the last tests are small such that the performance

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-09 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896620#action_12896620 ] Yan Zhou commented on PIG-1501: --- Unless there is any objection raised in the coming week,

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897005#action_12897005 ] Yan Zhou commented on PIG-1501: --- The default is *not* using the compression on

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch > need to investigate the impact of compression on pig performa

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897085#action_12897085 ] Yan Zhou commented on PIG-1518: --- The pseudo code of the combination op is as follows:

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-11 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897493#action_12897493 ] Yan Zhou commented on PIG-1518: --- Right, map side cogroup needs the sortness of the input,

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-11 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897496#action_12897496 ] Yan Zhou commented on PIG-1501: --- Please refer to HADOOP-3315 for overall Sequence Fil

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-12 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897887#action_12897887 ] Yan Zhou commented on PIG-1518: --- During the merge process, any empty splits will be ski

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898490#action_12898490 ] Yan Zhou commented on PIG-1518: --- There is a bigger question at hand. The semantic

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899445#action_12899445 ] Yan Zhou commented on PIG-1518: --- Another approach is to mark splits as uncombinable only

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899605#action_12899605 ] Yan Zhou commented on PIG-1518: --- One experimental result on a 15-node cluster of 2 x

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899609#action_12899609 ] Yan Zhou commented on PIG-1518: --- The formatting of the table of the last comment is a bit

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch > multi file input format for load

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899888#action_12899888 ] Yan Zhou commented on PIG-1518: --- In summary, the split combination's controllables ar

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900123#action_12900123 ] Yan Zhou commented on PIG-1518: --- No. It does not work inside an optimizer as logical/phys

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Style changes, Hudson pass, plus other minor changes. Internal Hudson results: [exec

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch rebased on the latest trunk. > Logical Optimizer: Expression optimizor r

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch the compression codec is configurable on gzip or lzo; plus some minor changes > n

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900950#action_12900950 ] Yan Zhou commented on PIG-1501: --- The internal Hudson results are as follows: [exec

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Internal Hudson results: [exec] +1 overall. [exec] [exec] +1

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch The add method if PigSplit is removed. The debug code is left to facilitate future

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Fix a typo; rebase on the latest trunk. > multi file input format for load

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Patch Available (was: Open) Release Note: Feature: combine splits of sizes smaller than the

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Open (was: Patch Available) > multi file input format for load

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Patch Available (was: Open) > multi file input format for load

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Minor polish of a debugging code inside comments > multi file input format

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch Address the review comments, code rebasing on the latest trunk. > need

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch rebasing on the latest trunk > Logical Optimizer: Expression optimizor r

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Improvement on logging info. > multi file input format for load

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Open (was: Patch Available) > multi file input format for load

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Status: Patch Available (was: Open) This feature will save HDFS space used to store the intermediate data used

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch > multi file input format for load

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch rebased on the latest trunk > multi file input format for load

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903102#action_12903102 ] Yan Zhou commented on PIG-1518: --- It is not combinable if the loader is a CollectableLoad

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903423#action_12903423 ] Yan Zhou commented on PIG-1518: --- MergeJoinIndexer and IndexableLoadFunc are both

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903525#action_12903525 ] Yan Zhou commented on PIG-1518: --- In summary, the following functionalities won't s

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903528#action_12903528 ] Yan Zhou commented on PIG-1518: --- All other functionalities except for the two mentioned in

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Addressing the review comments except for not making several optimization rules

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch rebased on the latest trunk. > Logical Optimizer: Expression optimizor r

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch I use findbugs 1.3.9 and it finds the patch clean. The attached findbugs results

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Status: Patch Available (was: Open) Release Note: This logical simplification contains the following

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-31 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Release Note: This feature will save HDFS space used to store the intermediate data used by PIG and potentially

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-31 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Release Note: This feature will save HDFS space used to store the intermediate data used by PIG and potentially

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-31 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904868#action_12904868 ] Yan Zhou commented on PIG-1501: --- To be more eaccurate, the default compression would be

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908926#action_12908926 ] Yan Zhou commented on PIG-366: -- Robert, first, thanks for your effort to pick up this fea

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908962#action_12908962 ] Yan Zhou commented on PIG-366: -- Yes. But the original patch by Shubham had hooked the plugi

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908971#action_12908971 ] Yan Zhou commented on PIG-366: -- One more clearification: by design example generator does

[jira] Resolved: (PIG-239) illustrate followed by dump gives a runtime exception

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou resolved PIG-239. -- Fix Version/s: 0.8.0 (was: 0.9.0) Resolution: Cannot Reproduce Can not reproduce

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-14 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909330#action_12909330 ] Yan Zhou commented on PIG-366: -- Robert, Could you put down a step-by-step instruction on ho

[jira] Commented: (PIG-1628) log this message at debug level : 'Pig Internal storage in use'

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913029#action_12913029 ] Yan Zhou commented on PIG-1628: --- +1. Patch looks good. > log this message at debu

[jira] Created: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
Key: PIG-1635 URL: https://issues.apache.org/jira/browse/PIG-1635 Project: Pig Issue Type: Bug Components: impl Reporter: Yan Zhou Assignee: Yan Zhou Priority: Minor b = FILTER a by (( f1 > 1) AND

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913036#action_12913036 ] Yan Zhou commented on PIG-1635: --- This is regarding a new feature (PIG-1399) added for

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Affects Version/s: 0.8.0 > Logical simplifier does not simplify away constants under AND and OR; af

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Attachment: PIG-1635.patch > Logical simplifier does not simplify away constants under AND and OR; af

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Status: Patch Available (was: Open) > Logical simplifier does not simplify away constants under AND and

[jira] Created: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-22 Thread Yan Zhou (JIRA)
/browse/PIG-1645 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee: Yan Zhou Fix For: 0.8.0 The stack looks like the following: java.lang.NullPointerException at java.util.Arrays.binarySearch(Arrays.java

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Release Note: Feature: combine splits of sizes smaller than the value of property "pig.maxCombinedSplitSiz

[jira] Commented: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914128#action_12914128 ] Yan Zhou commented on PIG-1645: --- The problem is that both RandomSampleLoader

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914145#action_12914145 ] Yan Zhou commented on PIG-1635: --- test-patch results: [exec] +1 overall. [

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914150#action_12914150 ] Yan Zhou commented on PIG-1635: --- All test-core tests also run clean. > Logical sim

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Attachment: PIG-1645.patch test-core passed. test-patch results: [exec] -1 overall. [exec

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Status: Patch Available (was: Open) > Using both small split combination and temporary file compression o

[jira] Commented: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914541#action_12914541 ] Yan Zhou commented on PIG-1645: --- The possibility of failure also depends upon the b

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914672#action_12914672 ] Yan Zhou commented on PIG-1635: --- I did a thorough check for this patch. Actually some of

[jira] Created: (PIG-1647) Logical simplifier throws a NPE

2010-09-24 Thread Yan Zhou (JIRA)
Logical simplifier throws a NPE --- Key: PIG-1647 URL: https://issues.apache.org/jira/browse/PIG-1647 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee

[jira] Created: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-24 Thread Yan Zhou (JIRA)
Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee: Yan Zhou Fix For: 0.8.0 For instance, if a small split has block locations h1, h2 and h3; another small split has h1, h3, h4. After combination, the composite split contains 4 block locations

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Attachment: PIG-1647.patch > Logical simplifier throws a

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Attachment: PIG-1647.patch passes test-core. test-patch results: [exec] +1 overall. [exec

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Status: Patch Available (was: Open) > Logical simplifier throws a

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Created: (PIG-1651) PIG class loading mishandled

2010-09-27 Thread Yan Zhou (JIRA)
PIG class loading mishandled Key: PIG-1651 URL: https://issues.apache.org/jira/browse/PIG-1651 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee

[jira] Commented: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915815#action_12915815 ] Yan Zhou commented on PIG-1648: --- Top 5 locations with most data will be used. This has

[jira] Updated: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1648: -- Attachment: PIG-1648.patch > Split combination may return too many block locations to map/reduce framew

[jira] Commented: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915852#action_12915852 ] Yan Zhou commented on PIG-1648: --- test-patch results: [exec] +1 overall. [

[jira] Updated: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1648: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1648: -- Status: Patch Available (was: Open) > Split combination may return too many block locations to map/red

[jira] Created: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
Reporter: Yan Zhou In fact, all these types of keys of values that are negative but within the byte or short's range would have the problem. Basic cally, a byte value of -1 & 0xff will return 255 not -1. -- This message is automatically generated by JIRA. - You can reply to this email

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Fix Version/s: 0.8.0 Affects Version/s: 0.8.0 > ORDER BY does not work properly on integer/short k

[jira] Assigned: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou reassigned PIG-1658: - Assignee: Yan Zhou > ORDER BY does not work properly on integer/short keys that are

[jira] Created: (PIG-1659) sortinfo is not set for store if there is a filter after ORDER BY

2010-09-30 Thread Yan Zhou (JIRA)
Versions: 0.8.0 Reporter: Yan Zhou Assignee: Daniel Dai Fix For: 0.8.0 This has caused 6 (of 7) failures in the Zebra test TestOrderPreserveVariableTable. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Status: Patch Available (was: Open) > ORDER BY does not work properly on integer/short keys that are

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Attachment: PIG-1658.patch This problem is caused by the PIG-1295 patch. test-core pass. Zebra's nightly

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Attachment: PIG-1658.patch Add Zebra test TestMergeJoinPartial to the "pigtest" target. > ORDE

[jira] Commented: (PIG-1659) sortinfo is not set for store if there is a filter after ORDER BY

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917012#action_12917012 ] Yan Zhou commented on PIG-1659: --- Need to make sure it is invoked after optimization in

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Status: Resolved (was: Patch Available) Resolution: Fixed Committed to both trunk and the 0.8 branch

[jira] Commented: (PIG-1057) [Zebra] Zebra does not support concurrent deletions of column groups now.

2009-10-29 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771679#action_12771679 ] Yan Zhou commented on PIG-1057: --- patch reviewed +1. This patch will address the concern R

[jira] Updated: (PIG-997) [zebra] Sorted Table Support by Zebra

2009-10-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-997: - Status: Patch Available (was: Open) > [zebra] Sorted Table Support by Ze

[jira] Assigned: (PIG-997) [zebra] Sorted Table Support by Zebra

2009-10-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou reassigned PIG-997: Assignee: Yan Zhou > [zebra] Sorted Table Support by Ze

[jira] Updated: (PIG-997) [zebra] Sorted Table Support by Zebra

2009-10-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-997: - Attachment: SortedTable.patch > [zebra] Sorted Table Support by Ze

[jira] Updated: (PIG-1026) [zebra] map split returns null

2009-10-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1026: -- Attachment: PIG_1026.patch > [zebra] map split returns n

  1   2   3   4   5   6   >