[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Attachment: PIG-1658.patch Add Zebra test TestMergeJoinPartial to the pigtest target. ORDER BY does not work

[jira] Commented: (PIG-1659) sortinfo is not set for store if there is a filter after ORDER BY

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12917012#action_12917012 ] Yan Zhou commented on PIG-1659: --- Need to make sure it is invoked after optimization in both old

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-10-01 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Status: Resolved (was: Patch Available) Resolution: Fixed Committed to both trunk and the 0.8 branch

[jira] Created: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
Reporter: Yan Zhou In fact, all these types of keys of values that are negative but within the byte or short's range would have the problem. Basic cally, a byte value of -1 0xff will return 255 not -1. -- This message is automatically generated by JIRA. - You can reply to this email to add

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Fix Version/s: 0.8.0 Affects Version/s: 0.8.0 ORDER BY does not work properly on integer/short keys

[jira] Assigned: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou reassigned PIG-1658: - Assignee: Yan Zhou ORDER BY does not work properly on integer/short keys that are -1

[jira] Created: (PIG-1659) sortinfo is not set for store if there is a filter after ORDER BY

2010-09-30 Thread Yan Zhou (JIRA)
Versions: 0.8.0 Reporter: Yan Zhou Assignee: Daniel Dai Fix For: 0.8.0 This has caused 6 (of 7) failures in the Zebra test TestOrderPreserveVariableTable. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Status: Patch Available (was: Open) ORDER BY does not work properly on integer/short keys that are -1

[jira] Updated: (PIG-1658) ORDER BY does not work properly on integer/short keys that are -1

2010-09-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1658: -- Attachment: PIG-1658.patch This problem is caused by the PIG-1295 patch. test-core pass. Zebra's nightly pass

[jira] Commented: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12915815#action_12915815 ] Yan Zhou commented on PIG-1648: --- Top 5 locations with most data will be used. This has been

[jira] Commented: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12915852#action_12915852 ] Yan Zhou commented on PIG-1648: --- test-patch results: [exec] +1 overall. [exec

[jira] Updated: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1648: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-28 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1648: -- Status: Patch Available (was: Open) Split combination may return too many block locations to map/reduce

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Created: (PIG-1651) PIG class loading mishandled

2010-09-27 Thread Yan Zhou (JIRA)
PIG class loading mishandled Key: PIG-1651 URL: https://issues.apache.org/jira/browse/PIG-1651 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Attachment: PIG-1647.patch passes test-core. test-patch results: [exec] +1 overall. [exec

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Status: Patch Available (was: Open) Logical simplifier throws a NPE

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Attachment: PIG-1645.patch test-core passed. test-patch results: [exec] -1 overall. [exec

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Status: Patch Available (was: Open) Using both small split combination and temporary file compression

[jira] Commented: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914541#action_12914541 ] Yan Zhou commented on PIG-1645: --- The possibility of failure also depends upon the block

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914672#action_12914672 ] Yan Zhou commented on PIG-1635: --- I did a thorough check for this patch. Actually some

[jira] Created: (PIG-1647) Logical simplifier throws a NPE

2010-09-24 Thread Yan Zhou (JIRA)
Logical simplifier throws a NPE --- Key: PIG-1647 URL: https://issues.apache.org/jira/browse/PIG-1647 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee

[jira] Created: (PIG-1648) Split combination may return too many block locations to map/reduce framework

2010-09-24 Thread Yan Zhou (JIRA)
Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee: Yan Zhou Fix For: 0.8.0 For instance, if a small split has block locations h1, h2 and h3; another small split has h1, h3, h4. After combination, the composite split contains 4 block locations

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1647) Logical simplifier throws a NPE

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1647: -- Attachment: PIG-1647.patch Logical simplifier throws a NPE

[jira] Updated: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1645: -- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to both trunk and the 0.8

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Release Note: Feature: combine splits of sizes smaller than the value of property pig.maxCombinedSplitSize

[jira] Commented: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914128#action_12914128 ] Yan Zhou commented on PIG-1645: --- The problem is that both RandomSampleLoader

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914145#action_12914145 ] Yan Zhou commented on PIG-1635: --- test-patch results: [exec] +1 overall. [exec

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12914150#action_12914150 ] Yan Zhou commented on PIG-1635: --- All test-core tests also run clean. Logical simplifier does

[jira] Created: (PIG-1645) Using both small split combination and temporary file compression on a query of ORDER BY may cause crash

2010-09-22 Thread Yan Zhou (JIRA)
/browse/PIG-1645 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Yan Zhou Assignee: Yan Zhou Fix For: 0.8.0 The stack looks like the following: java.lang.NullPointerException at java.util.Arrays.binarySearch(Arrays.java

[jira] Commented: (PIG-1628) log this message at debug level : 'Pig Internal storage in use'

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913029#action_12913029 ] Yan Zhou commented on PIG-1628: --- +1. Patch looks good. log this message at debug level : 'Pig

[jira] Created: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
Key: PIG-1635 URL: https://issues.apache.org/jira/browse/PIG-1635 Project: Pig Issue Type: Bug Components: impl Reporter: Yan Zhou Assignee: Yan Zhou Priority: Minor b = FILTER a by (( f1 1) AND (1

[jira] Commented: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913036#action_12913036 ] Yan Zhou commented on PIG-1635: --- This is regarding a new feature (PIG-1399) added for 0.8

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Affects Version/s: 0.8.0 Logical simplifier does not simplify away constants under AND and OR; after

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Attachment: PIG-1635.patch Logical simplifier does not simplify away constants under AND and OR; after

[jira] Updated: (PIG-1635) Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of AND and OR may get changed

2010-09-21 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1635: -- Status: Patch Available (was: Open) Logical simplifier does not simplify away constants under

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-14 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12909330#action_12909330 ] Yan Zhou commented on PIG-366: -- Robert, Could you put down a step-by-step instruction on how

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908926#action_12908926 ] Yan Zhou commented on PIG-366: -- Robert, first, thanks for your effort to pick up this feature

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908962#action_12908962 ] Yan Zhou commented on PIG-366: -- Yes. But the original patch by Shubham had hooked the plugin

[jira] Commented: (PIG-366) PigPen - Eclipse plugin for a graphical PigLatin editor

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12908971#action_12908971 ] Yan Zhou commented on PIG-366: -- One more clearification: by design example generator does

[jira] Resolved: (PIG-239) illustrate followed by dump gives a runtime exception

2010-09-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou resolved PIG-239. -- Fix Version/s: 0.8.0 (was: 0.9.0) Resolution: Cannot Reproduce Can not reproduce

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-31 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Release Note: This feature will save HDFS space used to store the intermediate data used by PIG and potentially

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch I use findbugs 1.3.9 and it finds the patch clean. The attached findbugs results

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Status: Patch Available (was: Open) Release Note: This logical simplification contains the following

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Addressing the review comments except for not making several optimization rules

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-27 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch rebased on the latest trunk. Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Status: Patch Available (was: Open) This feature will save HDFS space used to store the intermediate data used

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch rebased on the latest trunk multi file input format for loaders

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-26 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12903102#action_12903102 ] Yan Zhou commented on PIG-1518: --- It is not combinable if the loader is a CollectableLoadFunc

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch Address the review comments, code rebasing on the latest trunk. need to investigate

RE: [jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-25 Thread Yan Zhou
on pig performance Key: PIG-1501 URL: https://issues.apache.org/jira/browse/PIG-1501 Project: Pig Issue Type: Test Reporter: Olga Natkovich Assignee: Yan

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Improvement on logging info. multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-25 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Open (was: Patch Available) multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Open (was: Patch Available) multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Patch Available (was: Open) multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-24 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Minor polish of a debugging code inside comments multi file input format

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch The add method if PigSplit is removed. The debug code is left to facilitate future

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Fix a typo; rebase on the latest trunk. multi file input format for loaders

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Patch Available (was: Open) Release Note: Feature: combine splits of sizes smaller than

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Style changes, Hudson pass, plus other minor changes. Internal Hudson results: [exec

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch rebased on the latest trunk. Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch the compression codec is configurable on gzip or lzo; plus some minor changes need

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-20 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900950#action_12900950 ] Yan Zhou commented on PIG-1501: --- The internal Hudson results are as follows: [exec] -1

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch multi file input format for loaders

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899888#action_12899888 ] Yan Zhou commented on PIG-1518: --- In summary, the split combination's controllables are through

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-18 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12900123#action_12900123 ] Yan Zhou commented on PIG-1518: --- No. It does not work inside an optimizer as logical/physical

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899445#action_12899445 ] Yan Zhou commented on PIG-1518: --- Another approach is to mark splits as uncombinable only when

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899605#action_12899605 ] Yan Zhou commented on PIG-1518: --- One experimental result on a 15-node cluster of 2 x Xeon L5420

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-17 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899609#action_12899609 ] Yan Zhou commented on PIG-1518: --- The formatting of the table of the last comment is a bit off

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12898490#action_12898490 ] Yan Zhou commented on PIG-1518: --- There is a bigger question at hand. The semantics

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-12 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897887#action_12897887 ] Yan Zhou commented on PIG-1518: --- During the merge process, any empty splits will be skipped

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-11 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897493#action_12897493 ] Yan Zhou commented on PIG-1518: --- Right, map side cogroup needs the sortness of the input

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-11 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897496#action_12897496 ] Yan Zhou commented on PIG-1501: --- Please refer to HADOOP-3315 for overall Sequence File vs TFile

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897005#action_12897005 ] Yan Zhou commented on PIG-1501: --- The default is *not* using the compression on the intermediate

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: PIG-1501.patch need to investigate the impact of compression on pig performance

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-10 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12897085#action_12897085 ] Yan Zhou commented on PIG-1518: --- The pseudo code of the combination op is as follows: for each

[jira] Updated: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-09 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1501: -- Attachment: compress_perf_data_2.txt The data set in the last tests are small such that the performance

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-08-09 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12896620#action_12896620 ] Yan Zhou commented on PIG-1501: --- Unless there is any objection raised in the coming week, I'll

[jira] Updated: (PIG-1496) Mandatory rule ImplicitSplitInserter

2010-08-04 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1496: -- Attachment: PIG-1496.patch More comments in code per the reviewer's comment. Mandatory rule

[jira] Updated: (PIG-1496) Mandatory rule ImplicitSplitInserter

2010-08-04 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1496: -- Attachment: (was: PIG-1496.patch) Mandatory rule ImplicitSplitInserter

[jira] Updated: (PIG-1496) Mandatory rule ImplicitSplitInserter

2010-08-04 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1496: -- Status: Patch Available (was: Open) Mandatory rule ImplicitSplitInserter

[jira] Updated: (PIG-1496) Mandatory rule ImplicitSplitInserter

2010-08-04 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1496: -- Attachment: PIG-1496.patch Mandatory rule ImplicitSplitInserter

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-02 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894778#action_12894778 ] Yan Zhou commented on PIG-1518: --- In contrast with Hive, where the CombineFileInputFormat

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-07-30 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894205#action_12894205 ] Yan Zhou commented on PIG-1518: --- CombinedInputFormat, in lieu of the deprecated

[jira] Commented: (PIG-1501) need to investigate the impact of compression on pig performance

2010-07-29 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893746#action_12893746 ] Yan Zhou commented on PIG-1501: --- gzip and lzo2 are tried as the compression codecs; TFile

[jira] Updated: (PIG-1453) [zebra] Intermittent failure for TestOrderPreserveUnionHDFS

2010-07-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1453: -- Status: Resolved (was: Patch Available) Resolution: Fixed Committed to the trunk. [zebra

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-15 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: (was: PIG-1399.patch) Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-15 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-14 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Might not be appplicable to trunk yet as it depends upon a uncommitted patch

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-14 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: PIG-1399.patch Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-14 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1399: -- Attachment: (was: PIG-1399.patch) Logical Optimizer: Expression optimizor rule

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-13 Thread Yan Zhou
the JOIN (in the branch leading up to the source C) while ( (a3+b3 10) OR (c2 ==5) ) stays put below the JOIN. Please let me know if there is a way of doing the above optimization without converting the original expression to CNF. Thanks, Swati On Mon, Jul 12, 2010 at 4:26 PM, Yan Zhou y...@yahoo

RE: Add deepCopy in LogicalExpression

2010-07-13 Thread Yan Zhou
What I used in logic expression simplification, where a DNF plan is used, is a logical expression proxy that wraps up the original expression in the original plan. But it is not translated to physical plans, where I believe extra care would be needed. Yan -Original Message- From: Swati

[jira] Commented: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887912#action_12887912 ] Yan Zhou commented on PIG-1399: --- A couple of additional scenarios to be simplified: 5

[jira] Commented: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-13 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887923#action_12887923 ] Yan Zhou commented on PIG-1399: --- This work is not to optimize on the generic boolean logics

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
take a look and comment as you see appropriate. Thanks, Yan From: Swati Jain [mailto:swat...@aggiemail.usu.edu] Sent: Friday, July 09, 2010 11:00 PM To: Yan Zhou Cc: pig-dev@hadoop.apache.org Subject: Re: PIG Logical Optimization: Use CNF

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
be evaluated. Please let me know if I miss any of your points. Thanks, Yan From: Swati Jain [mailto:swat...@aggiemail.usu.edu] Sent: Monday, July 12, 2010 11:52 AM To: Yan Zhou Cc: pig-dev@hadoop.apache.org Subject: Re: PIG Logical Optimization: Use

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
On Mon, Jul 12, 2010 at 3:14 PM, Yan Zhou y...@yahoo-inc.com wrote: Hopefully by this week. I'm still in the debugging phase of the work. While you are welcome to reuse some of my algorithms, I doubt you can reuse the code as much as you want. It's basically for my DNF use. You might

  1   2   3   4   5   >