Folks,
Posting on the dev since this is regarding the new logical plan optimization
framework which is not enabled yet. I was interested in playing around with
the new optimization framework and try adding some simple rules to it.
I have attached two simple programs which do not work when the
Hi,
I am interested in implementing logical optimization rules and to target
this I have studied currently implemented logical rules and the rule
framework. In particular, I felt that rules dealing with LOfilter are not
able to handle complicated boolean expressions. I would like to share
-expressions.
Thanks,
Yan
-Original Message-
From: Swati Jain [mailto:swat...@aggiemail.usu.edu]
Sent: Monday, July 05, 2010 2:34 AM
To: pig-dev@hadoop.apache.org
Cc: Daniel Dai
Subject: PIG Logical Optimization: Use CNF in SplitFilter
Hi,
I am interested in implementing logical
,
Yan
--
*From:* Swati Jain [mailto:swat...@aggiemail.usu.edu]
*Sent:* Friday, July 09, 2010 11:00 PM
*To:* Yan Zhou
*Cc:* pig-dev@hadoop.apache.org
*Subject:* Re: PIG Logical Optimization: Use CNF in SplitFilter
Hi Yan,
I agree that the first scenario
.
Thanks,
Yan
--
*From:* Swati Jain [mailto:swat...@aggiemail.usu.edu]
*Sent:* Monday, July 12, 2010 11:52 AM
*To:* Yan Zhou
*Cc:* pig-dev@hadoop.apache.org
*Subject:* Re: PIG Logical Optimization: Use CNF in SplitFilter
I was wondering if you are not going
to support scenario 1, which will push up all possible
filters closer to source and have all benefits on pruned I/O, we should
not use CNF.
Thanks,
Yan
-Original Message-
From: Swati Jain [mailto:swat...@aggiemail.usu.edu]
Sent: Monday, July 12, 2010 4:04 PM
To: pig-dev
Hi,
I am working on ticket PIG -1494 (
https://issues.apache.org/jira/browse/PIG-1494 ).
While implementing this functionality (conversion of logical expression into
CNF), I need to construct the OperatorPlan for the base expressions of the
CNF. For example, given an expression (c1 10) AND
?
Alan.
On Jul 12, 2010, at 11:19 PM, Swati Jain wrote:
Hi,
I am working on ticket PIG -1494 (
https://issues.apache.org/jira/browse/PIG-1494 ).
While implementing this functionality (conversion of logical expression
into
CNF), I need to construct the OperatorPlan for the base expressions
I noticed a number of places in the code where the successors of a
LogicalRelationalOperator is accessed as op.successors.get(0). Is it
always the case that logical relational operators (in the new logical
optimizer framework) have only 1 successor? Why dont the rules iterate over
the successors
[
https://issues.apache.org/jira/browse/PIG-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887425#action_12887425
]
Swati Jain commented on PIG-1494:
-
Reply from Yan Zhou:
The filter logic split problem can
Affects Versions: 0.7.0
Reporter: Swati Jain
Priority: Minor
Fix For: 0.8.0
The PushUpFilter rule is not able to handle complicated boolean expressions.
For example, SplitFilter rule is splitting one LOFilter into two by AND.
However it will not be able to split
Reporter: Swati Jain
Assignee: Swati Jain
Fix For: 0.8.0
It would be useful to have a way to `deepCopy` an expression. `deepCopy` will
create a new object so that changes made to one object will not reflect in the
copy. There are 2 reasons why we don't override
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain reassigned PIG-1512:
---
Assignee: Swati Jain
PlanPrinter does not print LOJoin operator in the new logical optimization
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Attachment: printJoin.patch
PlanPrinter does not print LOJoin operator in the new logical optimization
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Attachment: printJoin.patch
Fix tab character
PlanPrinter does not print LOJoin operator in the new
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Attachment: printJoin.patch
Attach the right file, final upload.
PlanPrinter does not print LOJoin
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on PIG-1512 started by Swati Jain.
PlanPrinter does not print LOJoin operator in the new logical optimization
framework
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Swati Jain
Fix For: 0.8.0
Attachments: printJoin.patch
PlanPrinter does not print LOJoin relational operator. As such, the LOJoin
operator would not get printed when we do an explain
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Attachment: (was: printJoin.patch)
PlanPrinter does not print LOJoin operator in the new logical
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Attachment: (was: printJoin.patch)
PlanPrinter does not print LOJoin operator in the new logical
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Patch Info: [Patch Available]
PlanPrinter does not print LOJoin operator in the new logical optimization
[
https://issues.apache.org/jira/browse/PIG-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1512:
Status: Patch Available (was: In Progress)
PlanPrinter does not print LOJoin operator in the new logical
: impl
Reporter: Swati Jain
Assignee: Swati Jain
Priority: Minor
Fix For: 0.8.0
Consider the following:
{noformat}
A = load 'any file' USING PigStorage(',') as (a1:int,a2:int,a3:int);
B = load 'any file' USING PigStorage(',') as (b1:int,b2:int,b3:int
[
https://issues.apache.org/jira/browse/PIG-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894467#action_12894467
]
Swati Jain commented on PIG-1530:
-
a) This is not a developer coding issue. The example I
[
https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1510:
Patch Info: [Patch Available]
Add `deepCopy` for LogicalExpressions
[
https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1510:
Attachment: deepCopy.patch
Revised patch after merging changes. Also wrote a unit test.
Add `deepCopy
[
https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1510:
Status: Patch Available (was: In Progress)
Add `deepCopy` for LogicalExpressions
[
https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1510:
Status: In Progress (was: Patch Available)
Add `deepCopy` for LogicalExpressions
[
https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899282#action_12899282
]
Swati Jain commented on PIG-1510:
-
We make a copy of all instance variables, including
Affects Versions: 0.8.0
Reporter: Swati Jain
Assignee: Swati Jain
Fix For: 0.8.0
Provide utility to construct CNF form of predicates
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[
https://issues.apache.org/jira/browse/PIG-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899785#action_12899785
]
Swati Jain commented on PIG-1549:
-
Details about the benefits are listed in PIG-1494
[
https://issues.apache.org/jira/browse/PIG-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1549:
Attachment: 0001-Add-CNF-utility-class.patch
Added the patch file generated by git.
Provide utility
[
https://issues.apache.org/jira/browse/PIG-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swati Jain updated PIG-1549:
Status: Patch Available (was: Open)
Provide utility to construct CNF form of predicates
33 matches
Mail list logo
