Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Yingyi Bu (Code Review)]

Yingyi Bu has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 13:

(5 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1119/13//COMMIT_MSG
Commit Message:

Line 7: ASTERIXDB-1487: fix the wrong plan when we purning the selective branch.
purning -> prune


Line 10: 2. Disable the join branch pruning in case of unnestmap following 
datasourcescan.
"unnestmap" -> "operators that do not preserve cardinality"


https://asterix-gerrit.ics.uci.edu/#/c/1119/13/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/RemoveUnusedOneToOneEquiJoinRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/RemoveUnusedOneToOneEquiJoinRule.java:

Line 120: private int removeJoinFromInputBranch(Mutable 
opRef, int startBranch) throws AlgebricksException {
Is variable startBranch necessary?
Why it didn't work without this parameter?


Line 196: return removeJoinFromInputBranch(opRef, 1);
What's the purpose of the recursive call here?


Line 202: AbstractLogicalOperator op = (AbstractLogicalOperator) 
opRef.getValue();
Call op.getOperatorTag() once after line 202.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 13: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/695/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 13:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/695/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Wenhai Li (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1119

to look at the new patch set (#13).

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..

ASTERIXDB-1487: fix the wrong plan when we purning the selective branch.

1. Add the test case of ASTERIX-1487 with half records required.
2. Disable the join branch pruning in case of unnestmap following 
datasourcescan.
   - We need to prune the join branch when it is NOT required by the upstream 
operators and its generated join key is derived from the same DATASOURCE of the 
other branch.
   - We SHOULD NOT prune the join branch if there exists a selective operator 
(UNNESTMAP, LOUNNESTMAP, LIMIT, SELECT) located between the join operator and 
DATASOURCESCAN.

Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/RemoveUnusedOneToOneEquiJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
8 files changed, 182 insertions(+), 28 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/19/1119/13
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2723/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Wenhai Li (Code Review)]

Wenhai Li has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 12:

The condition 3 has been implicitly checked by gatherProducingDataScans, where 
the dataSourceScanOperator is added onto the list before we verify the join 
condition contains the PRIMARY KEY of this dataSource. It means that the join 
condition on the same dataSource will definitely join on the same field 
(PRIMARY KEY determines anything.).

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Taewoo Kim (Code Review)]

Taewoo Kim has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 12:

I discussed this with Wenhai. 

For a join such as A.x = B.y, we need to make sure that
1) The datasourcescan of A and B refers to the same dataset (A,B = the same 
dataset)
2) Operators after the join do not use any variables from one branch.
3) The fields X and Y should be the same field

I am not sure whether the current rule checks the condition 3) properly.

Wenhai also mentioned two issues:
#a. We have a SELECT condition besides to the above (e.g., A.x = B.y and A.z < 
100). In this case, We have a datasource scan, followed by a SELECT, then the 
join op. We can't prune the A branch since if we do so, A.z can't be checked. A 
solution could be moving A.z < 100 after the join operator and change it to B.z 
< 100. 

#b. For surrogate join case that utilizes a secondary inverted index-search, 
this rule always prune the inner branch (which does main job). How to let this 
recognize the inner branch is an issue.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2708/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1487: fix the wrong plan when we purning the selec...

[Wenhai Li (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1119

to look at the new patch set (#12).

Change subject: ASTERIXDB-1487: fix the wrong plan when we purning the 
selective branch.
..

ASTERIXDB-1487: fix the wrong plan when we purning the selective branch.

1. Add the test case of ASTERIX-1487 with half records required.
2. Disable the join branch pruning in case of unnestmap following 
datasourcescan.
   - We need to prune the join branch when it is NOT required by the upstream 
operators and its generated join key is derived from the same DATASOURCE of the 
other branch.
   - We SHOULD NOT prune the join branch if there exists a selective operator 
(UNNESTMAP, LOUNNESTMAP, LIMIT, SELECT) located between the join operator and 
DATASOURCESCAN.

Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/RemoveUnusedOneToOneEquiJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fuzzyjoin/dblp-csx-aqlplus_5/dblp-csx-aqlplus_5.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
7 files changed, 154 insertions(+), 14 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/19/1119/12
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1119
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1aef69a2278853fd9f8020da6639331b367ed5ad
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Yingyi Bu