Change in asterixdb[master]: [ASTERIXDB-2103][STO] Too many disk components for Correlate...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2103][STO] Too many disk components for 
CorrelatedPolicy
..


Patch Set 17: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2018
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Gerrit-PatchSet: 17
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2103][STO] Too many disk components for Correlate...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2103][STO] Too many disk components for 
CorrelatedPolicy
..


Patch Set 16: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2018
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2119][COMP] Fix variable ordering of project push...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2119][COMP] Fix variable ordering of project pushdown
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2048
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I21d0a2764e22482a9361b709e566d78acd33cf4d
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2119][COMP] Fix variable ordering of project push...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2119][COMP] Fix variable ordering of project pushdown
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/2048/3/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushProjectDownRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushProjectDownRule.java:

Line 81: private static Pair 
pushThroughOp(Set toPush, Mutable opRef2,
> MAJOR SonarQube violation:
Reformating issue?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2048
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I21d0a2764e22482a9361b709e566d78acd33cf4d
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2103][STO] Too many disk components for Correlate...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2103][STO] Too many disk components for 
CorrelatedPolicy
..


Patch Set 8:

(4 comments)

I left some minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/2018/8/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIndexAccessor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIndexAccessor.java:

PS8, Line 45: merge
"merge" -> "flush"?


https://asterix-gerrit.ics.uci.edu/#/c/2018/8/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIndexOperationContext.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIndexOperationContext.java:

PS8, Line 60: 
It looks a little weird to add the `getDependingOps` to the `context` 
interface. Any reasons to add the method here?


https://asterix-gerrit.ics.uci.edu/#/c/2018/8/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractIoOperation.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractIoOperation.java:

Line 58: isFinished.set(true);;
double ;; ?


https://asterix-gerrit.ics.uci.edu/#/c/2018/8/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexAccessor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexAccessor.java:

Line 89: ctx.getDependingOps().clear();
line 89 to 92 are used in many places. Maybe a `setDependingOps()` would save 
some efforts?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2018
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2103][STO] Too many disk components for Correlate...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2103][STO] Too many disk components for 
CorrelatedPolicy
..


Patch Set 3:

@Luo any ideas about the "cb-jenkins" test fail? I think you can ask @Adbullah 
about details.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2018
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2103][STO] Too many disk components for Correlate...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-2103][STO] Too many disk components for 
CorrelatedPolicy
..


Patch Set 2:

(2 comments)

I have two minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/2018/2/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMHarness.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMHarness.java:

Line 110:  */
add a comment about the returned value?


https://asterix-gerrit.ics.uci.edu/#/c/2018/2/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/MergeOperation.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/MergeOperation.java:

PS2, Line 81: 
should it only be set by itself? then maybe make it private?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2018
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for Corre...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for 
Correlated Datasets
..


Patch Set 6:

Cloudberry currently only need an inverted index. So I didn't test other 
indexes. I think it will be a good experiment to do and research about the 
performance when you have more time :-)
We can improve it in separate patches.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1845
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I100fc0b86b8a6fa36a95d77806107bad0307544e
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for Corre...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for 
Correlated Datasets
..


Patch Set 6:

@Chenluo,  not much about this patch on the code side. I want to update some 
finding after using this patch on my test data.

1. Too many components? 
There are a lot more 2ndary components than prefix-policy generated. In one 
partition I have 557 inverted index components (there are 282 primary index 
components), and half of them are very tiny.  (e.g., 2M). Previously we only 
have 60 inverted indexes. 
2. Performance is a bit slower than the prefix policy. (?)
I did a simple count test for tweet contains "election" and "happy".
 prefix  correlated
election88s  93s
happy  190s 241.351s

The performance is slower than prefix policy which contradicts with our 
conjecture. Maybe it also related to too many 2dnary components?

It's not an objection about this patch. I think we still need to merge this one 
to complete the "correlated" policy. After that, we need more performance test 
and deeper analysis to improve it in the future patches. :-)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1845
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I100fc0b86b8a6fa36a95d77806107bad0307544e
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for Corre...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-1946][STO][IDX]Create RTree/InvertedIdx for 
Correlated Datasets
..


Patch Set 5: Code-Review+1

Maybe Abdullah or Yingyi can have a pass finally?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1845
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I100fc0b86b8a6fa36a95d77806107bad0307544e
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1952][TX][IDX]Log incoming filter vals

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-1952][TX][IDX]Log incoming filter vals
..


Patch Set 10: Code-Review+1

I see. I will give a +1. I think the upsert issue as Murtadha's pointed out 
could be another CR. I will let him decide. :-)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1798
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I9268fe0b60145545c5933bab698d651c324397d7
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1952][TX][IDX]Log incoming filter vals

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: [ASTERIXDB-1952][TX][IDX]Log incoming filter vals
..


Patch Set 10:

(1 comment)

why is shows can't merge?

https://asterix-gerrit.ics.uci.edu/#/c/1798/5/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTree.java:

PS5, Line 824: 
should this function general for other indexes? or is just special for BTree?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1798
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I9268fe0b60145545c5933bab698d651c324397d7
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Avoid always merging old components in prefix policy

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Avoid always merging old components in prefix policy
..


Patch Set 5:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1818/5/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java:

Line 310: if (mergable) {
> MAJOR SonarQube violation:
I think this comment is valid. 

Now there are too many if-else blocks that are actually confusing, can you 
reduce it?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1818
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Avoid always merging old components in prefix policy

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Avoid always merging old components in prefix policy
..


Patch Set 4:

(7 comments)

some minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/1818/4/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/PrefixMergePolicy.java:

Line 49: private final static double MAX_MERGABLE_COMPONENT_SIZE_RATIO = 
1.2;
maybe it's good to also put the source of this magic number?


Line 143: Collections.reverse(immutableComponents);
could you explain the reason of the reverse?


Line 256: private Pair 
getMergableComponentsIndex(List immutableComponents) {
I think it's better to move this big chunk of comment before the function as 
Java doc?


Line 266: || immutableComponents.get(i).getState() != 
ComponentState.READABLE_UNWRITABLE) {
just curious, to make sure the state check is consistent, is this function (or 
its caller) synchronized or there is only one merge thread who also takes care 
of updating the state?


Line 276: // normally this shouldn't happen, but just make 
sure we skip unmergeable components
if it's a suspicious state, it's better to add a warning log.


Line 284: boolean isLastComponent = j + 1 == 
immutableComponents.size() ? true : false;
no need to use ternary?


Line 289: return Pair.of(i, j);
it feels like we should treat two part of the && differently for the else case? 

if `!canMerge` [i,j], we just increase j, 
if `canMerge` [i,j], 
if `shouldMerge[i,j]`, return
else increase i  and break

correct me if I'm wrong :-)


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1818
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I464da3fed38cded0aee7b319a35664eae069a2ba
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Jianfeng Jia has submitted this change and it was merged.

Change subject: Change logical plan to apply filter from 2ndary index
..


Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1727
Integration-Tests: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Reviewed-by: Yingyi Bu 
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#17).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Change logical plan to apply filter from 2ndary index
..


Patch Set 15:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1727/15/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java:

PS15, Line 166: ArrayList
> ArrayList -> List
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/15/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java:

PS15, Line 126:public void setExtraVariables(List 
extraVars, List extraOutputVars) {
  : this.extraVars = extraVars;
  : for (int i = 0; i < extraVars.size(); i++) {
  : this.inputVars.get(i).addAll(extraVars.get(i));
  : }
  : this.outputVars.addAll(extraOutputVars);
  : }
> Can we eliminate the setter and instead, add a constructor with these two a
Done. 

I created a new IntersectOperator in the IntroduceLSMFilterRule.


https://asterix-gerrit.ics.uci.edu/#/c/1727/15/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/intersect/IntersectOperatorDescriptor.java
File 
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/intersect/IntersectOperatorDescriptor.java:

PS15, Line 109:  throw new HyracksException("The given input extra fields is 
not equal");
> Error code?
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1727
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#16).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#13).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Change logical plan to apply filter from 2ndary index
..


Patch Set 12:

(27 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java:

PS11, Line 118: propagateIndexFilter();
> getPropageIndexFilter -> propagateIndexFilter
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java:

PS11, Line 66: getScanVariables
> Can it be just as.getVariables()?
The filter values are appended if using IntroducingFilterRule. For the case of 
the scan, I think we should not deliver those filter variables.


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java:

PS11, Line 125: 
> It looks that propagateIndexFilter can be retrieved from unnestMapOp so you
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java:

Line 226: 
> MAJOR SonarQube violation:
it won't be as clear as it of now.


PS11, Line 244: (Unnest
> Generalize the filter propagation using a loop?  In other words, make the c
I just found an issue for the intersection case, which requires NOT to compare 
the filter field, but has to carry the value out of the pipeline. 

so I think the code has to be operator specific then?


PS11, Line 271: ersect.
> Throw an exception?
Done


PS11, Line 313: 
> ArrayList -> List
Done


PS11, Line 317: 
> cast is not needed:
Done


PS11, Line 319: computeA
> It should never be null?
Done


PS11, Line 328: ndantOp = queue.poll(
> error code
Done


Line 333: AbstractFunctionCallExpression f = 
(AbstractFunctionCallExpression) unnestExpr;
> MAJOR SonarQube violation:
it won't be as clear as it of now.


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql:

PS11, Line 36:  in da
> write statement is not needed.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql:

PS11, Line 36: time("
> write statement is not needed.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql:

PS11, Line 36: 
> write statement is not needed.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql:

PS11, Line 36: s_star
> write statement is not needed.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql:

PS11, Line 38: rea
> write statement is not needed
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql:

PS11, Line 38: egion 
> write statement is not needed.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql:

PS11, Line 36: egion 
> write statement is not needed
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/11/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
File 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#12).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#10).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Change logical plan to apply filter from 2ndary index
..


Patch Set 9:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-app/data/twitter/real.2.adm
File asterixdb/asterix-app/data/twitter/real.2.adm:

Line 1: { "create_at": datetime("2016-01-02T10:00:00.000Z"), "id": 
683347019176501248, "text": "@e_tomkiewicz o I know", "in_reply_to_status": 
683306019192176642, "in_reply_to_user": 379427210, "favorite_count": 0, 
"retweet_count": 0, "lang": "en", "is_retweet": false, "user_mentions": {{ 
379427210 }}, "user": { "id": 51597840, "name": "Sydney Isroff", "screen_name": 
"SydneyIsroff", "lang": "en", "location": "null", "create_at": 
date("2009-06-27"), "description": "Kids are allowed but not welcomed", 
"followers_count": 408, "friends_count": 78, "statues_count": 13278 }, "place": 
{ "country": "United States", "country_code": "United States", "full_name": 
"Milwaukee, WI", "id": "2a93711775303f90", "name": "Milwaukee", "place_type": 
"city", "bounding_box": rectangle("-88.070827,42.920822 -87.863758,43.192623") 
}, "geo_tag": { "stateID": 55, "stateName": "Wisconsin", "countyID": 55079, 
"countyName": "Milwaukee", "cityID": 5553000, "cityName": "Milwaukee" } }
> Do we need 10,000 tuples in a test case?
reduced to 5000


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-app/data/twitter/real.adm
File asterixdb/asterix-app/data/twitter/real.adm:

Line 1: { "create_at": datetime("2015-11-23T16:14:03.000Z"), "id": 
668945640186101761, "text": "Just posted a photo @ Campus Martius Park 
https://t.co/5Ax4E2CdWZ;, "in_reply_to_status": -1, "in_reply_to_user": -1, 
"favorite_count": 0, "coordinate": point("-83.04647491,42.33170228"), 
"retweet_count": 0, "lang": "en", "is_retweet": false, "user": { "id": 
48121888, "name": "Kevin McKague", "screen_name": "KevinOfMI", "lang": "en", 
"location": "Davison, Michigan", "create_at": date("2009-06-17"), 
"description": "I need to ride my bike until my brain shuts up and my muscles 
are screaming. \nRight after these donuts. Dad of 3.\n Visit my blog 18 Wheels 
and a 12-Speed Bike.", "followers_count": 1178, "friends_count": 1780, 
"statues_count": 22263 }, "place": { "country": "United States", 
"country_code": "United States", "full_name": "Detroit, MI", "id": 
"b463d3bd6064861b", "name": "Detroit", "place_type": "city", "bounding_box": 
rectangle("-83.288056,42.255085 -82.91052,42.450488") 
 }, "geo_tag": { "stateID": 26, "stateName": "Michigan", "countyID": 26163, 
"countyName": "Wayne", "cityID": 2622000, "cityName": "Detroit" } }
> Do we need 20,000 tuples in a test case?
reduced to 5000


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1727
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Ian Maxon, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#9).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Change logical plan to apply filter from 2ndary index
..


Patch Set 7:

(9 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1727/7/.gitignore
File .gitignore:

> Why change this file?
removed.


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java:

Line 355: // TODO look at here about the variables
> Can you be more specific about this comment?
Removed.


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java:

PS7, Line 262: 
> Can these two switch() be combined?
Done


Line 291: private void propagateFilterInSecondaryUnnest(UnnestMapOperator 
secondaryUnnest, IAType filterType,
> propagateFilterInSecondaryUnnsetMap?
Done


Line 305: private void setFilterValueFromSecondaryUnnest(UnnestMapOperator 
primaryOp,
> setFilterValueFromSecondaryUnnestMap?
Done


PS7, Line 320: assignFilterFromSecondaryUnnest
> This function is not being used? Could it be deleted?
we used it in line 126.


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql:

Line 1: /*
> Where did you include this test?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql:

Line 1: /*
> It's better to add the SQLPP version of the tests?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1727/7/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java:

Line 26: import org.apache.hyracks.api.dataflow.value.*;
> Our principle is not doing *? Isn't it? Maybe Till needs to check this.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1727
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#8).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.7.server.aql
A 

Change in asterixdb[master]: ASTERIXDB-1905: Incorrect filter for post-load sidx

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1905: Incorrect filter for post-load sidx
..


Patch Set 4: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1743
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1905: Incorrect filter for post-load sidx

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1905: Incorrect filter for post-load sidx
..


Patch Set 3: Code-Review+1

LGTM. I think we should also fix that two formatting issues?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1743
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1905: Incorrect filter for post-load sidx

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1905: Incorrect filter for post-load sidx
..


Patch Set 2:

any reason for the verify fail?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1743
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#4).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M .gitignore
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A asterixdb/asterix-app/data/twitter/real.2.adm
A asterixdb/asterix-app/data/twitter/real.adm
M asterixdb/asterix-app/src/main/resources/asterix-build-configuration.xml
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.5.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.6.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.7.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.8.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/delete/delete.9.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.10.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.11.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/filters/upsert/upsert.5.server.aql
A 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#3).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DatasetDataSource.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/AbstractScanOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/AbstractUnnestMapOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestMapOperator.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
28 files changed, 747 insertions(+), 47 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/27/1727/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1727
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Change logical plan to apply filter from 2ndary index

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1727

to look at the new patch set (#2).

Change subject: Change logical plan to apply filter from 2ndary index
..

Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to
replace the constant filter value from the query to the value
carried from 2ndary index search.
- Can use 2ndary index filter even the query doens't contain
any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IndexSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/btree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/multi-index-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search-wo-query-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/rtree-btree-search.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/btree-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/multi-index-btree-search.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search-wo-query-filter.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/rtree-btree-search.plan
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/DatasetDataSource.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/AbstractScanOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/AbstractUnnestMapOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestMapOperator.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
28 files changed, 747 insertions(+), 47 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/27/1727/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1727
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Carry filter in 2ndary-to-primary index search
..


Patch Set 7:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1720/7/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java:

PS7, Line 34: DataSetConstants
> expands imports
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/7/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java:

PS7, Line 48: .*;
> expands imports
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/7/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java:

PS7, Line 48: DataSetConstants
> expands imports
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1720
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Yingyi Bu, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#8).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Carry filter in 2ndary-to-primary index search
..


Patch Set 6:

(20 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1720/6/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java:

PS6, Line 358: here
> What does this TODO item mean?
removed.


PS6, Line 358: // TODO look at here about the variables
> Is this TODO necessary to this patch? Or just a TODO marker on local branch
removed.


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java:

PS6, Line 66: .*;
> expand the imports?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java:

PS6, Line 43: .*;
> expand import?
Done


PS6, Line 101: createSecondaryDataFlowHelperFactory
> Is it necessary have two different methods that share the same code?
The main requirement is to give the valid btreeFields and the filterFields if 
filter appears. I refactor the code to add these two as parameters for one 
interface function instead of two interfaces.


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java:

PS6, Line 44: DataSetConstants
> expand import?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java:

PS6, Line 26: .*;
> expand imports
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java:

PS6, Line 82: maxFilterFieldIndexes
> if appendFilter is false, should we still have the two fields minFilterFiel
These two types of filters are orthogonal. `minFilterFieldIndexes` and 
`maxFilterFieldIndexes` are indexes of the filter values in the input frame. 
They are given by the query parameters.

`appendFilter` is to indicate if we need to append the index component filter 
values to the **output** frame.


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexDataflowHelper.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexDataflowHelper.java:

PS6, Line 44:  public int getNumFilterFields();
> Is this method used for supporting multiple filters on an index?
No. It returns the number of fields for a filter value. It mainly used for 
writing null filter values to the output. since we don't know how many fields 
should we write.


https://asterix-gerrit.ics.uci.edu/#/c/1720/6/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java:

PS6, Line 266:  int[] offsets = nonFilterTupleBuild.getFieldEndOffsets();
 : for (int i = 0; i < offsets.length; i++) {
 : int start = i > 0 ? offsets[i - 1] : 0;
 : tb.addField(nonFilterTupleBuild.getByteArray(), 
start, offsets[i]);
 : }
> Why would filter tuple == null?
In Hyracks, the outputRecordDescriptor (i.e. how many fields in the record 
after an operator) is predefined. So if it said `append`, there must be a field 
waiting to be filled up. If we skip it, the bytearray will be messed up. 

We rely on the compiler to make a 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#7).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorNodePushable.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Carry filter in 2ndary-to-primary index search
..


Patch Set 6:

(6 comments)

@Yingyi @Ian, this is patch expose the filter value and pass it along with 
index record in the dataflow pipeline.

The next patch will address the logical plan rule changes to create 
corresponding hyracks jobs.

https://asterix-gerrit.ics.uci.edu/#/c/1720/1/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java:

Line 276: for (int i = 0; i < numFields; i++) {
> CRITICAL SonarQube violation:
Done


Line 277: try {
> CRITICAL SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/4/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java:

Line 278: nonMatchWriter.writeMissing(out);
> CRITICAL SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java:

Line 244: if (foundTuple) {
> MAJOR SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeWithBuddySortedCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeWithBuddySortedCursor.java:

Line 86: if (foundIn < 0) {
> MAJOR SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/1720/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexSearchCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndexSearchCursor.java:

Line 208: if (accessorIndex < 0) {
> MAJOR SonarQube violation:
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1720
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#6).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/rtree/LSMRTreeWithAntiMatterTuplesSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/RTreeSecondaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#5).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeUpdateSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeCountingSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/test/java/org/apache/hyracks/storage/am/btree/test/FramewriterTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexDataflowHelper.java
D 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/ITreeIndexOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexDataflowHelper.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/impls/TreeIndexDiskOrderScanCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#4).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/common/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeUpdateSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeCountingSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/test/java/org/apache/hyracks/storage/am/btree/test/FramewriterTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexDataflowHelper.java
D 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/ITreeIndexOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexDataflowHelper.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/impls/TreeIndexDiskOrderScanCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1720

to look at the new patch set (#3).

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/common/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeUpdateSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeCountingSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/test/java/org/apache/hyracks/storage/am/btree/test/FramewriterTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexDataflowHelper.java
D 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/ITreeIndexOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexDataflowHelper.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/impls/TreeIndexDiskOrderScanCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
M 

Change in asterixdb[master]: Carry filter in 2ndary-to-primary index search

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1720

Change subject: Carry filter in 2ndary-to-primary index search
..

Carry filter in 2ndary-to-primary index search

Change-Id: I287f1dbd230aa649f1350114abf0a1d47e2bb53c
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/AbstractBTreeOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest.java
A 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/common/DataSetConstants.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexScanOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreePrimaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexInsertOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeSecondaryIndexSearchOperatorTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeUpdateSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeCountingSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/impls/BTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/test/java/org/apache/hyracks/storage/am/btree/test/FramewriterTest.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/api/IIndexCursor.java
D 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/ITreeIndexOperatorDescriptor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexDataflowHelper.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/TreeIndexDataflowHelper.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/impls/TreeIndexDiskOrderScanCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreePointSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
M 

Change in asterixdb[master]: Hyracks Unit tests for LSM Filters

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Hyracks Unit tests for LSM Filters
..


Patch Set 10: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1704
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie0e79c0a25d5d6be727b9ebad841801024c00e94
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Hyracks Unit tests for LSM Filters

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Hyracks Unit tests for LSM Filters
..


Patch Set 7: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1704
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie0e79c0a25d5d6be727b9ebad841801024c00e94
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Hyracks Unit tests for LSM Filters

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Hyracks Unit tests for LSM Filters
..


Patch Set 3:

(1 comment)

do we have the corresponding RTree/InvertedIndex merge filter test?

https://asterix-gerrit.ics.uci.edu/#/c/1704/3/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeFilterMergeTestDriver.java
File 
hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeFilterMergeTestDriver.java:

Line 90: List> minMaxes = new 
ArrayList<>();
could you modularize the following several for loop into several functions? 
e.g., `createFilter()`, `mergeFilters()`, `mergeComponents`, etc. I think soon 
we will reuse these functions.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1704
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie0e79c0a25d5d6be727b9ebad841801024c00e94
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Document the Query Service API

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Document the Query Service API
..


Patch Set 3: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1698
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcefb1671ea305a5958c9a74a588b4aaa17f399f
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: Document the Query Service API

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Document the Query Service API
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1698/1/asterixdb/asterix-doc/src/site/markdown/api.md
File asterixdb/asterix-doc/src/site/markdown/api.md:

PS1, Line 47: client_context_id
could we have more explanation about the `client_context_id` and its usage? I 
only know one use case of canceling a running query.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1698
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifcefb1671ea305a5958c9a74a588b4aaa17f399f
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix for ASTERIXDB-1886: Filter not selective

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1886: Filter not selective
..


Patch Set 2:

Any test case we can add to make filter safer?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1691
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I123438c7fe5fba64aff4e88b1bda0e2be1a32574
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix for ASTERIXDB-1879

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1879
..


Patch Set 7: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1672
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I63734838235e86ee9f0127663f73f65444c423bb
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Blow 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix for ASTERIXDB-1879

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1879
..


Patch Set 6:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1672/6/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/LSMComponentFilterReferenceTest.java
File 
hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/LSMComponentFilterReferenceTest.java:

PS6, Line 43: 
just curious, why not Assert.equal(filter.getLength(), 11) ?


Line 47: }
do we have the corresponding loading logic so that we can simulate the path of 
reading from a bytearray or something else?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1672
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I63734838235e86ee9f0127663f73f65444c423bb
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Blow 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Print filter information in logical plan

[Jianfeng Jia (Code Review)]

Jianfeng Jia has submitted this change and it was merged.

Change subject: Print filter information in logical plan
..


Print filter information in logical plan

Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1673
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Yingyi Bu 
BAD: Jenkins 
Reviewed-by: Ian Maxon 
Integration-Tests: Jenkins 
---
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
1 file changed, 18 insertions(+), 2 deletions(-)

Approvals:
  Yingyi Bu: Looks good to me, approved
  Ian Maxon: Looks good to me, but someone else must approve
  Jenkins: Verified; No violations found; No violations found; Verified



diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
index 71ac8f3..2139627 100644
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
@@ -313,15 +313,31 @@
 
 private Void printAbstractUnnestMapOperator(AbstractUnnestMapOperator op, 
Integer indent, String opSignature)
 throws AlgebricksException {
-addIndent(indent).append(opSignature + " " + op.getVariables() + " <- "
+AlgebricksAppendable plan = addIndent(indent).append(opSignature + " " 
+ op.getVariables() + " <- "
 + op.getExpressionRef().getValue().accept(exprVisitor, 
indent));
+appendFilterInformation(plan, op.getMinFilterVars(), 
op.getMaxFilterVars());
 return null;
 }
 
 @Override
 public Void visitDataScanOperator(DataSourceScanOperator op, Integer 
indent) throws AlgebricksException {
-addIndent(indent).append(
+AlgebricksAppendable plan = addIndent(indent).append(
 "data-scan " + op.getProjectVariables() + "<-" + 
op.getVariables() + " <- " + op.getDataSource());
+appendFilterInformation(plan, op.getMinFilterVars(), 
op.getMaxFilterVars());
+return null;
+}
+
+private Void appendFilterInformation(AlgebricksAppendable plan, 
List minFilterVars,
+List maxFilterVars) throws AlgebricksException {
+if (minFilterVars != null || maxFilterVars != null) {
+plan.append(" with filter on");
+}
+if (minFilterVars != null) {
+plan.append(" min:" + minFilterVars);
+}
+if (maxFilterVars != null) {
+plan.append(" max:" + maxFilterVars);
+}
 return null;
 }
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1673
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Print filter information in logical plan

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Print filter information in logical plan
..


Patch Set 2:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1673/2//COMMIT_MSG
Commit Message:

PS2, Line 9: working
> Remove local commit log from commit msg?
Done


https://asterix-gerrit.ics.uci.edu/#/c/1673/2/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java:

PS2, Line 330: printFilterInformation
> printFilterInformation
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1673
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Print filter information in logical plan

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1673

to look at the new patch set (#3).

Change subject: Print filter information in logical plan
..

Print filter information in logical plan

Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
---
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
1 file changed, 18 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/73/1673/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1673
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Fix for ASTERIXDB-1879

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1879
..


Patch Set 1:

I'm thinking an easier unit test. E.g., we can write a filter information to a 
page and read it later to see if we can get the correct information?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1672
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I63734838235e86ee9f0127663f73f65444c423bb
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Print filter information in logical plan

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1673

to look at the new patch set (#2).

Change subject: Print filter information in logical plan
..

Print filter information in logical plan

The following commits from your working branch will be included:

commit c3b9a8d2a8b7782290743e465c8ebdb5270d1ea6
Author: Jianfeng Jia 
Date:   Mon Apr 10 23:16:30 2017 -0700

print filter information inside the logical plan

Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
---
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
1 file changed, 18 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/73/1673/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1673
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Print filter information in logical plan

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1673

Change subject: Print filter information in logical plan
..

Print filter information in logical plan

The following commits from your working branch will be included:

commit c3b9a8d2a8b7782290743e465c8ebdb5270d1ea6
Author: Jianfeng Jia 
Date:   Mon Apr 10 23:16:30 2017 -0700

print filter information inside the logical plan

Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
---
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
1 file changed, 18 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/73/1673/1

diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
index 71ac8f3..81e18c1 100644
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
@@ -313,15 +313,31 @@
 
 private Void printAbstractUnnestMapOperator(AbstractUnnestMapOperator op, 
Integer indent, String opSignature)
 throws AlgebricksException {
-addIndent(indent).append(opSignature + " " + op.getVariables() + " <- "
+AlgebricksAppendable plan = addIndent(indent).append(opSignature + " " 
+ op.getVariables() + " <- "
 + op.getExpressionRef().getValue().accept(exprVisitor, 
indent));
+printFilterInformation(plan, op.getMinFilterVars(), 
op.getMaxFilterVars());
 return null;
 }
 
 @Override
 public Void visitDataScanOperator(DataSourceScanOperator op, Integer 
indent) throws AlgebricksException {
-addIndent(indent).append(
+AlgebricksAppendable plan = addIndent(indent).append(
 "data-scan " + op.getProjectVariables() + "<-" + 
op.getVariables() + " <- " + op.getDataSource());
+printFilterInformation(plan, op.getMinFilterVars(), 
op.getMaxFilterVars());
+return null;
+}
+
+private Void printFilterInformation(AlgebricksAppendable plan, 
List minFilterVars,
+List maxFilterVars) throws AlgebricksException {
+if (minFilterVars != null || maxFilterVars != null) {
+plan.append(" with filter on");
+}
+if (minFilterVars != null){
+plan.append(" min:" + minFilterVars);
+}
+if (maxFilterVars != null) {
+plan.append(" max:" + maxFilterVars);
+}
 return null;
 }
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1673
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8c3893fc54337695df4837e34ad7f7e89758718e
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 

Change in asterixdb[master]: Fix for ASTERIXDB-1879

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1879
..


Patch Set 1:

so this means we have to re-ingest the data since the filter is not write 
correctly 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1672
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I63734838235e86ee9f0127663f73f65444c423bb
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix for ASTERIXDB-1879

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1879
..


Patch Set 1:

should we have some sort of filter related unit test?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1672
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I63734838235e86ee9f0127663f73f65444c423bb
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1877: Tokenizer in FullText Search fix

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1877: Tokenizer in FullText Search fix
..


Patch Set 4: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1668
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic949ccc0cc3e66c489f4c8435672a74c70b7c0e7
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1327, ASTERIXDB-1362 Fixed circle-point intersect ...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1327, ASTERIXDB-1362 Fixed circle-point intersect 
function
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1558
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2512c73c9dcd593dc7e6690435da67f8086ff0db
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1778: Optimize the edit-distance-check function

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1778: Optimize the edit-distance-check function
..


Patch Set 6:

Looks good. But I still feel there should be some simple *JUnit* test for the 
edit distance, not the AQL ones. 
The AQL (or SQL++) tests are too far away and usually is very difficult to hit 
the corner cases.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1481
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibc8729c4514bb87c347dd7d50358fd897b769977
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1778: Optimize the edit-distance-check function

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1778: Optimize the edit-distance-check function
..


Patch Set 4:

(10 comments)

Just some minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/1481/4/asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/similarity/IGenericSimilarityMetric.java
File 
asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/similarity/IGenericSimilarityMetric.java:

PS4, Line 26:  
use javadoc syntax?


PS4, Line 29: float
this function doesn't has to be exposed.


PS4, Line 32: returns
use javadoc?


https://asterix-gerrit.ics.uci.edu/#/c/1481/3/asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/similarity/SimilarityMetricEditDistance.java
File 
asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/similarity/SimilarityMetricEditDistance.java:

PS3, Line 63: public
is it necessary to has an `public interface` ? 
I think it can just be a private function of this class.


Line 70: boolean canTerminateEarly = edThresh >= 0 ? true : false;
*boolean canTerminateEarly = edThresh >= 0* is enough.

and if edThresh > min(flLen, slLen) should also be false?


PS3, Line 131: 1
can you define a static variable and give `-1` a good name?


PS3, Line 144: Gets
do we really need this comments ? :-)


PS3, Line 157: -
it worth explain the meaning of -1


PS3, Line 168: public
is it necessary to be a public method?


PS3, Line 219: public
public -> private?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1481
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibc8729c4514bb87c347dd7d50358fd897b769977
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Support LSM-filter on multiple input paths

[Jianfeng Jia (Code Review)]

Jianfeng Jia has submitted this change and it was merged.

Change subject: Support LSM-filter on multiple input paths
..


Support LSM-filter on multiple input paths

The IntroduceLSMComponentFilterRule only checked one input path,
which does not work for the multiple-path cases (e.g., intersection case)
This patch fixes this bug.

Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1356
Reviewed-by: Taewoo Kim 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
3 files changed, 99 insertions(+), 6 deletions(-)

Approvals:
  Taewoo Kim: Looks good to me, approved
  Jenkins: Verified; No violations found



diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
index 0e54640..9b568b9 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
@@ -19,7 +19,9 @@
 package org.apache.asterix.optimizer.rules.am;
 
 import java.util.ArrayList;
+import java.util.LinkedList;
 import java.util.List;
+import java.util.Queue;
 
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.metadata.declared.AqlDataSource;
@@ -149,8 +151,12 @@
 private void changePlan(List optFuncExprs, 
AbstractLogicalOperator op, Dataset dataset,
 IOptimizationContext context) throws AlgebricksException {
 
-AbstractLogicalOperator descendantOp = (AbstractLogicalOperator) 
op.getInputs().get(0).getValue();
-while (descendantOp != null) {
+Queue queue = new 
LinkedList<>(op.getInputs());
+while (!queue.isEmpty()) {
+AbstractLogicalOperator descendantOp = (AbstractLogicalOperator) 
queue.poll().getValue();
+if (descendantOp == null) {
+continue;
+}
 if (descendantOp.getOperatorTag() == 
LogicalOperatorTag.DATASOURCESCAN) {
 DataSourceScanOperator dataSourceScanOp = 
(DataSourceScanOperator) descendantOp;
 AqlDataSource ds = (AqlDataSource) 
dataSourceScanOp.getDataSource();
@@ -208,10 +214,7 @@
 }
 }
 }
-if (descendantOp.getInputs().isEmpty()) {
-break;
-}
-descendantOp = (AbstractLogicalOperator) 
descendantOp.getInputs().get(0).getValue();
+queue.addAll(descendantOp.getInputs());
 }
 }
 
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
new file mode 100644
index 000..353129e
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description: Tests three types of secondary indexes should trigger 
intersection rule
+ * Success: Yes
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create type tTweet as closed {
+  id: int32,
+  location: point,
+  message: string,
+  create_at: datetime,
+  misc: string
+}
+
+create dataset dsTweet(tTweet) primary key id with filter on create_at;
+
+create index ngram_index on dsTweet(message) 

Change in asterixdb[master]: Support LSM-filter on multiple input paths

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Support LSM-filter on multiple input paths
..


Patch Set 3: -Code-Review

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1356
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Support LSM-filter on multiple input paths

[Jianfeng Jia (Code Review)]

Hello Taewoo Kim, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1356

to look at the new patch set (#3).

Change subject: Support LSM-filter on multiple input paths
..

Support LSM-filter on multiple input paths

The IntroduceLSMComponentFilterRule only checked one input path,
which does not work for the multiple-path cases (e.g., intersection case)
This patch fixes this bug.

Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
3 files changed, 99 insertions(+), 6 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/56/1356/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1356
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Support LSM-filter on multiple input paths

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Support LSM-filter on multiple input paths
..


Patch Set 3: Code-Review+2

Forward Taewoo's +2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1356
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Support filter on multiple input path

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1356

to look at the new patch set (#2).

Change subject: Support filter on multiple input path
..

Support filter on multiple input path

The following commits from your working branch will be included:

commit c80f97e85d9f620f65342ff95ca1c6a8c3e4f8ae
Author: Jianfeng Jia 
Date:   Tue Nov 22 21:13:20 2016 -0800

add one testcase

commit 5681c748867ff801406398a9784809d7ce77a973
Author: Jianfeng Jia 
Date:   Tue Nov 22 20:44:20 2016 -0800

change the lsm filter rule to compatible with multiple inputs scenario

Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
3 files changed, 99 insertions(+), 6 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/56/1356/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1356
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Not suppose to MERGE, for discussion purpose The following c...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has abandoned this change.

Change subject: Not suppose to MERGE, for discussion purpose The following 
commits from your working branch will be included:
..


Abandoned

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1135
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Id6d437ccff51adb236f4e69c78761a2c336ae71a
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Support filter on multiple input path

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1356

Change subject: Support filter on multiple input path
..

Support filter on multiple input path

The following commits from your working branch will be included:

commit c80f97e85d9f620f65342ff95ca1c6a8c3e4f8ae
Author: Jianfeng Jia 
Date:   Tue Nov 22 21:13:20 2016 -0800

add one testcase

commit 5681c748867ff801406398a9784809d7ce77a973
Author: Jianfeng Jia 
Date:   Tue Nov 22 20:44:20 2016 -0800

change the lsm filter rule to compatible with multiple inputs scenario

Change-Id: I390ac11f3a2781f10f726dce917d89ec640dd917
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/btree-rtree-ngram-intersect-with-filter.plan
3 files changed, 99 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/56/1356/1

diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
index 0e54640..7e99b6c 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
@@ -19,7 +19,9 @@
 package org.apache.asterix.optimizer.rules.am;
 
 import java.util.ArrayList;
+import java.util.LinkedList;
 import java.util.List;
+import java.util.Queue;
 
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.metadata.declared.AqlDataSource;
@@ -149,8 +151,12 @@
 private void changePlan(List optFuncExprs, 
AbstractLogicalOperator op, Dataset dataset,
 IOptimizationContext context) throws AlgebricksException {
 
-AbstractLogicalOperator descendantOp = (AbstractLogicalOperator) 
op.getInputs().get(0).getValue();
-while (descendantOp != null) {
+Queue queue = new 
LinkedList<>(op.getInputs());
+while (!queue.isEmpty()) {
+AbstractLogicalOperator descendantOp = (AbstractLogicalOperator) 
queue.poll().getValue();
+if (descendantOp == null) {
+continue;
+}
 if (descendantOp.getOperatorTag() == 
LogicalOperatorTag.DATASOURCESCAN) {
 DataSourceScanOperator dataSourceScanOp = 
(DataSourceScanOperator) descendantOp;
 AqlDataSource ds = (AqlDataSource) 
dataSourceScanOp.getDataSource();
@@ -211,7 +217,7 @@
 if (descendantOp.getInputs().isEmpty()) {
 break;
 }
-descendantOp = (AbstractLogicalOperator) 
descendantOp.getInputs().get(0).getValue();
+queue.addAll(descendantOp.getInputs());
 }
 }
 
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
new file mode 100644
index 000..353129e
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect-with-filter.aql
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description: Tests three types of secondary indexes should trigger 
intersection rule
+ * Success: Yes
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create type tTweet as closed {
+  id: int32,
+  location: point,
+  message: string,
+  create_at: datetime,
+  misc: string
+}
+
+create dataset dsTweet(tTweet) primary key id with filter on create_at;
+
+create index ngram_index on 

Change in asterixdb[master]: ASTERIXDB-1700: fixed multiple same type of index applicatio...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1700: fixed multiple same type of index application 
error on the same field
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1307
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I450f3adb20c777d5b9a8f638e010076b9d817942
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Make Lock Manager Statistics at FINE granularity

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Make Lock Manager Statistics at FINE granularity
..


Patch Set 4: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1303
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id26ac346a5fa1410e1b74a4fa88ae9f45f791be2
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Make Lock Manager Statistics at FINE granularity

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Make Lock Manager Statistics at FINE granularity
..


Patch Set 3: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1303
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id26ac346a5fa1410e1b74a4fa88ae9f45f791be2
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Make Lock Manager Statistics at FINE granularity

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Make Lock Manager Statistics at FINE granularity
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1303/3/asterixdb/asterix-events/src/main/resources/events/cc_start/cc_start.sh
File asterixdb/asterix-events/src/main/resources/events/cc_start/cc_start.sh:

Line 72: echo " 
"
 >> $LOG_DIR/cc.log
would it be a little cleaner by using cat? 


cat <<"EOF" >> $LOG_DIR

-
$date


EOF


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1303
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id26ac346a5fa1410e1b74a4fa88ae9f45f791be2
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Not suppose to MERGE, for discussion purpose The following c...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Not suppose to MERGE, for discussion purpose The following 
commits from your working branch will be included:
..


Patch Set 1:

That `println` only print if anything wrong happens. So it shouldn't be the 
problem.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1135
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id6d437ccff51adb236f4e69c78761a2c336ae71a
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Index-only plan step 2: Added SplitOperator

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Index-only plan step 2: Added SplitOperator
..


Patch Set 12: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1196
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ice190827513cd8632764b52c9d0338d65c830740
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Index-only plan step 2: Added SplitOperator

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Index-only plan step 2: Added SplitOperator
..


Patch Set 11:

(1 comment)

Just one comment

https://asterix-gerrit.ics.uci.edu/#/c/1196/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SplitOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SplitOperator.java:

Line 32: private final Mutable branchingExpression;
Can it be just ILogicalExpression than `Mutable`?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1196
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ice190827513cd8632764b52c9d0338d65c830740
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Index-only plan step 2: Added SplitOperator

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Index-only plan step 2: Added SplitOperator
..


Patch Set 10:

Like this kind of test? 
:https://github.com/apache/asterixdb/blob/bce00de839d844e41de2958e1494a63efdde/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/IntersectOperatorDescriptorTest.java

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1196
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ice190827513cd8632764b52c9d0338d65c830740
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Updated DeletableFrameTupleAppender to support reusing index...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Updated DeletableFrameTupleAppender to support reusing index 
slots.
..


Patch Set 2:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1214/2/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/AbstractRunGeneratorTest.java
File 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/AbstractRunGeneratorTest.java:

Line 119: // TODO: This randomly fails. Not sure what causes this, 
except that the value in bytes is a truncated
does this fail still occur?


Line 260: System.err.println(specialPair);
for debug purpose?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1214
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I05cda3360cc503bf847aeca5998be06f1f7d3c15
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Preston Carman 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Preston Carman 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix for ASTERIXDB-1636

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix for ASTERIXDB-1636
..


Patch Set 4: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1205
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib4bc413fcda9a5c98ae57f94e1c8a68fe9aacda3
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: Updated DeletableFrameTupleAppender to support reusing index...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Updated DeletableFrameTupleAppender to support reusing index 
slots.
..


Patch Set 1:

(9 comments)

Looks good in general. Just a few minor comments.

https://asterix-gerrit.ics.uci.edu/#/c/1214/1/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/util/DeletableFrameTupleAppender.java
File 
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/util/DeletableFrameTupleAppender.java:

Line 34:  * The offsets also store both the start and end values because tuples 
may be out of
out of "range"?


Line 41:  * tuple_append holds an int indicating the offset 
used to append the next tuple.
maybe it helps to explain the difference between `tuple_append` and the 
`next_index`?


Line 79: private IntegerPairPool ipp = new IntegerPairPool();
it's better to unify the initialization place, either all inside constructor or 
outside.


Line 80: 
ipp should be final variable as well?


Line 88: private int getIndexCount() {
it's often more clear to move the private functions to the end.
I just realized it recently ...


Line 128: if (getTupleEndOffset(nextIndex) <= 0) {
is it possible to merge this condition with the while loop?


Line 160: setTupleAppend(0);
If I understand correctly, this offset should add some paddings?


Line 279: }
do we need to set the member variable `nextIndex` as well?


https://asterix-gerrit.ics.uci.edu/#/c/1214/1/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/util/IntegerPairPool.java
File 
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/util/IntegerPairPool.java:

Line 26: list = new ArrayList<>();
since the behavior is just adding and removing to and from tail, it would be 
better to use LinkedList instead of ArrayList


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1214
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I05cda3360cc503bf847aeca5998be06f1f7d3c15
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Preston Carman 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Preston Carman 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Adding a new 101 tutorial for SQL++.

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Adding a new 101 tutorial for SQL++.
..


Patch Set 1:

Thanks Yingyi for the explaination!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1131
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I299b4947b928f301e8fdca8522d9a9f1e6a772cb
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Carey 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1628: Fixed an issue in External Hash Group by

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1628: Fixed an issue in External Hash Group by
..


Patch Set 4: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1144
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8901d2b64659fb0d2b97d73f45a9fe113232e860
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1628: Fixed an issue in External Hash Group by

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1628: Fixed an issue in External Hash Group by
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1144/3/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/HashSpillableTableFactory.java
File 
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/HashSpillableTableFactory.java:

Line 249: // Actually, at this stage, we know that this is not a 
in-memory hash (#frames required > #frameLimit).
would you please summarize the comments and put it on top of the function?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1144
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8901d2b64659fb0d2b97d73f45a9fe113232e860
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1628: Fixed an issue in External Hash Group by

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1628: Fixed an issue in External Hash Group by
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1144
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8901d2b64659fb0d2b97d73f45a9fe113232e860
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Not suppose to MERGE, for discussion purpose The following c...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Not suppose to MERGE, for discussion purpose The following 
commits from your working branch will be included:
..


Patch Set 1:

@Yingyi, I think there maybe the issue of the Math.pow() which gives an 
approximate result. Here I tested the Int number from 1 to Int.Max. The result 
is as expected, but this test gonna take looong to execute. Any suggestions?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1135
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id6d437ccff51adb236f4e69c78761a2c336ae71a
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb[master]: Fixes related to ASTERIXDB-1534

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fixes related to ASTERIXDB-1534
..


Patch Set 2: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If51e0cd183f9d5ed6edaebef4a0568a6c67062e3
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Blow 
Gerrit-HasComments: No

Change in asterixdb[master]: Add tests for ASTERIXDB-1300

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1093

Change subject: Add tests for ASTERIXDB-1300
..

Add tests for ASTERIXDB-1300

The intersection for multiple-secondary indexes will follow the skip-index hint.

Change-Id: I797bdb37fda1c29c9cfc45efb6fbcba4e7b6c29e
---
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-one-index.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-two-index.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/skip-one-index.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/multi-indexes/skip-two-index.plan
4 files changed, 143 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/93/1093/1

diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-one-index.aql
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-one-index.aql
new file mode 100644
index 000..7383edc
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-one-index.aql
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create type tTweet as closed {
+  id: int32,
+  location: point,
+  message: string,
+  create_at: datetime,
+  misc: string
+}
+
+create dataset dsTweet(tTweet) primary key id;
+
+create index ngram_index on dsTweet(message) type ngram(3);
+create index time_index on dsTweet(create_at) type btree;
+create index location_index on dsTweet(location) type rtree;
+
+write output to nc1:"rttest/btree-rtree-ngram-intersect.adm";
+
+let $region := 
create-rectangle(create-point(-128.4300781252,20.298506037222175), 
create-point(-64.2699218752,54.56902589732035))
+let $ts_start := datetime("2015-11-11T00:00:00Z")
+let $ts_end := datetime("2015-12-18T23:59:59Z")
+let $keyword := "hello"
+for $t in dataset dsTweet
+where $t.create_at >= $ts_start and $t.create_at < $ts_end
+  and /* +skip-index */ spatial-intersect($t.location, $region)
+  and contains($t.message, $keyword)
+return $t
+
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-two-index.aql
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-two-index.aql
new file mode 100644
index 000..de0807f
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/skip-two-index.aql
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description: Tests three types of secondary indexes should trigger 
intersection rule
+ * Success: Yes
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+use dataverse test;
+
+create type tTweet as closed {
+  id: int32,
+  location: point,
+  message: string,
+  create_at: datetime,
+  misc: string
+}
+
+create dataset dsTweet(tTweet) primary key id;
+
+create index ngram_index on dsTweet(message) type ngram(3);
+create index time_index on dsTweet(create_at) type btree;
+create index location_index on dsTweet(location) type rtree;
+
+write output to nc1:"rttest/btree-rtree-ngram-intersect.adm";
+
+let $region := 

Change in asterixdb[master]: Patch for the remaning issue inconsistent with UTF8.-> ASTER...

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Patch for the remaning issue inconsistent with UTF8.-> 
ASTERIXDB-1478: fix the utf8 reader.
..


Patch Set 5:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1077/5/asterixdb/asterix-external-data/src/test/resources/results/beer.txt
File asterixdb/asterix-external-data/src/test/resources/results/beer.txt:

Line 1453: { "name": "Basil T's Brew Pub and Italian Grill", "city": "Toms 
River", "state": "New Jersey", "code": "8753", "country": "United States", 
"phone": "1-732-244-7566", "website": "", "type": "brewery", "updated": 
"2010-07-22 20:00:20", "description": "", "address": [ "1171 Hooper Avenue" ], 
"geo": { "accuracy": "RANGE_INTERPOLATED", "lat": 39.9767, "lon": -74.1829 } }
why do we change this file?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1077
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idb302dc604fcd71811de550d3d4bd727c81a13ee
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Patch for the remaning issue inconsistent with UTF8.

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Patch for the remaning issue inconsistent with UTF8.
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1077/3/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AsterixInputStreamReader.java
File 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/AsterixInputStreamReader.java:

Line 102: if (remaining) {
I feel add extra if else could solve the problem but still error-prone. 
The ideal case may simplify all these `if,else, while` condition to no more 
than two levels to avoid these kind of problems thoroughly? 

Just a thought.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1077
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idb302dc604fcd71811de550d3d4bd727c81a13ee
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1463: add a regression test.

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: ASTERIXDB-1463: add a regression test.
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1067
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iaf3a5a616799af2ee58ba559e1dc1db8b6e010ad
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix ASTERIXDB-1566 fix UTF8 comparator and hash function.

[Jianfeng Jia (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1054

to look at the new patch set (#2).

Change subject: Fix ASTERIXDB-1566 fix UTF8 comparator and hash function.
..

Fix ASTERIXDB-1566 fix UTF8 comparator and hash function.

Change-Id: I187bf1243abf143b3b265fa8098614b9a72c65ad
---
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/string/string-equal-public/string-equal-public.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryComparatorFactoryProvider.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryHashFunctionFactoryProvider.java
M 
hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java
6 files changed, 44 insertions(+), 6 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/1054/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1054
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I187bf1243abf143b3b265fa8098614b9a72c65ad
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 

Change in asterixdb[master]: Fix ASTERIXDB-1566 fix UTF8 comparator and hash function.

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fix ASTERIXDB-1566 fix UTF8 comparator and hash function.
..


Patch Set 1:

(7 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1054/1//COMMIT_MSG
Commit Message:

Line 8: 
> "ASTERIXDB-1566"->"ASTERIXDB-1566: fix UTF8 comparator and hash function."
Done


https://asterix-gerrit.ics.uci.edu/#/c/1054/1/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql:

Line 19: drop dataverse test if exists;
> remove this file.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1054/1/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql:

Line 17:  * under the License.
> remove this file.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1054/1/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql:

Line 18:  */
> rename this file to "string-equal-public.1.query.aql".
Done


Line 19: use dataverse test;
> remove the dataverse reference.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1054/1/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryComparatorFactoryProvider.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryComparatorFactoryProvider.java:

Line 45: import org.apache.hyracks.data.std.primitive.*;
> import each individual class.
Done


https://asterix-gerrit.ics.uci.edu/#/c/1054/1/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryHashFunctionFactoryProvider.java
File 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryHashFunctionFactoryProvider.java:

Line 28: import org.apache.hyracks.data.std.primitive.*;
> import each individual class.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1054
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I187bf1243abf143b3b265fa8098614b9a72c65ad
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wenhai Li 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fix ASTERIXDB-1566

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1054

Change subject: Fix ASTERIXDB-1566
..

Fix ASTERIXDB-1566

Change the utf8 comparator from RawComparator to UTF8Comparator

Change-Id: I187bf1243abf143b3b265fa8098614b9a72c65ad
---
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/string/string-equal-public/string-equal-public.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryComparatorFactoryProvider.java
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/AqlBinaryHashFunctionFactoryProvider.java
M 
hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java
8 files changed, 82 insertions(+), 16 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/1054/1

diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql
new file mode 100644
index 000..d17ea60
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.1.ddl.aql
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse test if exists;
+create dataverse test;
+
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql
new file mode 100644
index 000..042f3ce
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.2.update.aql
@@ -0,0 +1,18 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql
new file mode 100644
index 000..4959846
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/string/string-equal-public/string-equal-public.3.query.aql
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * 

Change in asterixdb[master]: Fulltext search initial implementation

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fulltext search initial implementation
..


Patch Set 5:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-doc/src/site/markdown/aql/manual.md
File asterixdb/asterix-doc/src/site/markdown/aql/manual.md:

Line 720: `rtree` for spatial data, and `keyword`, `ngram`, and `fulltext` for 
textual (string) data.
what is the different between `keyword` and `fulltext`?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/989
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I71887c2ea847e4488f4c98a11f8a5bcad02cac5a
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Heri Ramampiaro 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fulltext search initial implementation

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Fulltext search initial implementation
..


Patch Set 5:

(18 comments)

https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java:

Line 298: // Full-text search consideration: an (un)ordered 
list of string type can be compatible with string
duplicate comments?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java:

Line 216: // Checks whether a proper constant expression is in place for 
the full-text search.
this comment doesn't give more information than the function name, better to 
remove it.


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java:

Line 1183: // For full-text search
can't understand the comment. better to remove it?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/metadata/queries/basic/dataset_with_meta-1/dataset_with_meta-1.1.ddl.aql
File 
asterixdb/asterix-app/src/test/resources/metadata/queries/basic/dataset_with_meta-1/dataset_with_meta-1.1.ddl.aql:

Line 30:   'text': string
just want to confirm that the single quote will work? i was thinking it only 
works for double quote.


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/ngram-contains-panic.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/ngram-contains-panic.aql:

Line 45: where string-contains($o.title, "Mu")
while `string-contains` check the index? There is an ngram_idx built on this 
field.


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/ngram-contains.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/ngram-contains.aql:

Line 44: where string-contains($o.title, "Multimedia")
will it check the inverted index?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/word-contains.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-basic/word-contains.aql:

Line 45: where string-contains($o.title, "Multimedia")
will it check the inverted index?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-join/ngram-contains.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/inverted-index-join/ngram-contains.aql:

Line 45: where string-contains($o1.title, $o2.title) and $o1.id < $o2.id
will it check the inverted index?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/btree-rtree-ngram-intersect.aql:

Line 51:   and string-contains($t.message, $keyword)
will it check the inverted index?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/with-primary-index-intersect.aql
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/multi-indexes/with-primary-index-intersect.aql:

Line 51:   and string-contains($t.message, $keyword)
will it check the inverted index?


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection/tinysocial-intersect.3.query.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-selection/intersection/tinysocial-intersect.3.query.aql:

Line 27: and string-contains($t.message-text, $keyword)
this test is supposed to trigger the inverted index search, so 
`string-contains` won't serve the purpose


https://asterix-gerrit.ics.uci.edu/#/c/989/5/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/base/Expression.java
File 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/base/Expression.java:

Line 35: FTCONTAINS_EXPRESSION,
why full text search is an expression than just a function?

Change in asterixdb[master]: Fix the binary documentation

[Jianfeng Jia (Code Review)]

Jianfeng Jia has submitted this change and it was merged.

Change subject: Fix the binary documentation
..


Fix the binary documentation

Change-Id: I509f0d4bc4b228d85226a85fe68d6ae97b52f5bd
Reviewed-on: https://asterix-gerrit.ics.uci.edu/970
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 
---
M asterixdb/asterix-doc/src/site/markdown/aql/functions.md
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
index 89dc311..b6f5538 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
@@ -591,7 +591,7 @@
 " the voice-command is bad:("
 " the voicemail-service is awesome"
 
-## String Functions [Back to TOC] ##
+## Binary Functions [Back to TOC] ##
 ### parse-binary ###
   * Syntax:
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/970
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I509f0d4bc4b228d85226a85fe68d6ae97b52f5bd
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Fix the binary documentation

[Jianfeng Jia (Code Review)]

Jianfeng Jia has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/970

Change subject: Fix the binary documentation
..

Fix the binary documentation

Change-Id: I509f0d4bc4b228d85226a85fe68d6ae97b52f5bd
---
M asterixdb/asterix-doc/src/site/markdown/aql/functions.md
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/70/970/1

diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
index 89dc311..b6f5538 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
@@ -591,7 +591,7 @@
 " the voice-command is bad:("
 " the voicemail-service is awesome"
 
-## String Functions [Back to TOC] ##
+## Binary Functions [Back to TOC] ##
 ### parse-binary ###
   * Syntax:
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/970
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I509f0d4bc4b228d85226a85fe68d6ae97b52f5bd
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 

Change in asterixdb[master]: Add the documentation for the `binary` data type

[Jianfeng Jia (Code Review)]

Jianfeng Jia has submitted this change and it was merged.

Change subject: Add the documentation for the `binary` data type
..


Add the documentation for the `binary` data type

Change-Id: Iea9b29a1a8ff37617fb94cd562a1f885f8867ad3
Reviewed-on: https://asterix-gerrit.ics.uci.edu/934
Reviewed-by: Till Westmann 
Reviewed-by: Jenkins 
Reviewed-by: Jianfeng Jia 
Tested-by: Jenkins 
---
M asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md
M asterixdb/asterix-doc/src/site/markdown/aql/functions.md
2 files changed, 128 insertions(+), 3 deletions(-)

Approvals:
  Jianfeng Jia: Looks good to me, approved
  Till Westmann: Looks good to me, approved
  Jenkins: Looks good to me, but someone else must approve; Verified



diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md
index 59ef5c9..09d9177 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md
@@ -27,6 +27,7 @@
 * [Float](#PrimitiveTypesFloat)
 * [Double](#PrimitiveTypesDouble)
 * [String](#PrimitiveTypesString)
+* [Binary](#PrimitiveTypesBinary)
 * [Point](#PrimitiveTypesPoint)
 * [Line](#PrimitiveTypesLine)
 * [Rectangle](#PrimitiveTypesRectangle)
@@ -120,7 +121,7 @@
 
 
 ### String [Back to TOC] ###
-`string` represents a sequence of characters.
+`string` represents a sequence of characters. The total length of the sequence 
can be up to 2,147,483,648.
 
  * Example:
 
@@ -132,6 +133,26 @@
  * The expected result is:
 
 { "v1": "This is a string.", "v2": "\"This is a quoted string\"" }
+
+
+### Binary [Back to TOC] ###
+`binary` represents a sequence of bytes. It can be constructed from a `hex` or 
a `base64` string sequence.
+The total length of the byte sequence can be up to 2,147,483,648.
+
+ * Example:
+
+let $hex1 := hex("ABCDEF0123456789")
+let $hex2 := hex("abcdef0123456789")
+let $base64_1 := base64("0123456789qwertyui+/")
+let $base64_2 := base64('QXN0ZXJpeA==')
+return { "hex1" : $hex1, "hex2": $hex2, "base64_1" : $base64_1, 
"base64_2" : $base64_2}
+
+ * The default output format is in `hex` format. Thus, the expected result is:
+
+  { "hex1": hex("ABCDEF0123456789"),
+"hex2": hex("ABCDEF0123456789"),
+"base64_1": hex("D35DB7E39EBBF3DAB07ABB72BA2FBF"),
+"base64_2": hex("41737465726978") }
 
 
 ### Point [Back 
to TOC] ###
@@ -344,4 +365,3 @@
 
 
 {{"hello", 9328, "world", [1, 2, null]}}
-
diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
index 3b78a93..89dc311 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
@@ -23,6 +23,7 @@
 
 * [Numeric Functions](#NumericFunctions)
 * [String Functions](#StringFunctions)
+* [Binary Functions](#BinaryFunctions)
 * [Aggregate Functions](#AggregateFunctions)
 * [Spatial Functions](#SpatialFunctions)
 * [Similarity Functions](#SimilarityFunctions)
@@ -589,6 +590,110 @@
 " its touch-screen is horrible"
 " the voice-command is bad:("
 " the voicemail-service is awesome"
+
+## String Functions [Back to TOC] ##
+### parse-binary ###
+  * Syntax:
+
+  parse-binary(string, encoding)
+
+  * Creates a `binary` from an string encoded in `encoding` format.
+  * Arguments:
+* `string` : An encoded `string`
+* `encoding` : A string notation specifies the encoding type of the given 
`string`.
+Currently we support `hex` and `base64` format.
+  * Return Value:
+* A `binary` that is decoded from the given `string`.
+
+  * Example:
+
+let $c1 := parse-binary("ABCDEF0123456789","hex")
+let $c2 := parse-binary("abcdef0123456789","HEX")
+let $c3 := parse-binary('QXN0ZXJpeAE=',"base64")
+return [ $c1, $c2, $c3 ]
+
+  * The expected result is:
+
+  [ hex("ABCDEF0123456789"), hex("ABCDEF0123456789"), 
hex("4173746572697801") ]
+
+### print-binary ###
+  * Syntax:
+
+  print-binary(binary, encoding)
+
+  * Prints a `binary` to the required encoding `string` format.
+  * Arguments:
+* `binary` : A `binary` data need to be printed.
+* `encoding` : A string notation specifies the expected encoding type.
+Currently we support `hex` and `base64` format.
+  * Return Value:
+* A `string` that represents the encoded format of a `binary`.
+
+  * Example:
+
+print-binary(hex("ABCDEF0123456789"), "base64")
+print-binary(base64("q83vASNFZ4k="), "hex")
+
+  * The expected result is:
+
+"q83vASNFZ4k="
+"ABCDEF0123456789"
+
+### binary-length ###
+  * Syntax:
+
+  binary-length(binary)
+
+  * Returns the number 

Change in asterixdb[master]: Add the documentation for the `binary` data type

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Add the documentation for the `binary` data type
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/934
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iea9b29a1a8ff37617fb94cd562a1f885f8867ad3
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add the documentation for the `binary` data type

[Jianfeng Jia (Code Review)]

Jianfeng Jia has posted comments on this change.

Change subject: Add the documentation for the `binary` data type
..


Patch Set 1:

(5 comments)

@Till, I removed the implementation sentences.

https://asterix-gerrit.ics.uci.edu/#/c/934/1/asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md
File asterixdb/asterix-doc/src/site/markdown/aql/datamodel.md:

Line 129: The total length of the sequence can be up to 2,147,483,648.
> It seems to me that know about the maximal length is useful for an AQL user
Done


Line 161: "base64_2": hex("41737465726978") }
> The documentation looks good to me but, again, I would not talk about the i
Done


https://asterix-gerrit.ics.uci.edu/#/c/934/1/asterixdb/asterix-doc/src/site/markdown/aql/functions.md
File asterixdb/asterix-doc/src/site/markdown/aql/functions.md:

Line 598:   parse-binary(string, encoded_type)  
> WS
Done


Line 603: * `encoded_type` : A string notation specifies the encoding type 
of the given `string`. Currently we support `hex` and `base64` format.
> Could we put a line-break here to stick to 120 chars?
Done


Line 626: * `encoding_type` : A string notation specifies the expected 
encoding type. Currently we support `hex` and `base64` format.
> Could we put a line-break here to stick to 120 chars?
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/934
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iea9b29a1a8ff37617fb94cd562a1f885f8867ad3
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Jianfeng Jia 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Jianfeng Jia 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: Yes