Change in asterixdb[master]: [ASTERIXDB-2468] Extend CountVarToCountOneRule to support no...

2018-11-20 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2468] Extend CountVarToCountOneRule to support no 
group-by
..


Patch Set 9:

@Dmitry: Chen checked that the CountVartoCountOneRule only removes the ASSIGN 
operator from the generated index-only plans when the rule is fired. I also 
checked the code and it looks fine to me.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/3024
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie0808912cf45d3a914ebf7f69ec42f73b33c071f
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2454] Remove non-unique AQL tests (pt.1)

2018-10-29 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2454] Remove non-unique AQL tests (pt.1)
..


Patch Set 4: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/3007
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I13f676e9322f8b94e8c9686b261e8752c7100724
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2466][FUN] Implement window functions

2018-10-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2466][FUN] Implement window functions
..


Patch Set 2:

(12 comments)

Ali already gave good comments. I have some questions. (1) What does 
"materialization" do for the window operator? When calculating the required 
capacity, I saw that part. (2) I think we need to add more run time test cases 
without using "function".  (3) Generally, I think it would be nice if we could 
add a brief explanation at the beginning of each class.

https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/AbstractRankRunningAggregateEvaluator.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/AbstractRankRunningAggregateEvaluator.java:

Line 34: public abstract class AbstractRankRunningAggregateEvaluator implements 
IWindowAggregateEvaluator {
I think it would be better to put a brief class comment.


Line 56: AbstractRankRunningAggregateEvaluator(IScalarEvaluator[] args, 
boolean dense, SourceLocation sourceLoc) {
Do we use this "dense" variable?


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/DenseRankRunningAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/DenseRankRunningAggregateDescriptor.java:

Line 33: public class DenseRankRunningAggregateDescriptor extends 
AbstractRunningAggregateFunctionDynamicDescriptor {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/NtileRunningAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/NtileRunningAggregateDescriptor.java:

Line 32: public class NtileRunningAggregateDescriptor extends 
AbstractRunningAggregateFunctionDynamicDescriptor {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/NtileRunningAggregateEvaluator.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/NtileRunningAggregateEvaluator.java:

Line 37: public class NtileRunningAggregateEvaluator implements 
IWindowAggregateEvaluator {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/PercentRankRunningAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/PercentRankRunningAggregateDescriptor.java:

Line 33: public class PercentRankRunningAggregateDescriptor extends 
AbstractRunningAggregateFunctionDynamicDescriptor {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/PercentRankRunningAggregateEvaluator.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/PercentRankRunningAggregateEvaluator.java:

Line 33: class PercentRankRunningAggregateEvaluator extends 
AbstractRankRunningAggregateEvaluator {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RankRunningAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RankRunningAggregateDescriptor.java:

Line 33: public class RankRunningAggregateDescriptor extends 
AbstractRunningAggregateFunctionDynamicDescriptor {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RankRunningAggregateEvaluator.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RankRunningAggregateEvaluator.java:

Line 33: class RankRunningAggregateEvaluator extends 
AbstractRankRunningAggregateEvaluator {
I think it would be better to put a brief class comment.


https://asterix-gerrit.ics.uci.edu/#/c/3002/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RowNumberRunningAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/runningaggregates/std/RowNumberRunningAggregateDescriptor.java:

Line 31: public class RowNumberRunningAggregateDescriptor extends 
AbstractRunningAggregateFunctionDynamicDescriptor {
I think it

Change in asterixdb[master]: [ASTERIXDB-2454] Remove non-unique AQL tests

2018-10-17 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2454] Remove non-unique AQL tests
..


Patch Set 7: Code-Review+2

LGTM. Thanks!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2979
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I13f676e9322f8b94e8c9686b261e8752a7100722
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases

2018-10-17 Thread Taewoo Kim (Code Review) 
Taewoo Kim has abandoned this change.

Change subject: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases
..


Abandoned

Ian Maxon will take care of this issue.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2976
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I475092bda3efea9953c06f93146e19816efef7eb
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-09 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..


[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1076
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
Reviewed-by: Taewoo Kim 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard_01.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-09 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..


Patch Set 100: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1076
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
Gerrit-PatchSet: 100
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Wenhai Li 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#100).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard_01.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#99).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard_01.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#98).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard_01.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#97).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..


Patch Set 93:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1076/93/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java:

Line 269
> Hm, RuleCollections.buildLoadFieldsRuleCollection() runs again after the bu
Reverting to the original version would not break any run-time test cases. 
However, since the logical plan that the fuzzy join generates a logical plan 
that contains "subplan" in the newly generated plan, the plan needs to be 
transformed once again using this rule. 

In fact, as the current patch set shows, no test cases fail now. And what this 
rule does is detecting "subplan" and transforms it. I think a repetitive 
application would not cause an issue if we set "context.addToDontApplySet(this, 
subplanOp);" properly.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1076
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
Gerrit-PatchSet: 93
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Wenhai Li 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-08 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#96).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-03 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..


Patch Set 93:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1076/93/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java:

Line 269
> Wenhai, could you reply to this question?
I think the reason is that this inline rule needs to be applied to the logical 
plans that are generated by AQL+ framework in buildFuzzyJoinRuleCollection(). 
If you see the RuleCollections class, buildLoadFieldsRuleCollection() includes 
this rule. And right after that, buildFuzzyJoinRuleCollection() is performed. 
Thus, if we just execute this once, the logical plans from AQL+ framework 
cannot be transformed using this rule. Wenhai, plaease correct me if I'm wrong.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1076
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
Gerrit-PatchSet: 93
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Wenhai Li 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-03 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#94).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-03 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..


Patch Set 93:

(7 comments)

https://asterix-gerrit.ics.uci.edu/#/c/1076/5//COMMIT_MSG
Commit Message:

Line 17:   - Use similarity-jaccard-prefix to enable the pp+ join strategy.
Please check the other commit message and please follow the format.


https://asterix-gerrit.ics.uci.edu/#/c/1076/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java:

Line 55: case UNION:
Why do we add UNION here?


https://asterix-gerrit.ics.uci.edu/#/c/1076/27/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java:

Line 353: groupByRight += ", ";
counter.set(context.getVarCounter())


https://asterix-gerrit.ics.uci.edu/#/c/1076/93/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java:

Line 269
> Why this was removed? the rule scans the whole plan starting from the root.
Wenhai, could you reply to this question?


https://asterix-gerrit.ics.uci.edu/#/c/1076/93/asterixdb/asterix-app/src/test/resources/optimizerts/only.txt
File asterixdb/asterix-app/src/test/resources/optimizerts/only.txt:

Line 20: #inverted-index-join-noeqjoin/ngram-jaccard-inline.sqlpp
> why are these added and commented out? Is this intentional?
Needs to be removed.


https://asterix-gerrit.ics.uci.edu/#/c/1076/38/asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-4.1.1/word-jaccard.1.ddl.aql
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries/fuzzyjoin/dblp-csx-4.1.1/word-jaccard.1.ddl.aql:

Line 21:  *  We expect the join to be transformed into an 
prefix-based fuzzy join following with an < select.
> "an 'less than' select" ?
Sorry. I was not accurate. an prefix -> a prefix.


https://asterix-gerrit.ics.uci.edu/#/c/1076/93/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
File asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml:

Line 2453:

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-10-02 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#93).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
M asterixdb/asterix-app/src/test/resources/optimizerts/only.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M

Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update

2018-09-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] User-defined Function Documentation update
..


Patch Set 16: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2804
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1
Gerrit-PatchSet: 16
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update

2018-09-27 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] User-defined Function Documentation update
..


Patch Set 15: Code-Review+1

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2804
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update

2018-09-27 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] User-defined Function Documentation update
..


Patch Set 15:

(1 comment)

Just one question.

https://asterix-gerrit.ics.uci.edu/#/c/2804/15/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp:

Line 26: connect feed TweetFeed to dataset ProcessedTweets apply function 
testlib#addMentionedUsers;
Is there a reason to remove the parallelism hint?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2804
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update

2018-09-25 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] User-defined Function Documentation update
..


Patch Set 11:

Can you rebase the patch set first? :-)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2804
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian2 Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Carey 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Wail Alkowaileet 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2455][DOC] Deprecate AQL documentations

2018-09-24 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2455][DOC] Deprecate AQL documentations
..


[ASTERIXDB-2455][DOC] Deprecate AQL documentations

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Create [Deprecated] section and move AQL docs to there.
- Move some docs from /aql directory to /sqlpp directory.

Change-Id: I677dd7a8d114197eaa2ae93e0405184526b31a03
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2977
Sonar-Qube: Jenkins 
Reviewed-by: Ian Maxon 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
---
R asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
R asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
R asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
M asterixdb/asterix-doc/src/site/site.xml
4 files changed, 11 insertions(+), 11 deletions(-)

Approvals:
  Jenkins: Verified; No violations found; ; Verified
  Ian Maxon: Looks good to me, approved



diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/filters.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/filters.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/similarity.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/similarity.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
diff --git a/asterixdb/asterix-doc/src/site/site.xml 
b/asterixdb/asterix-doc/src/site/site.xml
index 99947ad..1167c37 100644
--- a/asterixdb/asterix-doc/src/site/site.xml
+++ b/asterixdb/asterix-doc/src/site/site.xml
@@ -71,22 +71,16 @@
 
 
 
-  
-  
+  
 
 
 
   
 
 
-
+
   
   
-
-
-
-  
-  
 
 
 
@@ -95,12 +89,18 @@
 
 
 
-  
   
   
   
-  
-  
+  
+  
+  
+
+
+
+  
+  
+  
 
 
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2977
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I677dd7a8d114197eaa2ae93e0405184526b31a03
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim

Change in asterixdb[master]: [ASTERIXDB-2455][DOC] Deprecate AQL documentations

2018-09-24 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2977

Change subject: [ASTERIXDB-2455][DOC] Deprecate AQL documentations
..

[ASTERIXDB-2455][DOC] Deprecate AQL documentations

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Create [Deprecated] section and move AQL docs to there.
- Move some docs from /aql directory to /sqlpp directory.

Change-Id: I677dd7a8d114197eaa2ae93e0405184526b31a03
---
R asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
R asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
R asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
M asterixdb/asterix-doc/src/site/site.xml
4 files changed, 11 insertions(+), 11 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/77/2977/1

diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/filters.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/filters.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md
diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md
diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/similarity.md 
b/asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
similarity index 100%
rename from asterixdb/asterix-doc/src/site/markdown/aql/similarity.md
rename to asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md
diff --git a/asterixdb/asterix-doc/src/site/site.xml 
b/asterixdb/asterix-doc/src/site/site.xml
index 99947ad..1167c37 100644
--- a/asterixdb/asterix-doc/src/site/site.xml
+++ b/asterixdb/asterix-doc/src/site/site.xml
@@ -71,22 +71,16 @@
 
 
 
-  
-  
+  
 
 
 
   
 
 
-
+
   
   
-
-
-
-  
-  
 
 
 
@@ -95,12 +89,18 @@
 
 
 
-  
   
   
   
-  
-  
+  
+  
+  
+
+
+
+  
+  
+  
 
 
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2977
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I677dd7a8d114197eaa2ae93e0405184526b31a03
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-09-24 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..


[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2854
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
8 files changed, 278 insertions(+), 0 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  Jenkins: Verified; No violations found; Verified
  Dmitry Lychagin: Looks good to me, approved



diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
index c925b55..10037f0 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
@@ -1196,6 +1196,30 @@
 origVarToSIdxUnnestMapOpVarMap.put(tVar, 
skVarsFromSIdxUnnestMap.get(sIndexIdx));
 }
 
+// For B-Tree case: if the given secondary key field variable is used 
only in the select or
+// join condition, we were not able to catch the mapping between the 
the SK from the original
+// data-scan and the SK from the secondary index search since they are 
different logical variables.
+// (E.g., we are sending a query on a composite index but returns only 
one field.)
+List varsUsedInTopOpButNotAfterwards = new 
ArrayList<>();
+copyVarsToAnotherList(uniqueUsedVarsInTopOp, 
varsUsedInTopOpButNotAfterwards);
+varsUsedInTopOpButNotAfterwards.removeAll(usedVarsAfterTopOp);
+if (idxType == IndexType.BTREE) {
+for (LogicalVariable v : varsUsedInTopOpButNotAfterwards) {
+int sIndexIdx = 
chosenIndexFieldNames.indexOf(subTree.getVarsToFieldNameMap().get(v));
+// For the join-case, the match might not exist.
+// In this case, we just propagate the variables later.
+if (sIndexIdx == -1) {
+continue;
+}
+LogicalVariable replacedVar = context.newVar();
+origPKRecAndSKVarToleftPathMap.put(v, replacedVar);
+
origVarToOutputVarMap.put(skVarsFromSIdxUnnestMap.get(sIndexIdx), v);
+// Constructs the mapping between the SK from the original 
data-scan
+// and the SK from the secondary index search since they are 
different logical variables.
+origVarToSIdxUnnestMapOpVarMap.put(v, 
skVarsFromSIdxUnnestMap.get(sIndexIdx));
+}
+}
+
 // For R-Tree case: if the given secondary key field variable is used 
only in the select or join condition,
 // we were not able to catch the mapping between the original 
secondary key field and the newly restored
 // secondary key field in the assign operator in the right path.
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
new file mode 100644
index 000..3259a28
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Enable multiway similarity joins

2018-09-24 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1076

to look at the new patch set (#91).

Change subject: [NO ISSUE][COMP][RT] Enable multiway similarity joins
..

[NO ISSUE][COMP][RT] Enable multiway similarity joins

- Enable the FuzzyJoinRule that transforms
  a nested-loop-similarity-join plan to a three-stage-similarity join.
- Modify FuzzyJoinRuleCollections.
  - Add the ExtractCommonExpressionRule to extract common expressions
in the star-like multiple similarity join substitutions.
  - Add the InlineSubplanInputForNestedTupleSourceRule to translate
the generated subplan from the similarity function-derived
substitution into join in case of nested schemas.
  - Use similarity-jaccard-prefix to enable the pp+ join strategy.
  - Use the right side to build the heavy hash join on
the prefix tokens from both sides.
  - Add RemoveAssign/Variables/AggRules to iteratively remove unused
assign/vars once FuzzyJoinRule is applied in each round.
- Add three new optimization cases for multiway similarity joins.
  - link-like multiway similarity joins
  - star-like multiway similarity joins
  - hybrid multiway similarity joins with the both styles of similarity joins.
- Add a check whether a similarity function is on
  a select over an existing similarity join.
- Change the inverted-index-based similarity join to the three-stage-similarity 
join
  due to efficiency considerations.

Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/compiler/provider/DefaultRuleSetFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/FuzzyUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/FuzzyJoinRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
A asterixdb/asterix-app/data/dblp-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/dblp-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csx-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/csxauthors.adm
A asterixdb/asterix-app/data/pub-small/dblp-small-multi-id.txt
A asterixdb/asterix-app/data/pub-small/dblpauthors.adm
M asterixdb/asterix-app/src/test/resources/optimizerts/ignore.txt
M asterixdb/asterix-app/src/test/resources/optimizerts/only.txt
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-hybrid.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-selflink.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-simple.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx-star.aql
M asterixdb/asterix-app/src/test/resources/optimizerts/queries/fj-dblp-csx.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-dual-order.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/similarity/jaccard-similarity-join-right-ahead.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-hybrid.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-selflink.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-simple.plan
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx-star.plan
M asterixdb/asterix-app/src/test/resources/optimizerts/results/fj-dblp-csx.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/ngram-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join-noeqjoin/word-jaccard-inline.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/issue741.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/leftouterjoin-probe-pidx-with-join-jaccard-check-idx_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-fuzzyeq-jaccard_03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-join/ngram-jaccard-check_03.plan
M

Change in asterixdb[master]: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases

2018-09-23 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new patch set (#3).

Change subject: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases
..

[ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Remove most of AQL test cases that are not directly related to the similarity 
join
  feature. Those test cases are needed to guard the AQL+ framework.

Change-Id: I475092bda3efea9953c06f93146e19816efef7eb
---
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it.xml
2 files changed, 4 insertions(+), 6,972 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/76/2976/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2976
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I475092bda3efea9953c06f93146e19816efef7eb
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases

2018-09-23 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new patch set (#2).

Change subject: [ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases
..

[ASTERIXDB-2454][TEST] Remove most of AQL runtime test cases

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Remove most of AQL test cases that are not directly related to the similarity 
join
  feature. Those test cases are needed to guard the AQL+ framework.

Change-Id: I475092bda3efea9953c06f93146e19816efef7eb
---
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it.xml
2 files changed, 4 insertions(+), 6,972 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/76/2976/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2976
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I475092bda3efea9953c06f93146e19816efef7eb
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-09-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..


Patch Set 3:

(2 comments)

@Dmitry: thanks. I have addressed your comments.

https://asterix-gerrit.ics.uci.edu/#/c/2854/3/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java:

Line 1205: for (LogicalVariable v : usedVarsAfterTopOp) {
> This loop can be replaced by varsUsedInTopOpButNotAfterwards.removeAll(used
Done


https://asterix-gerrit.ics.uci.edu/#/c/2854/3/asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
File 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp:

Line 48: statues_count: int64
> should be statuses_count here and everywhere below
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2854
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-09-22 Thread Taewoo Kim (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2854

to look at the new patch set (#5).

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
8 files changed, 278 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/2854/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2854
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-09-22 Thread Taewoo Kim (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2854

to look at the new patch set (#4).

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/btree-sidx-composite-idxonly-04/btree-sidx-composite-idxonly-04.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
8 files changed, 282 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/2854/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2854
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2440][COMP] Compiler failure with IN expression

2018-08-13 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2440][COMP] Compiler failure with IN expression
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2890
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f9734c37b074ef9b16bc7ae403ae079b86b0e9a
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2438][COMP] Allow assigns when pushing limit into...

2018-08-08 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2438][COMP] Allow assigns when pushing limit into 
data scan
..


Patch Set 1: Code-Review+2

(1 comment)

Can you remove a trailing white space? Just a minor comment.

https://asterix-gerrit.ics.uci.edu/#/c/2856/1/asterixdb/asterix-app/src/test/resources/runtimets/results/limit/push-limit-to-primary-lookup-select/push-limit-to-primary-lookup-select.5.adm
File 
asterixdb/asterix-app/src/test/resources/runtimets/results/limit/push-limit-to-primary-lookup-select/push-limit-to-primary-lookup-select.5.adm:

Line 25: order (ASC, $$24) (ASC, $$25) 
Please remove this whitespace.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2856
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3b68890d034dded3db7572462f47161721aa18fa
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-08-07 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new patch set (#3).

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
3 files changed, 165 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/2854/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2854
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-08-07 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new patch set (#2).

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
3 files changed, 166 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/2854/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2854
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a compo...

2018-08-07 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2854

Change subject: [ASTERIXDB-2437][COMP] Ensure the index-only plan on a 
composite index
..

[ASTERIXDB-2437][COMP] Ensure the index-only plan on a composite index

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the proper build of an index-only plan on a composite index
  where both fields are used in the SELECT condition and only one field
  is returned.

Change-Id: Idcc4cbe08323e0c6edb4a01637b2017128da1ab5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-sidx-idxonly-10.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-10.plan
3 files changed, 177 insertions(+), 11 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/54/2854/1

diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
index c925b55..b1de3a1 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
@@ -1053,7 +1053,7 @@
 constAssignOp.setSourceLocation(sourceLoc);
 if (constantAssignVarUsedInTopOp) {
 // Places this assign after the secondary index-search op.
-constAssignOp.getInputs().add(new 
MutableObject(inputOp));
+constAssignOp.getInputs().add(new 
MutableObject<>(inputOp));
 constAssignOp.setExecutionMode(ExecutionMode.PARTITIONED);
 
context.computeAndSetTypeEnvironmentForOperator(constAssignOp);
 currentOp = constAssignOp;
@@ -1082,7 +1082,7 @@
 splitOp = new SplitOperator(2,
 new MutableObject(new 
VariableReferenceExpression(condSplitVars.get(0;
 splitOp.setSourceLocation(sourceLoc);
-splitOp.getInputs().add(new 
MutableObject(currentOp));
+splitOp.getInputs().add(new MutableObject<>(currentOp));
 splitOp.setExecutionMode(ExecutionMode.PARTITIONED);
 context.computeAndSetTypeEnvironmentForOperator(splitOp);
 
@@ -1189,11 +1189,40 @@
 LogicalVariable replacedVar = context.newVar();
 origPKRecAndSKVarToleftPathMap.put(tVar, replacedVar);
 origVarToOutputVarMap.put(skVarsFromSIdxUnnestMap.get(sIndexIdx), 
tVar);
-unionVarMap.add(new Triple(replacedVar,
+unionVarMap.add(new Triple<>(replacedVar,
 skVarsFromSIdxUnnestMap.get(sIndexIdx), tVar));
 // Constructs the mapping between the SK from the original 
data-scan
 // and the SK from the secondary index search since they are 
different logical variables.
 origVarToSIdxUnnestMapOpVarMap.put(tVar, 
skVarsFromSIdxUnnestMap.get(sIndexIdx));
+}
+
+// For B-Tree case: if the given secondary key field variable is used 
only in the select or
+// join condition, we were not able to catch the mapping between the 
the SK from the original
+// data-scan and the SK from the secondary index search since they are 
different logical variables.
+// (E.g., we are sending a query on a composite index but returns only 
one field.)
+List varsUsedInTopOpButNotAfterwards = new 
ArrayList<>();
+copyVarsToAnotherList(uniqueUsedVarsInTopOp, 
varsUsedInTopOpButNotAfterwards);
+for (LogicalVariable v : usedVarsAfterTopOp) {
+if (varsUsedInTopOpButNotAfterwards.contains(v)) {
+varsUsedInTopOpButNotAfterwards.remove(v);
+}
+}
+if (idxType == IndexType.BTREE) {
+for (LogicalVariable v : varsUsedInTopOpButNotAfterwards) {
+int sIndexIdx = 
chosenIndexFieldNames.indexOf(subTree.getVarsToFieldNameMap().get(v));
+// For the join-case, the match might not exist.
+// In this case, we just propagate the variables later.
+if (sIndexIdx == -1) {
+continue;
+}
+// B-Tree case:
+LogicalVariable replacedVar = context.newVar();
+origPKRecAndSKVarToleftPathMap.put(v, replacedVar);
+
origVarToOutputVarMap.put(skVarsFromSIdxUnnestMap.get(sIndexIdx), v);
+// Constructs the mapping between the SK from the original 
data-scan
+// and the SK from the secondary index search since they are 
different logical variables.
+

Change in asterixdb[master]: [ASTERIXDB-2434][COMP] Ensure execution mode before EnforceS...

2018-08-07 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2434][COMP] Ensure execution mode before 
EnforceStructuralPropertiesRule
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2846
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iafe8d1dc0ae75735ba7f52a2ec5c82085c9e7f73
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2428][COMP] Incorrect result with limit if offset...

2018-07-27 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2428][COMP] Incorrect result with limit if offset is 
negative
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2821
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I2238cc5d8f48e14aa2b74d120248a4848dd35691
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2424][COMP] No limit pushdown if multiple limit c...

2018-07-26 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2424][COMP] No limit pushdown if multiple limit 
clauses
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2815
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id92bb99c57d1363740b0ccbb09bceb59da968ca0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2420][COMP] Internal error with limit clause

2018-07-24 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2420][COMP] Internal error with limit clause
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2805
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6c77b7e24d6d0a95f3c772301d592c5b5c478119
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2417][COMP] Fix issues with LIMIT clause

2018-07-20 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2417][COMP] Fix issues with LIMIT clause
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2788
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I986fab3e79b072aa2441af293f3e16c3f37bf508
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-2418][COMP] Fix NPE in SqlppExpressionToPlanTransl...

2018-07-19 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: ASTERIXDB-2418][COMP] Fix NPE in SqlppExpressionToPlanTranslator
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2789
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie3207d077480a90fd00fdd59923ea76e2ca6db25
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2411][COMP] Error when aggregate inside between o...

2018-07-13 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2411][COMP] Error when aggregate inside between 
operator
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2759
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0839e4ce89da524050ce30542fc8788963c4cbe7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2380][COMP] Incorrect comparison of exchange oper...

2018-07-12 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2380][COMP] Incorrect comparison of exchange 
operators
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2754
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib9948531cdbed7ecddd9ace1d179c97523dbaec4
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2409][RT] Make the full-text search return entire...

2018-07-10 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2409][RT] Make the full-text search return entire 
results
..


[ASTERIXDB-2409][RT] Make the full-text search return entire results

- user model changes: no
- storage format changes: no
- interface changes: no

details:
Currently, the full-text search returns a subset of true results
(only one frame) for multiple keywords queries if the result size
is greater than one frame. This patch fixes the issue.

Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2742
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M asterixdb/asterix-app/data/fulltext/cloudberry_sample_tweet.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.4.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.3.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.4.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.5.adm
R 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-08/fulltext-index-08.8.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.3.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
13 files changed, 15,667 insertions(+), 109 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; No violations found; ; Verified




-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2742
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: [ASTERIXDB-2409][RT] Make the full-text search return entire...

2018-07-09 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2742

to look at the new patch set (#2).

Change subject: [ASTERIXDB-2409][RT] Make the full-text search return entire 
results
..

[ASTERIXDB-2409][RT] Make the full-text search return entire results

- user model changes: no
- storage format changes: no
- interface changes: no

details:
Currently, the full-text search returns a subset of true results
(only one frame) for multiple keywords queries if the result size
is greater than one frame. This patch fixes the issue.

Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
---
M asterixdb/asterix-app/data/fulltext/cloudberry_sample_tweet.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.4.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.3.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.4.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.5.adm
R 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-08/fulltext-index-08.8.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.3.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
13 files changed, 15,667 insertions(+), 109 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/42/2742/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2742
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2409][RT] Make the full-text search return entire...

2018-07-09 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2742

Change subject: [ASTERIXDB-2409][RT] Make the full-text search return entire 
results
..

[ASTERIXDB-2409][RT] Make the full-text search return entire results

- user model changes: no
- storage format changes: no
- interface changes: no

details:
Currently, the full-text search returns a subset of true results
(only one frame) for multiple keywords queries if the result size
is greater than one frame. This patch fixes the issue.

Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
---
M asterixdb/asterix-app/data/fulltext/cloudberry_sample_tweet.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-09/fulltext-index-09.4.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.3.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.4.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-08/fulltext-08.5.adm
R 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-08/fulltext-index-08.8.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.3.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-09/fulltext-index-09.4.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
13 files changed, 15,375 insertions(+), 109 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/42/2742/1


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2742
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8e4a6b949017259b015798fac767bbee35e37b20
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim

Change in asterixdb[master]: Index-only plan

2018-06-20 Thread Taewoo Kim (Code Review) 
Taewoo Kim has restored this change.

Change subject: Index-only plan
..


Restored

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/744
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: restore
Gerrit-Change-Id: Ifa02c13d4fddd880e1ee9e85eef6577301fb4560
Gerrit-PatchSet: 26
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Young-Seok Kim

Change in asterixdb[master]: [NO ISSUE][RT] Eliminate HashMap lookup during comparision

2018-05-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][RT] Eliminate HashMap lookup during comparision
..


Patch Set 1: Code-Review+2

LGTM too.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2655
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie5b8efdcb226a8818deaf57a6305a18107544feb
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Relax JAVA_HOME check for sample cluster

2018-05-09 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: Relax JAVA_HOME check for sample cluster
..


Patch Set 3: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2638
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6b9421b9b480eeea94644a227faf614c2ccf2fb2
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Eliminate S Lock for Disk Components

2018-05-08 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][STO] Eliminate S Lock for Disk Components
..


Patch Set 1:

(1 comment)

Thanks for taking care of this.

https://asterix-gerrit.ics.uci.edu/#/c/2637/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java:

Line 109: if (includeMutableComponent && 
queueHead.getCursorIndex() == 0) {
Why this additional check (queueHead.getCursorIndex()) is needed? Isn't 
includeMutableComponent variable enough, since we set it to false once we 
finish fetching all tuples from the in-memory component.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2637
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id6ec999b131cd6609d588966d7ae7788f429ab9d
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Index-only plan

2018-05-08 Thread Taewoo Kim (Code Review) 
Taewoo Kim has abandoned this change.

Change subject: Index-only plan
..


Abandoned

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/744
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ifa02c13d4fddd880e1ee9e85eef6577301fb4560
Gerrit-PatchSet: 26
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-Reviewer: Young-Seok Kim

Change in asterixdb[master]: [ASTERIXDB-2344] LIMIT pushdown for primary index

2018-05-01 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2344] LIMIT pushdown for primary index
..


Patch Set 17: Code-Review+1

LGTM. I only have a few minor comments.

#1. Do we want to keep the optimizer tests? or just use runtime test framework? 
It looks like you are using the runtime test to generate the plan and check it 
without introducing the optimizer tests.

#2. Can you add a description to each test? What are they supposed to check?

#3. Before declaring outputLimit, selectCondition, and tupleFilterFactory 
variables in each class, can we add a simple comment about them? What's the 
purpose of them?

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2541
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I824fcad79995325e12a1a81d629160025294b915
Gerrit-PatchSet: 17
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim (please use wangs...@gmail.com) 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

2018-04-27 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components
..


[ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Let the index-only plan properly work on the disk components of B+Tree.
Currently, only the records from in-memory components has been applied because
searchCallback.proceed() is only called for those.

Change-Id: I655eacc5517352a382d1b61f7b630f0f307b7c7b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2623
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
---
M asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.3.metrics.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.4.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index/secondary-index.3.metrics.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/metrics/secondary-index-index-only/secondary-index-index-only.3.regexadm
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
9 files changed, 169 insertions(+), 41 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  Jenkins: Verified; ; Verified
  Dmitry Lychagin: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml
index f65ede3..da7ba31 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml
@@ -35,5 +35,10 @@
 secondary-index
   
 
+
+  
+secondary-index-index-only
+  
+
   
 
\ No newline at end of file
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.1.ddl.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.1.ddl.sqlpp
new file mode 100644
index 000..c4d77ca
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.1.ddl.sqlpp
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*
+ * Description  : Processed objects metrics on secondary index scan when it's 
an index-only plan
+ * Expected Res : Success
+ * Date : 27 Apr 2018
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+
+use test;
+
+create type test.AddressType as
+{
+  number : bigint,
+  street : string,
+  city : string
+};
+
+create type test.CustomerType as
+ closed {
+  cid : bigint,
+  name : string,
+  age : bigint?,
+  address : AddressType?,
+  lastorder : {
+  oid : bigint,
+  total : float
+  }
+};
+
+create dataset Customers(CustomerType) primary key cid;
+create index customer_name_idx on Customers(name);
\ No newline at end of file
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.2.update.sqlpp

Change in asterixdb[master]: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

2018-04-27 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2623

to look at the new patch set (#2).

Change subject: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components
..

[ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Let the index-only plan properly work on the disk components of B+Tree.
Currently, only the records from in-memory components has been applied because
searchCallback.proceed() is only called for those.

Change-Id: I655eacc5517352a382d1b61f7b630f0f307b7c7b
---
M asterixdb/asterix-app/src/test/resources/runtimets/metrics.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.3.metrics.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index-index-only/secondary-index-index-only.4.ddl.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/metrics/secondary-index/secondary-index.3.metrics.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/metrics/secondary-index-index-only/secondary-index-index-only.3.regexadm
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
9 files changed, 169 insertions(+), 41 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/23/2623/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2623
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I655eacc5517352a382d1b61f7b630f0f307b7c7b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

2018-04-27 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2623

Change subject: [ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components
..

[ASTERIXDB-2374][RT] Index-only plan on B+Tree disk components

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Let the index-only plan properly work on the disk components of B+Tree.
Currently, only the records from in-memory components has been applied because
searchCallback.proceed() is only called for those.

Change-Id: I655eacc5517352a382d1b61f7b630f0f307b7c7b
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeSearchCursor.java
2 files changed, 43 insertions(+), 41 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/23/2623/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
index a675047..6eef5a9 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeRangeSearchCursor.java
@@ -105,48 +105,50 @@
 while (!outputPriorityQueue.isEmpty() || needPushElementIntoQueue) {
 if (!outputPriorityQueue.isEmpty()) {
 PriorityQueueElement queueHead = outputPriorityQueue.peek();
-if (canCallProceed && includeMutableComponent) {
+if (canCallProceed) {
 resultOfSearchCallbackProceed = 
searchCallback.proceed(queueHead.getTuple());
-if (!resultOfSearchCallbackProceed) {
-// In case proceed() fails and there is an in-memory 
component,
-// we can't simply use this element since there might 
be a change.
-PriorityQueueElement mutableElement = 
remove(outputPriorityQueue, 0);
-if (mutableElement != null) {
-// Copies the current queue head
-if (tupleBuilder == null) {
-tupleBuilder = new 
ArrayTupleBuilder(cmp.getKeyFieldCount());
+if (includeMutableComponent) {
+if (!resultOfSearchCallbackProceed) {
+// In case proceed() fails and there is an 
in-memory component,
+// we can't simply use this element since there 
might be a change.
+PriorityQueueElement mutableElement = 
remove(outputPriorityQueue, 0);
+if (mutableElement != null) {
+// Copies the current queue head
+if (tupleBuilder == null) {
+tupleBuilder = new 
ArrayTupleBuilder(cmp.getKeyFieldCount());
+}
+TupleUtils.copyTuple(tupleBuilder, 
queueHead.getTuple(), cmp.getKeyFieldCount());
+
copyTuple.reset(tupleBuilder.getFieldEndOffsets(), tupleBuilder.getByteArray());
+// Unlatches/unpins the leaf page of the index.
+rangeCursors[0].close();
+// Reconcile.
+searchCallback.reconcile(copyTuple);
+// Re-traverses the index.
+reusablePred.setLowKey(copyTuple, true);
+btreeAccessors[0].search(rangeCursors[0], 
reusablePred);
+
pushIntoQueueFromCursorAndReplaceThisElement(mutableElement);
+// now that we have completed the search and 
we have latches over the pages,
+// it is safe to complete the operation.. but 
as per the API of the callback
+// we only complete if we're producing this 
tuple
+// get head again
+queueHead = outputPriorityQueue.peek();
+/*
+ * We need to restart in one of two cases:
+ * 1. no more elements in the priority queue.
+ * 2. the key of the head has

Change in asterixdb[master]: [ASTERIXDB-2356][SQL] Syntax error when parsing CASE expression

2018-04-19 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2356][SQL] Syntax error when parsing CASE expression
..


Patch Set 2: Code-Review+2

(1 comment)

LGTM - only one minor comment

https://asterix-gerrit.ics.uci.edu/#/c/2571/2/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
File asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml:

Line 3598: case_05
I know this is not related to your change. But, this should be fixed? case_05 
--> case_07?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2571
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia9ded0d7a20aaa9cd81240821f5b903f262b0b6a
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2349][SITE] Revise fulltext and similarity docume...

2018-04-11 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2349][SITE] Revise fulltext and similarity 
documentation
..


[ASTERIXDB-2349][SITE] Revise fulltext and similarity documentation

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Update all examples in the fulltext and similarity documentation
using SQLPP.

Change-Id: Icd9c5bc6249feb03b4297bdc84b5f3aa0efcdc47
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2567
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 
---
M asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
M asterixdb/asterix-doc/src/site/markdown/aql/similarity.md
2 files changed, 105 insertions(+), 133 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  Jenkins: Verified; No violations found; ; Verified
  Ian Maxon: Looks good to me, approved



diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
index bc0b398..1328ed9 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
@@ -44,73 +44,61 @@
 ftcontains(Expression1, Expression2, {FullTextOption})
 ftcontains(Expression1, Expression2)
 
-For example, we can execute the following query to find tweet messages where 
the `message-text` field includes
+For example, we can execute the following query to find Chirp messages where 
the `messageText` field includes
 “voice” as a word. Please note that an FTS search is case-insensitive.
 Thus, "Voice" or "voice" will be evaluated as the same word.
 
-use dataverse TinySocial;
-
-for $msg in dataset TweetMessages
-where ftcontains($msg.message-text, "voice", {"mode":"any"})
-return {"id": $msg.id}
-
-The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructed Data 
in AsterixDB](primer.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB).
-
-The same query can be also expressed in the SQL++.
-
 use TinySocial;
 
-select element {"id":msg.id}
-from TweetMessages as msg
-where TinySocial.ftcontains(msg.`message-text`, "voice", 
{"mode":"any"})
+select element {"chirpId": msg.chirpId}
+from ChirpMessages msg
+where ftcontains(msg.messageText, "voice", {"mode":"any"});
+
+The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructed Data 
in 
AsterixDB](../sqlpp/primer-sqlpp.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB).
 
 The `Expression1` is an expression that should be evaluable as a string at 
runtime as in the above example
-where `$msg.message-text` is a string field. The `Expression2` can be a 
string, an (un)ordered list
+where `msg.messageText` is a string field. The `Expression2` can be a string, 
an (un)ordered list
 of string value(s), or an expression. In the last case, the given expression 
should be evaluable
 into one of the first two types, i.e., into a string value or an (un)ordered 
list of string value(s).
 
 The following examples are all valid expressions.
 
-   ... where ftcontains($msg.message-text, "sound")
-   ... where ftcontains($msg.message-text, "sound", {"mode":"any"})
-   ... where ftcontains($msg.message-text, ["sound", "system"], 
{"mode":"any"})
-   ... where ftcontains($msg.message-text, {{"speed", "stand", 
"customization"}}, {"mode":"all"})
-   ... where ftcontains($msg.message-text, let $keyword_list := ["voice", 
"system"] return $keyword_list, {"mode":"all"})
-   ... where ftcontains($msg.message-text, $keyword_list, {"mode":"any"})
-
-In the last example above, `$keyword_list` should evaluate to a string or an 
(un)ordered list of string value(s).
+   ... where ftcontains(msg.messageText, "sound")
+   ... where ftcontains(msg.messageText, "sound", {"mode":"any"})
+   ... where ftcontains(msg.messageText, ["sound", "system"], 
{"mode":"any"})
+   ... where ftcontains(msg.messageText, {{"speed", "stand", 
"customization"}}, {"mode":"all"})
 
 The last `FullTextOption` parameter clarifies the given FTS request. If you 
omit the `FullTextOption` parameter,
 then the default value will be set for each possible option. Currently, we 
only have one option named `mode`.
 And as we extend the FTS feature, more options will be added. Please note that 
the format of `FullTextOption`
 is a record, thus you need to put the option(s) in a record `{}`.
 The `mode` option indicates whether the given FTS query is a conjunctive (AND) 
or disjunctive (OR) search request.
-This option can be either `“any”` or `“all”`. The default value for `mode` is 
`“all”`. If one specifies `“any”`,
-a disjunctive

Change in asterixdb[master]: [ASTERIXDB-2349][SITE] Revise fulltext and similarity docume...

2018-04-04 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new patch set (#2).

Change subject: [ASTERIXDB-2349][SITE] Revise fulltext and similarity 
documentation
..

[ASTERIXDB-2349][SITE] Revise fulltext and similarity documentation

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Update all examples in the fulltext and similarity documentation
using SQLPP.

Change-Id: Icd9c5bc6249feb03b4297bdc84b5f3aa0efcdc47
---
M asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
M asterixdb/asterix-doc/src/site/markdown/aql/similarity.md
2 files changed, 105 insertions(+), 133 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/67/2567/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2567
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icd9c5bc6249feb03b4297bdc84b5f3aa0efcdc47
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [ASTERIXDB-2349][SITE] Revise fulltext and similarity docume...

2018-04-04 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2567

Change subject: [ASTERIXDB-2349][SITE] Revise fulltext and similarity 
documentation
..

[ASTERIXDB-2349][SITE] Revise fulltext and similarity documentation

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Update all examples in the fulltext and similarity documentation
using SQLPP.

Change-Id: Icd9c5bc6249feb03b4297bdc84b5f3aa0efcdc47
---
M asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
M asterixdb/asterix-doc/src/site/markdown/aql/similarity.md
2 files changed, 104 insertions(+), 132 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/67/2567/1

diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md 
b/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
index bc0b398..1328ed9 100644
--- a/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
+++ b/asterixdb/asterix-doc/src/site/markdown/aql/fulltext.md
@@ -44,73 +44,61 @@
 ftcontains(Expression1, Expression2, {FullTextOption})
 ftcontains(Expression1, Expression2)
 
-For example, we can execute the following query to find tweet messages where 
the `message-text` field includes
+For example, we can execute the following query to find Chirp messages where 
the `messageText` field includes
 “voice” as a word. Please note that an FTS search is case-insensitive.
 Thus, "Voice" or "voice" will be evaluated as the same word.
 
-use dataverse TinySocial;
-
-for $msg in dataset TweetMessages
-where ftcontains($msg.message-text, "voice", {"mode":"any"})
-return {"id": $msg.id}
-
-The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructed Data 
in AsterixDB](primer.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB).
-
-The same query can be also expressed in the SQL++.
-
 use TinySocial;
 
-select element {"id":msg.id}
-from TweetMessages as msg
-where TinySocial.ftcontains(msg.`message-text`, "voice", 
{"mode":"any"})
+select element {"chirpId": msg.chirpId}
+from ChirpMessages msg
+where ftcontains(msg.messageText, "voice", {"mode":"any"});
+
+The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructed Data 
in 
AsterixDB](../sqlpp/primer-sqlpp.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB).
 
 The `Expression1` is an expression that should be evaluable as a string at 
runtime as in the above example
-where `$msg.message-text` is a string field. The `Expression2` can be a 
string, an (un)ordered list
+where `msg.messageText` is a string field. The `Expression2` can be a string, 
an (un)ordered list
 of string value(s), or an expression. In the last case, the given expression 
should be evaluable
 into one of the first two types, i.e., into a string value or an (un)ordered 
list of string value(s).
 
 The following examples are all valid expressions.
 
-   ... where ftcontains($msg.message-text, "sound")
-   ... where ftcontains($msg.message-text, "sound", {"mode":"any"})
-   ... where ftcontains($msg.message-text, ["sound", "system"], 
{"mode":"any"})
-   ... where ftcontains($msg.message-text, {{"speed", "stand", 
"customization"}}, {"mode":"all"})
-   ... where ftcontains($msg.message-text, let $keyword_list := ["voice", 
"system"] return $keyword_list, {"mode":"all"})
-   ... where ftcontains($msg.message-text, $keyword_list, {"mode":"any"})
-
-In the last example above, `$keyword_list` should evaluate to a string or an 
(un)ordered list of string value(s).
+   ... where ftcontains(msg.messageText, "sound")
+   ... where ftcontains(msg.messageText, "sound", {"mode":"any"})
+   ... where ftcontains(msg.messageText, ["sound", "system"], 
{"mode":"any"})
+   ... where ftcontains(msg.messageText, {{"speed", "stand", 
"customization"}}, {"mode":"all"})
 
 The last `FullTextOption` parameter clarifies the given FTS request. If you 
omit the `FullTextOption` parameter,
 then the default value will be set for each possible option. Currently, we 
only have one option named `mode`.
 And as we extend the FTS feature, more options will be added. Please note that 
the format of `FullTextOption`
 is a record, thus you need to put the option(s) in a record `{}`.
 The `mode` option indicates whether the given FTS query is a conjunctive (AND) 
or disjunctive (OR) search request.
-This option can be either `“any”` or `“all”`. The default value for `mode` is 
`“all”`. If one specifies `“any”`,
-a disjunctive search will be conducted. For example, the following query will 
find documents whose `message-text`
+This option can be either `“all”` (AND) or `“any”` (OR). The default value for 
`mode` is `“all”`. If one specifies `“any”`,
+a disjunctive search will be conducted. For example, the following query will 
find documents whose

Change in asterixdb[master]: [ASTERIXDB-2344] LIMIT pushdown for primary index

2018-04-03 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2344] LIMIT pushdown for primary index
..


Patch Set 5:

(13 comments)

Overall, it looks good.

https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java:

Line 127: if (unnestMap instanceof UnnestMapOperator) {
1. unnestmap.getOperatorTag() == ... would be better.
2. We also have LeftOuterUnnestMapOperator. So, I think you need to consider 
both cases. AbstractUnnestMapOperator is safe.


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushLimitIntoPrimaryRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushLimitIntoPrimaryRule.java:

Line 45:  */
Again, it should be AbstractUnnestMapOperator.


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/limit/limit-scan-select/limit-scan-select.1.ddl.sqlpp
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/limit/limit-scan-select/limit-scan-select.1.ddl.sqlpp:

Line 36: create  dataset DBLP1(DBLPType) primary key id;
Test case name: push-limit-to-primary-scan-select ?


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/limit/limit-secondary-index-select/limit-secondary-index-select.1.ddl.sqlpp
File 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/limit/limit-secondary-index-select/limit-secondary-index-select.1.ddl.sqlpp:

Line 49: 
Why is this case added?


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-app/src/test/resources/runtimets/results/limit/limit-scan-select/limit-scan-select.3.adm
File 
asterixdb/asterix-app/src/test/resources/runtimets/results/limit/limit-scan-select/limit-scan-select.3.adm:

Line 20:   -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
Is this actual result that we expect?


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/asterixdb/asterix-app/src/test/resources/runtimets/results/limit/limit-scan/limit-scan.3.adm
File 
asterixdb/asterix-app/src/test/resources/runtimets/results/limit/limit-scan/limit-scan.3.adm:

Line 20:   -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
Is this actual result that we expect?


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestMapOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/UnnestMapOperator.java:

Line 36: private long outputLimit = -1;
Should be moved to the parent.


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/UsedVariableVisitor.java:

Line 310: }
Should be applied to visitLeftUnnestMapOperator too.


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitor.java:

Line 336: plan.append(" select condition 
(").append(selectCondition.getValue().accept(exprVisitor, indent))
maybe "condition"?


Line 345: plan.append(" output limit 
").append(String.valueOf(outputLimit));
Maybe just "limit"?


https://asterix-gerrit.ics.uci.edu/#/c/2541/5/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/dataflow/IndexSearchOperatorNodePushable.java:

Line 278: try {
Maybe we need to add one more if() here to not go through the following for 
loop?

if (!finished) { }


Line 385: 
Why is the following class needed in this class? Also, can you put a comment 
about this class?


Line 417: "getFrameTupleAccessor is not supported by 
ReferenceFrameTupleReference");
Let's use ErrorCode?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2541
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I824fcad79995325e12a1a81d629160025294b915
Gerrit-PatchSet:

Change in asterixdb[master]: [ASTERIXDB-2339] Add a new inverted index merge cursor

2018-03-24 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2339] Add a new inverted index merge cursor
..


Patch Set 5:

(3 comments)

Generally, how opType is used in the bulk loader?

https://asterix-gerrit.ics.uci.edu/#/c/2519/5/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex.java:

Line 394: 
Let's not make an unnecessary change?


Line 411: componentBulkLoader.end();
Let's not make an unnecessary change?


https://asterix-gerrit.ics.uci.edu/#/c/2519/5/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-invertedindex-test/src/test/java/org/apache/hyracks/storage/am/lsm/invertedindex/common/AbstractInvertedIndexTest.java
File 
hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-invertedindex-test/src/test/java/org/apache/hyracks/storage/am/lsm/invertedindex/common/AbstractInvertedIndexTest.java:

Line 87: 
//LSMInvertedIndexTestUtils.compareActualAndExpectedIndexesRangeSearch(testCtx);
Remove this comment?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2519
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I57d039cd7e08033884529a204bff9acffd96d9bb
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverte...

2018-03-23 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted 
list
..


[ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted list

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Fix a bug that when a page of an inverted list
that is pinned to the buffer cache is accessed concurrently.

Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2517
Reviewed-by: Luo Chen 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
1 file changed, 5 insertions(+), 3 deletions(-)

Approvals:
  Luo Chen: Looks good to me, approved
  Anon. E. Moose #1000171: 
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; No violations found; ; Verified



diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
index da3f079..7f3d12f 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
@@ -220,9 +220,11 @@
 // Assumption: processing inverted list takes time; so, we don't 
want to keep them on the buffer cache.
 // Rather, we utilize the assigned working memory (buffers).
 tmpBuffer = page.getBuffer();
-tmpBuffer.rewind();
-buffers.get(currentBufferIdx).rewind();
-buffers.get(currentBufferIdx).put(tmpBuffer);
+
+// Copies the entire content of the page to the current buffer in 
the working memory.
+System.arraycopy(tmpBuffer.array(), 0, 
buffers.get(currentBufferIdx).array(), 0,
+buffers.get(currentBufferIdx).capacity());
+
buffers.get(currentBufferIdx).position(buffers.get(currentBufferIdx).capacity());
 
 currentBufferIdx++;
 bufferCache.unpin(page);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2517
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverte...

2018-03-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted 
list
..


Patch Set 1:

(1 comment)

Thanks!!!

https://asterix-gerrit.ics.uci.edu/#/c/2517/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java:

PS1, Line 225: tmpBuffer.limit()
> To be safer, it's better to use buffers.get(currentBufferIdx).limit() inste
Yes. That makes sense. I replaced this with capacity to be safe. Limit() can be 
changed...


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2517
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverte...

2018-03-22 Thread Taewoo Kim (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2517

to look at the new patch set (#2).

Change subject: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted 
list
..

[ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted list

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Fix a bug that when a page of an inverted list
that is pinned to the buffer cache is accessed concurrently.

Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
1 file changed, 5 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/17/2517/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2517
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen

Change in asterixdb[master]: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverte...

2018-03-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2517

Change subject: [ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted 
list
..

[ASTERIXDB-2338][RT] Allow concurrent accesses to an inverted list

- user model changes: no
- storage format changes: no
- interface changes: no

Details: Fix a bug that when a page of an inverted list
that is pinned to the buffer cache is accessed concurrently.

Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
1 file changed, 4 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/17/2517/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
index da3f079..dffa8c2 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
@@ -220,9 +220,10 @@
 // Assumption: processing inverted list takes time; so, we don't 
want to keep them on the buffer cache.
 // Rather, we utilize the assigned working memory (buffers).
 tmpBuffer = page.getBuffer();
-tmpBuffer.rewind();
-buffers.get(currentBufferIdx).rewind();
-buffers.get(currentBufferIdx).put(tmpBuffer);
+
+// Copies the entire content of the page to the current buffer in 
the working memory.
+System.arraycopy(tmpBuffer.array(), 0, 
buffers.get(currentBufferIdx).array(), 0, tmpBuffer.limit());
+buffers.get(currentBufferIdx).position(tmpBuffer.limit());
 
 currentBufferIdx++;
 bufferCache.unpin(page);

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2517
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0d6e7e7188efe1f08016af0ab1840bc0cb59d49c
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim

Change in asterixdb[master]: [NO ISSUE][COMP][RT] Rename index-only test case directory

2018-03-10 Thread Taewoo Kim (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2472

to look at the new patch set (#2).

Change subject: [NO ISSUE][COMP][RT] Rename index-only test case directory
..

[NO ISSUE][COMP][RT] Rename index-only test case directory

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Rename index-only test case directories to a shorter version
  not to cause issues on Windows environment.

Change-Id: I360bd650391586a0101cb2eb639c330534753112
---
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-pidx-to-sidx-idxonly-equi-join_01/btree-pidx-to-sidx-idxonly-equi-join_01.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-pidx-to-sidx-idxonly-equi-join_01/btree-pidx-to-sidx-idxonly-equi-join_01.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-pidx-to-sidx-idxonly-equi-join_01/btree-pidx-to-sidx-idxonly-equi-join_01.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-pidx-equi-join_01/btree-sidx-idxonly-to-pidx-equi-join_01.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-pidx-equi-join_01/btree-sidx-idxonly-to-pidx-equi-join_01.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-pidx-equi-join_01/btree-sidx-idxonly-to-pidx-equi-join_01.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-idxonly-to-sidx-idxonly-equi-join_01.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-pidx-equi-join_01/btree-sidx-non-idxonly-to-pidx-equi-join_01.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-pidx-equi-join_01/btree-sidx-non-idxonly-to-pidx-equi-join_01.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-pidx-equi-join_01/btree-sidx-non-idxonly-to-pidx-equi-join_01.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-join/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01/btree-sidx-non-idxonly-to-sidx-idxonly-equi-join_01.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-pidx-join-btree-sidx3-idxonly/probe-pidx-join-btree-sidx3-idxonly.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-pidx-join-btree-sidx3-idxonly/probe-pidx-join-btree-sidx3-idxonly.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-pidx-join-btree-sidx3-idxonly/probe-pidx-join-btree-sidx3-idxonly.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-pidx1/probe-sidx-btree-idxonly-join-btree-pidx1.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-pidx1/probe-sidx-btree-idxonly-join-btree-pidx1.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-pidx1/probe-sidx-btree-idxonly-join-btree-pidx1.3.query.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly.1.ddl.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly.2.update.aql
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/index-leftouterjoin/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly/probe-sidx-btree-idxonly-join-btree-sidx1-idxonly.3.query.aql
R

Change in asterixdb[master]: [NO ISSUE][RT] no latching on an inverted list during merge

2018-03-01 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [NO ISSUE][RT] no latching on an inverted list during merge
..


[NO ISSUE][RT] no latching on an inverted list during merge

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not conduct a latching on an inverted list during
  a disk component merge of an inverted index since
  a disk component is immutable.

Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2444
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
1 file changed, 1 insertion(+), 3 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; No violations found; ; Verified



diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
index ca0f40b..b033b99 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
@@ -30,7 +30,7 @@
 import org.apache.hyracks.storage.common.file.BufferedFileHandle;
 
 /**
- * A simple scan cursor that only reads a frame by frame from the inverted 
list. This cursor does not
+ * A simple scan cursor that only reads a frame by frame from the inverted 
list on disk. This cursor does not
  * conduct a binary search. It only supports the scan operation. The main 
purpose of this cursor is
  * doing a full-scan of an inverted list during a storage-component-merge 
process.
  */
@@ -111,14 +111,12 @@
 }
 currentPageId++;
 page = bufferCache.pin(BufferedFileHandle.getDiskPageId(fileId, 
currentPageId), false);
-page.acquireReadLatch();
 pinned = true;
 }
 
 @Override
 public void unloadPages() throws HyracksDataException {
 if (pinned) {
-page.releaseReadLatch();
 bufferCache.unpin(page);
 pinned = false;
 }

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: [ASTERIXDB-2307][COMP] Incorrect result with quantified expr...

2018-03-01 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2307][COMP] Incorrect result with quantified 
expression
..


Patch Set 2: Code-Review+2

LGTM! Only one minor comment: it would be wonderful if you could put a comment 
regarding how EVERY SATISFIES works.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2441
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I9e282071c87a9551829c31af43909970d307e417
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][RT] no latching on an inverted list during merge

2018-03-01 Thread Taewoo Kim (Code Review) 
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2444

to look at the new patch set (#2).

Change subject: [NO ISSUE][RT] no latching on an inverted list during merge
..

[NO ISSUE][RT] no latching on an inverted list during merge

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not conduct a latching on an inverted list during
  a disk component merge of an inverted index since
  a disk component is immutable.

Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
1 file changed, 1 insertion(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/44/2444/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [NO ISSUE][RT] no latching on an inverted list during merge

2018-03-01 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2444

Change subject: [NO ISSUE][RT] no latching on an inverted list during merge
..

[NO ISSUE][RT] no latching on an inverted list during merge

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not conduct a latching on an inverted list during
  a disk component merge of an inverted index since
  a disk component is immutable.

Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
1 file changed, 4 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/44/2444/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
index ca0f40b..1792275 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListScanCursor.java
@@ -30,7 +30,7 @@
 import org.apache.hyracks.storage.common.file.BufferedFileHandle;
 
 /**
- * A simple scan cursor that only reads a frame by frame from the inverted 
list. This cursor does not
+ * A simple scan cursor that only reads a frame by frame from the inverted 
list on disk. This cursor does not
  * conduct a binary search. It only supports the scan operation. The main 
purpose of this cursor is
  * doing a full-scan of an inverted list during a storage-component-merge 
process.
  */
@@ -105,20 +105,19 @@
 public void loadPages() throws HyracksDataException {
 if (pinned) {
 unloadPages();
+pinned = false;
 }
 if (currentPageId == endPageId) {
 return;
 }
 currentPageId++;
 page = bufferCache.pin(BufferedFileHandle.getDiskPageId(fileId, 
currentPageId), false);
-page.acquireReadLatch();
 pinned = true;
 }
 
 @Override
 public void unloadPages() throws HyracksDataException {
 if (pinned) {
-page.releaseReadLatch();
 bufferCache.unpin(page);
 pinned = false;
 }
@@ -162,6 +161,7 @@
 public void doClose() throws HyracksDataException {
 if (pinned) {
 unloadPages();
+pinned = false;
 }
 }
 
@@ -169,6 +169,7 @@
 public void doDestroy() throws HyracksDataException {
 if (pinned) {
 unloadPages();
+pinned = false;
 }
 }
 

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I266135596bbd5158d5fcf5ce3d1735d407be6f0f
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim

Change in asterixdb[master]: [ASTERIXDB-2302][COMP] Incorrect result with non-enforced index

2018-02-26 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2302][COMP] Incorrect result with non-enforced index
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I236c4c92d82bc3292193b2eb0f9b771042b3ca1a
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2296][COMP] proper handling of an optional subfie...

2018-02-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2296][COMP] proper handling of an optional subfield 
type
..


[ASTERIXDB-2296][COMP] proper handling of an optional subfield type

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Handle an optional subfield type properly in
  AbstractIntroduceAccessMethodRule.getFieldNameFromSubTree()

Change-Id: Ia80b6dbf549b77295f310b7c6f3a3f3f5a323114
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2419
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
Tested-by: Jenkins 
Contrib: Jenkins 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-return-optional-field.plan
3 files changed, 146 insertions(+), 3 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  Jenkins: Verified; No violations found; ; Verified
  Dmitry Lychagin: Looks good to me, approved



diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
index f4178d7..4f9b4df 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
@@ -30,6 +30,8 @@
 
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.common.config.DatasetConfig.IndexType;
+import org.apache.asterix.common.exceptions.CompilationException;
+import org.apache.asterix.common.exceptions.ErrorCode;
 import org.apache.asterix.dataflow.data.common.ExpressionTypeComputer;
 import org.apache.asterix.metadata.declared.MetadataProvider;
 import org.apache.asterix.metadata.entities.Index;
@@ -38,6 +40,7 @@
 import org.apache.asterix.om.base.AString;
 import org.apache.asterix.om.constants.AsterixConstantValue;
 import org.apache.asterix.om.functions.BuiltinFunctions;
+import org.apache.asterix.om.typecomputer.impl.TypeComputeUtils;
 import org.apache.asterix.om.types.ARecordType;
 import org.apache.asterix.om.types.ATypeTag;
 import org.apache.asterix.om.types.AbstractCollectionType;
@@ -872,9 +875,15 @@
 }
 
 if (!isByName) {
-fieldName = sourceVar.equals(metaVar)
-? ((ARecordType) 
metaType.getSubFieldType(parentFieldNames)).getFieldNames()[fieldIndex]
-: ((ARecordType) 
recordType.getSubFieldType(parentFieldNames)).getFieldNames()[fieldIndex];
+IAType subFieldType = sourceVar.equals(metaVar) ? 
metaType.getSubFieldType(parentFieldNames)
+: recordType.getSubFieldType(parentFieldNames);
+// Sub-field type can be AUnionType in case if it's 
optional. Thus, needs to get the actual type.
+subFieldType = 
TypeComputeUtils.getActualType(subFieldType);
+if (subFieldType.getTypeTag() != ATypeTag.OBJECT) {
+throw 
CompilationException.create(ErrorCode.TYPE_CONVERT, subFieldType,
+ARecordType.class.getName());
+}
+fieldName = ((ARecordType) 
subFieldType).getFieldNames()[fieldIndex];
 }
 if (optFuncExpr != null) {
 optFuncExpr.setSourceVar(funcVarIndex, ((AssignOperator) 
op).getVariables().get(assignVarIndex));
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
new file mode 100644
index 000..3221cc3
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may

Change in asterixdb[master]: [NO ISSUE] Fix template, again

2018-02-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] Fix template, again
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2420
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iaac1c2df7b2f1c2d70e869a47d4fc4ab90d5f5f2
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ian Maxon 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2296][COMP] proper handling of an optional subfie...

2018-02-22 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2419

to look at the new patch set (#3).

Change subject: [ASTERIXDB-2296][COMP] proper handling of an optional subfield 
type
..

[ASTERIXDB-2296][COMP] proper handling of an optional subfield type

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Handle an optional subfield type properly in
  AbstractIntroduceAccessMethodRule.getFieldNameFromSubTree()

Change-Id: Ia80b6dbf549b77295f310b7c6f3a3f3f5a323114
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-return-optional-field.plan
3 files changed, 146 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/19/2419/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2419
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia80b6dbf549b77295f310b7c6f3a3f3f5a323114
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim

Change in asterixdb[master]: [ASTERIXDB-2296][COMP] proper handling of an optional subfie...

2018-02-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2296][COMP] proper handling of an optional subfield 
type
..


Patch Set 1:

(3 comments)

Thanks!

https://asterix-gerrit.ics.uci.edu/#/c/2419/1/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java:

Line 881: if (subFieldType instanceof AUnionType) {
> There's a helper method for extracting actual type from a union: TypeComput
Done


Line 884: if (!(subFieldType instanceof ARecordType)) {
> Can we do getTypeTag() != OBJECT  here instead of instanceof?
Done


Line 888: fieldName = sourceVar.equals(metaVar) ? 
((ARecordType) subFieldType).getFieldNames()[fieldIndex]
> why conditional (?) expression here? both branches seem to be the same.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2419
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia80b6dbf549b77295f310b7c6f3a3f3f5a323114
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2296][COMP] proper handling of an optional subfie...

2018-02-22 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2419

Change subject: [ASTERIXDB-2296][COMP] proper handling of an optional subfield 
type
..

[ASTERIXDB-2296][COMP] proper handling of an optional subfield type

- user-model changes: no
- storage format changes: no
- interface changes: no

Details:
- Handle an optional subfield type properly in
  AbstractIntroduceAccessMethodRule.getFieldNameFromSubTree()

Change-Id: Ia80b6dbf549b77295f310b7c6f3a3f3f5a323114
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/results/filter/inverted-btree-search-return-optional-field.plan
3 files changed, 149 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/19/2419/1

diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
index f4178d7..41593dc 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
@@ -30,6 +30,8 @@
 
 import org.apache.asterix.common.config.DatasetConfig.DatasetType;
 import org.apache.asterix.common.config.DatasetConfig.IndexType;
+import org.apache.asterix.common.exceptions.CompilationException;
+import org.apache.asterix.common.exceptions.ErrorCode;
 import org.apache.asterix.dataflow.data.common.ExpressionTypeComputer;
 import org.apache.asterix.metadata.declared.MetadataProvider;
 import org.apache.asterix.metadata.entities.Index;
@@ -40,6 +42,7 @@
 import org.apache.asterix.om.functions.BuiltinFunctions;
 import org.apache.asterix.om.types.ARecordType;
 import org.apache.asterix.om.types.ATypeTag;
+import org.apache.asterix.om.types.AUnionType;
 import org.apache.asterix.om.types.AbstractCollectionType;
 import org.apache.asterix.om.types.BuiltinType;
 import org.apache.asterix.om.types.IAType;
@@ -872,9 +875,18 @@
 }
 
 if (!isByName) {
-fieldName = sourceVar.equals(metaVar)
-? ((ARecordType) 
metaType.getSubFieldType(parentFieldNames)).getFieldNames()[fieldIndex]
-: ((ARecordType) 
recordType.getSubFieldType(parentFieldNames)).getFieldNames()[fieldIndex];
+// Sub-field type can be AUnionType in case if it's 
optional.
+IAType subFieldType = sourceVar.equals(metaVar) ? 
metaType.getSubFieldType(parentFieldNames)
+: recordType.getSubFieldType(parentFieldNames);
+if (subFieldType instanceof AUnionType) {
+subFieldType = ((AUnionType) 
subFieldType).getActualType();
+}
+if (!(subFieldType instanceof ARecordType)) {
+throw 
CompilationException.create(ErrorCode.TYPE_CONVERT, AUnionType.class.getName(),
+ARecordType.class.getName());
+}
+fieldName = sourceVar.equals(metaVar) ? ((ARecordType) 
subFieldType).getFieldNames()[fieldIndex]
+: ((ARecordType) 
subFieldType).getFieldNames()[fieldIndex];
 }
 if (optFuncExpr != null) {
 optFuncExpr.setSourceVar(funcVarIndex, ((AssignOperator) 
op).getVariables().get(assignVarIndex));
diff --git 
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
new file mode 100644
index 000..3221cc3
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/filter/inverted-btree-search-return-optional-field.sqlpp
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS

Change in asterixdb[master]: [NO ISSUE][RT] Ensure an inverted list cursor.close()

2018-02-21 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [NO ISSUE][RT] Ensure an inverted list cursor.close()
..


[NO ISSUE][RT] Ensure an inverted list cursor.close()

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Ensure to always execute an inverted list cursor.close().

Change-Id: I78c7908830be810b1d40abbd5f1978818869
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2410
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
4 files changed, 88 insertions(+), 50 deletions(-)

Approvals:
  abdullah alamoudi: Looks good to me, approved
  Jenkins: Verified; No violations found; ; Verified

Objections:
  Anon. E. Moose #1000171: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
index 085f8d5..6e8689b 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
@@ -172,6 +172,7 @@
 btreePred.setHighKey(btreeSearchTuple, true);
 try {
 btreeAccessor.search(btreeCursor, btreePred);
+cursorNeedsClose = true;
 } catch (Exception e) {
 btreeSearchTuple.removeLastTuple();
 throw HyracksDataException.create(e);
diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
index a33b045..d9e7d34 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
@@ -87,8 +87,11 @@
 return true;
 }
 // The current inverted-list-range-search cursor is exhausted.
-invListRangeSearchCursor.unloadPages();
-invListRangeSearchCursor.close();
+try {
+invListRangeSearchCursor.unloadPages();
+} finally {
+invListRangeSearchCursor.close();
+}
 isInvListCursorOpen = false;
 openInvListRangeSearchCursor();
 return isInvListCursorOpen;
@@ -105,22 +108,34 @@
 
 @Override
 public void doDestroy() throws HyracksDataException {
-if (isInvListCursorOpen) {
-invListRangeSearchCursor.unloadPages();
-invListRangeSearchCursor.destroy();
-isInvListCursorOpen = false;
+try {
+if (isInvListCursorOpen) {
+try {
+invListRangeSearchCursor.unloadPages();
+} finally {
+isInvListCursorOpen = false;
+invListRangeSearchCursor.destroy();
+}
+}
+} finally {
+btreeCursor.destroy();
 }
-btreeCursor.destroy();
 }
 
 @Override
 public void doClose() throws HyracksDataException {
-if (isInvListCursorOpen) {
-invListRangeSearchCursor.unloadPages();
-invListRangeSearchCursor.close();
-isInvListCursorOpen = false;
+try {
+if (isInvListCursorOpen) {
+

Change in asterixdb[master]: [NO ISSUE][RT] Ensure an inverted list cursor.close()

2018-02-21 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][RT] Ensure an inverted list cursor.close()
..


Patch Set 4:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/2410/2/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java:

PS2, Line 157: try {
> If this throws an exception, close will not be called.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2410
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I78c7908830be810b1d40abbd5f1978818869
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][RT] Ensure an inverted list cursor.close()

2018-02-21 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2410

to look at the new patch set (#3).

Change subject: [NO ISSUE][RT] Ensure an inverted list cursor.close()
..

[NO ISSUE][RT] Ensure an inverted list cursor.close()

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Ensure to always execute an inverted list cursor.close().

Change-Id: I78c7908830be810b1d40abbd5f1978818869
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
3 files changed, 41 insertions(+), 30 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/2410/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2410
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I78c7908830be810b1d40abbd5f1978818869
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: [NO ISSUE][RT] Ensure an inverted list cursor.close()

2018-02-21 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2410

to look at the new patch set (#4).

Change subject: [NO ISSUE][RT] Ensure an inverted list cursor.close()
..

[NO ISSUE][RT] Ensure an inverted list cursor.close()

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Ensure to always execute an inverted list cursor.close().

Change-Id: I78c7908830be810b1d40abbd5f1978818869
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndexRangeSearchCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
4 files changed, 88 insertions(+), 50 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/2410/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2410
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I78c7908830be810b1d40abbd5f1978818869
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: [NO ISSUE][RT] Ensure a cursor.close()

2018-02-20 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2410

to look at the new patch set (#2).

Change subject: [NO ISSUE][RT] Ensure a cursor.close()
..

[NO ISSUE][RT] Ensure a cursor.close()

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Ensure a cursor.close() is always executed.

Change-Id: I78c7908830be810b1d40abbd5f1978818869
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
3 files changed, 37 insertions(+), 29 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/2410/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2410
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I78c7908830be810b1d40abbd5f1978818869
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins

Change in asterixdb[master]: [NO ISSUE][RT] Ensure a cursor.close()

2018-02-20 Thread Taewoo Kim (Code Review) 
Taewoo Kim has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2410

Change subject: [NO ISSUE][RT] Ensure a cursor.close()
..

[NO ISSUE][RT] Ensure a cursor.close()

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Ensure a cursor.close() is always executed.

Change-Id: I78c7908830be810b1d40abbd5f1978818869
---
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
3 files changed, 33 insertions(+), 29 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/2410/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
index 085f8d5..6e8689b 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedListCursor.java
@@ -172,6 +172,7 @@
 btreePred.setHighKey(btreeSearchTuple, true);
 try {
 btreeAccessor.search(btreeCursor, btreePred);
+cursorNeedsClose = true;
 } catch (Exception e) {
 btreeSearchTuple.removeLastTuple();
 throw HyracksDataException.create(e);
diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
index ff9a2f1..4e16d15 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/AbstractTOccurrenceSearcher.java
@@ -253,7 +253,7 @@
 ((BufferManagerBackedVSizeFrame) queryTokenFrame).destroy();
 
 // Releases the frames of the cursor.
-if (isSingleInvertedList && singleInvListCursor != null) {
+if (singleInvListCursor != null) {
 singleInvListCursor.unloadPages();
 singleInvListCursor.close();
 }
diff --git 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
index 0bfc140..b24819a 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
@@ -126,41 +126,41 @@
 isFinalList = true;
 }
 InvertedListCursor invListCursor = invListCursors.get(i);
-// Loads the inverted list (at least some part of it).
-invListCursor.prepareLoadPages();
-invListCursor.loadPages();
-if (i < numPrefixLists) {
-// Merges a prefix list.
-doneMerge = mergePrefixList(invListCursor, prevSearchResult, 
result, isFinalList);
-} else {
-// Merge suffix list.
-int numInvListElements = invListCursor.size();
-int currentNumResults = prevSearchResult.getNumResults();
-// Should we binary search the next list or should we 
sort-merge it?
-if (currentNumResults * Math.log(numInvListElements) < 
currentNumResults + numInvListElements) {
-doneMerge = mergeSuffixListProbe(invListCursor, 
prevSearchResult, result, i, numInvLists,
-occurrenceThreshold, isFinalList);
+try {
+// Loads

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-19 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..


[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2251
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 
Integration-Tests: Jenkins 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M

Change in asterixdb[master]: [NO ISSUE][RT] follow IFrameWriter protocol in SplitOperator...

2018-02-18 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE][RT] follow IFrameWriter protocol in 
SplitOperatorDescriptor
..


Patch Set 4: Code-Review+2

LGTM.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2399
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3166895589e1ab7355d689397f676f7da5c9809f
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-18 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#37).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-16 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..


Patch Set 35:

Done. Thanks!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2251
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
Gerrit-PatchSet: 35
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-16 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#36).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [NO ISSUE] Correct import of InvokeUtil in BufferCache

2018-02-16 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [NO ISSUE] Correct import of InvokeUtil in BufferCache
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2396
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4c9c1829db05ac691f7b0e6dc459d7646496ab6b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: abdullah alamoudi 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1972][COMP][RT][TX] index-only plan

2018-02-16 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: [ASTERIXDB-1972][COMP][RT][TX] index-only plan
..


[ASTERIXDB-1972][COMP][RT][TX] index-only plan

- user model changes: no
- storage format changes: no
- interface changes: IAccessMethod, ILSMIndexOperationContext,
 IIndexAccessor

Details:
- Implement an index-only plan
- Add a SET option that disables the index-only plan

Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1866
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Taewoo Kim 
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodAnalysisContext.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroducePrimaryIndexForAggregationRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceSelectAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/OptimizableOperatorSubTree.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
A asterixdb/asterix-app/data/nontagged/customerData2.json
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/primary-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_04.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_05.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_06.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-primary-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-non-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-composite-index-indexonly-plan-01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01-disable-indexonly-plan.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-02.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-03.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-04.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-05.aql
A

Change in asterixdb[master]: [ASTERIXDB-1972][COMP][RT][TX] index-only plan

2018-02-16 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-1972][COMP][RT][TX] index-only plan
..


Patch Set 66: Code-Review+2

Forwarding Dmitry's +2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1866
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51
Gerrit-PatchSet: 66
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: [ASTERIXDB-1972] Coordinated Change

2018-02-16 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-1972] Coordinated Change
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2376
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I191af547f3142fe5446ead4a2d053ea71e3f1d97
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-1972][COMP][RT][TX] index-only plan

2018-02-15 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-1972][COMP][RT][TX] index-only plan
..


Patch Set 65:

(2 comments)

Done!

https://asterix-gerrit.ics.uci.edu/#/c/1866/65/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java:

Line 106: enum secondaryUnnestMapOutputVarType {
> class/enum names start with a capital letter, so should be "SecondaryUnnest
Done


Line 424: 
CompilationException.create(ErrorCode.CANNOT_GET_CONDITIONAL_SPLIT_KEY_VARIABLE);
> Need to throw this exception.
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1866
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51
Gerrit-PatchSet: 65
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Yingyi Bu 
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-1972][COMP][RT][TX] index-only plan

2018-02-15 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins, Dmitry Lychagin,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1866

to look at the new patch set (#66).

Change subject: [ASTERIXDB-1972][COMP][RT][TX] index-only plan
..

[ASTERIXDB-1972][COMP][RT][TX] index-only plan

- user model changes: no
- storage format changes: no
- interface changes: IAccessMethod, ILSMIndexOperationContext,
 IIndexAccessor

Details:
- Implement an index-only plan
- Add a SET option that disables the index-only plan

Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodAnalysisContext.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroducePrimaryIndexForAggregationRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceSelectAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/OptimizableOperatorSubTree.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
A asterixdb/asterix-app/data/nontagged/customerData2.json
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/primary-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_04.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_05.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_06.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-primary-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-non-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-composite-index-indexonly-plan-01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01-disable-indexonly-plan.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-02.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-03.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-04.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-05.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-06.aql
A

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-15 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#35).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-14 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#34).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [ASTERIXDB-1972][COMP][RT][TX] index-only plan

2018-02-14 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1866

to look at the new patch set (#64).

Change subject: [ASTERIXDB-1972][COMP][RT][TX] index-only plan
..

[ASTERIXDB-1972][COMP][RT][TX] index-only plan

- user model changes: no
- storage format changes: no
- interface changes: IAccessMethod, ILSMIndexOperationContext,
 IIndexAccessor

Details:
- Implement an index-only plan
- Add a SET option that disables the index-only plan

Change-Id: Ifd5c9ab1cf2e4bedb7d8db582441919875e74d51
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/BTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/RTreeSearchPOperator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodAnalysisContext.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AccessMethodUtils.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/BTreeAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceJoinAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceLSMComponentFilterRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroducePrimaryIndexForAggregationRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/IntroduceSelectAccessMethodRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/InvertedIndexAccessMethod.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/OptimizableOperatorSubTree.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/RTreeAccessMethod.java
A asterixdb/asterix-app/data/nontagged/customerData2.json
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/primary-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_04.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_05.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-equi-join_06.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-primary-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-non-indexonly-plan-to-secondary-indexonly-plan-equi-join_01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join-index-only.aql
M 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index-join/secondary-self-equi-join.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-composite-index-indexonly-plan-01.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01-disable-indexonly-plan.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-01.sqlpp
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-02.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-03.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-04.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-05.aql
A 
asterixdb/asterix-app/src/test/resources/optimizerts/queries/btree-index/btree-secondary-index-indexonly-plan-06.aql
A

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-14 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#33).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
R 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/PLEASE_ADD_SQLPP_TEST_CASE
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-after-ingest/change-feed-with-meta-pk-in-meta-index-after-ingest.4.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest/change-feed-with-meta-pk-in-meta-index-with-missing-after-ingest.4.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed-with-function/connect-feed-with-function.1.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed-with-function/connect-feed-with-function.2.update.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed-with-function/connect-feed-with-function.3.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/connect-feed/connect-feed.3.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/create-policy-from-file/create-policy-from-file.1.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/create-policy-from-file/create-policy-from-file.2.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/create-policy-from-file/create-policy-from-file.3.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/disconnect-live-feed/disconnect-live-feed.1.update.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/drop-function-used-by-feed/drop-function-used-by-feed.3.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-push-socket/feed-push-socket.6.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.1.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.2.lib.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.3.update.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-function/feed-with-external-function.6.lib.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-external-parser/feed-with-external-parser.4.update.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.3.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feed-with-multiple-indexes/feed-with-multiple-indexes.5.query.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_01/feeds_01.1.ddl.aql
D 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_01/feeds_01.2.update.aql
D

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-14 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#32).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-14 Thread Taewoo Kim (Code Review) 
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2251

to look at the new patch set (#31).

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..

[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter
- storage format changes: no
- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,
 IInvertedIndexAccessor, IInvertedListCursor
 IObjectFactory, IPartitionedInvertedIndex,
 IIndexAccessor

Details:
- Introduce text.searchmemory parameter in the configuration
  to conduct budget-constrained inverted index search to prevent
  a possible OOM exception
- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java
A asterixdb/asterix-app/data/csv/fragile_sample.csv
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/ResourceUtils.java
M asterixdb/asterix-app/src/main/resources/cc.conf
M asterixdb/asterix-app/src/main/resources/cc2.conf
M asterixdb/asterix-app/src/main/resources/cc3.conf
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M asterixdb/asterix-app/src/test/resources/cc.conf
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-03/fulltext-index-03.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.3.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.query.sqlpp
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/fulltext/fulltext-index-large-data/fulltext-index-large-data.5.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.4.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/inverted-index-ngram-edit-distance-large-data/inverted-index-ngram-edit-distance-large-data.5.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java
M

Change in asterixdb[master]: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inv...

2018-02-14 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: [ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained 
Inverted index search
..


Patch Set 30:

(10 comments)

Thanks Abdullah.

https://asterix-gerrit.ics.uci.edu/#/c/2251/29/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java
File 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java:

PS29, Line 144: CANNOT_CONTINUE_TEXT_SEARCH
> this seems to be used in many places and for different reasons. can we have
Good idea. I have separated the code.


https://asterix-gerrit.ics.uci.edu/#/c/2251/29/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/api/IInvertedListCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/api/IInvertedListCursor.java:

PS29, Line 27: IInvertedListCursor
> just a suggestion.. if we want this to be another cursor, let's extend Enfo
OK. Done.


https://asterix-gerrit.ics.uci.edu/#/c/2251/6/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeElementInvertedListCursor.java:

Line 148:  */
> MAJOR SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/2251/29/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeFrameTupleAppender.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/FixedSizeFrameTupleAppender.java:

PS29, Line 34: FixedSizeFrameTupleAppender
> There is another class with the same name  (used for hash join I believe) F
This class only supports a fixed size tuple. Thus, it's simple. The class used 
for hash join actually supports variable length tuples since it has 
tupleEndOffset for each tuple at the end of a frame. So, 
this one: "FixedSizeFrameTuple"Appender
That one: "FixedSizeFrame"TupleAppender


https://asterix-gerrit.ics.uci.edu/#/c/2251/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedIndexSearchResult.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedIndexSearchResult.java:

Line 60: 
> MAJOR SonarQube violation:
Done


Line 86: this.bufferManager = bufferManager;
> MAJOR SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/2251/1/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java
File 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/InvertedListMerger.java:

Line 42: 
> MAJOR SonarQube violation:
Done


Line 100:  */
> MAJOR SonarQube violation:
Done


Line 369: // For a final result, needs to pause when a 
frame becomes full to let the caller
> MAJOR SonarQube violation:
Done


https://asterix-gerrit.ics.uci.edu/#/c/2251/1/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IIndexAccessor.java
File 
hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IIndexAccessor.java:

Line 88:  */
> MAJOR SonarQube violation:
Done


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2251
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5
Gerrit-PatchSet: 30
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: Yes

  1   2   3   4   5   6   7   8   >