date:20180318

[jira] [Updated] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

2018-03-18 Thread RenZhiMin (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

RenZhiMin updated KYLIN-3296:
-
Labels: patch  (was: )

> cube合并时，java.lang.ArrayIndexOutOfBoundsException at 
> java.lang.System.arraycopy(Native Method)
> -
>
> Key: KYLIN-3296
> URL: https://issues.apache.org/jira/browse/KYLIN-3296
> Project: Kylin
>  Issue Type: Bug
>  Components: Others
>Affects Versions: v2.3.0
>Reporter: RenZhiMin
>Priority: Major
>  Labels: patch
> Attachments: RowKeySplitter.patch
>
>
> cube中，设计rowkey时，有个维度设置编码方式是固定长度500。每天采用内存构建算法。在合并cube时，在生成的mr中的map任务执行中出现“java.lang.ArrayIndexOutOfBoundsException
>  at java.lang.System.arraycopy(Native Method)” 
> 错误。经查看在生成的mr中的map任务中需要对要合并的cuboiddata数据的rowkey进行切分，切分时，根据每个维度的编码方式获取对应的长度，然后从rowkey中获取，并赋值给SplittedBytes的value中，由于value数组初始化时设置的固定值255，所以在切分大于255的维度值时，出现下标越界错误。



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

2018-03-18 Thread RenZhiMin (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

RenZhiMin updated KYLIN-3296:
-
Attachment: RowKeySplitter.patch

> cube合并时，java.lang.ArrayIndexOutOfBoundsException at 
> java.lang.System.arraycopy(Native Method)
> -
>
> Key: KYLIN-3296
> URL: https://issues.apache.org/jira/browse/KYLIN-3296
> Project: Kylin
>  Issue Type: Bug
>  Components: Others
>Affects Versions: v2.3.0
>Reporter: RenZhiMin
>Priority: Major
> Attachments: RowKeySplitter.patch
>
>
> cube中，设计rowkey时，有个维度设置编码方式是固定长度500。每天采用内存构建算法。在合并cube时，在生成的mr中的map任务执行中出现“java.lang.ArrayIndexOutOfBoundsException
>  at java.lang.System.arraycopy(Native Method)” 
> 错误。经查看在生成的mr中的map任务中需要对要合并的cuboiddata数据的rowkey进行切分，切分时，根据每个维度的编码方式获取对应的长度，然后从rowkey中获取，并赋值给SplittedBytes的value中，由于value数组初始化时设置的固定值255，所以在切分大于255的维度值时，出现下标越界错误。



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

2018-03-18 Thread RenZhiMin (JIRA)

RenZhiMin created KYLIN-3296:


 Summary: cube合并时，java.lang.ArrayIndexOutOfBoundsException at 
java.lang.System.arraycopy(Native Method)
 Key: KYLIN-3296
 URL: https://issues.apache.org/jira/browse/KYLIN-3296
 Project: Kylin
  Issue Type: Bug
  Components: Others
Affects Versions: v2.3.0
Reporter: RenZhiMin
 Attachments: RowKeySplitter.patch

cube中，设计rowkey时，有个维度设置编码方式是固定长度500。每天采用内存构建算法。在合并cube时，在生成的mr中的map任务执行中出现“java.lang.ArrayIndexOutOfBoundsException
 at java.lang.System.arraycopy(Native Method)” 
错误。经查看在生成的mr中的map任务中需要对要合并的cuboiddata数据的rowkey进行切分，切分时，根据每个维度的编码方式获取对应的长度，然后从rowkey中获取，并赋值给SplittedBytes的value中，由于value数组初始化时设置的固定值255，所以在切分大于255的维度值时，出现下标越界错误。



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404334#comment-16404334
 ] 

ASF subversion and git services commented on KYLIN-3257:


Commit e7343f8459fd7d85ba40076f17146c2cf9cd758e in kylin's branch 
refs/heads/master from lwz9103
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e7343f8 ]

KYLIN-3257 remove useless call in FuzzyValueCombination


> Useless call in FuzzyValueCombination
> -
>
> Key: KYLIN-3257
> URL: https://issues.apache.org/jira/browse/KYLIN-3257
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Wenzheng Liu
>Priority: Minor
>
> {code}
> public static  List> calculate(Map 
> fuzzyValues, long cap) {
> Collections.emptyMap();
> {code}
> The empty map has no effect to method execution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

2018-03-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404333#comment-16404333
 ] 

ASF GitHub Bot commented on KYLIN-3257:
---

lidongsjtu closed pull request #120: KYLIN-3257 remove useless call in 
FuzzyValueCombination
URL: https://github.com/apache/kylin/pull/120
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/core-cube/src/main/java/org/apache/kylin/cube/common/FuzzyValueCombination.java
 
b/core-cube/src/main/java/org/apache/kylin/cube/common/FuzzyValueCombination.java
index 74012a4e62..1be3cccd81 100644
--- 
a/core-cube/src/main/java/org/apache/kylin/cube/common/FuzzyValueCombination.java
+++ 
b/core-cube/src/main/java/org/apache/kylin/cube/common/FuzzyValueCombination.java
@@ -44,7 +44,6 @@
 }
 
 public static  List> calculate(Map fuzzyValues, 
long cap) {
-Collections.emptyMap();
 Dim[] dims = toDims(fuzzyValues);
 // If a query has many IN clause and each IN clause has many values, 
then it will easily generate 
 // thousands of fuzzy keys. When there are lots of fuzzy keys, the 
scan performance is bottle necked 
diff --git 
a/core-storage/src/main/java/org/apache/kylin/storage/translate/FuzzyValueCombination.java
 
b/core-storage/src/main/java/org/apache/kylin/storage/translate/FuzzyValueCombination.java
index bfa398ed4a..c6fa4b4531 100644
--- 
a/core-storage/src/main/java/org/apache/kylin/storage/translate/FuzzyValueCombination.java
+++ 
b/core-storage/src/main/java/org/apache/kylin/storage/translate/FuzzyValueCombination.java
@@ -46,7 +46,6 @@
 }
 
 public static  List> calculate(Map 
fuzzyValues, long cap) {
-Collections.emptyMap();
 Dim[] dims = toDims(fuzzyValues);
 // If a query has many IN clause and each IN clause has many values, 
then it will easily generate 
 // thousands of fuzzy keys. When there are lots of fuzzy keys, the 
scan performance is bottle necked 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Useless call in FuzzyValueCombination
> -
>
> Key: KYLIN-3257
> URL: https://issues.apache.org/jira/browse/KYLIN-3257
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Wenzheng Liu
>Priority: Minor
>
> {code}
> public static  List> calculate(Map 
> fuzzyValues, long cap) {
> Collections.emptyMap();
> {code}
> The empty map has no effect to method execution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

2018-03-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404331#comment-16404331
 ] 

ASF GitHub Bot commented on KYLIN-3257:
---

coveralls commented on issue #120: KYLIN-3257 remove useless call in 
FuzzyValueCombination
URL: https://github.com/apache/kylin/pull/120#issuecomment-374088945
 
 
   ## Pull Request Test Coverage Report for [Build 
3055](https://coveralls.io/builds/16044147)
   
   * **0** of **0**  **(NaN%)** changed or added relevant lines in **0** files 
are covered.
   * **19** unchanged lines in **4** files lost coverage.
   * Overall coverage decreased (**-0.03%**) to **23.847%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-common/src/main/java/org/apache/kylin/common/KylinConfigCannotInitException.java](https://coveralls.io/builds/16044147/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfigCannotInitException.java#L24)
 | 2 | 0.0% |
   | 
[core-common/src/main/java/org/apache/kylin/common/KylinConfig.java](https://coveralls.io/builds/16044147/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfig.java#L278)
 | 4 | 33.19% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/16044147/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439)
 | 5 | 77.81% |
   | 
[core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java](https://coveralls.io/builds/16044147/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfigBase.java#L67)
 | 8 | 16.12% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/16044147/badge)](https://coveralls.io/builds/16044147)
 |
   | :-- | --: |
   | Change from base [Build 3053](https://coveralls.io/builds/16037244): |  
-0.03% |
   | Covered Lines: | 14323 |
   | Relevant Lines: | 60062 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Useless call in FuzzyValueCombination
> -
>
> Key: KYLIN-3257
> URL: https://issues.apache.org/jira/browse/KYLIN-3257
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Wenzheng Liu
>Priority: Minor
>
> {code}
> public static  List> calculate(Map 
> fuzzyValues, long cap) {
> Collections.emptyMap();
> {code}
> The empty map has no effect to method execution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

2018-03-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404330#comment-16404330
 ] 

ASF GitHub Bot commented on KYLIN-3257:
---

codecov-io commented on issue #120: KYLIN-3257 remove useless call in 
FuzzyValueCombination
URL: https://github.com/apache/kylin/pull/120#issuecomment-374088896
 
 
   # [Codecov](https://codecov.io/gh/apache/kylin/pull/120?src=pr=h1) Report
   > Merging [#120](https://codecov.io/gh/apache/kylin/pull/120?src=pr=desc) 
into 
[master](https://codecov.io/gh/apache/kylin/commit/4bcd8a14d3fcce385057275fcae10520abe2ead2?src=pr=desc)
 will **decrease** coverage by `0.02%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/kylin/pull/120/graphs/tree.svg?height=150=JawVgbgsVo=650=pr)](https://codecov.io/gh/apache/kylin/pull/120?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master #120  +/-   ##
   
   - Coverage 21.83%   21.81%   -0.03% 
   + Complexity 3888 3884   -4 
   
 Files   999  999  
 Lines 6006460062   -2 
 Branches   8673 8673  
   
   - Hits  1311713102  -15 
   - Misses4572745741  +14 
   + Partials   1220 1219   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/kylin/pull/120?src=pr=tree) | Coverage Δ 
| Complexity Δ | |
   |---|---|---|---|
   | 
[...pache/kylin/cube/common/FuzzyValueCombination.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2NvbW1vbi9GdXp6eVZhbHVlQ29tYmluYXRpb24uamF2YQ==)
 | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...kylin/storage/translate/FuzzyValueCombination.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL3RyYW5zbGF0ZS9GdXp6eVZhbHVlQ29tYmluYXRpb24uamF2YQ==)
 | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...e/kylin/common/KylinConfigCannotInitException.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Nhbm5vdEluaXRFeGNlcHRpb24uamF2YQ==)
 | `0% <0%> (-100%)` | `0% <0%> (-1%)` | |
   | 
[...main/java/org/apache/kylin/common/KylinConfig.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZy5qYXZh)
 | `30.67% <0%> (-1.69%)` | `20% <0%> (-1%)` | |
   | 
[.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==)
 | `14.89% <0%> (-1.23%)` | `35% <0%> (-2%)` | |
   | 
[...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/120/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh)
 | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/kylin/pull/120?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/kylin/pull/120?src=pr=footer). Last 
update 
[4bcd8a1...f743946](https://codecov.io/gh/apache/kylin/pull/120?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Useless call in FuzzyValueCombination
> -
>
> Key: KYLIN-3257
> URL: https://issues.apache.org/jira/browse/KYLIN-3257
> Project: Kylin
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Wenzheng Liu
>Priority: Minor
>
> {code}
> public static  List> calculate(Map 
> fuzzyValues, long cap) {
> Collections.emptyMap();
> {code}
> The empty map has no effect to method execution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3295) Unused method SQLDigestUtil#appendTsFilterToExecute

2018-03-18 Thread Ted Yu (JIRA)

Ted Yu created KYLIN-3295:
-

 Summary: Unused method SQLDigestUtil#appendTsFilterToExecute
 Key: KYLIN-3295
 URL: https://issues.apache.org/jira/browse/KYLIN-3295
 Project: Kylin
  Issue Type: Bug
Reporter: Ted Yu


SQLDigestUtil#appendTsFilterToExecute is not called anywhere.

Also
{code}
   T ret = action.apply(null);
{code}
Passing null to {{apply}} seems incorrect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404104#comment-16404104
 ] 

ASF subversion and git services commented on KYLIN-3276:


Commit d9561d06e8e49f87d9fa9ff41a53c20e3291d162 in kylin's branch 
refs/heads/2.3.x from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d9561d0 ]

KYLIN-3276 fix unstable it test case


> query with dynamic parameter get the same query result
> --
>
> Key: KYLIN-3276
> URL: https://issues.apache.org/jira/browse/KYLIN-3276
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC, Query Engine
>Affects Versions: v2.2.0
> Environment: Kylin 2.2.0, JDK 1.8
>Reporter: ZHAOXIXIN
>Assignee: nichunen
>Priority: Major
> Fix For: v2.3.1
>
>
> sql:"select distinct uid from table where daytime=? limit 70" 
> ps.setString(1,"2018-01-03") 
> store the result to ArrayList, and print the size of ArrayList. 
> when I set the daytime to different dates, the same result I got. 
> I changed the sql, just change limit value for example 
> sql:"select distinct uid from table where daytime=? limit 60" 
> then I got the right value the same like query from GUI:count (distinct uid) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404103#comment-16404103
 ] 

ASF subversion and git services commented on KYLIN-3276:


Commit 60639456423a44ea6d88b0548fa3c1541db57e98 in kylin's branch 
refs/heads/2.3.x from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6063945 ]

KYLIN-3276 Fix query with different dynamic parameters get the same query result


> query with dynamic parameter get the same query result
> --
>
> Key: KYLIN-3276
> URL: https://issues.apache.org/jira/browse/KYLIN-3276
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC, Query Engine
>Affects Versions: v2.2.0
> Environment: Kylin 2.2.0, JDK 1.8
>Reporter: ZHAOXIXIN
>Assignee: nichunen
>Priority: Major
> Fix For: v2.3.1
>
>
> sql:"select distinct uid from table where daytime=? limit 70" 
> ps.setString(1,"2018-01-03") 
> store the result to ArrayList, and print the size of ArrayList. 
> when I set the daytime to different dates, the same result I got. 
> I changed the sql, just change limit value for example 
> sql:"select distinct uid from table where daytime=? limit 60" 
> then I got the right value the same like query from GUI:count (distinct uid) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3276) query with dynamic parameter get the same query result

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu resolved KYLIN-3276.
--
Resolution: Fixed

> query with dynamic parameter get the same query result
> --
>
> Key: KYLIN-3276
> URL: https://issues.apache.org/jira/browse/KYLIN-3276
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC, Query Engine
>Affects Versions: v2.2.0
> Environment: Kylin 2.2.0, JDK 1.8
>Reporter: ZHAOXIXIN
>Assignee: nichunen
>Priority: Major
> Fix For: v2.3.1
>
>
> sql:"select distinct uid from table where daytime=? limit 70" 
> ps.setString(1,"2018-01-03") 
> store the result to ArrayList, and print the size of ArrayList. 
> when I set the daytime to different dates, the same result I got. 
> I changed the sql, just change limit value for example 
> sql:"select distinct uid from table where daytime=? limit 60" 
> then I got the right value the same like query from GUI:count (distinct uid) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3279) Util Class for encryption and decryption

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu resolved KYLIN-3279.
--
Resolution: Fixed

> Util Class for encryption and decryption
> 
>
> Key: KYLIN-3279
> URL: https://issues.apache.org/jira/browse/KYLIN-3279
> Project: Kylin
>  Issue Type: Improvement
>  Components: Others
>Affects Versions: v2.3.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> Extract methods of "encrypt" and "decrypt" in 
> org.apache.kylin.rest.security.PasswordPlaceholderConfigurer to a Util class 
> in core-common package. This will make them more reusable. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu resolved KYLIN-3281.
--
   Resolution: Fixed
Fix Version/s: v2.4.0

> OLAPProjectRule can't normal working with  projectRel[input=sortRel]
> 
>
> Key: KYLIN-3281
> URL: https://issues.apache.org/jira/browse/KYLIN-3281
> Project: Kylin
>  Issue Type: Bug
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Critical
> Fix For: v2.4.0
>
>
> for queries like:
> select sum("count") from (SELECT LSTG_FORMAT_NAME ,  COUNT(*) AS "count" FROM 
> TEST_KYLIN_FACT  GROUP BY  "LSTG_FORMAT_NAME" ORDER BY "count" DESC) as a
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPAggregateRel(group=[{}], EXPR$0=[SUM($0)], ctx=[])
>   OLAPProjectRel(count=[$1], ctx=[])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
>   OLAPAggregateRel(group=[{0}], count=[COUNT()], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableAggregate(group=[{}], EXPR$0=[SUM($0)])
> EnumerableCalc(expr#0..1=[{inputs}], count=[$t1])
>   EnumerableSort(sort0=[$1], dir0=[DESC])
> EnumerableAggregate(group=[{0}], count=[COUNT()])
>   OLAPToEnumerableConverter
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> VolcanoPlanner apply OLAPProjectRule to logicalProject,convert child node 
> should use child traitset not current proejct traitset.
> If we use current project trait set, the child node trait set is  [olap][0 
> DESC], it will create a new subset but the subset doesn't have any rel. the 
> cost is inf,
> so volcanoPlanner choose a  the EnumerableAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu resolved KYLIN-3284.
--
   Resolution: Fixed
Fix Version/s: v2.4.0

> Refactor all OLAPRel computeSelfCost
> 
>
> Key: KYLIN-3284
> URL: https://issues.apache.org/jira/browse/KYLIN-3284
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
> Fix For: v2.4.0
>
>
> for queries like:
>  select lstg_format_name from test_kylin_fact order by case 
>  when 1=1 then 
>  cal_dt 
>  ELSE 
>  seller_id 
>  end 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
>  EnumerableSort(sort0=[$1], dir0=[ASC])
>   OLAPToEnumerableConverter
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> This is the RelOptPlanner:
> {code:java}
> Sets:
> Set#0, type: RecordType(BIGINT TRANS_ID, BIGINT ORDER_ID, DATE CAL_DT, 
> VARCHAR(256) LSTG_FORMAT_NAME, BIGINT LEAF_CATEG_ID, INTEGER LSTG_SITE_ID, 
> SMALLINT SLR_SEGMENT_CD, INTEGER SELLER_ID, DECIMAL(19, 4) PRICE, INTEGER 
> ITEM_COUNT, VARCHAR(256) TEST_COUNT_DISTINCT_BITMAP, DECIMAL(19, 4) 
> DEAL_AMOUNT, INTEGER DEAL_YEAR, VARCHAR(256) 
> LEFTJOIN_BUYER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_SELLER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_BUYER_COUNTRY_ABBR, VARCHAR(256) LEFTJOIN_SELLER_COUNTRY_ABBR, 
> BIGINT _KY_COUNT__, INTEGER _KY_SUM_TEST_KYLIN_FACT_ITEM_COUNT_, DECIMAL(19, 
> 4) _KY_SUM_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MIN_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MAX_TEST_KYLIN_FACT_PRICE_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_SELLER_ID_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME_TEST_KYLIN_FACT_SELLER_ID_,
>  ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_TEST_COUNT_DISTINCT_BITMAP_, 
> DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_DEAL_AMOUNT_, ANY 
> _KY_PERCENTILE_APPROX_TEST_KYLIN_FACT_PRICE_)
>   rel#140:Subset#0.OLAP.[], best=rel#131, importance=0.7291
>   rel#131:OLAPTableScan.OLAP.[](table=[DEFAULT, 
> TEST_KYLIN_FACT],ctx=,fields=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 
> 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]), rowcount=100.0, 
> cumulative cost={5.0 rows, 5.051 cpu, 0.0 io}
>   
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
>  rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
>   rel#169:Subset#0.NONE.[2], best=null, importance=0.7291
>   
> rel#167:LogicalSort.NONE.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={inf}
>   rel#171:Subset#0.ENUMERABLE.[2], best=rel#182, importance=0.81
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   rel#173:Subset#0.OLAP.[1], best=null, importance=0.6561
>   rel#177:Subset#0.ENUMERABLE.[], best=rel#182, 
> importance=0.7291
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   
> rel#185:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#140:Subset#0.OLAP.[]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 5.051 cpu, 0.0 io}
>   rel#180:Subset#0.OLAP.[2], best=rel#179, importance=0.6561
>   
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
>  rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
> Set#1,

[jira] [Updated] (KYLIN-3149) Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu updated KYLIN-3149:
-
Fix Version/s: v2.4.0

> Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected
> 
>
> Key: KYLIN-3149
> URL: https://issues.apache.org/jira/browse/KYLIN-3149
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.2.0
>Reporter: hongbin ma
>Assignee: yiming.xu
>Priority: Major
> Fix For: v2.4.0
>
> Attachments: dump.txt
>
>
> for queries like:
> {code:sql}
> select TRANS_ID from kylin_sales group by cast (case 
> WHEN  '1030101' = '1030101' then substring(COALESCE(OPS_USER_ID, 
> ''), 1, 1)
> when  '1030101' = '1030102' then substring(COALESCE(OPS_REGION, 
> ''), 1, 1)  
> when  '1030101' = '1030103' then substring(COALESCE(LSTG_FORMAT_NAME, 
> ''), 1, 1)
> when  '1030101' = '1030104' then substring(COALESCE(LSTG_FORMAT_NAME, 
> ''), 1, 1)
> end as varchar(256)), TRANS_ID;
> {code}
> the expected logical plan after volcano is:
> {code}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
>   OLAPProjectRel(TRANS_ID=[$1], ctx=[])
> OLAPLimitRel(ctx=[], fetch=[5])
>   OLAPAggregateRel(group=[{0, 1}], ctx=[])
> OLAPProjectRel($f0=[SUBSTRING(CASE(IS NOT NULL($9), $9, 
> ''), 1, 1)], TRANS_ID=[$0], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
> {code}
> however the actual is:
> {code}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPProjectRel(TRANS_ID=[$1], ctx=[])
>   OLAPAggregateRel(group=[{0, 1}], ctx=[])
> OLAPProjectRel($f0=[CAST(CASE(=('1030101', '1030101'), 
> SUBSTRING(CASE(IS NOT NULL($9), $9, ''), 1, 1), =('1030101', 
> '1030102'), SUBSTRING(CASE(IS NOT NULL($10), $10, ''), 1, 1), 
> =('1030101', '1030103'), SUBSTRING(CASE(IS NOT NULL($2), $2, ''), 
> 1, 1), =('1030101', '1030104'), SUBSTRING(CASE(IS NOT NULL($2), $2, 
> ''), 1, 1), null)):VARCHAR(256) CHARACTER SET "UTF-16LE" COLLATE 
> "UTF-16LE$en_US$primary"], TRANS_ID=[$0], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
> {code}
> looks like Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as 
> expected. If we dump the internal state of this VolcanoPlanner 
> (org.apache.calcite.plan.volcano.VolcanoPlanner#dump), line 19-21 from the 
> complete dump is attached:
> {code}
>   rel#337:Subset#1.OLAP.[], best=rel#339, importance=0.6561
>   
> rel#339:OLAPProjectRel.OLAP.[](input=rel#303:Subset#0.OLAP.[],$f0=CAST(CASE(=('1030101',
>  '1030101'), SUBSTRING(CASE(IS NOT NULL($9), $9, ''), 1, 1), 
> =('1030101', '1030102'), SUBSTRING(CASE(IS NOT NULL($10), $10, 
> ''), 1, 1), =('1030101', '1030103'), SUBSTRING(CASE(IS NOT 
> NULL($2), $2, ''), 1, 1), =('1030101', '1030104'), 
> SUBSTRING(CASE(IS NOT NULL($2), $2, ''), 1, 1), 
> null)):VARCHAR(256) CHARACTER SET "UTF-16LE" COLLATE 
> "UTF-16LE$en_US$primary",TRANS_ID=$0,ctx=), rowcount=100.0, cumulative 
> cost={15.0 rows, 25.05 cpu, 0.0 io}
>   
> rel#348:OLAPProjectRel.OLAP.[](input=rel#303:Subset#0.OLAP.[],$f0=SUBSTRING(CASE(IS
>  NOT NULL($9), $9, ''), 1, 1),TRANS_ID=$0,ctx=), rowcount=100.0, 
> cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> {code}
> we see two rels with same cost:  #339 and #348, where #339 is created from 
> LogicalProject = (OLAPProjectRule)=> OLAPProject, and #348 is created from 
> LogicalProject =( ReduceExpressionsRule) => Reduced LogicalProject 
> =(OLAPProjectRule)=> Reduced OLAPProject . Since ReduceExpressionsRule 
> require Logical Project rather than OLAP Project, #339 is never reduced.
> The worse thing is that cost of #339 and #348 are same. By current volcano 
> planner algorithm  the first met rel will be chosen, so unexpected rel is 
> chosen
> A simple approach to fix this is to refine the rel choosing algorithm: when 
> two rels are equal in cost, choose a "simpler" one. Since we don't have a 
> perfect measurement of "simple", we simply choose the rel with smaller 
> toString() length



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3283) Support values RelNode

2018-03-18 Thread Billy Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/KYLIN-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Billy Liu updated KYLIN-3283:
-
Fix Version/s: v2.4.0

> Support values RelNode
> --
>
> Key: KYLIN-3283
> URL: https://issues.apache.org/jira/browse/KYLIN-3283
> Project: Kylin
>  Issue Type: New Feature
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
> Fix For: v2.4.0
>
>
> for queries like:
> select leaf_categ_id, sum(price) as sum_price from test_kylin_fact  group by 
> leaf_categ_id
> union all
> select 1 as leaf_categ_id2, 2 as sum_price2 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPUnionRel(all=[true], ctx=[], all=[true])
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
>   OLAPProjectRel(LEAF_CATEG_ID2=[1], SUM_PRICE2=[2], ctx=[])
> OLAPValuesRel(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableUnion(all=[true])
> OLAPToEnumerableConverter
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> EnumerableCalc(expr#0=[{inputs}], expr#1=[1], expr#2=[2], 
> LEAF_CATEG_ID2=[$t1], SUM_PRICE2=[$t2])
>   EnumerableValues(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KYLIN-3294) Remove HBaseMROutput.java, RangeKeyDistributionJob.java and other sunset classes

2018-03-18 Thread Shaofeng SHI (JIRA)

Shaofeng SHI created KYLIN-3294:
---

 Summary: Remove HBaseMROutput.java, RangeKeyDistributionJob.java 
and other sunset classes
 Key: KYLIN-3294
 URL: https://issues.apache.org/jira/browse/KYLIN-3294
 Project: Kylin
  Issue Type: Improvement
  Components: Job Engine
Reporter: Shaofeng SHI


They were legacy classes, keeping them will add maintainence effort especially 
when upgrade HBase version. Should delete them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403929#comment-16403929
 ] 

ASF subversion and git services commented on KYLIN-3284:


Commit 4e0bb70f72e6fcc11bacd7d8ede996ede5449560 in kylin's branch 
refs/heads/master from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4e0bb70 ]

KYLIN-3284 Refactor all OLAPRel computeSelfCost


> Refactor all OLAPRel computeSelfCost
> 
>
> Key: KYLIN-3284
> URL: https://issues.apache.org/jira/browse/KYLIN-3284
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
>
> for queries like:
>  select lstg_format_name from test_kylin_fact order by case 
>  when 1=1 then 
>  cal_dt 
>  ELSE 
>  seller_id 
>  end 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
>  EnumerableSort(sort0=[$1], dir0=[ASC])
>   OLAPToEnumerableConverter
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> This is the RelOptPlanner:
> {code:java}
> Sets:
> Set#0, type: RecordType(BIGINT TRANS_ID, BIGINT ORDER_ID, DATE CAL_DT, 
> VARCHAR(256) LSTG_FORMAT_NAME, BIGINT LEAF_CATEG_ID, INTEGER LSTG_SITE_ID, 
> SMALLINT SLR_SEGMENT_CD, INTEGER SELLER_ID, DECIMAL(19, 4) PRICE, INTEGER 
> ITEM_COUNT, VARCHAR(256) TEST_COUNT_DISTINCT_BITMAP, DECIMAL(19, 4) 
> DEAL_AMOUNT, INTEGER DEAL_YEAR, VARCHAR(256) 
> LEFTJOIN_BUYER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_SELLER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_BUYER_COUNTRY_ABBR, VARCHAR(256) LEFTJOIN_SELLER_COUNTRY_ABBR, 
> BIGINT _KY_COUNT__, INTEGER _KY_SUM_TEST_KYLIN_FACT_ITEM_COUNT_, DECIMAL(19, 
> 4) _KY_SUM_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MIN_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MAX_TEST_KYLIN_FACT_PRICE_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_SELLER_ID_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME_TEST_KYLIN_FACT_SELLER_ID_,
>  ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_TEST_COUNT_DISTINCT_BITMAP_, 
> DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_DEAL_AMOUNT_, ANY 
> _KY_PERCENTILE_APPROX_TEST_KYLIN_FACT_PRICE_)
>   rel#140:Subset#0.OLAP.[], best=rel#131, importance=0.7291
>   rel#131:OLAPTableScan.OLAP.[](table=[DEFAULT, 
> TEST_KYLIN_FACT],ctx=,fields=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 
> 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]), rowcount=100.0, 
> cumulative cost={5.0 rows, 5.051 cpu, 0.0 io}
>   
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
>  rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
>   rel#169:Subset#0.NONE.[2], best=null, importance=0.7291
>   
> rel#167:LogicalSort.NONE.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={inf}
>   rel#171:Subset#0.ENUMERABLE.[2], best=rel#182, importance=0.81
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   rel#173:Subset#0.OLAP.[1], best=null, importance=0.6561
>   rel#177:Subset#0.ENUMERABLE.[], best=rel#182, 
> importance=0.7291
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   
> rel#185:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#140:Subset#0.OLAP.[]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 5.051 cpu, 0.0 io}
>   rel#180:Subset#0.OLAP.[2], best=rel#179,

[jira] [Commented] (KYLIN-3283) Support values RelNode

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403930#comment-16403930
 ] 

ASF subversion and git services commented on KYLIN-3283:


Commit 6122f0675d05e10ade86c45a0dc64dfb20e728f3 in kylin's branch 
refs/heads/master from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6122f06 ]

KYLIN-3283 Support values RelNode


> Support values RelNode
> --
>
> Key: KYLIN-3283
> URL: https://issues.apache.org/jira/browse/KYLIN-3283
> Project: Kylin
>  Issue Type: New Feature
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
>
> for queries like:
> select leaf_categ_id, sum(price) as sum_price from test_kylin_fact  group by 
> leaf_categ_id
> union all
> select 1 as leaf_categ_id2, 2 as sum_price2 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPUnionRel(all=[true], ctx=[], all=[true])
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
>   OLAPProjectRel(LEAF_CATEG_ID2=[1], SUM_PRICE2=[2], ctx=[])
> OLAPValuesRel(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableUnion(all=[true])
> OLAPToEnumerableConverter
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> EnumerableCalc(expr#0=[{inputs}], expr#1=[1], expr#2=[2], 
> LEAF_CATEG_ID2=[$t1], SUM_PRICE2=[$t2])
>   EnumerableValues(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403928#comment-16403928
 ] 

ASF subversion and git services commented on KYLIN-3281:


Commit 8a9ed59213bb2f2d06ef99e836d7b21e37413112 in kylin's branch 
refs/heads/master from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8a9ed59 ]

KYLIN-3281 OLAPProjectRule can't normal working with projectRel[input=sortRel]


> OLAPProjectRule can't normal working with  projectRel[input=sortRel]
> 
>
> Key: KYLIN-3281
> URL: https://issues.apache.org/jira/browse/KYLIN-3281
> Project: Kylin
>  Issue Type: Bug
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Critical
>
> for queries like:
> select sum("count") from (SELECT LSTG_FORMAT_NAME ,  COUNT(*) AS "count" FROM 
> TEST_KYLIN_FACT  GROUP BY  "LSTG_FORMAT_NAME" ORDER BY "count" DESC) as a
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPAggregateRel(group=[{}], EXPR$0=[SUM($0)], ctx=[])
>   OLAPProjectRel(count=[$1], ctx=[])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
>   OLAPAggregateRel(group=[{0}], count=[COUNT()], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableAggregate(group=[{}], EXPR$0=[SUM($0)])
> EnumerableCalc(expr#0..1=[{inputs}], count=[$t1])
>   EnumerableSort(sort0=[$1], dir0=[DESC])
> EnumerableAggregate(group=[{0}], count=[COUNT()])
>   OLAPToEnumerableConverter
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> VolcanoPlanner apply OLAPProjectRule to logicalProject,convert child node 
> should use child traitset not current proejct traitset.
> If we use current project trait set, the child node trait set is  [olap][0 
> DESC], it will create a new subset but the subset doesn't have any rel. the 
> cost is inf,
> so volcanoPlanner choose a  the EnumerableAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403927#comment-16403927
 ] 

ASF subversion and git services commented on KYLIN-3276:


Commit 09689e24855a6c4ed57f0f55c226f4929fc180e7 in kylin's branch 
refs/heads/master from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=09689e2 ]

KYLIN-3276 fix unstable it test case


> query with dynamic parameter get the same query result
> --
>
> Key: KYLIN-3276
> URL: https://issues.apache.org/jira/browse/KYLIN-3276
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC, Query Engine
>Affects Versions: v2.2.0
> Environment: Kylin 2.2.0, JDK 1.8
>Reporter: ZHAOXIXIN
>Assignee: nichunen
>Priority: Major
> Fix For: v2.3.1
>
>
> sql:"select distinct uid from table where daytime=? limit 70" 
> ps.setString(1,"2018-01-03") 
> store the result to ArrayList, and print the size of ArrayList. 
> when I set the daytime to different dates, the same result I got. 
> I changed the sql, just change limit value for example 
> sql:"select distinct uid from table where daytime=? limit 60" 
> then I got the right value the same like query from GUI:count (distinct uid) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3279) Util Class for encryption and decryption

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403925#comment-16403925
 ] 

ASF subversion and git services commented on KYLIN-3279:


Commit 0b4e35b817d908280fd399722dfbb69a3162c6b6 in kylin's branch 
refs/heads/master from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0b4e35b ]

KYLIN-3279 Util Class for encryption and decryption


> Util Class for encryption and decryption
> 
>
> Key: KYLIN-3279
> URL: https://issues.apache.org/jira/browse/KYLIN-3279
> Project: Kylin
>  Issue Type: Improvement
>  Components: Others
>Affects Versions: v2.3.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> Extract methods of "encrypt" and "decrypt" in 
> org.apache.kylin.rest.security.PasswordPlaceholderConfigurer to a Util class 
> in core-common package. This will make them more reusable. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403917#comment-16403917
 ] 

ASF subversion and git services commented on KYLIN-3284:


Commit 4e0bb70f72e6fcc11bacd7d8ede996ede5449560 in kylin's branch 
refs/heads/sync from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4e0bb70 ]

KYLIN-3284 Refactor all OLAPRel computeSelfCost


> Refactor all OLAPRel computeSelfCost
> 
>
> Key: KYLIN-3284
> URL: https://issues.apache.org/jira/browse/KYLIN-3284
> Project: Kylin
>  Issue Type: Improvement
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
>
> for queries like:
>  select lstg_format_name from test_kylin_fact order by case 
>  when 1=1 then 
>  cal_dt 
>  ELSE 
>  seller_id 
>  end 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
>  EnumerableSort(sort0=[$1], dir0=[ASC])
>   OLAPToEnumerableConverter
> OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>   OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, 
> CAST($7):DATE)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 
> 22, 23, 24, 25, 26]])
> {code}
> This is the RelOptPlanner:
> {code:java}
> Sets:
> Set#0, type: RecordType(BIGINT TRANS_ID, BIGINT ORDER_ID, DATE CAL_DT, 
> VARCHAR(256) LSTG_FORMAT_NAME, BIGINT LEAF_CATEG_ID, INTEGER LSTG_SITE_ID, 
> SMALLINT SLR_SEGMENT_CD, INTEGER SELLER_ID, DECIMAL(19, 4) PRICE, INTEGER 
> ITEM_COUNT, VARCHAR(256) TEST_COUNT_DISTINCT_BITMAP, DECIMAL(19, 4) 
> DEAL_AMOUNT, INTEGER DEAL_YEAR, VARCHAR(256) 
> LEFTJOIN_BUYER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_SELLER_ID_AND_COUNTRY_NAME, VARCHAR(256) 
> LEFTJOIN_BUYER_COUNTRY_ABBR, VARCHAR(256) LEFTJOIN_SELLER_COUNTRY_ABBR, 
> BIGINT _KY_COUNT__, INTEGER _KY_SUM_TEST_KYLIN_FACT_ITEM_COUNT_, DECIMAL(19, 
> 4) _KY_SUM_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MIN_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) 
> _KY_MAX_TEST_KYLIN_FACT_PRICE_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_SELLER_ID_, ANY 
> _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME_TEST_KYLIN_FACT_SELLER_ID_,
>  ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_TEST_COUNT_DISTINCT_BITMAP_, 
> DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_DEAL_AMOUNT_, ANY 
> _KY_PERCENTILE_APPROX_TEST_KYLIN_FACT_PRICE_)
>   rel#140:Subset#0.OLAP.[], best=rel#131, importance=0.7291
>   rel#131:OLAPTableScan.OLAP.[](table=[DEFAULT, 
> TEST_KYLIN_FACT],ctx=,fields=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 
> 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]), rowcount=100.0, 
> cumulative cost={5.0 rows, 5.051 cpu, 0.0 io}
>   
> rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
>  rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
>   rel#169:Subset#0.NONE.[2], best=null, importance=0.7291
>   
> rel#167:LogicalSort.NONE.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={inf}
>   rel#171:Subset#0.ENUMERABLE.[2], best=rel#182, importance=0.81
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   rel#173:Subset#0.OLAP.[1], best=null, importance=0.6561
>   rel#177:Subset#0.ENUMERABLE.[], best=rel#182, 
> importance=0.7291
>   
> rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
>   
> rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
>   
> rel#185:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#140:Subset#0.OLAP.[]),
>  rowcount=100.0, cumulative cost={1.0E100 rows, 5.051 cpu, 0.0 io}
>   rel#180:Subset#0.OLAP.[2], best=rel#179,

[jira] [Commented] (KYLIN-3283) Support values RelNode

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403918#comment-16403918
 ] 

ASF subversion and git services commented on KYLIN-3283:


Commit 6122f0675d05e10ade86c45a0dc64dfb20e728f3 in kylin's branch 
refs/heads/sync from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6122f06 ]

KYLIN-3283 Support values RelNode


> Support values RelNode
> --
>
> Key: KYLIN-3283
> URL: https://issues.apache.org/jira/browse/KYLIN-3283
> Project: Kylin
>  Issue Type: New Feature
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Major
>
> for queries like:
> select leaf_categ_id, sum(price) as sum_price from test_kylin_fact  group by 
> leaf_categ_id
> union all
> select 1 as leaf_categ_id2, 2 as sum_price2 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPUnionRel(all=[true], ctx=[], all=[true])
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
>   OLAPProjectRel(LEAF_CATEG_ID2=[1], SUM_PRICE2=[2], ctx=[])
> OLAPValuesRel(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableUnion(all=[true])
> OLAPToEnumerableConverter
>   OLAPAggregateRel(group=[{0}], SUM_PRICE=[SUM($1)], ctx=[])
> OLAPProjectRel(LEAF_CATEG_ID=[$4], PRICE=[$8], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> EnumerableCalc(expr#0=[{inputs}], expr#1=[1], expr#2=[2], 
> LEAF_CATEG_ID2=[$t1], SUM_PRICE2=[$t2])
>   EnumerableValues(type=[RecordType(INTEGER ZERO)], tuples=[[{ 0 }]])
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403916#comment-16403916
 ] 

ASF subversion and git services commented on KYLIN-3281:


Commit 8a9ed59213bb2f2d06ef99e836d7b21e37413112 in kylin's branch 
refs/heads/sync from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8a9ed59 ]

KYLIN-3281 OLAPProjectRule can't normal working with projectRel[input=sortRel]


> OLAPProjectRule can't normal working with  projectRel[input=sortRel]
> 
>
> Key: KYLIN-3281
> URL: https://issues.apache.org/jira/browse/KYLIN-3281
> Project: Kylin
>  Issue Type: Bug
>  Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: yiming.xu
>Assignee: yiming.xu
>Priority: Critical
>
> for queries like:
> select sum("count") from (SELECT LSTG_FORMAT_NAME ,  COUNT(*) AS "count" FROM 
> TEST_KYLIN_FACT  GROUP BY  "LSTG_FORMAT_NAME" ORDER BY "count" DESC) as a
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPAggregateRel(group=[{}], EXPR$0=[SUM($0)], ctx=[])
>   OLAPProjectRel(count=[$1], ctx=[])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
>   OLAPAggregateRel(group=[{0}], count=[COUNT()], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
> EnumerableLimit(fetch=[5])
>   EnumerableAggregate(group=[{}], EXPR$0=[SUM($0)])
> EnumerableCalc(expr#0..1=[{inputs}], count=[$t1])
>   EnumerableSort(sort0=[$1], dir0=[DESC])
> EnumerableAggregate(group=[{0}], count=[COUNT()])
>   OLAPToEnumerableConverter
> OLAPProjectRel(LSTG_FORMAT_NAME=[$3], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], 
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 
> 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> VolcanoPlanner apply OLAPProjectRule to logicalProject,convert child node 
> should use child traitset not current proejct traitset.
> If we use current project trait set, the child node trait set is  [olap][0 
> DESC], it will create a new subset but the subset doesn't have any rel. the 
> cost is inf,
> so volcanoPlanner choose a  the EnumerableAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3279) Util Class for encryption and decryption

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403913#comment-16403913
 ] 

ASF subversion and git services commented on KYLIN-3279:


Commit 0b4e35b817d908280fd399722dfbb69a3162c6b6 in kylin's branch 
refs/heads/sync from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0b4e35b ]

KYLIN-3279 Util Class for encryption and decryption


> Util Class for encryption and decryption
> 
>
> Key: KYLIN-3279
> URL: https://issues.apache.org/jira/browse/KYLIN-3279
> Project: Kylin
>  Issue Type: Improvement
>  Components: Others
>Affects Versions: v2.3.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> Extract methods of "encrypt" and "decrypt" in 
> org.apache.kylin.rest.security.PasswordPlaceholderConfigurer to a Util class 
> in core-common package. This will make them more reusable. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403914#comment-16403914
 ] 

ASF subversion and git services commented on KYLIN-3276:


Commit 6ba2fcaafc3c849851eed1e790318d1bc015fd7f in kylin's branch 
refs/heads/sync from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6ba2fca ]

KYLIN-3276 Fix query with different dynamic parameters get the same query result


> query with dynamic parameter get the same query result
> --
>
> Key: KYLIN-3276
> URL: https://issues.apache.org/jira/browse/KYLIN-3276
> Project: Kylin
>  Issue Type: Bug
>  Components: Driver - JDBC, Query Engine
>Affects Versions: v2.2.0
> Environment: Kylin 2.2.0, JDK 1.8
>Reporter: ZHAOXIXIN
>Assignee: nichunen
>Priority: Major
> Fix For: v2.3.1
>
>
> sql:"select distinct uid from table where daytime=? limit 70" 
> ps.setString(1,"2018-01-03") 
> store the result to ArrayList, and print the size of ArrayList. 
> when I set the daytime to different dates, the same result I got. 
> I changed the sql, just change limit value for example 
> sql:"select distinct uid from table where daytime=? limit 60" 
> then I got the right value the same like query from GUI:count (distinct uid) 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3262) Produce jacoco exec files for integration test

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403911#comment-16403911
 ] 

ASF subversion and git services commented on KYLIN-3262:


Commit cb3913b681a1230130376215cfa30477eb3692f0 in kylin's branch 
refs/heads/sync from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cb3913b ]

APACHE-KYLIN-3262: add on-the-fly instrumentation with using JaCoCo Java Agent 
for integration test

Signed-off-by: Billy Liu 


> Produce jacoco exec files for integration test
> --
>
> Key: KYLIN-3262
> URL: https://issues.apache.org/jira/browse/KYLIN-3262
> Project: Kylin
>  Issue Type: Task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.0
>
> Attachments: APACHE-KYLIN-3262.patch
>
>
> Currently to calculate code coverage, kylin only considers UT. However, IT 
> also covers lots of code. It's better to include this for code coverage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3262) Produce jacoco exec files for integration test

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403912#comment-16403912
 ] 

ASF subversion and git services commented on KYLIN-3262:


Commit 537ae8afd1c60d374980c099e5c5fead2c215684 in kylin's branch 
refs/heads/sync from [~yimingliu]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=537ae8a ]

KYLIN-3262 add JaCoCo Java Agent for integration test


> Produce jacoco exec files for integration test
> --
>
> Key: KYLIN-3262
> URL: https://issues.apache.org/jira/browse/KYLIN-3262
> Project: Kylin
>  Issue Type: Task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.0
>
> Attachments: APACHE-KYLIN-3262.patch
>
>
> Currently to calculate code coverage, kylin only considers UT. However, IT 
> also covers lots of code. It's better to include this for code coverage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KYLIN-3149) Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected

2018-03-18 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/KYLIN-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403919#comment-16403919
 ] 

ASF subversion and git services commented on KYLIN-3149:


Commit b81d7a2df289029768edd6532e63eae251f28b41 in kylin's branch 
refs/heads/sync from [~hn5092]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b81d7a2 ]

KYLIN-3149 Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as 
expected


> Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected
> 
>
> Key: KYLIN-3149
> URL: https://issues.apache.org/jira/browse/KYLIN-3149
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.2.0
>Reporter: hongbin ma
>Assignee: yiming.xu
>Priority: Major
> Attachments: dump.txt
>
>
> for queries like:
> {code:sql}
> select TRANS_ID from kylin_sales group by cast (case 
> WHEN  '1030101' = '1030101' then substring(COALESCE(OPS_USER_ID, 
> ''), 1, 1)
> when  '1030101' = '1030102' then substring(COALESCE(OPS_REGION, 
> ''), 1, 1)  
> when  '1030101' = '1030103' then substring(COALESCE(LSTG_FORMAT_NAME, 
> ''), 1, 1)
> when  '1030101' = '1030104' then substring(COALESCE(LSTG_FORMAT_NAME, 
> ''), 1, 1)
> end as varchar(256)), TRANS_ID;
> {code}
> the expected logical plan after volcano is:
> {code}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
>   OLAPProjectRel(TRANS_ID=[$1], ctx=[])
> OLAPLimitRel(ctx=[], fetch=[5])
>   OLAPAggregateRel(group=[{0, 1}], ctx=[])
> OLAPProjectRel($f0=[SUBSTRING(CASE(IS NOT NULL($9), $9, 
> ''), 1, 1)], TRANS_ID=[$0], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
> {code}
> however the actual is:
> {code}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[5])
> OLAPProjectRel(TRANS_ID=[$1], ctx=[])
>   OLAPAggregateRel(group=[{0, 1}], ctx=[])
> OLAPProjectRel($f0=[CAST(CASE(=('1030101', '1030101'), 
> SUBSTRING(CASE(IS NOT NULL($9), $9, ''), 1, 1), =('1030101', 
> '1030102'), SUBSTRING(CASE(IS NOT NULL($10), $10, ''), 1, 1), 
> =('1030101', '1030103'), SUBSTRING(CASE(IS NOT NULL($2), $2, ''), 
> 1, 1), =('1030101', '1030104'), SUBSTRING(CASE(IS NOT NULL($2), $2, 
> ''), 1, 1), null)):VARCHAR(256) CHARACTER SET "UTF-16LE" COLLATE 
> "UTF-16LE$en_US$primary"], TRANS_ID=[$0], ctx=[])
>   OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[], fields=[[0, 
> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
> {code}
> looks like Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as 
> expected. If we dump the internal state of this VolcanoPlanner 
> (org.apache.calcite.plan.volcano.VolcanoPlanner#dump), line 19-21 from the 
> complete dump is attached:
> {code}
>   rel#337:Subset#1.OLAP.[], best=rel#339, importance=0.6561
>   
> rel#339:OLAPProjectRel.OLAP.[](input=rel#303:Subset#0.OLAP.[],$f0=CAST(CASE(=('1030101',
>  '1030101'), SUBSTRING(CASE(IS NOT NULL($9), $9, ''), 1, 1), 
> =('1030101', '1030102'), SUBSTRING(CASE(IS NOT NULL($10), $10, 
> ''), 1, 1), =('1030101', '1030103'), SUBSTRING(CASE(IS NOT 
> NULL($2), $2, ''), 1, 1), =('1030101', '1030104'), 
> SUBSTRING(CASE(IS NOT NULL($2), $2, ''), 1, 1), 
> null)):VARCHAR(256) CHARACTER SET "UTF-16LE" COLLATE 
> "UTF-16LE$en_US$primary",TRANS_ID=$0,ctx=), rowcount=100.0, cumulative 
> cost={15.0 rows, 25.05 cpu, 0.0 io}
>   
> rel#348:OLAPProjectRel.OLAP.[](input=rel#303:Subset#0.OLAP.[],$f0=SUBSTRING(CASE(IS
>  NOT NULL($9), $9, ''), 1, 1),TRANS_ID=$0,ctx=), rowcount=100.0, 
> cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> {code}
> we see two rels with same cost:  #339 and #348, where #339 is created from 
> LogicalProject = (OLAPProjectRule)=> OLAPProject, and #348 is created from 
> LogicalProject =( ReduceExpressionsRule) => Reduced LogicalProject 
> =(OLAPProjectRule)=> Reduced OLAPProject . Since ReduceExpressionsRule 
> require Logical Project rather than OLAP Project, #339 is never reduced.
> The worse thing is that cost of #339 and #348 are same. By current volcano 
> planner algorithm  the first met rel will be chosen, so unexpected rel is 
> chosen
> A simple approach to fix this is to refine the rel choosing algorithm: when 
> two rels are equal in cost, choose a "simpler" one. Since we don't have a 
> perfect measurement of "simple", we simply choose the rel with smaller 
> toString() length



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

[jira] [Updated] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

[jira] [Created] (KYLIN-3296) cube合并时，java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method)

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

[jira] [Commented] (KYLIN-3257) Useless call in FuzzyValueCombination

[jira] [Created] (KYLIN-3295) Unused method SQLDigestUtil#appendTsFilterToExecute

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

[jira] [Resolved] (KYLIN-3276) query with dynamic parameter get the same query result

[jira] [Resolved] (KYLIN-3279) Util Class for encryption and decryption

[jira] [Resolved] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

[jira] [Resolved] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

[jira] [Updated] (KYLIN-3149) Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected

[jira] [Updated] (KYLIN-3283) Support values RelNode

[jira] [Created] (KYLIN-3294) Remove HBaseMROutput.java, RangeKeyDistributionJob.java and other sunset classes

[jira] [Commented] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

[jira] [Commented] (KYLIN-3283) Support values RelNode

[jira] [Commented] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

[jira] [Commented] (KYLIN-3279) Util Class for encryption and decryption

[jira] [Commented] (KYLIN-3284) Refactor all OLAPRel computeSelfCost

[jira] [Commented] (KYLIN-3283) Support values RelNode

[jira] [Commented] (KYLIN-3281) OLAPProjectRule can't normal working with projectRel[input=sortRel]

[jira] [Commented] (KYLIN-3279) Util Class for encryption and decryption

[jira] [Commented] (KYLIN-3276) query with dynamic parameter get the same query result

[jira] [Commented] (KYLIN-3262) Produce jacoco exec files for integration test

[jira] [Commented] (KYLIN-3262) Produce jacoco exec files for integration test

[jira] [Commented] (KYLIN-3149) Calcite's ReduceExpressionsRule.PROJECT_INSTANCE not working as expected

30 matches

Site Navigation

Mail list logo

Footer information