[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571140#comment-16571140
]
ASF subversion and git services commented on KYLIN-3476:
Commit e5162aad45f3bbc6bcc99afb08bd9bf836650642 in kylin's branch
refs/heads/2.4.x from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e5162aa ]
KYLIN-3476 fix TupleExpression verification when parsing sql
Signed-off-by: shaofengshi
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) get malformed token with Cluster Mode(multi-query)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570974#comment-16570974
]
Temple Zhou commented on KYLIN-3474:
[~albertoramon]
Oh, maybe you are right, I will check my LB.
> get malformed token with Cluster Mode(multi-query)
> --
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Priority: Major
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570315#comment-16570315
]
ASF subversion and git services commented on KYLIN-3476:
Commit b938a1c53e5b5e894b9ca0cd6c6b3b757bdeca6a in kylin's branch
refs/heads/master from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b938a1c ]
KYLIN-3476 fix TupleExpression verification when parsing sql
Signed-off-by: shaofengshi
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570217#comment-16570217
]
ASF subversion and git services commented on KYLIN-3476:
Commit b938a1c53e5b5e894b9ca0cd6c6b3b757bdeca6a in kylin's branch
refs/heads/KYLIN-3476 from Zhong
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b938a1c ]
KYLIN-3476 fix TupleExpression verification when parsing sql
Signed-off-by: shaofengshi
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3476:
Fix Version/s: v2.4.1
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570003#comment-16570003
]
Zhong Yanghong commented on KYLIN-3476:
---
Hi [~Shaofengshi], this patch is for fixing this kind of issue. Once the patch
applied, the issue should be fixed.
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569989#comment-16569989
]
Shaofeng SHI commented on KYLIN-3476:
-
I uploaded the error message in "b134-12adb0383568.png"
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3476:
Attachment: b134-12adb0383568.png
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch, b134-12adb0383568.png,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569978#comment-16569978
]
Shaofeng SHI commented on KYLIN-3476:
-
!image-2018-08-06-17-58-18-760.png!
Yanghong, a user reported this error; It seems a measure value couldn't be used
in "division", but that should be accepted as it is not in a "sum" or other
calculation. Is it a bug?
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3476) Fix TupleExpression verification when parsing sql
[
https://issues.apache.org/jira/browse/KYLIN-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3476:
Attachment: image-2018-08-06-17-58-18-760.png
> Fix TupleExpression verification when parsing sql
> -
>
> Key: KYLIN-3476
> URL: https://issues.apache.org/jira/browse/KYLIN-3476
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: APACHE-KYLIN-3476.patch,
> image-2018-08-06-17-58-18-760.png
>
>
> Previously there's a validation check for sql, since dynamic column does not
> support {{sum(col1/col2)}}. However, if the sql has a subquery and
> {{sum(col1/col2)}} is in the main query rather than the sub-query, this
> should be OK. One example is as follows:
> {code}
> select T1.D1, sum(T1.gmv/T2.gmv)
> from
> (select D1, D2, sum(M1) as gmv
> from T
> group by D1, D2) T1
> inner join
> (select D1, D2, sum(M2) as gmv
> from T
> group by D1, D2) T2
> on T1.D1 = T2.D1
> group by T1.D1
> {code}
> Since the outside {{sum(T1.gmv/T2.gmv)}} is handled by calcite rather than
> Kylin, it's OK and should not go through the verification.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3483.
-
Resolution: Fixed
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.5.0
>
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569972#comment-16569972
]
ASF subversion and git services commented on KYLIN-3483:
Commit 4d61c30ebd52c555632933ccdc928f85cc806cfd in kylin's branch
refs/heads/master from tttMelody
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4d61c30 ]
KYLIN-3483, fix the loss of precision in HLLCSerializer.
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.5.0
>
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569971#comment-16569971
]
ASF GitHub Bot commented on KYLIN-3483:
---
shaofengshi closed pull request #187: KYLIN-3483, fix the loss of precision in
HLLCSerializer.
URL: https://github.com/apache/kylin/pull/187
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-metadata/src/main/java/org/apache/kylin/measure/hllc/HLLCSerializer.java
b/core-metadata/src/main/java/org/apache/kylin/measure/hllc/HLLCSerializer.java
index 9310864916..9ad16952aa 100644
---
a/core-metadata/src/main/java/org/apache/kylin/measure/hllc/HLLCSerializer.java
+++
b/core-metadata/src/main/java/org/apache/kylin/measure/hllc/HLLCSerializer.java
@@ -85,7 +85,7 @@ protected double getStorageBytesEstimate(double
averageNumOfElementsInCounter) {
int registerIndexSize = current().getRegisterIndexSize();
int m = 1 << precision;
if (!current().isDense((int) averageNumOfElementsInCounter)
-|| averageNumOfElementsInCounter < (m - 5) / (1 +
registerIndexSize)) {
+|| averageNumOfElementsInCounter < (m - 5d) / (1d +
registerIndexSize)) {
// 5 = 1 + 4 for scheme and size
// size * (getRegisterIndexSize + 1)
return 5 + averageNumOfElementsInCounter * (registerIndexSize + 1);
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.5.0
>
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3379) timestampadd bug fix and add test
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-3379: - Assignee: jiatao.tao (was: hongbin ma) > timestampadd bug fix and add test > - > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: jiatao.tao >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3379) timestampadd bug fix and add test
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-3379: -- Summary: timestampadd bug fix and add test (was: timestampadd test coverage is not enough) > timestampadd bug fix and add test > - > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: hongbin ma >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3485) Make unloading table more flexible
[ https://issues.apache.org/jira/browse/KYLIN-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3485: --- Assignee: rongchuan.jin > Make unloading table more flexible > -- > > Key: KYLIN-3485 > URL: https://issues.apache.org/jira/browse/KYLIN-3485 > Project: Kylin > Issue Type: Improvement > Components: RDBMS Source, REST Service, Streaming >Reporter: rongchuan.jin >Assignee: rongchuan.jin >Priority: Major > > Now Kylin use fixed code to unload streaming table , it only works with > Kafka.If I want to use other streaming datasource to extend Kylin like Aliyun > Loghub Service.It seems to be difficult. > I would like contribute improvment to make unloading table more flexible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3485) Make unloading table more flexible
rongchuan.jin created KYLIN-3485: Summary: Make unloading table more flexible Key: KYLIN-3485 URL: https://issues.apache.org/jira/browse/KYLIN-3485 Project: Kylin Issue Type: Improvement Components: RDBMS Source, REST Service, Streaming Reporter: rongchuan.jin Now Kylin use fixed code to unload streaming table , it only works with Kafka.If I want to use other streaming datasource to extend Kylin like Aliyun Loghub Service.It seems to be difficult. I would like contribute improvment to make unloading table more flexible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3483) Imprecise comparison between double and integer division
[
https://issues.apache.org/jira/browse/KYLIN-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569916#comment-16569916
]
ASF GitHub Bot commented on KYLIN-3483:
---
tttMelody commented on issue #187: KYLIN-3483, fix the loss of precision in
HLLCSerializer.
URL: https://github.com/apache/kylin/pull/187#issuecomment-410639736
Thanks @shaofengshi !
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Imprecise comparison between double and integer division
>
>
> Key: KYLIN-3483
> URL: https://issues.apache.org/jira/browse/KYLIN-3483
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.5.0
>
>
> In HLLCSerializer :
> {code}
> || averageNumOfElementsInCounter < (m - 5) / (1 +
> registerIndexSize)) {
> {code}
> The integer division on the right side of < would lose remainder, or
> fractional part of the quotient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3479) Model can save when kafka partition date column not select
[ https://issues.apache.org/jira/browse/KYLIN-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569908#comment-16569908 ] ASF subversion and git services commented on KYLIN-3479: Commit 122b4d0cf0d0917438eb1c05e9d132e068bb481b in kylin's branch refs/heads/master from GinaZhai [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=122b4d0 ] KYLIN-3479 Model can save when Kafka partition date column not select KYLIN-3479 Model can save when kafka partition date column not select. KYLIN-3479 Model can save when kafka partition date column not select fix ugly code Remove unrelated code > Model can save when kafka partition date column not select > -- > > Key: KYLIN-3479 > URL: https://issues.apache.org/jira/browse/KYLIN-3479 > Project: Kylin > Issue Type: Improvement > Components: Streaming, Web >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.5.0 > > Attachments: kafka partition column not select.png > > > I didn't select *Partition Date Column* when creating model with streaming > table. When I create cube and click save button, it will pop up a popup > window with error message "Must define a partition column". I think this > error message should show up early. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3479) Model can save when kafka partition date column not select
[
https://issues.apache.org/jira/browse/KYLIN-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569906#comment-16569906
]
ASF GitHub Bot commented on KYLIN-3479:
---
shaofengshi closed pull request #186: KYLIN-3479 Model can save when kafka
partition date column not select
URL: https://github.com/apache/kylin/pull/186
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/server-base/src/main/java/org/apache/kylin/rest/service/ModelService.java
b/server-base/src/main/java/org/apache/kylin/rest/service/ModelService.java
index 23c0085f21..b3f6e2d4dd 100644
--- a/server-base/src/main/java/org/apache/kylin/rest/service/ModelService.java
+++ b/server-base/src/main/java/org/apache/kylin/rest/service/ModelService.java
@@ -33,6 +33,7 @@
import org.apache.kylin.metadata.ModifiedOrder;
import org.apache.kylin.metadata.draft.Draft;
import org.apache.kylin.metadata.model.DataModelDesc;
+import org.apache.kylin.metadata.model.ISourceAware;
import org.apache.kylin.metadata.model.JoinsTree;
import org.apache.kylin.metadata.model.ModelDimensionDesc;
import org.apache.kylin.metadata.model.TableDesc;
@@ -132,6 +133,13 @@ public DataModelDesc createModelDesc(String projectName,
DataModelDesc desc) thr
throw new
BadRequestException(String.format(msg.getDUPLICATE_MODEL_NAME(),
desc.getName()));
}
+String factTableName = desc.getRootFactTableName();
+TableDesc tableDesc = getTableManager().getTableDesc(factTableName,
projectName);
+if (tableDesc.getSourceType() == ISourceAware.ID_STREAMING
+&& (desc.getPartitionDesc() == null ||
desc.getPartitionDesc().getPartitionDateColumn() == null)) {
+throw new IllegalArgumentException("Must define a partition
column.");
+}
+
DataModelDesc createdDesc = null;
String owner =
SecurityContextHolder.getContext().getAuthentication().getName();
createdDesc = getDataModelManager().createDataModelDesc(desc,
projectName, owner);
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Model can save when kafka partition date column not select
> --
>
> Key: KYLIN-3479
> URL: https://issues.apache.org/jira/browse/KYLIN-3479
> Project: Kylin
> Issue Type: Improvement
> Components: Streaming, Web
>Reporter: Na Zhai
>Assignee: Na Zhai
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: kafka partition column not select.png
>
>
> I didn't select *Partition Date Column* when creating model with streaming
> table. When I create cube and click save button, it will pop up a popup
> window with error message "Must define a partition column". I think this
> error message should show up early.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3479) Model can save when kafka partition date column not select
[ https://issues.apache.org/jira/browse/KYLIN-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569907#comment-16569907 ] ASF subversion and git services commented on KYLIN-3479: Commit 122b4d0cf0d0917438eb1c05e9d132e068bb481b in kylin's branch refs/heads/master from GinaZhai [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=122b4d0 ] KYLIN-3479 Model can save when Kafka partition date column not select KYLIN-3479 Model can save when kafka partition date column not select. KYLIN-3479 Model can save when kafka partition date column not select fix ugly code Remove unrelated code > Model can save when kafka partition date column not select > -- > > Key: KYLIN-3479 > URL: https://issues.apache.org/jira/browse/KYLIN-3479 > Project: Kylin > Issue Type: Improvement > Components: Streaming, Web >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.5.0 > > Attachments: kafka partition column not select.png > > > I didn't select *Partition Date Column* when creating model with streaming > table. When I create cube and click save button, it will pop up a popup > window with error message "Must define a partition column". I think this > error message should show up early. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3479) Model can save when kafka partition date column not select
[ https://issues.apache.org/jira/browse/KYLIN-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569909#comment-16569909 ] ASF subversion and git services commented on KYLIN-3479: Commit 122b4d0cf0d0917438eb1c05e9d132e068bb481b in kylin's branch refs/heads/master from GinaZhai [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=122b4d0 ] KYLIN-3479 Model can save when Kafka partition date column not select KYLIN-3479 Model can save when kafka partition date column not select. KYLIN-3479 Model can save when kafka partition date column not select fix ugly code Remove unrelated code > Model can save when kafka partition date column not select > -- > > Key: KYLIN-3479 > URL: https://issues.apache.org/jira/browse/KYLIN-3479 > Project: Kylin > Issue Type: Improvement > Components: Streaming, Web >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.5.0 > > Attachments: kafka partition column not select.png > > > I didn't select *Partition Date Column* when creating model with streaming > table. When I create cube and click save button, it will pop up a popup > window with error message "Must define a partition column". I think this > error message should show up early. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3479) Model can save when kafka partition date column not select
[ https://issues.apache.org/jira/browse/KYLIN-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3479. - Resolution: Fixed > Model can save when kafka partition date column not select > -- > > Key: KYLIN-3479 > URL: https://issues.apache.org/jira/browse/KYLIN-3479 > Project: Kylin > Issue Type: Improvement > Components: Streaming, Web >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.5.0 > > Attachments: kafka partition column not select.png > > > I didn't select *Partition Date Column* when creating model with streaming > table. When I create cube and click save button, it will pop up a popup > window with error message "Must define a partition column". I think this > error message should show up early. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3379) timestampadd test coverage is not enough
[
https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569901#comment-16569901
]
ASF GitHub Bot commented on KYLIN-3379:
---
shaofengshi closed pull request #188: KYLIN-3379 fix ts add bug and enhance the
test coverage of timestamp add.
URL: https://github.com/apache/kylin/pull/188
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/atopcalcite/pom.xml b/atopcalcite/pom.xml
index 56b6d413ee..41848461ac 100644
--- a/atopcalcite/pom.xml
+++ b/atopcalcite/pom.xml
@@ -47,5 +47,10 @@
org.apache.calcite.avatica
avatica
+
+junit
+junit
+test
+
diff --git
a/atopcalcite/src/main/java/org/apache/calcite/runtime/SqlFunctions.java
b/atopcalcite/src/main/java/org/apache/calcite/runtime/SqlFunctions.java
new file mode 100644
index 00..38bd223e05
--- /dev/null
+++ b/atopcalcite/src/main/java/org/apache/calcite/runtime/SqlFunctions.java
@@ -0,0 +1,2238 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.runtime;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.sql.SQLException;
+import java.sql.Timestamp;
+import java.text.DecimalFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.TimeZone;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.regex.Pattern;
+
+import org.apache.calcite.DataContext;
+import org.apache.calcite.avatica.util.ByteString;
+import org.apache.calcite.avatica.util.DateTimeUtils;
+import org.apache.calcite.avatica.util.Spaces;
+import org.apache.calcite.avatica.util.TimeUnitRange;
+import org.apache.calcite.linq4j.AbstractEnumerable;
+import org.apache.calcite.linq4j.CartesianProductEnumerator;
+import org.apache.calcite.linq4j.Enumerable;
+import org.apache.calcite.linq4j.Enumerator;
+import org.apache.calcite.linq4j.Linq4j;
+import org.apache.calcite.linq4j.function.Deterministic;
+import org.apache.calcite.linq4j.function.Function1;
+import org.apache.calcite.linq4j.function.NonDeterministic;
+import org.apache.calcite.linq4j.tree.Primitive;
+import org.apache.calcite.runtime.FlatLists.ComparableList;
+import org.apache.calcite.util.Bug;
+import org.apache.calcite.util.NumberUtil;
+
+/*
+ * OVERRIDE POINT:
+ * - more power() overloads
+ * - refined org.apache.calcite.runtime.SqlFunctions#addMonths(int, int)
+ * - corner case subString()
+ * - corner case trim_()
+ * - upper()
+ * - lower()
+ * - charLength()
+ * - addMonths()
+ */
+
+/**
+ * Helper methods to implement SQL functions in generated code.
+ *
+ * Not present: and, or, not (builtin operators are better, because they
+ * use lazy evaluation. Implementations do not check for null values; the
+ * calling code must do that.
+ *
+ * Many of the functions do not check for null values. This is intentional.
+ * If null arguments are possible, the code-generation framework checks for
+ * nulls before calling the functions.
+ */
+
+@SuppressWarnings("UnnecessaryUnboxing")
+@Deterministic
+public class SqlFunctions {
+private static final DecimalFormat DOUBLE_FORMAT =
NumberUtil.decimalFormat("0.0E0");
+
+private static final TimeZone LOCAL_TZ = TimeZone.getDefault();
+
+private static final Function1, Enumerable>
LIST_AS_ENUMERABLE = new Function1, Enumerable>() {
+public Enumerable apply(List list) {
+return Linq4j.asEnumerable(list);
+}
+};
+
+private static final Function1>
ARRAY_CARTESIAN_PRODUCT = new Function1>() {
+public Enumerable apply(Object[] lists) {
+final List> enumerators = new ArrayList<>();
+for (Object list : lists) {
+
[jira] [Commented] (KYLIN-3379) timestampadd test coverage is not enough
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569902#comment-16569902 ] ASF subversion and git services commented on KYLIN-3379: Commit 95f35fc4e60a2085d8e2e6272e1579eed3a76913 in kylin's branch refs/heads/master from tttMelody [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=95f35fc ] KYLIN-3379 enhance the test coverage of timestamp add. > timestampadd test coverage is not enough > > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: hongbin ma >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3379) timestampadd test coverage is not enough
[ https://issues.apache.org/jira/browse/KYLIN-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3379: Fix Version/s: v2.5.0 > timestampadd test coverage is not enough > > > Key: KYLIN-3379 > URL: https://issues.apache.org/jira/browse/KYLIN-3379 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.3.1 >Reporter: hongbin ma >Assignee: hongbin ma >Priority: Major > Fix For: v2.5.0 > > > complex cases like > timestampadd(MONTH,23,test_kylin_fact.cal_dt) or > timestampadd(MONTH,-23,test_kylin_fact.cal_dt) is not covered. > > And my tests shows this kind of queries will fail IT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
