[jira] [Updated] (KYLIN-4902) Avoid using weak cryptographic algorithm
[ https://issues.apache.org/jira/browse/KYLIN-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-4902: Fix Version/s: v3.1.2 > Avoid using weak cryptographic algorithm > - > > Key: KYLIN-4902 > URL: https://issues.apache.org/jira/browse/KYLIN-4902 > Project: Kylin > Issue Type: Improvement >Reporter: Vicky Zhang >Priority: Major > Fix For: v3.1.2 > > > in file > kylin/blob/master/core-common/src/main/java/org/apache/kylin/common/util/EncryptUtil.java, > AES/ECB/PKCS5PADDING is used for encrypting and decrypting information, line > 36 and 49 > *Security impact*: > ECB is a deprecated encryption mode used with AES, it may cause inefficient > encryption, here is the comparison example that encrypts a picture with ECB > mode and CBC mode > [https://datalocker.com/what-is-the-difference-between-ecb-mode-versus-cbc-mode-aes-encryption/] > some links here: [https://cwe.mitre.org/data/definitions/327.html] > suggestions: we recommend you use AES/CBC/PKCS5Padding, but CBC mode would > require an iv in the encryption process for security concerns. > *Please share with us your opinions/comments if there is any:* > Is the bug report helpful? > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4906) support query/job server dynamic register and discovery in kylin4
[ https://issues.apache.org/jira/browse/KYLIN-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288206#comment-17288206 ] ASF GitHub Bot commented on KYLIN-4906: --- hit-lacus merged pull request #1585: URL: https://github.com/apache/kylin/pull/1585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > support query/job server dynamic register and discovery in kylin4 > - > > Key: KYLIN-4906 > URL: https://issues.apache.org/jira/browse/KYLIN-4906 > Project: Kylin > Issue Type: Improvement > Components: Integration >Affects Versions: v4.0.0-beta >Reporter: ShengJun Zheng >Assignee: ShengJun Zheng >Priority: Major > Fix For: v4.0.0-GA > > Attachments: screenshot-1.png, screenshot-2.png > > > Hi，currently we are troubled by a static configuration: > kylin.server.cluster-servers. There tow situations not so friendly to us. > 1. when adding query node to cluster, we forgot to update this > configuration in job server. It caused the newly added server was not > notified of all metadata update events, causing wrong query result. > 2. we plan to deploy query server in k8s , this static configuration is not > cloud native (having to update k8s configmap) > We use kylin 2.x，seems this is still a problem in kylin 3.x (relavent code: > org.apache.kylin.metadata.cachesync.Broadcaster). > I wonder if i was wrong or is there already a solution . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4906) support query/job server dynamic register and discovery in kylin4
[ https://issues.apache.org/jira/browse/KYLIN-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288205#comment-17288205 ] ASF subversion and git services commented on KYLIN-4906: Commit 8a21389f685dfbbcd9c9254e578d7e8dcf70f078 in kylin's branch refs/heads/kylin-on-parquet-v2 from zhengshengjun [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8a21389 ] KYLIN-4906 Support query/job server dynamic register and discovery in kylin4 (cherry picked from commit fa37255b15bdf69e347ec747baf0731ce6634b39) (cherry picked from commit fa37255b15bdf69e347ec747baf0731ce6634b39) (cherry picked from commit fa37255b15bdf69e347ec747baf0731ce6634b39) > support query/job server dynamic register and discovery in kylin4 > - > > Key: KYLIN-4906 > URL: https://issues.apache.org/jira/browse/KYLIN-4906 > Project: Kylin > Issue Type: Improvement > Components: Integration >Affects Versions: v4.0.0-beta >Reporter: ShengJun Zheng >Assignee: ShengJun Zheng >Priority: Major > Fix For: v4.0.0-GA > > Attachments: screenshot-1.png, screenshot-2.png > > > Hi，currently we are troubled by a static configuration: > kylin.server.cluster-servers. There tow situations not so friendly to us. > 1. when adding query node to cluster, we forgot to update this > configuration in job server. It caused the newly added server was not > notified of all metadata update events, causing wrong query result. > 2. we plan to deploy query server in k8s , this static configuration is not > cloud native (having to update k8s configmap) > We use kylin 2.x，seems this is still a problem in kylin 3.x (relavent code: > org.apache.kylin.metadata.cachesync.Broadcaster). > I wonder if i was wrong or is there already a solution . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] hit-lacus merged pull request #1585: KYLIN-4906 Support query/job server dynamic register and discovery in kylin4
hit-lacus merged pull request #1585: URL: https://github.com/apache/kylin/pull/1585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4906) support query/job server dynamic register and discovery in kylin4
[ https://issues.apache.org/jira/browse/KYLIN-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288204#comment-17288204 ] ASF GitHub Bot commented on KYLIN-4906: --- hit-lacus commented on pull request #1585: URL: https://github.com/apache/kylin/pull/1585#issuecomment-783128322 This patch looks to me, will merge. Maybe we should update the offical document about JobScheduler, @zhangayqian could you please add a question and answer in https://cwiki.apache.org/confluence/display/KYLIN/FAQ+Kylin+4.X ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > support query/job server dynamic register and discovery in kylin4 > - > > Key: KYLIN-4906 > URL: https://issues.apache.org/jira/browse/KYLIN-4906 > Project: Kylin > Issue Type: Improvement > Components: Integration >Affects Versions: v4.0.0-beta >Reporter: ShengJun Zheng >Assignee: ShengJun Zheng >Priority: Major > Fix For: v4.0.0-GA > > Attachments: screenshot-1.png, screenshot-2.png > > > Hi，currently we are troubled by a static configuration: > kylin.server.cluster-servers. There tow situations not so friendly to us. > 1. when adding query node to cluster, we forgot to update this > configuration in job server. It caused the newly added server was not > notified of all metadata update events, causing wrong query result. > 2. we plan to deploy query server in k8s , this static configuration is not > cloud native (having to update k8s configmap) > We use kylin 2.x，seems this is still a problem in kylin 3.x (relavent code: > org.apache.kylin.metadata.cachesync.Broadcaster). > I wonder if i was wrong or is there already a solution . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] hit-lacus commented on pull request #1585: KYLIN-4906 Support query/job server dynamic register and discovery in kylin4
hit-lacus commented on pull request #1585: URL: https://github.com/apache/kylin/pull/1585#issuecomment-783128322 This patch looks to me, will merge. Maybe we should update the offical document about JobScheduler, @zhangayqian could you please add a question and answer in https://cwiki.apache.org/confluence/display/KYLIN/FAQ+Kylin+4.X ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (KYLIN-4821) I use kylin to build a cube ,it provides errors
[
https://issues.apache.org/jira/browse/KYLIN-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu reassigned KYLIN-4821:
---
Assignee: Xiaoxiang Yu
> I use kylin to build a cube ,it provides errors
> ---
>
> Key: KYLIN-4821
> URL: https://issues.apache.org/jira/browse/KYLIN-4821
> Project: Kylin
> Issue Type: Bug
> Components: Environment
>Affects Versions: v3.1.0
> Environment: hadoop-2..6.0
> hbase-1.2.0
> hive-1.1.0
> kylin-3.1.0
>Reporter: Angel
>Assignee: Xiaoxiang Yu
>Priority: Major
> Attachments: 11X~0QUTYTB7YAPQB@87{KW.png, 8$~W`)S]1t3...@1n3339n6.png
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4823) Push down having filter error when group by dynamic column
[
https://issues.apache.org/jira/browse/KYLIN-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4823:
Fix Version/s: v3.1.2
> Push down having filter error when group by dynamic column
> ---
>
> Key: KYLIN-4823
> URL: https://issues.apache.org/jira/browse/KYLIN-4823
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.1.0
>Reporter: hcy
>Priority: Major
> Fix For: v3.1.2
>
>
> 如果cube只有一个segment，且shard by的列存在于group by中，满足having filter push
> down的条件时，如果group by中存在动态列，并且case when then
> 中的表达是为column而不是常量时会报数组越界的错误。配置kylin.query.enable-dynamic-column=true无效，也会报错。
> 测试Cube如下:
> 模型为kylin
> example中的kylin_sales_model，cube为kylin_sales_cube，为了重现错误把BUYER_ID的rowkey设置为shard
> by
> 测试SQL如下:
> SELECT BUYER_ID,
> CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END AS
> dyna_group,
> SUM(PRICE)
> FROM KYLIN_SALES
> GROUP BY
> BUYER_ID,
> CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END
> HAVING SUM(PRICE)>10
> 报错如下：
> {color:#b94a48}Index: 4, Size: 1 while executing SQL: "select * from (SELECT
> BUYER_ID, CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END
> AS dyna_group, SUM(PRICE) FROM KYLIN_SALES GROUP BY BUYER_ID, CASE WHEN
> LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END HAVING
> SUM(PRICE)>10) limit 5"{color}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 4, Size: 1
> at java.util.ArrayList.rangeCheck(ArrayList.java:657)
> at java.util.ArrayList.get(ArrayList.java:433)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.checkHavingCanPushDown(GTCubeStorageQueryBase.java:552)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.getStorageQueryRequest(GTCubeStorageQueryBase.java:189)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.search(GTCubeStorageQueryBase.java:89)
> at
> org.apache.kylin.query.enumerator.OLAPEnumerator.queryStorage(OLAPEnumerator.java:117)
> at
> org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:60)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4855) kylin metrics prefix bug in system-cube.sh
[ https://issues.apache.org/jira/browse/KYLIN-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288168#comment-17288168 ] ASF subversion and git services commented on KYLIN-4855: Commit d1c99d754e8d6fe1fe820fe75a89dae1d18c2dc9 in kylin's branch refs/heads/master from benjobs [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d1c99d7 ] KYLIN-4855 kylin.metrics.prefix bug fix (#1536) * KYLIN-4855 kylin.metrics.prefix bug fix * KYLIN-4855 kylin.metrics.prefix bug fix > kylin metrics prefix bug in system-cube.sh > -- > > Key: KYLIN-4855 > URL: https://issues.apache.org/jira/browse/KYLIN-4855 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.6.0 > Environment: 2.6.0+ >Reporter: Huajie Wang >Priority: Minor > Labels: easyfix > Fix For: v3.1.2 > > Original Estimate: 1h > Remaining Estimate: 1h > > After executed "system-cube.sh cron", the system cube will be added to the > crontab, but the cube name is wrong, and it does not match the actual > situation. After testing, it is found that the system-cube is > There is a bug in the handling of kylin metrics prefix > In the system-cube.sh script, the prefix is "KYLIN". The logic is wrong. You > should get "kylin.metrics.prefix" from kylin.properties. If this item is set, > take the value set by the user, if not set, go to the default value. "KYLIN" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4855) kylin metrics prefix bug in system-cube.sh
[ https://issues.apache.org/jira/browse/KYLIN-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288165#comment-17288165 ] ASF subversion and git services commented on KYLIN-4855: Commit d1c99d754e8d6fe1fe820fe75a89dae1d18c2dc9 in kylin's branch refs/heads/master from benjobs [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d1c99d7 ] KYLIN-4855 kylin.metrics.prefix bug fix (#1536) * KYLIN-4855 kylin.metrics.prefix bug fix * KYLIN-4855 kylin.metrics.prefix bug fix > kylin metrics prefix bug in system-cube.sh > -- > > Key: KYLIN-4855 > URL: https://issues.apache.org/jira/browse/KYLIN-4855 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.6.0 > Environment: 2.6.0+ >Reporter: Huajie Wang >Priority: Minor > Labels: easyfix > Fix For: v3.1.2 > > Original Estimate: 1h > Remaining Estimate: 1h > > After executed "system-cube.sh cron", the system cube will be added to the > crontab, but the cube name is wrong, and it does not match the actual > situation. After testing, it is found that the system-cube is > There is a bug in the handling of kylin metrics prefix > In the system-cube.sh script, the prefix is "KYLIN". The logic is wrong. You > should get "kylin.metrics.prefix" from kylin.properties. If this item is set, > take the value set by the user, if not set, go to the default value. "KYLIN" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4855) kylin metrics prefix bug in system-cube.sh
[ https://issues.apache.org/jira/browse/KYLIN-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288167#comment-17288167 ] ASF subversion and git services commented on KYLIN-4855: Commit d1c99d754e8d6fe1fe820fe75a89dae1d18c2dc9 in kylin's branch refs/heads/master from benjobs [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d1c99d7 ] KYLIN-4855 kylin.metrics.prefix bug fix (#1536) * KYLIN-4855 kylin.metrics.prefix bug fix * KYLIN-4855 kylin.metrics.prefix bug fix > kylin metrics prefix bug in system-cube.sh > -- > > Key: KYLIN-4855 > URL: https://issues.apache.org/jira/browse/KYLIN-4855 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.6.0 > Environment: 2.6.0+ >Reporter: Huajie Wang >Priority: Minor > Labels: easyfix > Fix For: v3.1.2 > > Original Estimate: 1h > Remaining Estimate: 1h > > After executed "system-cube.sh cron", the system cube will be added to the > crontab, but the cube name is wrong, and it does not match the actual > situation. After testing, it is found that the system-cube is > There is a bug in the handling of kylin metrics prefix > In the system-cube.sh script, the prefix is "KYLIN". The logic is wrong. You > should get "kylin.metrics.prefix" from kylin.properties. If this item is set, > take the value set by the user, if not set, go to the default value. "KYLIN" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4855) kylin metrics prefix bug in system-cube.sh
[jira] [Commented] (KYLIN-4855) kylin metrics prefix bug in system-cube.sh
[ https://issues.apache.org/jira/browse/KYLIN-4855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288164#comment-17288164 ] ASF GitHub Bot commented on KYLIN-4855: --- hit-lacus commented on pull request #1536: URL: https://github.com/apache/kylin/pull/1536#issuecomment-783071773 Thanks @wolfboys for contributions, thanks @zhangayqian for verification. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > kylin metrics prefix bug in system-cube.sh > -- > > Key: KYLIN-4855 > URL: https://issues.apache.org/jira/browse/KYLIN-4855 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.6.0 > Environment: 2.6.0+ >Reporter: Huajie Wang >Priority: Minor > Labels: easyfix > Fix For: v3.1.2 > > Original Estimate: 1h > Remaining Estimate: 1h > > After executed "system-cube.sh cron", the system cube will be added to the > crontab, but the cube name is wrong, and it does not match the actual > situation. After testing, it is found that the system-cube is > There is a bug in the handling of kylin metrics prefix > In the system-cube.sh script, the prefix is "KYLIN". The logic is wrong. You > should get "kylin.metrics.prefix" from kylin.properties. If this item is set, > take the value set by the user, if not set, go to the default value. "KYLIN" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] hit-lacus commented on pull request #1536: KYLIN-4855 kylin.metrics.prefix bug fix
hit-lacus commented on pull request #1536: URL: https://github.com/apache/kylin/pull/1536#issuecomment-783071773 Thanks @wolfboys for contributions, thanks @zhangayqian for verification. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] hit-lacus merged pull request #1536: KYLIN-4855 kylin.metrics.prefix bug fix
hit-lacus merged pull request #1536: URL: https://github.com/apache/kylin/pull/1536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KYLIN-4908) Segment pruner support integer partition col in spark query engine
ShengJun Zheng created KYLIN-4908: - Summary: Segment pruner support integer partition col in spark query engine Key: KYLIN-4908 URL: https://issues.apache.org/jira/browse/KYLIN-4908 Project: Kylin Issue Type: Improvement Reporter: ShengJun Zheng It's allowed to use int/bigint partition column from hive table to divide KYLIN's segments, but segment pruner doesn't support prune segments based on integer-type partition. -- This message was sent by Atlassian Jira (v8.3.4#803005)
