[GitHub] [kylin] zhoukangcn opened a new pull request #1033: KYLIN-4322: set storage.hbase.endpoint-compress-result default value …
zhoukangcn opened a new pull request #1033: KYLIN-4322: set storage.hbase.endpoint-compress-result default value … URL: https://github.com/apache/kylin/pull/1033 …false see: https://issues.apache.org/jira/browse/KYLIN-4322 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4322) Cost–benefit of compression HBase result
[ https://issues.apache.org/jira/browse/KYLIN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005968#comment-17005968 ] ASF GitHub Bot commented on KYLIN-4322: --- zhoukangcn commented on pull request #1033: KYLIN-4322: set storage.hbase.endpoint-compress-result default value … URL: https://github.com/apache/kylin/pull/1033 …false see: https://issues.apache.org/jira/browse/KYLIN-4322 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cost–benefit of compression HBase result > > > Key: KYLIN-4322 > URL: https://issues.apache.org/jira/browse/KYLIN-4322 > Project: Kylin > Issue Type: Bug >Reporter: ZhouKang >Priority: Major > > kylin.storage.hbase.endpoint-compress-result is TRUE as default. > In our production environment, when the hbase scan result is larger than > 200M, it will take more than 10s to compress data. > We can find this by hbase's log: > ||Size||avg rate||max rate||avg time||max time|| > |<1M|0.12|0.25|0.18ms|0.7s| > |1M ~ 10M|0.39|0.97|0.2s|0.6s| > |10M ~ 100M|0.47|0.81|2s|6.3s| > |>100M|0.95|0.96|15.7s|24.8s| > rate: compressed data size / origin data size > AND please NOTICE that, > when the source data size is less than 1M, 65% compression data is larger > than source data. > When source data is less then 10M, the latency of data transmission is > acceptability. When data is larger then 100M, it will take a long time to > compress data. > > So, I think kylin.storage.hbase.endpoint-compress-result should be FALSE by > default; > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4322) Cost–benefit of compression HBase result
[ https://issues.apache.org/jira/browse/KYLIN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhouKang updated KYLIN-4322: Summary: Cost–benefit of compression HBase result (was: ROA of compression HBase result) > Cost–benefit of compression HBase result > > > Key: KYLIN-4322 > URL: https://issues.apache.org/jira/browse/KYLIN-4322 > Project: Kylin > Issue Type: Bug >Reporter: ZhouKang >Priority: Major > > kylin.storage.hbase.endpoint-compress-result is TRUE as default. > In our production environment, when the hbase scan result is larger than > 200M, it will take more than 10s to compress data. > We can find this by hbase's log: > ||Size||avg rate||max rate||avg time||max time|| > |<1M|0.12|0.25|0.18ms|0.7s| > |1M ~ 10M|0.39|0.97|0.2s|0.6s| > |10M ~ 100M|0.47|0.81|2s|6.3s| > |>100M|0.95|0.96|15.7s|24.8s| > rate: compressed data size / origin data size > AND please NOTICE that, > when the source data size is less than 1M, 65% compression data is larger > than source data. > When source data is less then 10M, the latency of data transmission is > acceptability. When data is larger then 100M, it will take a long time to > compress data. > > So, I think kylin.storage.hbase.endpoint-compress-result should be FALSE by > default; > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4322) ROA of compression HBase result
[ https://issues.apache.org/jira/browse/KYLIN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhouKang updated KYLIN-4322: Summary: ROA of compression HBase result (was: ROC of compression HBase result) > ROA of compression HBase result > --- > > Key: KYLIN-4322 > URL: https://issues.apache.org/jira/browse/KYLIN-4322 > Project: Kylin > Issue Type: Bug >Reporter: ZhouKang >Priority: Major > > kylin.storage.hbase.endpoint-compress-result is TRUE as default. > In our production environment, when the hbase scan result is larger than > 200M, it will take more than 10s to compress data. > We can find this by hbase's log: > ||Size||avg rate||max rate||avg time||max time|| > |<1M|0.12|0.25|0.18ms|0.7s| > |1M ~ 10M|0.39|0.97|0.2s|0.6s| > |10M ~ 100M|0.47|0.81|2s|6.3s| > |>100M|0.95|0.96|15.7s|24.8s| > rate: compressed data size / origin data size > AND please NOTICE that, > when the source data size is less than 1M, 65% compression data is larger > than source data. > When source data is less then 10M, the latency of data transmission is > acceptability. When data is larger then 100M, it will take a long time to > compress data. > > So, I think kylin.storage.hbase.endpoint-compress-result should be FALSE by > default; > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4322) ROC of compression HBase result
[ https://issues.apache.org/jira/browse/KYLIN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhouKang updated KYLIN-4322: Description: kylin.storage.hbase.endpoint-compress-result is TRUE as default. In our production environment, when the hbase scan result is larger than 200M, it will take more than 10s to compress data. We can find this by hbase's log: ||Size||avg rate||max rate||avg time||max time|| |<1M|0.12|0.25|0.18ms|0.7s| |1M ~ 10M|0.39|0.97|0.2s|0.6s| |10M ~ 100M|0.47|0.81|2s|6.3s| |>100M|0.95|0.96|15.7s|24.8s| rate: compressed data size / origin data size AND please NOTICE that, when the source data size is less than 1M, 65% compression data is larger than source data. When source data is less then 10M, the latency of data transmission is acceptability. When data is larger then 100M, it will take a long time to compress data. So, I think kylin.storage.hbase.endpoint-compress-result should be FALSE by default; > ROC of compression HBase result > --- > > Key: KYLIN-4322 > URL: https://issues.apache.org/jira/browse/KYLIN-4322 > Project: Kylin > Issue Type: Bug >Reporter: ZhouKang >Priority: Major > > kylin.storage.hbase.endpoint-compress-result is TRUE as default. > In our production environment, when the hbase scan result is larger than > 200M, it will take more than 10s to compress data. > We can find this by hbase's log: > ||Size||avg rate||max rate||avg time||max time|| > |<1M|0.12|0.25|0.18ms|0.7s| > |1M ~ 10M|0.39|0.97|0.2s|0.6s| > |10M ~ 100M|0.47|0.81|2s|6.3s| > |>100M|0.95|0.96|15.7s|24.8s| > rate: compressed data size / origin data size > AND please NOTICE that, > when the source data size is less than 1M, 65% compression data is larger > than source data. > When source data is less then 10M, the latency of data transmission is > acceptability. When data is larger then 100M, it will take a long time to > compress data. > > So, I think kylin.storage.hbase.endpoint-compress-result should be FALSE by > default; > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4321: Fix Version/s: v3.1.0 > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4321. - Resolution: Fixed > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4080) Project schema update event casues error reload NEW DataModelDesc
[ https://issues.apache.org/jira/browse/KYLIN-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005945#comment-17005945 ] ASF GitHub Bot commented on KYLIN-4080: --- nichunen commented on pull request #976: KYLIN-4080 Project schema update event casues error reload NEW DataModelDesc URL: https://github.com/apache/kylin/pull/976 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Project schema update event casues error reload NEW DataModelDesc > - > > Key: KYLIN-4080 > URL: https://issues.apache.org/jira/browse/KYLIN-4080 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Assignee: Yuzhang QIU >Priority: Blocker > > Hi, dear Kylin dev team: >When create new DataModelDesc, DataModelManager.createDataModelDese:246 > will temporarily add the new model name into selected project(project1) > cache, but won't persist it. The TEMPORARY ADD operation will make the model > reloading successful, rather than throw "No project found for model ..." > exception(at ProjectManager:391). >However, If there have another threads are processing "Broadcasting > update project_schema, project1", it will clean up cache of project1 and > reload it, which will reset the "TEMPORARY ADD" operation. Meanwhile, the > model saving thread has persisted the DataModelDesc and start to reload it, > but will find there have "No project for this model". > The new model can't be created again because the conflict timestamp and > can't be reloaded into cache because the abrove problem. >How do you think about this?? > >Best regards > >yuzhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4080) Project schema update event casues error reload NEW DataModelDesc
[ https://issues.apache.org/jira/browse/KYLIN-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005946#comment-17005946 ] ASF subversion and git services commented on KYLIN-4080: Commit 34a21fa4976213460bf875bc96dcc0f500fe2bb5 in kylin's branch refs/heads/master from yaqian.zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=34a21fa ] KYLIN-4080 Project schema update event casues error reload NEW DataModelDesc > Project schema update event casues error reload NEW DataModelDesc > - > > Key: KYLIN-4080 > URL: https://issues.apache.org/jira/browse/KYLIN-4080 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Assignee: Yuzhang QIU >Priority: Blocker > > Hi, dear Kylin dev team: >When create new DataModelDesc, DataModelManager.createDataModelDese:246 > will temporarily add the new model name into selected project(project1) > cache, but won't persist it. The TEMPORARY ADD operation will make the model > reloading successful, rather than throw "No project found for model ..." > exception(at ProjectManager:391). >However, If there have another threads are processing "Broadcasting > update project_schema, project1", it will clean up cache of project1 and > reload it, which will reset the "TEMPORARY ADD" operation. Meanwhile, the > model saving thread has persisted the DataModelDesc and start to reload it, > but will find there have "No project for this model". > The new model can't be created again because the conflict timestamp and > can't be reloaded into cache because the abrove problem. >How do you think about this?? > >Best regards > >yuzhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #1032: Fix CI test
nichunen merged pull request #1032: Fix CI test URL: https://github.com/apache/kylin/pull/1032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #976: KYLIN-4080 Project schema update event casues error reload NEW DataModelDesc
nichunen merged pull request #976: KYLIN-4080 Project schema update event casues error reload NEW DataModelDesc URL: https://github.com/apache/kylin/pull/976 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #1032: Fix CI test
coveralls commented on issue #1032: Fix CI test URL: https://github.com/apache/kylin/pull/1032#issuecomment-569871682 ## Pull Request Test Coverage Report for [Build 5467](https://coveralls.io/builds/27856103) * **0** of **1** **(0.0%)** changed or added relevant line in **1** file are covered. * **7** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.002%**) to **27.471%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [query/src/main/java/org/apache/kylin/query/util/PushDownExecutor.java](https://coveralls.io/builds/27856103/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Futil%2FPushDownExecutor.java#L56) | 0 | 1 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/27856103/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/27856103/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/27856103) | | :-- | --: | | Change from base [Build 5466](https://coveralls.io/builds/27855903): | -0.002% | | Covered Lines: | 24070 | | Relevant Lines: | 87620 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #1032: Fix CI test
codecov-io commented on issue #1032: Fix CI test URL: https://github.com/apache/kylin/pull/1032#issuecomment-569871616 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=h1) Report > Merging [#1032](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/b7183e04f1cf8afe16851c6e026d2eaff5a2127a?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1032 +/- ## - Coverage 25.01% 25.01% -0.01% - Complexity 6179 6180 +1 Files 1437 1437 Lines 8762087620 Branches 1224812248 - Hits 2191921917 -2 - Misses6355163553 +2 Partials 2150 2150 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../org/apache/kylin/query/util/PushDownExecutor.java](https://codecov.io/gh/apache/kylin/pull/1032/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3V0aWwvUHVzaERvd25FeGVjdXRvci5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1032/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1032/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.31%)` | `7% <0%> (ø)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1032/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=footer). Last update [b7183e0...73fc90e](https://codecov.io/gh/apache/kylin/pull/1032?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4322) ROC of compression HBase result
[ https://issues.apache.org/jira/browse/KYLIN-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhouKang updated KYLIN-4322: Summary: ROC of compression HBase result (was: HBase) > ROC of compression HBase result > --- > > Key: KYLIN-4322 > URL: https://issues.apache.org/jira/browse/KYLIN-4322 > Project: Kylin > Issue Type: Bug >Reporter: ZhouKang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4322) HBase
ZhouKang created KYLIN-4322: --- Summary: HBase Key: KYLIN-4322 URL: https://issues.apache.org/jira/browse/KYLIN-4322 Project: Kylin Issue Type: Bug Reporter: ZhouKang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] zhangayqian opened a new pull request #1032: Fix CI test
zhangayqian opened a new pull request #1032: Fix CI test URL: https://github.com/apache/kylin/pull/1032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005911#comment-17005911 ] ASF GitHub Bot commented on KYLIN-4321: --- nichunen commented on pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark URL: https://github.com/apache/kylin/pull/1031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005912#comment-17005912 ] ASF subversion and git services commented on KYLIN-4321: Commit b7183e04f1cf8afe16851c6e026d2eaff5a2127a in kylin's branch refs/heads/master from weibin0516 [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b7183e0 ] KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark
nichunen merged pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark URL: https://github.com/apache/kylin/pull/1031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4104) Support multi jdbc pushdown runners to execute query/update
[ https://issues.apache.org/jira/browse/KYLIN-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005896#comment-17005896 ] weibin0516 commented on KYLIN-4104: --- Hi, [~shaofengshi], please see previous discussions http://apache-kylin.74782.x6.nabble.com/DISCUSS-Support-multiple-pushdown-query-engines-td13454.html > Support multi jdbc pushdown runners to execute query/update > --- > > Key: KYLIN-4104 > URL: https://issues.apache.org/jira/browse/KYLIN-4104 > Project: Kylin > Issue Type: New Feature >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > > Current (version 3.0.0-SNAPSHOT), kylin support only one kind of pushdown > query engine. In some user's scenario, need pushdown query to mysql, spark > sql，hive etc. > I think kylin need support multiple pushdowns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4104) Support multi jdbc pushdown runners to execute query/update
[ https://issues.apache.org/jira/browse/KYLIN-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005887#comment-17005887 ] Shao Feng Shi commented on KYLIN-4104: -- Hi Weibin, I'm not quite understand this feature. Usually one table is from one specific data source, there is no need to pushdown to multiple engines. The engines which doesn't have this table will report error, that is confusing. > Support multi jdbc pushdown runners to execute query/update > --- > > Key: KYLIN-4104 > URL: https://issues.apache.org/jira/browse/KYLIN-4104 > Project: Kylin > Issue Type: New Feature >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > > Current (version 3.0.0-SNAPSHOT), kylin support only one kind of pushdown > query engine. In some user's scenario, need pushdown query to mysql, spark > sql，hive etc. > I think kylin need support multiple pushdowns. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005397#comment-17005397 ] ASF GitHub Bot commented on KYLIN-4321: --- weibin0516 commented on pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark URL: https://github.com/apache/kylin/pull/1031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] weibin0516 opened a new pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark
weibin0516 opened a new pull request #1031: KYLIN-4321 Create fact distinct columns using spark by default when build engine is spark URL: https://github.com/apache/kylin/pull/1031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4321) Create fact distinct columns using spark by default when build engine is spark
[ https://issues.apache.org/jira/browse/KYLIN-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weibin0516 updated KYLIN-4321: -- Summary: Create fact distinct columns using spark by default when build engine is spark (was: Create fact distinct columns by spark when build engine is spark) > Create fact distinct columns using spark by default when build engine is spark > -- > > Key: KYLIN-4321 > URL: https://issues.apache.org/jira/browse/KYLIN-4321 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4321) Create fact distinct columns by spark when build engine is spark
weibin0516 created KYLIN-4321: - Summary: Create fact distinct columns by spark when build engine is spark Key: KYLIN-4321 URL: https://issues.apache.org/jira/browse/KYLIN-4321 Project: Kylin Issue Type: Improvement Reporter: weibin0516 Assignee: weibin0516 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] codecov-io commented on issue #1030: KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only …
codecov-io commented on issue #1030: KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only … URL: https://github.com/apache/kylin/pull/1030#issuecomment-569673413 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@9288f58`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=tree) ```diff @@Coverage Diff@@ ## master#1030 +/- ## = Coverage ? 25.02% Complexity? 6178 = Files ? 1437 Lines ?87621 Branches ?12248 = Hits ?21923 Misses?63549 Partials ? 2149 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ne/mr/steps/ExtractDictionaryFromGlobalMapper.java](https://codecov.io/gh/apache/kylin/pull/1030/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvRXh0cmFjdERpY3Rpb25hcnlGcm9tR2xvYmFsTWFwcGVyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1030/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `49.34% <0%> (ø)` | `64 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=footer). Last update [9288f58...5b065cc](https://codecov.io/gh/apache/kylin/pull/1030?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #1026: delete /execute_output metadata at the same time when deleting a job
coveralls edited a comment on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569659337 ## Pull Request Test Coverage Report for [Build 5462](https://coveralls.io/builds/27844295) * **0** of **14** **(0.0%)** changed or added relevant lines in **1** file are covered. * **511** unchanged lines in **11** files lost coverage. * Overall coverage decreased (**-0.05%**) to **27.468%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java](https://coveralls.io/builds/27844295/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fdao%2FExecutableDao.java#L381) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarStoreMetricsDesc.java](https://coveralls.io/builds/27844295/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FColumnarStoreMetricsDesc.java#L80) | 3 | 69.57% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/FragmentFilesMerger.java](https://coveralls.io/builds/27844295/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FFragmentFilesMerger.java#L341) | 4 | 97.73% | | [query/src/main/java/org/apache/kylin/query/adhoc/JdbcPushDownConnectionManager.java](https://coveralls.io/builds/27844295/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Fadhoc%2FJdbcPushDownConnectionManager.java#L38) | 8 | 0.0% | | [query/src/main/java/org/apache/kylin/query/util/PushDownUtil.java](https://coveralls.io/builds/27844295/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Futil%2FPushDownUtil.java#L66) | 13 | 75.68% | | [server-base/src/main/java/org/apache/kylin/rest/controller/ServiceDiscoveryStateController.java](https://coveralls.io/builds/27844295/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FServiceDiscoveryStateController.java#L66) | 25 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/streaming/RowRecordReader.java](https://coveralls.io/builds/27844295/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fstreaming%2FRowRecordReader.java#L131) | 38 | 0.0% | | [source-hive/src/main/java/org/apache/kylin/source/hive/BeelineHiveClient.java](https://coveralls.io/builds/27844295/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FBeelineHiveClient.java#L85) | 50 | 5.13% | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java](https://coveralls.io/builds/27844295/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkExecutable.java#L255) | 59 | 0.0% | | [query/src/main/java/org/apache/kylin/query/adhoc/PushDownRunnerJdbcImpl.java](https://coveralls.io/builds/27844295/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Fadhoc%2FPushDownRunnerJdbcImpl.java#L79) | 81 | 0.0% | | [core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java](https://coveralls.io/builds/27844295/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2FKylinConfigBase.java#L1852) | 108 | 13.6% | | Totals | [](https://coveralls.io/builds/27844295) | | :-- | --: | | Change from base [Build 5457](https://coveralls.io/builds/27821714): | -0.05% | | Covered Lines: | 24071 | | Relevant Lines: | 87634 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #1026: delete /execute_output metadata at the same time when deleting a job
codecov-io edited a comment on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569658483 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=h1) Report > Merging [#1026](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/6e4e456f701f1bdc5d5be7e0f738dae65215a782?src=pr=desc) will **decrease** coverage by `0.1%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #1026 +/- ## === - Coverage 25.11% 25% -0.11% Complexity 61786178 === Files 14331437 +4 Lines 87270 87634 +364 Branches 12218 12250 +32 === Hits 21917 21917 - Misses63205 63566 +361 - Partials 21482151 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...n/java/org/apache/kylin/job/dao/ExecutableDao.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9kYW8vRXhlY3V0YWJsZURhby5qYXZh) | `27.51% <0%> (-1.8%)` | `24 <0> (ø)` | | | [...core/storage/columnar/GeneralColumnDataReader.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvR2VuZXJhbENvbHVtbkRhdGFSZWFkZXIuamF2YQ==) | `89.47% <0%> (-5.27%)` | `6% <0%> (-1%)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [.../java/org/apache/kylin/common/util/HadoopUtil.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0hhZG9vcFV0aWwuamF2YQ==) | `14.56% <0%> (-1.23%)` | `10% <0%> (ø)` | | | [...va/org/apache/kylin/source/hive/HiveInputBase.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0hpdmVJbnB1dEJhc2UuamF2YQ==) | `6.69% <0%> (-1.07%)` | `4% <0%> (ø)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | | | [...eam/core/storage/columnar/FragmentFilesMerger.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvRnJhZ21lbnRGaWxlc01lcmdlci5qYXZh) | `96.02% <0%> (-0.53%)` | `30% <0%> (ø)` | | | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.36% <0%> (-0.5%)` | `44% <0%> (ø)` | | | [...rg/apache/kylin/source/hive/BeelineHiveClient.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0JlZWxpbmVIaXZlQ2xpZW50LmphdmE=) | `3.84% <0%> (-0.03%)` | `5% <0%> (ø)` | | | [...che/kylin/engine/mr/streaming/RowRecordReader.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RyZWFtaW5nL1Jvd1JlY29yZFJlYWRlci5qYXZh) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | ... and [15 more](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=footer). Last update [6e4e456...48365dd](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[jira] [Commented] (KYLIN-4319) in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries.
[ https://issues.apache.org/jira/browse/KYLIN-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005294#comment-17005294 ] ASF GitHub Bot commented on KYLIN-4319: --- YangZhen007 commented on pull request #1030: KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only … URL: https://github.com/apache/kylin/pull/1030 KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > in "Extract Dictionary from Global Dictionary" step, only need load global > dictonaries instead of all dictonaries. > -- > > Key: KYLIN-4319 > URL: https://issues.apache.org/jira/browse/KYLIN-4319 > Project: Kylin > Issue Type: Improvement >Reporter: YangZhen007 >Priority: Minor > > in "Extract Dictionary from Global Dictionary" step, only need load global > dictonaries instead of all dictonaries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] YangZhen007 opened a new pull request #1030: KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only …
YangZhen007 opened a new pull request #1030: KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only … URL: https://github.com/apache/kylin/pull/1030 KYLIN-4319 in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
coveralls commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569659337 ## Pull Request Test Coverage Report for [Build 5461](https://coveralls.io/builds/27843836) * **0** of **14** **(0.0%)** changed or added relevant lines in **1** file are covered. * **516** unchanged lines in **12** files lost coverage. * Overall coverage decreased (**-0.05%**) to **27.465%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-job/src/main/java/org/apache/kylin/job/dao/ExecutableDao.java](https://coveralls.io/builds/27843836/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fdao%2FExecutableDao.java#L383) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarStoreMetricsDesc.java](https://coveralls.io/builds/27843836/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FColumnarStoreMetricsDesc.java#L80) | 3 | 69.57% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/FragmentFilesMerger.java](https://coveralls.io/builds/27843836/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FFragmentFilesMerger.java#L341) | 4 | 97.73% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/27843836/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | [query/src/main/java/org/apache/kylin/query/adhoc/JdbcPushDownConnectionManager.java](https://coveralls.io/builds/27843836/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Fadhoc%2FJdbcPushDownConnectionManager.java#L38) | 8 | 0.0% | | [query/src/main/java/org/apache/kylin/query/util/PushDownUtil.java](https://coveralls.io/builds/27843836/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Futil%2FPushDownUtil.java#L66) | 13 | 75.68% | | [server-base/src/main/java/org/apache/kylin/rest/controller/ServiceDiscoveryStateController.java](https://coveralls.io/builds/27843836/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FServiceDiscoveryStateController.java#L66) | 25 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/streaming/RowRecordReader.java](https://coveralls.io/builds/27843836/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fstreaming%2FRowRecordReader.java#L131) | 38 | 0.0% | | [source-hive/src/main/java/org/apache/kylin/source/hive/BeelineHiveClient.java](https://coveralls.io/builds/27843836/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FBeelineHiveClient.java#L85) | 50 | 5.13% | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java](https://coveralls.io/builds/27843836/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkExecutable.java#L255) | 59 | 0.0% | | [query/src/main/java/org/apache/kylin/query/adhoc/PushDownRunnerJdbcImpl.java](https://coveralls.io/builds/27843836/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Fadhoc%2FPushDownRunnerJdbcImpl.java#L79) | 81 | 0.0% | | Totals | [](https://coveralls.io/builds/27843836) | | :-- | --: | | Change from base [Build 5457](https://coveralls.io/builds/27821714): | -0.05% | | Covered Lines: | 24069 | | Relevant Lines: | 87634 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
codecov-io commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569658483 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=h1) Report > Merging [#1026](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/4c8380549e10befd5cfb266eff6db9973582061f?src=pr=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #1026 +/- ## === - Coverage 25.05% 25% -0.05% + Complexity 61796178 -1 === Files 14341437 +3 Lines 87457 87634 +177 Branches 12234 12250 +16 === + Hits 21913 21915 +2 - Misses63396 63568 +172 - Partials 21482151 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...n/java/org/apache/kylin/job/dao/ExecutableDao.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9kYW8vRXhlY3V0YWJsZURhby5qYXZh) | `27.51% <0%> (-1.8%)` | `24 <0> (ø)` | | | [...va/org/apache/kylin/source/hive/HiveInputBase.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0hpdmVJbnB1dEJhc2UuamF2YQ==) | `6.69% <0%> (-1.07%)` | `4% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | | | [...eam/core/storage/columnar/FragmentFilesMerger.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvRnJhZ21lbnRGaWxlc01lcmdlci5qYXZh) | `96.02% <0%> (-0.53%)` | `30% <0%> (ø)` | | | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.36% <0%> (-0.48%)` | `44% <0%> (ø)` | | | [...rg/apache/kylin/source/hive/BeelineHiveClient.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0JlZWxpbmVIaXZlQ2xpZW50LmphdmE=) | `3.84% <0%> (-0.03%)` | `5% <0%> (ø)` | | | [...che/kylin/engine/mr/streaming/RowRecordReader.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RyZWFtaW5nL1Jvd1JlY29yZFJlYWRlci5qYXZh) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | [...ache/kylin/query/adhoc/PushDownRunnerJdbcImpl.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L2FkaG9jL1B1c2hEb3duUnVubmVySmRiY0ltcGwuamF2YQ==) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | [...st/controller/ServiceDiscoveryStateController.java](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TZXJ2aWNlRGlzY292ZXJ5U3RhdGVDb250cm9sbGVyLmphdmE=) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | ... and [8 more](https://codecov.io/gh/apache/kylin/pull/1026/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=footer). Last update [4c83805...7d26445](https://codecov.io/gh/apache/kylin/pull/1026?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[GitHub] [kylin] ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569651906 Or delete output separately? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #1028: KYLIN-4271
nichunen merged pull request #1028: KYLIN-4271 URL: https://github.com/apache/kylin/pull/1028 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-4271) Use configurable certificate to support LDAPs authentication of Kylin
[
https://issues.apache.org/jira/browse/KYLIN-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4271.
-
Resolution: Fixed
> Use configurable certificate to support LDAPs authentication of Kylin
> -
>
> Key: KYLIN-4271
> URL: https://issues.apache.org/jira/browse/KYLIN-4271
> Project: Kylin
> Issue Type: Improvement
> Components: Security
>Affects Versions: v3.0.0
>Reporter: wu.kehua
>Assignee: wu.kehua
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: kylin.log
>
>
> Kylin's user authentication is normal when connecting to an LDAP server with
> the LDAP protocol enabled. However, the LDAP protocol is transmitted in plain
> text and there are security risks. Therefore, the LDAP server that uses the
> LDAPs protocol needs to be enabled. The LDAPs protocol supports encrypted
> transmission. After configuring the LDAP related configuration in
> kylin.properties, Kylin server cannot connect to the LDAP server for user
> authentication.
> The Kylin log shows the error log, as follows, you can also see the detail
> log in attachment.
> {code:java}
> Root exception is javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> {code}
> So we add "kylin.security.ldap.connection-truststore" parameter which is set
> to be value of "javax.net.ssl.trustStore", so we can use configurable
> certificate to support LDAPs authentication of Kylin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4271) Use configurable certificate to support LDAPs authentication of Kylin
[
https://issues.apache.org/jira/browse/KYLIN-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005261#comment-17005261
]
ASF subversion and git services commented on KYLIN-4271:
Commit 9288f58caece490c1f44c3a67b6a28fc98e0d2c8 in kylin's branch
refs/heads/master from wukehua
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=9288f58 ]
KYLIN-4271 Use configurable certificate to support LDAPs authentication of Kylin
> Use configurable certificate to support LDAPs authentication of Kylin
> -
>
> Key: KYLIN-4271
> URL: https://issues.apache.org/jira/browse/KYLIN-4271
> Project: Kylin
> Issue Type: Improvement
> Components: Security
>Affects Versions: v3.0.0
>Reporter: wu.kehua
>Assignee: wu.kehua
>Priority: Major
> Attachments: kylin.log
>
>
> Kylin's user authentication is normal when connecting to an LDAP server with
> the LDAP protocol enabled. However, the LDAP protocol is transmitted in plain
> text and there are security risks. Therefore, the LDAP server that uses the
> LDAPs protocol needs to be enabled. The LDAPs protocol supports encrypted
> transmission. After configuring the LDAP related configuration in
> kylin.properties, Kylin server cannot connect to the LDAP server for user
> authentication.
> The Kylin log shows the error log, as follows, you can also see the detail
> log in attachment.
> {code:java}
> Root exception is javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> {code}
> So we add "kylin.security.ldap.connection-truststore" parameter which is set
> to be value of "javax.net.ssl.trustStore", so we can use configurable
> certificate to support LDAPs authentication of Kylin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4271) Use configurable certificate to support LDAPs authentication of Kylin
[
https://issues.apache.org/jira/browse/KYLIN-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4271:
Fix Version/s: v3.1.0
> Use configurable certificate to support LDAPs authentication of Kylin
> -
>
> Key: KYLIN-4271
> URL: https://issues.apache.org/jira/browse/KYLIN-4271
> Project: Kylin
> Issue Type: Improvement
> Components: Security
>Affects Versions: v3.0.0
>Reporter: wu.kehua
>Assignee: wu.kehua
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: kylin.log
>
>
> Kylin's user authentication is normal when connecting to an LDAP server with
> the LDAP protocol enabled. However, the LDAP protocol is transmitted in plain
> text and there are security risks. Therefore, the LDAP server that uses the
> LDAPs protocol needs to be enabled. The LDAPs protocol supports encrypted
> transmission. After configuring the LDAP related configuration in
> kylin.properties, Kylin server cannot connect to the LDAP server for user
> authentication.
> The Kylin log shows the error log, as follows, you can also see the detail
> log in attachment.
> {code:java}
> Root exception is javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> {code}
> So we add "kylin.security.ldap.connection-truststore" parameter which is set
> to be value of "javax.net.ssl.trustStore", so we can use configurable
> certificate to support LDAPs authentication of Kylin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4271) Use configurable certificate to support LDAPs authentication of Kylin
[
https://issues.apache.org/jira/browse/KYLIN-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005260#comment-17005260
]
ASF GitHub Bot commented on KYLIN-4271:
---
nichunen commented on pull request #1028: KYLIN-4271
URL: https://github.com/apache/kylin/pull/1028
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Use configurable certificate to support LDAPs authentication of Kylin
> -
>
> Key: KYLIN-4271
> URL: https://issues.apache.org/jira/browse/KYLIN-4271
> Project: Kylin
> Issue Type: Improvement
> Components: Security
>Affects Versions: v3.0.0
>Reporter: wu.kehua
>Assignee: wu.kehua
>Priority: Major
> Attachments: kylin.log
>
>
> Kylin's user authentication is normal when connecting to an LDAP server with
> the LDAP protocol enabled. However, the LDAP protocol is transmitted in plain
> text and there are security risks. Therefore, the LDAP server that uses the
> LDAPs protocol needs to be enabled. The LDAPs protocol supports encrypted
> transmission. After configuring the LDAP related configuration in
> kylin.properties, Kylin server cannot connect to the LDAP server for user
> authentication.
> The Kylin log shows the error log, as follows, you can also see the detail
> log in attachment.
> {code:java}
> Root exception is javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> {code}
> So we add "kylin.security.ldap.connection-truststore" parameter which is set
> to be value of "javax.net.ssl.trustStore", so we can use configurable
> certificate to support LDAPs authentication of Kylin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4319) in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries.
[ https://issues.apache.org/jira/browse/KYLIN-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YangZhen007 updated KYLIN-4319: --- Priority: Minor (was: Major) > in "Extract Dictionary from Global Dictionary" step, only need load global > dictonaries instead of all dictonaries. > -- > > Key: KYLIN-4319 > URL: https://issues.apache.org/jira/browse/KYLIN-4319 > Project: Kylin > Issue Type: Improvement >Reporter: YangZhen007 >Priority: Minor > > in "Extract Dictionary from Global Dictionary" step, only need load global > dictonaries instead of all dictonaries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4302) Fix the bug that InputStream is not closed properly
[ https://issues.apache.org/jira/browse/KYLIN-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005254#comment-17005254 ] ASF subversion and git services commented on KYLIN-4302: Commit c3b14e58712f0a5b7967f4ea07a3cb6821ab8c43 in kylin's branch refs/heads/master from weibin0516 [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c3b14e5 ] KYLIN-4302 Fix the bug that InputStream is not closed properly > Fix the bug that InputStream is not closed properly > --- > > Key: KYLIN-4302 > URL: https://issues.apache.org/jira/browse/KYLIN-4302 > Project: Kylin > Issue Type: Bug >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4302) Fix the bug that InputStream is not closed properly
[ https://issues.apache.org/jira/browse/KYLIN-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4302. - Resolution: Fixed > Fix the bug that InputStream is not closed properly > --- > > Key: KYLIN-4302 > URL: https://issues.apache.org/jira/browse/KYLIN-4302 > Project: Kylin > Issue Type: Bug >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4302) Fix the bug that InputStream is not closed properly
[ https://issues.apache.org/jira/browse/KYLIN-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4302: Fix Version/s: v3.1.0 > Fix the bug that InputStream is not closed properly > --- > > Key: KYLIN-4302 > URL: https://issues.apache.org/jira/browse/KYLIN-4302 > Project: Kylin > Issue Type: Bug >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4302) Fix the bug that InputStream is not closed properly
[ https://issues.apache.org/jira/browse/KYLIN-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005253#comment-17005253 ] ASF GitHub Bot commented on KYLIN-4302: --- nichunen commented on pull request #1000: KYLIN-4302 Fix the bug that InputStream is not closed properly URL: https://github.com/apache/kylin/pull/1000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix the bug that InputStream is not closed properly > --- > > Key: KYLIN-4302 > URL: https://issues.apache.org/jira/browse/KYLIN-4302 > Project: Kylin > Issue Type: Bug >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #1000: KYLIN-4302 Fix the bug that InputStream is not closed properly
nichunen merged pull request #1000: KYLIN-4302 Fix the bug that InputStream is not closed properly URL: https://github.com/apache/kylin/pull/1000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
[
https://issues.apache.org/jira/browse/KYLIN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4295.
-
Resolution: Fixed
> Instances displayed on Query Node are inconsistent with Job Node
>
>
> Key: KYLIN-4295
> URL: https://issues.apache.org/jira/browse/KYLIN-4295
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.0.0-alpha2
>Reporter: Temple Zhou
>Assignee: Yaqian Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg
>
>
> As shown in attachments.
> BTW: It is a backend issue, as the results return by restful API are
> inconsistent.
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
[
https://issues.apache.org/jira/browse/KYLIN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005250#comment-17005250
]
ASF subversion and git services commented on KYLIN-4295:
Commit e28395c9b92d76735c293526c5b472140ed0a59f in kylin's branch
refs/heads/master from yaqian.zhang
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e28395c ]
KYLIN-4295 Instances displayed on Query Node are inconsistent with Job Node
> Instances displayed on Query Node are inconsistent with Job Node
>
>
> Key: KYLIN-4295
> URL: https://issues.apache.org/jira/browse/KYLIN-4295
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.0.0-alpha2
>Reporter: Temple Zhou
>Assignee: Yaqian Zhang
>Priority: Minor
> Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg
>
>
> As shown in attachments.
> BTW: It is a backend issue, as the results return by restful API are
> inconsistent.
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
[
https://issues.apache.org/jira/browse/KYLIN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005249#comment-17005249
]
ASF GitHub Bot commented on KYLIN-4295:
---
nichunen commented on pull request #1014: KYLIN-4295 Instances displayed on
Query Node are inconsistent with JobNode
URL: https://github.com/apache/kylin/pull/1014
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Instances displayed on Query Node are inconsistent with Job Node
>
>
> Key: KYLIN-4295
> URL: https://issues.apache.org/jira/browse/KYLIN-4295
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.0.0-alpha2
>Reporter: Temple Zhou
>Assignee: Yaqian Zhang
>Priority: Minor
> Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg
>
>
> As shown in attachments.
> BTW: It is a backend issue, as the results return by restful API are
> inconsistent.
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
[
https://issues.apache.org/jira/browse/KYLIN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4295:
Fix Version/s: v3.1.0
> Instances displayed on Query Node are inconsistent with Job Node
>
>
> Key: KYLIN-4295
> URL: https://issues.apache.org/jira/browse/KYLIN-4295
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.0.0-alpha2
>Reporter: Temple Zhou
>Assignee: Yaqian Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg
>
>
> As shown in attachments.
> BTW: It is a backend issue, as the results return by restful API are
> inconsistent.
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #1014: KYLIN-4295 Instances displayed on Query Node are inconsistent with JobNode
nichunen merged pull request #1014: KYLIN-4295 Instances displayed on Query Node are inconsistent with JobNode URL: https://github.com/apache/kylin/pull/1014 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569638030 How about adding a verification in the catch statement? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4243) read function in NoCompressedColumnReader is wrong.
[ https://issues.apache.org/jira/browse/KYLIN-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005244#comment-17005244 ] nichunen commented on KYLIN-4243: - [~zengrui] Thanks, pr merged > read function in NoCompressedColumnReader is wrong. > --- > > Key: KYLIN-4243 > URL: https://issues.apache.org/jira/browse/KYLIN-4243 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: zengrui >Priority: Minor > Fix For: v3.1.0 > > Attachments: fix_read_bug.patch > > > I found that the read function in NoCompressedColumnReader maybe wrong > because the parameter colValLength is the length of all this dimension > values, not the length of one dimension value. > Then I try to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4243) read function in NoCompressedColumnReader is wrong.
[ https://issues.apache.org/jira/browse/KYLIN-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4243. - Resolution: Fixed > read function in NoCompressedColumnReader is wrong. > --- > > Key: KYLIN-4243 > URL: https://issues.apache.org/jira/browse/KYLIN-4243 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: zengrui >Priority: Minor > Fix For: v3.1.0 > > Attachments: fix_read_bug.patch > > > I found that the read function in NoCompressedColumnReader maybe wrong > because the parameter colValLength is the length of all this dimension > values, not the length of one dimension value. > Then I try to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #999: fix bug in KYLIN-4243
nichunen merged pull request #999: fix bug in KYLIN-4243 URL: https://github.com/apache/kylin/pull/999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4243) read function in NoCompressedColumnReader is wrong.
[ https://issues.apache.org/jira/browse/KYLIN-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005243#comment-17005243 ] ASF subversion and git services commented on KYLIN-4243: Commit 581831d5844ccc29be659955e8c06f31a43c270b in kylin's branch refs/heads/master from zengruios [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=581831d ] fix bug in KYLIN-4243 > read function in NoCompressedColumnReader is wrong. > --- > > Key: KYLIN-4243 > URL: https://issues.apache.org/jira/browse/KYLIN-4243 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: zengrui >Priority: Minor > Fix For: v3.1.0 > > Attachments: fix_read_bug.patch > > > I found that the read function in NoCompressedColumnReader maybe wrong > because the parameter colValLength is the length of all this dimension > values, not the length of one dimension value. > Then I try to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4243) read function in NoCompressedColumnReader is wrong.
[ https://issues.apache.org/jira/browse/KYLIN-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4243: Fix Version/s: (was: Future) v3.1.0 > read function in NoCompressedColumnReader is wrong. > --- > > Key: KYLIN-4243 > URL: https://issues.apache.org/jira/browse/KYLIN-4243 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: zengrui >Priority: Minor > Fix For: v3.1.0 > > Attachments: fix_read_bug.patch > > > I found that the read function in NoCompressedColumnReader maybe wrong > because the parameter colValLength is the length of all this dimension > values, not the length of one dimension value. > Then I try to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
ChangjiGuo commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569637531 Hi @nichunen I agree with your doubt. Do you think it is necessary to delete /execute_output? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4243) read function in NoCompressedColumnReader is wrong.
[ https://issues.apache.org/jira/browse/KYLIN-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005242#comment-17005242 ] ASF GitHub Bot commented on KYLIN-4243: --- nichunen commented on pull request #999: fix bug in KYLIN-4243 URL: https://github.com/apache/kylin/pull/999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > read function in NoCompressedColumnReader is wrong. > --- > > Key: KYLIN-4243 > URL: https://issues.apache.org/jira/browse/KYLIN-4243 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: zengrui >Priority: Minor > Fix For: v3.1.0 > > Attachments: fix_read_bug.patch > > > I found that the read function in NoCompressedColumnReader maybe wrong > because the parameter colValLength is the length of all this dimension > values, not the length of one dimension value. > Then I try to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] RupengWang opened a new pull request #1029: spark build distinct value, dimension dic, uhc dic
RupengWang opened a new pull request #1029: spark build distinct value, dimension dic, uhc dic URL: https://github.com/apache/kylin/pull/1029 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
nichunen commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569616496 @ChangjiGuo Hi, I think your point of this PR is right, job's tasks' should also be deleted once the job is deleted (they will be deleted by MetadataCleanupJob). But I doubt the code change will also bring instability of the job deleting operation. Because before this change, dropping a job only needs one time of metadata deleting, but after this change, it needs n+1 times(n is the number of tasks). If one task failed to be deleted, an exception will be thrown which indicates the deletion is failed, but actually, the job's metadata has been deleted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asf-ci commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
asf-ci commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569616515 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asf-ci commented on issue #1026: delete /execute_output metadata at the same time when deleting a job
asf-ci commented on issue #1026: delete /execute_output metadata at the same time when deleting a job URL: https://github.com/apache/kylin/pull/1026#issuecomment-569616514 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #1028: KYLIN-4271
coveralls commented on issue #1028: KYLIN-4271 URL: https://github.com/apache/kylin/pull/1028#issuecomment-569613538 ## Pull Request Test Coverage Report for [Build 5460](https://coveralls.io/builds/27841404) * **0** of **0** changed or added relevant lines in **0** files are covered. * **10** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.007%**) to **27.467%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/27841404/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/27841404/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/27841404/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/27841404/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/27841404) | | :-- | --: | | Change from base [Build 5459](https://coveralls.io/builds/27840232): | -0.007% | | Covered Lines: | 24063 | | Relevant Lines: | 87608 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #1028: KYLIN-4271
codecov-io commented on issue #1028: KYLIN-4271 URL: https://github.com/apache/kylin/pull/1028#issuecomment-569612992 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=h1) Report > Merging [#1028](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/95fe225b70339ca67a29a4f1fc550346a8e04000?src=pr=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #1028 +/- ## === - Coverage 25.01% 25% -0.02% + Complexity 61796178 -1 === Files 14371437 Lines 87608 87608 Branches 12242 12242 === - Hits 21919 21910 -9 - Misses63542 63548 +6 - Partials 21472150 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...ain/java/org/apache/kylin/dict/TrieDictionary.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L1RyaWVEaWN0aW9uYXJ5LmphdmE=) | `72.55% <0%> (-0.47%)` | `56% <0%> (-1%)` | | | [...ream/core/storage/columnar/ColumnarStoreCache.java](https://codecov.io/gh/apache/kylin/pull/1028/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTdG9yZUNhY2hlLmphdmE=) | `52.27% <0%> (+2.27%)` | `8% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=footer). Last update [95fe225...c022cef](https://codecov.io/gh/apache/kylin/pull/1028?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3628) Query with lookup table always use latest snapshot
[ https://issues.apache.org/jira/browse/KYLIN-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi updated KYLIN-3628: - Fix Version/s: (was: v3.0.0-beta) > Query with lookup table always use latest snapshot > -- > > Key: KYLIN-3628 > URL: https://issues.apache.org/jira/browse/KYLIN-3628 > Project: Kylin > Issue Type: Improvement >Reporter: Na Zhai >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.5 > > > If user queries a lookup table, Kylin will randomly selects a Cube (which has > the snapshot of this lookup table) to answer it. This causes uncertainty when > there are multiple cubes (share the same lookup): some cubes are newly built, > some not. If Kylin picks an old cube, the query result is old. > To remove this uncertainty, for such queries, either always use latest > snapshot, or use earlist snapshot. We believe the "latest" version is better. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4320) number of replicas of Cuboid files cannot be configured for Spark engine
[
https://issues.apache.org/jira/browse/KYLIN-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Congling Xia updated KYLIN-4320:
Description:
Hi, team. I try to change `dfs.replication` to 3 by adding the following config
override
{code:java}
kylin.engine.spark-conf.spark.hadoop.dfs.replication=3
{code}
Then, I get a strange result - numbers of replicas of cuboid files varies even
though they are in the same level.
!cuboid_replications.png!
I guess it is due to the conflicting settings in SparkUtil:
{code:java}
public static void modifySparkHadoopConfiguration(SparkContext sc) throws
Exception {
sc.hadoopConfiguration().set("dfs.replication", "2"); // cuboid
intermediate files, replication=2
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress",
"true");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.type",
"BLOCK");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.codec",
"org.apache.hadoop.io.compress.DefaultCodec"); // or
org.apache.hadoop.io.compress.SnappyCodec
}
{code}
It may be a bug for Spark property precedence. After checking [Spark
documents|#dynamically-loading-spark-properties]], it seems that some
programmatically set properties may not take effect and it is not a recommended
way for Spark job configuration.
Anyway, cuboid files may survive for weeks until expired or been merged, the
configuration rewrite in
`org.apache.kylin.engine.spark.SparkUtil#modifySparkHadoopConfiguration` makes
those files less reliable.
Is there any way to force cuboid files to remain 3 replicas? or shall we remove
the code in SparkUtil to make
kylin.engine.spark-conf.spark.hadoop.dfs.replication work properly?
was:
Hi, team. I try to change `dfs.replication` to 3 by adding the following config
override
{code:java}
kylin.engine.spark-conf.spark.hadoop.dfs.replication=3
{code}
Then, I get a strange result - numbers of replicas of cuboid files varies even
though they are in the same level.
!cuboid_replications.png!
I guess it is due to the conflicting settings is SparkUtil:
{code:java}
public static void modifySparkHadoopConfiguration(SparkContext sc) throws
Exception {
sc.hadoopConfiguration().set("dfs.replication", "2"); // cuboid
intermediate files, replication=2
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress",
"true");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.type",
"BLOCK");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.codec",
"org.apache.hadoop.io.compress.DefaultCodec"); // or
org.apache.hadoop.io.compress.SnappyCodec
}
{code}
It may be a bug for Spark property precedence. After checking [Spark
documents|[http://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties]],
it seems that some programmatically set properties may not take effect and it
is not a recommended way for Spark job configuration.
Anyway, cuboid files may survive for weeks until expired or been merged, the
configuration rewrite in
`org.apache.kylin.engine.spark.SparkUtil#modifySparkHadoopConfiguration` makes
those files less reliable.
Is there any way to force cuboid files to remain 3 replicas? or shall we remove
the code in SparkUtil to make
kylin.engine.spark-conf.spark.hadoop.dfs.replication work properly?
> number of replicas of Cuboid files cannot be configured for Spark engine
>
>
> Key: KYLIN-4320
> URL: https://issues.apache.org/jira/browse/KYLIN-4320
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Reporter: Congling Xia
>Priority: Major
> Attachments: cuboid_replications.png
>
>
> Hi, team. I try to change `dfs.replication` to 3 by adding the following
> config override
> {code:java}
> kylin.engine.spark-conf.spark.hadoop.dfs.replication=3
> {code}
> Then, I get a strange result - numbers of replicas of cuboid files varies
> even though they are in the same level.
> !cuboid_replications.png!
> I guess it is due to the conflicting settings in SparkUtil:
> {code:java}
> public static void modifySparkHadoopConfiguration(SparkContext sc) throws
> Exception {
> sc.hadoopConfiguration().set("dfs.replication", "2"); // cuboid
> intermediate files, replication=2
>
> sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress",
> "true");
>
> sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.type",
> "BLOCK");
>
> sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.codec",
> "org.apache.hadoop.io.compress.DefaultCodec"); // or
> org.apache.hadoop.io.compress.SnappyCodec
> }
> {code}
> It may be a bug for Spark property precedence. After checking
[jira] [Created] (KYLIN-4320) number of replicas of Cuboid files cannot be configured for Spark engine
Congling Xia created KYLIN-4320:
---
Summary: number of replicas of Cuboid files cannot be configured
for Spark engine
Key: KYLIN-4320
URL: https://issues.apache.org/jira/browse/KYLIN-4320
Project: Kylin
Issue Type: Bug
Components: Job Engine
Reporter: Congling Xia
Attachments: cuboid_replications.png
Hi, team. I try to change `dfs.replication` to 3 by adding the following config
override
{code:java}
kylin.engine.spark-conf.spark.hadoop.dfs.replication=3
{code}
Then, I get a strange result - numbers of replicas of cuboid files varies even
though they are in the same level.
!cuboid_replications.png!
I guess it is due to the conflicting settings is SparkUtil:
{code:java}
public static void modifySparkHadoopConfiguration(SparkContext sc) throws
Exception {
sc.hadoopConfiguration().set("dfs.replication", "2"); // cuboid
intermediate files, replication=2
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress",
"true");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.type",
"BLOCK");
sc.hadoopConfiguration().set("mapreduce.output.fileoutputformat.compress.codec",
"org.apache.hadoop.io.compress.DefaultCodec"); // or
org.apache.hadoop.io.compress.SnappyCodec
}
{code}
It may be a bug for Spark property precedence. After checking [Spark
documents|[http://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties]],
it seems that some programmatically set properties may not take effect and it
is not a recommended way for Spark job configuration.
Anyway, cuboid files may survive for weeks until expired or been merged, the
configuration rewrite in
`org.apache.kylin.engine.spark.SparkUtil#modifySparkHadoopConfiguration` makes
those files less reliable.
Is there any way to force cuboid files to remain 3 replicas? or shall we remove
the code in SparkUtil to make
kylin.engine.spark-conf.spark.hadoop.dfs.replication work properly?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4319) in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries.
YangZhen007 created KYLIN-4319: -- Summary: in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries. Key: KYLIN-4319 URL: https://issues.apache.org/jira/browse/KYLIN-4319 Project: Kylin Issue Type: Improvement Reporter: YangZhen007 in "Extract Dictionary from Global Dictionary" step, only need load global dictonaries instead of all dictonaries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4271) Use configurable certificate to support LDAPs authentication of Kylin
[
https://issues.apache.org/jira/browse/KYLIN-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005186#comment-17005186
]
ASF GitHub Bot commented on KYLIN-4271:
---
wkh8011 commented on pull request #1028: KYLIN-4271
URL: https://github.com/apache/kylin/pull/1028
KYLIN-4271 Use configurable certificate to support LDAPs authentication of
Kylin
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Use configurable certificate to support LDAPs authentication of Kylin
> -
>
> Key: KYLIN-4271
> URL: https://issues.apache.org/jira/browse/KYLIN-4271
> Project: Kylin
> Issue Type: Improvement
> Components: Security
>Affects Versions: v3.0.0
>Reporter: wu.kehua
>Assignee: wu.kehua
>Priority: Major
> Attachments: kylin.log
>
>
> Kylin's user authentication is normal when connecting to an LDAP server with
> the LDAP protocol enabled. However, the LDAP protocol is transmitted in plain
> text and there are security risks. Therefore, the LDAP server that uses the
> LDAPs protocol needs to be enabled. The LDAPs protocol supports encrypted
> transmission. After configuring the LDAP related configuration in
> kylin.properties, Kylin server cannot connect to the LDAP server for user
> authentication.
> The Kylin log shows the error log, as follows, you can also see the detail
> log in attachment.
> {code:java}
> Root exception is javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> {code}
> So we add "kylin.security.ldap.connection-truststore" parameter which is set
> to be value of "javax.net.ssl.trustStore", so we can use configurable
> certificate to support LDAPs authentication of Kylin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] wkh8011 opened a new pull request #1028: KYLIN-4271
wkh8011 opened a new pull request #1028: KYLIN-4271 URL: https://github.com/apache/kylin/pull/1028 KYLIN-4271 Use configurable certificate to support LDAPs authentication of Kylin This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
