[GitHub] [kylin] codecov-commenter commented on pull request #1228: KYLIN-4508 Add more unit tests to improve test coverage
codecov-commenter commented on pull request #1228: URL: https://github.com/apache/kylin/pull/1228#issuecomment-632572024 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=h1) Report > Merging [#1228](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **increase** coverage by `1.10%`. > The diff coverage is `56.52%`. [](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1228 +/- ## + Coverage 24.85% 25.96% +1.10% - Complexity 6266 6520 +254 Files 1453 1453 Lines 8922589243 +18 Branches 1245612458 +2 + Hits 2218123173 +992 + Misses6486963810-1059 - Partials 2175 2260 +85 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...org/apache/kylin/metrics/lib/impl/RecordEvent.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL1JlY29yZEV2ZW50LmphdmE=) | `72.04% <0.00%> (+72.04%)` | `24.00 <0.00> (+24.00)` | | | [...pache/kylin/metrics/lib/impl/TimePropertyEnum.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL1RpbWVQcm9wZXJ0eUVudW0uamF2YQ==) | `73.68% <ø> (+73.68%)` | `4.00 <0.00> (+4.00)` | | | [...pache/kylin/engine/mr/common/JobInfoConverter.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0pvYkluZm9Db252ZXJ0ZXIuamF2YQ==) | `60.29% <58.33%> (+31.61%)` | `23.00 <0.00> (+2.00)` | | | [...ache/kylin/metrics/lib/impl/BlockingReservoir.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL0Jsb2NraW5nUmVzZXJ2b2lyLmphdmE=) | `53.33% <66.66%> (+53.33%)` | `13.00 <2.00> (+13.00)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0.00%> (-5.41%)` | `6.00% <0.00%> (-1.00%)` | | | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0.00%> (-2.33%)` | `12.00% <0.00%> (ø%)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0.00%> (-2.31%)` | `0.00% <0.00%> (ø%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.30% <0.00%> (-1.83%)` | `7.00% <0.00%> (ø%)` | | | [...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=) | `15.94% <0.00%> (+0.50%)` | `3.00% <0.00%> (+1.00%)` | | | ... and [35 more](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=footer). Last update [7886a24...984dc81](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=lastupdated). Read the
[GitHub] [kylin] kyotoYaho opened a new pull request #1228: KYLIN-4508 Add more unit tests to improve test coverage
kyotoYaho opened a new pull request #1228: URL: https://github.com/apache/kylin/pull/1228 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113792#comment-17113792 ] ASF GitHub Bot commented on KYLIN-4508: --- kyotoYaho opened a new pull request #1228: URL: https://github.com/apache/kylin/pull/1228 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4513) Introduce query call trace
Zhong Yanghong created KYLIN-4513: - Summary: Introduce query call trace Key: KYLIN-4513 URL: https://issues.apache.org/jira/browse/KYLIN-4513 Project: Kylin Issue Type: New Feature Reporter: Zhong Yanghong Assignee: Zhong Yanghong -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls edited a comment on pull request #1226: KYLIN-4495 Support custom date format for partition date column
coveralls edited a comment on pull request #1226: URL: https://github.com/apache/kylin/pull/1226#issuecomment-632233973 ## Pull Request Test Coverage Report for [Build 5934](https://coveralls.io/builds/30965423) * **0** of **0** changed or added relevant lines in **0** files are covered. * **9** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.004%**) to **27.289%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/30965423/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/30965423/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/30965423/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/30965423/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 4 | 78.12% | | Totals | [](https://coveralls.io/builds/30965423) | | :-- | --: | | Change from base [Build 5911](https://coveralls.io/builds/30861301): | -0.004% | | Covered Lines: | 24349 | | Relevant Lines: | 89225 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4495) Support custom date formats for partition date column
[ https://issues.apache.org/jira/browse/KYLIN-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113862#comment-17113862 ] ASF GitHub Bot commented on KYLIN-4495: --- coveralls edited a comment on pull request #1226: URL: https://github.com/apache/kylin/pull/1226#issuecomment-632233973 ## Pull Request Test Coverage Report for [Build 5934](https://coveralls.io/builds/30965423) * **0** of **0** changed or added relevant lines in **0** files are covered. * **9** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.004%**) to **27.289%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/30965423/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/30965423/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/30965423/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/30965423/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 4 | 78.12% | | Totals | [](https://coveralls.io/builds/30965423) | | :-- | --: | | Change from base [Build 5911](https://coveralls.io/builds/30861301): | -0.004% | | Covered Lines: | 24349 | | Relevant Lines: | 89225 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support custom date formats for partition date column > - > > Key: KYLIN-4495 > URL: https://issues.apache.org/jira/browse/KYLIN-4495 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v3.1.0 > > > Support custom date formats for partition date column. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] codecov-commenter edited a comment on pull request #1226: KYLIN-4495 Support custom date format for partition date column
codecov-commenter edited a comment on pull request #1226: URL: https://github.com/apache/kylin/pull/1226#issuecomment-632231689 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=h1) Report > Merging [#1226](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1226 +/- ## - Coverage 24.85% 24.84% -0.02% + Complexity 6266 6263 -3 Files 1453 1453 Lines 8922589225 Branches 1245612456 - Hits 2218122166 -15 - Misses6486964879 +10 - Partials 2175 2180 +5 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0.00%> (-5.41%)` | `6.00% <0.00%> (-1.00%)` | | | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0.00%> (-2.33%)` | `12.00% <0.00%> (ø%)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0.00%> (-2.31%)` | `0.00% <0.00%> (ø%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.60% <0.00%> (-1.52%)` | `7.00% <0.00%> (ø%)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0.00%> (-0.80%)` | `17.00% <0.00%> (-1.00%)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=footer). Last update [7886a24...a11cdb5](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4495) Support custom date formats for partition date column
[ https://issues.apache.org/jira/browse/KYLIN-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113875#comment-17113875 ] ASF GitHub Bot commented on KYLIN-4495: --- codecov-commenter edited a comment on pull request #1226: URL: https://github.com/apache/kylin/pull/1226#issuecomment-632231689 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=h1) Report > Merging [#1226](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1226 +/- ## - Coverage 24.85% 24.84% -0.02% + Complexity 6266 6263 -3 Files 1453 1453 Lines 8922589225 Branches 1245612456 - Hits 2218122166 -15 - Misses6486964879 +10 - Partials 2175 2180 +5 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0.00%> (-5.41%)` | `6.00% <0.00%> (-1.00%)` | | | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0.00%> (-2.33%)` | `12.00% <0.00%> (ø%)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0.00%> (-2.31%)` | `0.00% <0.00%> (ø%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.60% <0.00%> (-1.52%)` | `7.00% <0.00%> (ø%)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/1226/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0.00%> (-0.80%)` | `17.00% <0.00%> (-1.00%)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=footer). Last update [7886a24...a11cdb5](https://codecov.io/gh/apache/kylin/pull/1226?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support custom date formats for partition date column > - > > Key: KYLIN-4495 > URL: https://issues.apache.org/jira/browse/KYLIN-4495 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v3.1.0 > > > Support custom date formats for partition date column. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3487) Create a new measure for precise count distinct
[ https://issues.apache.org/jira/browse/KYLIN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3487: Fix Version/s: (was: Backlog) v3.1.0 > Create a new measure for precise count distinct > --- > > Key: KYLIN-3487 > URL: https://issues.apache.org/jira/browse/KYLIN-3487 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v3.1.0 > > > To compute the precise count distinct, we can use bitmap and global > dictionary. However, there's a limitation for the global dictionary. It maps > from values to ids whose type is integer, which means the number of ids will > be less than 2B. And it's like a Pixiu for which there's increase but no > decrease. > In eBay, there's a requirement of calculating precise count distinct of > session. The session cardinality is large and will grow as time goes on. It > will not be feasible to use the global dictionary when its cardinality > exceeds the upper bound 2B. How can we deal with this? > There's good news that a session never crosses days. With this feature, we > don't need to merge bitmap across days. To calculate precise session > cardinality, we can assign each day a bitmap and directly summarize the > cardinalities estimated by each bitmap. No bitmap merge is needed. > To use bitmap for cardinality calculation, we need to map raw data from value > to an integer id, which is achieved by encoding the value with a dictionary. > Previously, for the ability of merging bitmaps from multiple segments, global > dictionary is used. However, in this case, there's no need of bitmap merge, > the global dictionary is not needed. > And we don't need to filter by or group by session. Then there's no need to > map from value to id and from id to value after the related bitmap is > constructed. Therefore, we don't need to store dictionaries for session. Only > the bitmap is enough. > To deal with segment merge, since bitmaps of each segment are not able to > merge to one bitmap, we use a map for storing multiple bitmaps. In the map, > the key is the segment name and the value is the segment-level bitmap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3487) Create a new measure for precise count distinct
[ https://issues.apache.org/jira/browse/KYLIN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3487: Sprint: Sprint 51 > Create a new measure for precise count distinct > --- > > Key: KYLIN-3487 > URL: https://issues.apache.org/jira/browse/KYLIN-3487 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v3.1.0 > > > To compute the precise count distinct, we can use bitmap and global > dictionary. However, there's a limitation for the global dictionary. It maps > from values to ids whose type is integer, which means the number of ids will > be less than 2B. And it's like a Pixiu for which there's increase but no > decrease. > In eBay, there's a requirement of calculating precise count distinct of > session. The session cardinality is large and will grow as time goes on. It > will not be feasible to use the global dictionary when its cardinality > exceeds the upper bound 2B. How can we deal with this? > There's good news that a session never crosses days. With this feature, we > don't need to merge bitmap across days. To calculate precise session > cardinality, we can assign each day a bitmap and directly summarize the > cardinalities estimated by each bitmap. No bitmap merge is needed. > To use bitmap for cardinality calculation, we need to map raw data from value > to an integer id, which is achieved by encoding the value with a dictionary. > Previously, for the ability of merging bitmaps from multiple segments, global > dictionary is used. However, in this case, there's no need of bitmap merge, > the global dictionary is not needed. > And we don't need to filter by or group by session. Then there's no need to > map from value to id and from id to value after the related bitmap is > constructed. Therefore, we don't need to store dictionaries for session. Only > the bitmap is enough. > To deal with segment merge, since bitmaps of each segment are not able to > merge to one bitmap, we use a map for storing multiple bitmaps. In the map, > the key is the segment name and the value is the segment-level bitmap. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113851#comment-17113851 ] ASF GitHub Bot commented on KYLIN-4508: --- codecov-commenter commented on pull request #1228: URL: https://github.com/apache/kylin/pull/1228#issuecomment-632572024 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=h1) Report > Merging [#1228](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **increase** coverage by `1.10%`. > The diff coverage is `56.52%`. [](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1228 +/- ## + Coverage 24.85% 25.96% +1.10% - Complexity 6266 6520 +254 Files 1453 1453 Lines 8922589243 +18 Branches 1245612458 +2 + Hits 2218123173 +992 + Misses6486963810-1059 - Partials 2175 2260 +85 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...org/apache/kylin/metrics/lib/impl/RecordEvent.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL1JlY29yZEV2ZW50LmphdmE=) | `72.04% <0.00%> (+72.04%)` | `24.00 <0.00> (+24.00)` | | | [...pache/kylin/metrics/lib/impl/TimePropertyEnum.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL1RpbWVQcm9wZXJ0eUVudW0uamF2YQ==) | `73.68% <ø> (+73.68%)` | `4.00 <0.00> (+4.00)` | | | [...pache/kylin/engine/mr/common/JobInfoConverter.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0pvYkluZm9Db252ZXJ0ZXIuamF2YQ==) | `60.29% <58.33%> (+31.61%)` | `23.00 <0.00> (+2.00)` | | | [...ache/kylin/metrics/lib/impl/BlockingReservoir.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL0Jsb2NraW5nUmVzZXJ2b2lyLmphdmE=) | `53.33% <66.66%> (+53.33%)` | `13.00 <2.00> (+13.00)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0.00%> (-5.41%)` | `6.00% <0.00%> (-1.00%)` | | | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0.00%> (-2.33%)` | `12.00% <0.00%> (ø%)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0.00%> (-2.31%)` | `0.00% <0.00%> (ø%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.30% <0.00%> (-1.83%)` | `7.00% <0.00%> (ø%)` | | | [...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=) | `15.94% <0.00%> (+0.50%)` | `3.00% <0.00%> (+1.00%)` | | | ... and [35 more](https://codecov.io/gh/apache/kylin/pull/1228/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1228?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø
[GitHub] [kylin] codecov-commenter commented on pull request #1227: KYLIN-3698 Fix kylin.sh can't export spark/flink/kafka dependency
codecov-commenter commented on pull request #1227: URL: https://github.com/apache/kylin/pull/1227#issuecomment-632504460 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=h1) Report > Merging [#1227](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/e7ab5dfd4431a5b410813311fa5e5a983e9aa4f2=desc) will **decrease** coverage by `0.03%`. > The diff coverage is `0.71%`. [](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1227 +/- ## - Coverage 24.87% 24.84% -0.04% - Complexity 6264 6265 +1 Files 1453 1453 Lines 8912589225 +100 Branches 1244012456 +16 - Hits 2217422171 -3 - Misses6477464877 +103 Partials 2177 2177 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.12% <0.00%> (-0.04%)` | `44.00 <0.00> (ø)` | | | [...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=) | `15.44% <0.00%> (+0.23%)` | `2.00 <0.00> (ø)` | | | [...engine/mr/steps/ConvergeCuboidDataPartitioner.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvQ29udmVyZ2VDdWJvaWREYXRhUGFydGl0aW9uZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...lin/engine/mr/steps/FactDistinctColumnsMapper.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvRmFjdERpc3RpbmN0Q29sdW1uc01hcHBlci5qYXZh) | `5.17% <0.00%> (-0.05%)` | `0.00 <0.00> (ø)` | | | [...che/kylin/storage/hbase/steps/CreateHTableJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DcmVhdGVIVGFibGVKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...apache/kylin/storage/hbase/steps/CubeHFileJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DdWJlSEZpbGVKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...ache/kylin/storage/hbase/steps/CubeHTableUtil.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DdWJlSFRhYmxlVXRpbC5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../kylin/storage/hbase/steps/HFileOutputFormat3.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IRmlsZU91dHB1dEZvcm1hdDMuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...in/storage/hbase/lookup/HBaseLookupRowEncoder.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9sb29rdXAvSEJhc2VMb29rdXBSb3dFbmNvZGVyLmphdmE=) | `92.59% <100.00%> (ø)` | `11.00 <0.00> (ø)` | | | [...ream/core/storage/columnar/ColumnarStoreCache.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTdG9yZUNhY2hlLmphdmE=) | `52.27% <0.00%> (-3.41%)` | `8.00% <0.00%> (ø%)` | | | ... and [2 more](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=footer). Last update [a471808...4538140](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=lastupdated). Read the [comment
[jira] [Commented] (KYLIN-3698) check-env.sh should print more details about checking items
[ https://issues.apache.org/jira/browse/KYLIN-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113780#comment-17113780 ] ASF GitHub Bot commented on KYLIN-3698: --- codecov-commenter commented on pull request #1227: URL: https://github.com/apache/kylin/pull/1227#issuecomment-632504460 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=h1) Report > Merging [#1227](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/e7ab5dfd4431a5b410813311fa5e5a983e9aa4f2=desc) will **decrease** coverage by `0.03%`. > The diff coverage is `0.71%`. [](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1227 +/- ## - Coverage 24.87% 24.84% -0.04% - Complexity 6264 6265 +1 Files 1453 1453 Lines 8912589225 +100 Branches 1244012456 +16 - Hits 2217422171 -3 - Misses6477464877 +103 Partials 2177 2177 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.12% <0.00%> (-0.04%)` | `44.00 <0.00> (ø)` | | | [...ache/kylin/engine/mr/common/AbstractHadoopJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0Fic3RyYWN0SGFkb29wSm9iLmphdmE=) | `15.44% <0.00%> (+0.23%)` | `2.00 <0.00> (ø)` | | | [...engine/mr/steps/ConvergeCuboidDataPartitioner.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvQ29udmVyZ2VDdWJvaWREYXRhUGFydGl0aW9uZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...lin/engine/mr/steps/FactDistinctColumnsMapper.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvRmFjdERpc3RpbmN0Q29sdW1uc01hcHBlci5qYXZh) | `5.17% <0.00%> (-0.05%)` | `0.00 <0.00> (ø)` | | | [...che/kylin/storage/hbase/steps/CreateHTableJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DcmVhdGVIVGFibGVKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...apache/kylin/storage/hbase/steps/CubeHFileJob.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DdWJlSEZpbGVKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...ache/kylin/storage/hbase/steps/CubeHTableUtil.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9DdWJlSFRhYmxlVXRpbC5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../kylin/storage/hbase/steps/HFileOutputFormat3.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IRmlsZU91dHB1dEZvcm1hdDMuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...in/storage/hbase/lookup/HBaseLookupRowEncoder.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9sb29rdXAvSEJhc2VMb29rdXBSb3dFbmNvZGVyLmphdmE=) | `92.59% <100.00%> (ø)` | `11.00 <0.00> (ø)` | | | [...ream/core/storage/columnar/ColumnarStoreCache.java](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTdG9yZUNhY2hlLmphdmE=) | `52.27% <0.00%> (-3.41%)` | `8.00% <0.00%> (ø%)` | | | ... and [2 more](https://codecov.io/gh/apache/kylin/pull/1227/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1227?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not
[jira] [Created] (KYLIN-4514) kylin通过jdbc连接apache phoenix，CUBE中的部分日期转换错误
dongwanfu created KYLIN-4514: Summary: kylin通过jdbc连接apache phoenix，CUBE中的部分日期转换错误 Key: KYLIN-4514 URL: https://issues.apache.org/jira/browse/KYLIN-4514 Project: Kylin Issue Type: Bug Components: Driver - JDBC Affects Versions: v3.0.1 Environment: hadoop-2.6.5 hbase-1.3.1 jdk1.8.0_181 sqoop-1.4.7 phoenix-4.14.3-HBase-1.3 Reporter: dongwanfu Attachments: DataInKylin1.jpg, DataInKylin2.jpg, DataInPhoenix.jpg kylin通过jdbc连接Phoenix，kylin创建的CUBE中的日期出现了错误时间，我的phoenix中导入了1条数据，其中有个字段birthtime（phoenix timestamp），时间范围在1970-2000年之间，kylin创建的Cube中的年份出现了很大的年份如9532年，也有很小的年份如1002年，我不知道怎么解决这个问题，只能提交这个issue。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4501) Support alluxio to adapt more storage like s3, oss ...
[ https://issues.apache.org/jira/browse/KYLIN-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113896#comment-17113896 ] Harvey Yue edited comment on KYLIN-4501 at 5/22/20, 9:52 AM: - Kylin and Alluxio related configuration changes: alluxio-site.properties ## fix delete cuboid temp directories issue: alluxio.exception.DirectoryNotEmptyException alluxio.user.file.delete.unchecked=true kylin.properties kylin.env.hdfs-working-dir=*alluxio://192.168.xxx.xxx:19998/kylin* kylin.storage.url=*alluxio* kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history was (Author: harveyyue): Configurations: kylin.properties # kylin.env.hdfs-working-dir=alluxio://192.168.xxx.xxx:19998/kylin # kylin.storage.url=alluxio # kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history # kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history > Support alluxio to adapt more storage like s3, oss ... > -- > > Key: KYLIN-4501 > URL: https://issues.apache.org/jira/browse/KYLIN-4501 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v4.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4501) Support alluxio to adapt more storage like s3, oss ...
[ https://issues.apache.org/jira/browse/KYLIN-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113896#comment-17113896 ] Harvey Yue edited comment on KYLIN-4501 at 5/22/20, 9:52 AM: - Kylin and Alluxio related configuration changes: alluxio-site.properties // fix delete cuboid temp directories issue: alluxio.exception.DirectoryNotEmptyException alluxio.user.file.delete.unchecked=true kylin.properties kylin.env.hdfs-working-dir=*alluxio://192.168.xxx.xxx:19998/kylin* kylin.storage.url=*alluxio* kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history was (Author: harveyyue): Kylin and Alluxio related configuration changes: alluxio-site.properties ## fix delete cuboid temp directories issue: alluxio.exception.DirectoryNotEmptyException alluxio.user.file.delete.unchecked=true kylin.properties kylin.env.hdfs-working-dir=*alluxio://192.168.xxx.xxx:19998/kylin* kylin.storage.url=*alluxio* kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history > Support alluxio to adapt more storage like s3, oss ... > -- > > Key: KYLIN-4501 > URL: https://issues.apache.org/jira/browse/KYLIN-4501 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v4.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1198: KYLIN-4464 Not apply SortProjectTransposeRule when window func in pro…
shaofengshi merged pull request #1198: URL: https://github.com/apache/kylin/pull/1198 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-3849) Flink cubing step : convert to HFile
[ https://issues.apache.org/jira/browse/KYLIN-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113928#comment-17113928 ] ASF subversion and git services commented on KYLIN-3849: Commit 51cb30e7d7e6818f79b5c8f59a7356dd1bc0deba in kylin's branch refs/heads/master from harveyyue [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=51cb30e ] KYLIN-3849 Flink cubing step : convert to HFile > Flink cubing step : convert to HFile > > > Key: KYLIN-3849 > URL: https://issues.apache.org/jira/browse/KYLIN-3849 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: Harvey Yue >Priority: Major > Attachments: flink_cube_build.jpg, hfile.jpg > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3849) Flink cubing step : convert to HFile
[ https://issues.apache.org/jira/browse/KYLIN-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113927#comment-17113927 ] ASF GitHub Bot commented on KYLIN-3849: --- shaofengshi merged pull request #1201: URL: https://github.com/apache/kylin/pull/1201 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flink cubing step : convert to HFile > > > Key: KYLIN-3849 > URL: https://issues.apache.org/jira/browse/KYLIN-3849 > Project: Kylin > Issue Type: Sub-task > Components: Flink Engine >Reporter: vinoyang >Assignee: Harvey Yue >Priority: Major > Attachments: flink_cube_build.jpg, hfile.jpg > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4464) Query ... row_number over(order by c1) ... order by c2 ... get wrong order result
[
https://issues.apache.org/jira/browse/KYLIN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113929#comment-17113929
]
ASF GitHub Bot commented on KYLIN-4464:
---
shaofengshi merged pull request #1198:
URL: https://github.com/apache/kylin/pull/1198
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Query ... row_number over(order by c1) ... order by c2 ... get wrong order
> result
> -
>
> Key: KYLIN-4464
> URL: https://issues.apache.org/jira/browse/KYLIN-4464
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-04-20-18-19-40-925.png,
> image-2020-04-20-18-22-34-765.png, image-2020-05-07-10-32-39-983.png
>
>
> *SQL*
> {code:java}
> select t.*, row_number() over (order by t.lstg_format_name) as row_num from (
> select lstg_format_name, sum(price) as GMV from
> KYLIN_SALES
> where lstg_format_name is not null
> group by lstg_format_name
> ) as t
> order by t.GMV desc limit 5{quote}
> {code}
>
> *Kylin result (order by lstg_format_name)*
> !image-2020-04-20-18-19-40-925.png!
>
> *Calcite execution plan*
> {code:java}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
> OLAPWindowRel(window#0=[window(partition {} order by [0 DESC] rows between
> UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])], ctx=[],
> groups=[[window(partition {} order by [0 DESC] rows between UNBOUNDED
> PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])]])
> OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
> OLAPAggregateRel(group=[{0}], GMV=[SUM($1)], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$2], PRICE=[$5], ctx=[])
> OLAPFilterRel(condition=[IS NOT NULL($2)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
> {code}
>
> *Hive result(order by GMV)*
> !image-2020-04-20-18-22-34-765.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1201: KYLIN-3849 Flink cubing step : convert to HFile
shaofengshi merged pull request #1201: URL: https://github.com/apache/kylin/pull/1201 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4501) Support alluxio to adapt more storage like s3, oss ...
[ https://issues.apache.org/jira/browse/KYLIN-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113896#comment-17113896 ] Harvey Yue commented on KYLIN-4501: --- Configurations: kylin.properties # kylin.env.hdfs-working-dir=alluxio://192.168.xxx.xxx:19998/kylin # kylin.storage.url=alluxio # kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history # kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history > Support alluxio to adapt more storage like s3, oss ... > -- > > Key: KYLIN-4501 > URL: https://issues.apache.org/jira/browse/KYLIN-4501 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v4.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4515) could not send mail on ssl port
[
https://issues.apache.org/jira/browse/KYLIN-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113899#comment-17113899
]
CHEN Qing commented on KYLIN-4515:
--
The root cause is, with such config above, HTMLEmail client doesn't enable ssl
connection, so we need to enable ssl connection by {{setSSLOnConnect}} as well
when using ssl port.
PR: [https://github.com/apache/kylin/pull/1214]
> could not send mail on ssl port
> ---
>
> Key: KYLIN-4515
> URL: https://issues.apache.org/jira/browse/KYLIN-4515
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v3.0.1
>Reporter: CHEN Qing
>Priority: Minor
> Fix For: v3.0.1
>
>
> Here is context of this
> problem：[https://www.mail-archive.com/user@kylin.apache.org/msg04009.html]
> When I set {{starttlsEnabled}} to true, and specify a ssl port as 465, the
> email could not be sent by this ssl port. It still use 25 default port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] chenqingspring commented on pull request #1214: [Bugfix] fix error with sending mail on ssl
chenqingspring commented on pull request #1214: URL: https://github.com/apache/kylin/pull/1214#issuecomment-632603162 @nichunen Sure, with my pleasure :) [KYLIN-4515](https://issues.apache.org/jira/browse/KYLIN-4515) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KYLIN-4515) could not send mail on ssl port
CHEN Qing created KYLIN-4515:
Summary: could not send mail on ssl port
Key: KYLIN-4515
URL: https://issues.apache.org/jira/browse/KYLIN-4515
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v3.0.1
Reporter: CHEN Qing
Fix For: v3.0.1
Here is context of this
problem：[https://www.mail-archive.com/user@kylin.apache.org/msg04009.html]
When I set {{starttlsEnabled}} to true, and specify a ssl port as 465, the
email could not be sent by this ssl port. It still use 25 default port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4464) Query ... row_number over(order by c1) ... order by c2 ... get wrong order result
[
https://issues.apache.org/jira/browse/KYLIN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113930#comment-17113930
]
ASF subversion and git services commented on KYLIN-4464:
Commit c5573dc9be7ad86b3a8c71bc4a17092594bdf8f9 in kylin's branch
refs/heads/master from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c5573dc ]
KYLIN-4464 Not apply SortProjectTransposeRule when window func in projection
> Query ... row_number over(order by c1) ... order by c2 ... get wrong order
> result
> -
>
> Key: KYLIN-4464
> URL: https://issues.apache.org/jira/browse/KYLIN-4464
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-04-20-18-19-40-925.png,
> image-2020-04-20-18-22-34-765.png, image-2020-05-07-10-32-39-983.png
>
>
> *SQL*
> {code:java}
> select t.*, row_number() over (order by t.lstg_format_name) as row_num from (
> select lstg_format_name, sum(price) as GMV from
> KYLIN_SALES
> where lstg_format_name is not null
> group by lstg_format_name
> ) as t
> order by t.GMV desc limit 5{quote}
> {code}
>
> *Kylin result (order by lstg_format_name)*
> !image-2020-04-20-18-19-40-925.png!
>
> *Calcite execution plan*
> {code:java}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
> OLAPWindowRel(window#0=[window(partition {} order by [0 DESC] rows between
> UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])], ctx=[],
> groups=[[window(partition {} order by [0 DESC] rows between UNBOUNDED
> PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])]])
> OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
> OLAPAggregateRel(group=[{0}], GMV=[SUM($1)], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$2], PRICE=[$5], ctx=[])
> OLAPFilterRel(condition=[IS NOT NULL($2)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
> {code}
>
> *Hive result(order by GMV)*
> !image-2020-04-20-18-22-34-765.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4517) Pruning cuboids with greedy algorithm
wangrupeng created KYLIN-4517: - Summary: Pruning cuboids with greedy algorithm Key: KYLIN-4517 URL: https://issues.apache.org/jira/browse/KYLIN-4517 Project: Kylin Issue Type: Sub-task Reporter: wangrupeng Assignee: wangrupeng Fix For: v4.0.0-beta -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4518) Pruning cuboids with genetic algorithm
wangrupeng created KYLIN-4518: - Summary: Pruning cuboids with genetic algorithm Key: KYLIN-4518 URL: https://issues.apache.org/jira/browse/KYLIN-4518 Project: Kylin Issue Type: Sub-task Reporter: wangrupeng Assignee: wangrupeng Fix For: v4.0.0-beta -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4519) Analyse user query history and offer suggestion about the removing of unused cuboids
wangrupeng created KYLIN-4519: - Summary: Analyse user query history and offer suggestion about the removing of unused cuboids Key: KYLIN-4519 URL: https://issues.apache.org/jira/browse/KYLIN-4519 Project: Kylin Issue Type: Sub-task Reporter: wangrupeng Assignee: wangrupeng Fix For: v4.0.0-beta -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KYLIN-4496) Metric data missing
[ https://issues.apache.org/jira/browse/KYLIN-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi reassigned KYLIN-4496: Assignee: tianhui > Metric data missing > --- > > Key: KYLIN-4496 > URL: https://issues.apache.org/jira/browse/KYLIN-4496 > Project: Kylin > Issue Type: Bug > Components: Metrics >Reporter: tianhui >Assignee: tianhui >Priority: Critical > Fix For: v3.1.0 > > > Metric data in memory is never flushed into file system when kylin closing, > So there is some metric data missing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KYLIN-4501) Support alluxio to adapt more storage like s3, oss ...
[ https://issues.apache.org/jira/browse/KYLIN-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113896#comment-17113896 ] Harvey Yue edited comment on KYLIN-4501 at 5/22/20, 9:56 AM: - *Kylin and Alluxio related configuration changes:* alluxio-site.properties // fix delete cuboid temp directories issue: alluxio.exception.DirectoryNotEmptyException alluxio.user.file.delete.unchecked=true kylin.properties kylin.env.hdfs-working-dir=*alluxio://192.168.xxx.xxx:19998/kylin* kylin.storage.url=*alluxio* kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history *Spark jars:* copy ALLUXIO_HOME/client/alluxio-2.2.0-client.jar to SPARK_HOME/jars was (Author: harveyyue): Kylin and Alluxio related configuration changes: alluxio-site.properties // fix delete cuboid temp directories issue: alluxio.exception.DirectoryNotEmptyException alluxio.user.file.delete.unchecked=true kylin.properties kylin.env.hdfs-working-dir=*alluxio://192.168.xxx.xxx:19998/kylin* kylin.storage.url=*alluxio* kylin.engine.spark-conf.spark.eventLog.dir=alluxio://192.168.xxx.xxx:19998/kylin/spark-history kylin.engine.spark-conf.spark.history.fs.logDirectory=alluxio://192.168.xxx.xxx:19998/kylin/spark-history > Support alluxio to adapt more storage like s3, oss ... > -- > > Key: KYLIN-4501 > URL: https://issues.apache.org/jira/browse/KYLIN-4501 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v4.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4516) Support System Cube
wangrupeng created KYLIN-4516: - Summary: Support System Cube Key: KYLIN-4516 URL: https://issues.apache.org/jira/browse/KYLIN-4516 Project: Kylin Issue Type: Sub-task Reporter: wangrupeng Assignee: wangrupeng Fix For: v4.0.0-beta -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4394) Upgrade dependency version for several CVEs
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113917#comment-17113917 ] ASF subversion and git services commented on KYLIN-4394: Commit 9164cc54768d961275e7f2533ce4a6dbf6cb9c36 in kylin's branch refs/heads/master from yaqian.zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=9164cc5 ] KYLIN-4394 Update kylin dependency Signed-off-by: shaofengshi > Upgrade dependency version for several CVEs > --- > > Key: KYLIN-4394 > URL: https://issues.apache.org/jira/browse/KYLIN-4394 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.1.0 > > Attachments: apache-kylin_CVE-report.md > > > I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 > *CVE ID:* > [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] > *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 > *CVE ID:* > [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], > [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], > [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], > [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], > [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], > [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] > *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* > [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], > > [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] > *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* > 1.4.198, 1.4.199, 1.4.200 > * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* > [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] > *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, > server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* > 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, > 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, > 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, > 28.2-android, 28.2-jre > * *Vulnerable Library Version:* org.apache.hive.hcatalog : > hive-hcatalog-core : 1.2.1 *CVE ID:* > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] > *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, > 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, > 3.1.1, 3.1.2 > * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 > *CVE ID:* > [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], > [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], > [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], > > [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] > *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 > * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE > ID:* > [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], > [CVE-2019-17196|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196] > *Import Path:* assembly/pom.xml, source-kafka/pom.xml, kylin-it/pom.xml, > stream-source-kafka/pom.xml, metrics-reporter-kafka/pom.xml *Suggested Safe > Versions:* 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0,
[GitHub] [kylin] zhangayqian opened a new pull request #1229: Update website RESTful API page
zhangayqian opened a new pull request #1229: URL: https://github.com/apache/kylin/pull/1229 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (KYLIN-4464) Query ... row_number over(order by c1) ... order by c2 ... get wrong order result
[
https://issues.apache.org/jira/browse/KYLIN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi resolved KYLIN-4464.
--
Fix Version/s: v3.0.2
Resolution: Fixed
> Query ... row_number over(order by c1) ... order by c2 ... get wrong order
> result
> -
>
> Key: KYLIN-4464
> URL: https://issues.apache.org/jira/browse/KYLIN-4464
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.1.0, v3.0.2
>
> Attachments: image-2020-04-20-18-19-40-925.png,
> image-2020-04-20-18-22-34-765.png, image-2020-05-07-10-32-39-983.png
>
>
> *SQL*
> {code:java}
> select t.*, row_number() over (order by t.lstg_format_name) as row_num from (
> select lstg_format_name, sum(price) as GMV from
> KYLIN_SALES
> where lstg_format_name is not null
> group by lstg_format_name
> ) as t
> order by t.GMV desc limit 5{quote}
> {code}
>
> *Kylin result (order by lstg_format_name)*
> !image-2020-04-20-18-19-40-925.png!
>
> *Calcite execution plan*
> {code:java}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
> OLAPWindowRel(window#0=[window(partition {} order by [0 DESC] rows between
> UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])], ctx=[],
> groups=[[window(partition {} order by [0 DESC] rows between UNBOUNDED
> PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])]])
> OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
> OLAPAggregateRel(group=[{0}], GMV=[SUM($1)], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$2], PRICE=[$5], ctx=[])
> OLAPFilterRel(condition=[IS NOT NULL($2)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
> {code}
>
> *Hive result(order by GMV)*
> !image-2020-04-20-18-22-34-765.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4464) Query ... row_number over(order by c1) ... order by c2 ... get wrong order result
[
https://issues.apache.org/jira/browse/KYLIN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113931#comment-17113931
]
ASF subversion and git services commented on KYLIN-4464:
Commit e2cac72b16d5a446eccb9699a44414087f0fb00e in kylin's branch
refs/heads/3.0.x from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e2cac72 ]
KYLIN-4464 Not apply SortProjectTransposeRule when window func in projection
> Query ... row_number over(order by c1) ... order by c2 ... get wrong order
> result
> -
>
> Key: KYLIN-4464
> URL: https://issues.apache.org/jira/browse/KYLIN-4464
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.1.0
>
> Attachments: image-2020-04-20-18-19-40-925.png,
> image-2020-04-20-18-22-34-765.png, image-2020-05-07-10-32-39-983.png
>
>
> *SQL*
> {code:java}
> select t.*, row_number() over (order by t.lstg_format_name) as row_num from (
> select lstg_format_name, sum(price) as GMV from
> KYLIN_SALES
> where lstg_format_name is not null
> group by lstg_format_name
> ) as t
> order by t.GMV desc limit 5{quote}
> {code}
>
> *Kylin result (order by lstg_format_name)*
> !image-2020-04-20-18-19-40-925.png!
>
> *Calcite execution plan*
> {code:java}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
> OLAPWindowRel(window#0=[window(partition {} order by [0 DESC] rows between
> UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])], ctx=[],
> groups=[[window(partition {} order by [0 DESC] rows between UNBOUNDED
> PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])]])
> OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
> OLAPAggregateRel(group=[{0}], GMV=[SUM($1)], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$2], PRICE=[$5], ctx=[])
> OLAPFilterRel(condition=[IS NOT NULL($2)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
> {code}
>
> *Hive result(order by GMV)*
> !image-2020-04-20-18-22-34-765.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4464) Query ... row_number over(order by c1) ... order by c2 ... get wrong order result
[
https://issues.apache.org/jira/browse/KYLIN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113933#comment-17113933
]
ASF subversion and git services commented on KYLIN-4464:
Commit d036da1bda725e7150b65e4eb387ddc0f22522f4 in kylin's branch
refs/heads/2.6.x from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d036da1 ]
KYLIN-4464 Not apply SortProjectTransposeRule when window func in projection
> Query ... row_number over(order by c1) ... order by c2 ... get wrong order
> result
> -
>
> Key: KYLIN-4464
> URL: https://issues.apache.org/jira/browse/KYLIN-4464
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.1.0, v3.0.2
>
> Attachments: image-2020-04-20-18-19-40-925.png,
> image-2020-04-20-18-22-34-765.png, image-2020-05-07-10-32-39-983.png
>
>
> *SQL*
> {code:java}
> select t.*, row_number() over (order by t.lstg_format_name) as row_num from (
> select lstg_format_name, sum(price) as GMV from
> KYLIN_SALES
> where lstg_format_name is not null
> group by lstg_format_name
> ) as t
> order by t.GMV desc limit 5{quote}
> {code}
>
> *Kylin result (order by lstg_format_name)*
> !image-2020-04-20-18-19-40-925.png!
>
> *Calcite execution plan*
> {code:java}
> EXECUTION PLAN BEFORE REWRITE
> OLAPToEnumerableConverter
> OLAPWindowRel(window#0=[window(partition {} order by [0 DESC] rows between
> UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])], ctx=[],
> groups=[[window(partition {} order by [0 DESC] rows between UNBOUNDED
> PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])]])
> OLAPLimitRel(ctx=[], fetch=[5])
> OLAPSortRel(sort0=[$1], dir0=[DESC], ctx=[])
> OLAPAggregateRel(group=[{0}], GMV=[SUM($1)], ctx=[])
> OLAPProjectRel(LSTG_FORMAT_NAME=[$2], PRICE=[$5], ctx=[])
> OLAPFilterRel(condition=[IS NOT NULL($2)], ctx=[])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], ctx=[],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
> {code}
>
> *Hive result(order by GMV)*
> !image-2020-04-20-18-22-34-765.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4511) Support project admin user to calculate column cardinality by web
[ https://issues.apache.org/jira/browse/KYLIN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114514#comment-17114514 ] ASF GitHub Bot commented on KYLIN-4511: --- codecov-commenter edited a comment on pull request #1225: URL: https://github.com/apache/kylin/pull/1225#issuecomment-631867241 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=h1) Report > Merging [#1225](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **increase** coverage by `1.05%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1225 +/- ## + Coverage 24.85% 25.91% +1.05% - Complexity 6266 6520 +254 Files 1453 1456 +3 Lines 8922589459 +234 Branches 1245612482 +26 + Hits 2218123179 +998 + Misses6486964022 -847 - Partials 2175 2258 +83 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...che/kylin/stream/core/storage/CheckPointStore.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvQ2hlY2tQb2ludFN0b3JlLmphdmE=) | `63.63% <0.00%> (-3.04%)` | `21.00% <0.00%> (-1.00%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0.00%> (-0.92%)` | `7.00% <0.00%> (ø%)` | | | [.../kylin/cache/memcached/MemcachedChunkingCache.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y2FjaGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NhY2hlL21lbWNhY2hlZC9NZW1jYWNoZWRDaHVua2luZ0NhY2hlLmphdmE=) | `39.70% <0.00%> (-0.45%)` | `15.00% <0.00%> (ø%)` | | | [...ava/org/apache/kylin/tool/JobDiagnosisInfoCLI.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-dG9vbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vdG9vbC9Kb2JEaWFnbm9zaXNJbmZvQ0xJLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [.../org/apache/kylin/query/relnode/OLAPTableScan.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3JlbG5vZGUvT0xBUFRhYmxlU2Nhbi5qYXZh) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/rest/service/QueryService.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9RdWVyeVNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/source/hive/CLIHiveClient.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0NMSUhpdmVDbGllbnQuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/source/hive/HiveTableMeta.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0hpdmVUYWJsZU1ldGEuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...apache/kylin/job/execution/AbstractExecutable.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vQWJzdHJhY3RFeGVjdXRhYmxlLmphdmE=) | `47.93% <0.00%> (ø)` | `51.00% <0.00%> (ø%)` | | | ... and [49 more](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by
[GitHub] [kylin] codecov-commenter edited a comment on pull request #1225: KYLIN-4511 Support project admin user to calculate column cardinality…
codecov-commenter edited a comment on pull request #1225: URL: https://github.com/apache/kylin/pull/1225#issuecomment-631867241 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=h1) Report > Merging [#1225](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/7886a2498178ded085497d6204b4b367f56ce0c8=desc) will **increase** coverage by `1.05%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1225 +/- ## + Coverage 24.85% 25.91% +1.05% - Complexity 6266 6520 +254 Files 1453 1456 +3 Lines 8922589459 +234 Branches 1245612482 +26 + Hits 2218123179 +998 + Misses6486964022 -847 - Partials 2175 2258 +83 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `35.71% <0.00%> (-4.77%)` | `5.00% <0.00%> (-1.00%)` | | | [...che/kylin/stream/core/storage/CheckPointStore.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvQ2hlY2tQb2ludFN0b3JlLmphdmE=) | `63.63% <0.00%> (-3.04%)` | `21.00% <0.00%> (-1.00%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0.00%> (-0.92%)` | `7.00% <0.00%> (ø%)` | | | [.../kylin/cache/memcached/MemcachedChunkingCache.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y2FjaGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NhY2hlL21lbWNhY2hlZC9NZW1jYWNoZWRDaHVua2luZ0NhY2hlLmphdmE=) | `39.70% <0.00%> (-0.45%)` | `15.00% <0.00%> (ø%)` | | | [...ava/org/apache/kylin/tool/JobDiagnosisInfoCLI.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-dG9vbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vdG9vbC9Kb2JEaWFnbm9zaXNJbmZvQ0xJLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [.../org/apache/kylin/query/relnode/OLAPTableScan.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3JlbG5vZGUvT0xBUFRhYmxlU2Nhbi5qYXZh) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/rest/service/QueryService.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9RdWVyeVNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/source/hive/CLIHiveClient.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0NMSUhpdmVDbGllbnQuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...va/org/apache/kylin/source/hive/HiveTableMeta.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0hpdmVUYWJsZU1ldGEuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00% <0.00%> (ø%)` | | | [...apache/kylin/job/execution/AbstractExecutable.java](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vQWJzdHJhY3RFeGVjdXRhYmxlLmphdmE=) | `47.93% <0.00%> (ø)` | `51.00% <0.00%> (ø%)` | | | ... and [49 more](https://codecov.io/gh/apache/kylin/pull/1225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=footer). Last update [7886a24...5b7b0ed](https://codecov.io/gh/apache/kylin/pull/1225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[jira] [Commented] (KYLIN-4496) Metric data missing
[ https://issues.apache.org/jira/browse/KYLIN-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114048#comment-17114048 ] Shao Feng Shi commented on KYLIN-4496: -- [~tianhui5] thanks for the reporting! [~nju_yaho] please review. > Metric data missing > --- > > Key: KYLIN-4496 > URL: https://issues.apache.org/jira/browse/KYLIN-4496 > Project: Kylin > Issue Type: Bug > Components: Metrics >Reporter: tianhui >Assignee: tianhui >Priority: Critical > Fix For: v3.1.0 > > > Metric data in memory is never flushed into file system when kylin closing, > So there is some metric data missing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114094#comment-17114094 ] ASF subversion and git services commented on KYLIN-4508: Commit e220b9a20c4eae09c8c240c066b730595eb03693 in kylin's branch refs/heads/master from Ma,Gang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=e220b9a ] KYLIN-4508 Add more unit tests for MR module Signed-off-by: shaofengshi > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114093#comment-17114093 ] ASF subversion and git services commented on KYLIN-4508: Commit 2d718c83db2abd91d053b77a44ed4a98648adcdc in kylin's branch refs/heads/master from Zhong, Yanghong [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=2d718c8 ] KYLIN-4508 Add more unit tests for core-metrics module Signed-off-by: shaofengshi > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi commented on pull request #1228: KYLIN-4508 Add more unit tests to improve test coverage
shaofengshi commented on pull request #1228: URL: https://github.com/apache/kylin/pull/1228#issuecomment-632722692 Merged by manual. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-4432) Duplicated queries with sytax error take unexpect long time when lazy query enabled
[
https://issues.apache.org/jira/browse/KYLIN-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi updated KYLIN-4432:
-
Summary: Duplicated queries with sytax error take unexpect long time when
lazy query enabled (was: duplicated queries with sytax error take unexpect
long time when lazy query enabled)
> Duplicated queries with sytax error take unexpect long time when lazy query
> enabled
> ---
>
> Key: KYLIN-4432
> URL: https://issues.apache.org/jira/browse/KYLIN-4432
> Project: Kylin
> Issue Type: Bug
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.0
>
>
> Hi! Our Kylin server becomes unresponsive recently. All query threads were
> being TIMED_WAITING so that no more thread in the thread pool of Tomcat could
> response new requests. The server logs are full of "Duplicate SQL request"s,
> and many requests retrying for so long time:
> {code:java}
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head
> -3
> 2020-03-23 12:15:32,279 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,379 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,479 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail
> -3
> 2020-03-23 12:26:56,920 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,020 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,121 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> {code}
> This can be re-produced easily:
> * enable lazy query (KYLIN-2897) by setting environment variable
> kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
> * send the same query with syntax error many times in several minutes
> The first query will quickly response with error message, and the following
> queries will executing for long time and will repeatly output "Duplicated SQL
> request is running, waiting...".
> The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache
> indicates duplicated query will wait at most one minute with default setting
> of LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than
> that.
> {code:java}
> while (response.isRunning()) {
> // Wait at most one minute
> if (System.currentTimeMillis() - response.getLazyQueryStartTime() >=
> getConfig()
> .getLazyQueryWaitingTimeoutMilliSeconds()) {
> cache.evict(sqlRequest.getCacheKey());
> return null;
> }
> logger.info("Duplicated SQL request is running, waiting...");
> try {
> Thread.sleep(100L);
> } catch (InterruptedException e) {
> }
> wrapper = cache.get(sqlRequest.getCacheKey());
> if (wrapper == null) {
> return null;
> }
> response = (SQLResponse) wrapper.get();
> if (response == null) {
> return null;
> }
> }
> {code}
> After some work-around, I find that the dummy response status remains
> unchanged after execution with exception. So the 2nd query will wait
> LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the
> 1st query from cache, and put a new dummy response into the cache. And the
> 3rd query may use the new dummy response's start time to compute timeout, and
> will wait for another LazyQueryWaitingTimeoutMilliSeconds.
> Suppose we send _n_ identical bad queries to the server quickly. The last
> query may wait for nearly _(n-1)_ times of
> LazyQueryWaitingTimeoutMilliSeconds. That's why the Web container's thread
> pool was exhausted – user program uses bad queries to check availability of
> Kylin service and retries many times in a short time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114187#comment-17114187 ] ASF GitHub Bot commented on KYLIN-4509: --- guangxuCheng commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632762551 @shaofengshi yes，It is a duplicate of #1024，I'm sorry I didn't notice this PR before. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Affects Versions: v2.6.5, v3.0.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4496) Metric data missing
[ https://issues.apache.org/jira/browse/KYLIN-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114033#comment-17114033 ] ASF GitHub Bot commented on KYLIN-4496: --- shaofengshi commented on pull request #1210: URL: https://github.com/apache/kylin/pull/1210#issuecomment-632691337 +1 the FSDataOutputStream object should be closed on closing. @kyotoYaho yanghong, please review this as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Metric data missing > --- > > Key: KYLIN-4496 > URL: https://issues.apache.org/jira/browse/KYLIN-4496 > Project: Kylin > Issue Type: Bug > Components: Metrics >Reporter: tianhui >Assignee: tianhui >Priority: Critical > Fix For: v3.1.0 > > > Metric data in memory is never flushed into file system when kylin closing, > So there is some metric data missing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1226: KYLIN-4495 Support custom date format for partition date column
shaofengshi merged pull request #1226: URL: https://github.com/apache/kylin/pull/1226 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4495) Support custom date formats for partition date column
[ https://issues.apache.org/jira/browse/KYLIN-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114052#comment-17114052 ] ASF GitHub Bot commented on KYLIN-4495: --- shaofengshi merged pull request #1226: URL: https://github.com/apache/kylin/pull/1226 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support custom date formats for partition date column > - > > Key: KYLIN-4495 > URL: https://issues.apache.org/jira/browse/KYLIN-4495 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v3.1.0 > > > Support custom date formats for partition date column. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4495) Support custom date formats for partition date column
[ https://issues.apache.org/jira/browse/KYLIN-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114053#comment-17114053 ] ASF subversion and git services commented on KYLIN-4495: Commit ca5255c58e4dfff023ee54764601fed2f51a81cd in kylin's branch refs/heads/master from Zhichao Zhang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=ca5255c ] KYLIN-4495 Support custom date formats for partition date column Support custom date formats for partition date column. > Support custom date formats for partition date column > - > > Key: KYLIN-4495 > URL: https://issues.apache.org/jira/browse/KYLIN-4495 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v3.1.0 > > > Support custom date formats for partition date column. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4394) Upgrade dependency version for several CVEs
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4394. -- Resolution: Fixed > Upgrade dependency version for several CVEs > --- > > Key: KYLIN-4394 > URL: https://issues.apache.org/jira/browse/KYLIN-4394 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.1.0 > > Attachments: apache-kylin_CVE-report.md > > > I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 > *CVE ID:* > [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] > *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 > *CVE ID:* > [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], > [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], > [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], > [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], > [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], > [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] > *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* > [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], > > [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] > *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* > 1.4.198, 1.4.199, 1.4.200 > * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* > [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] > *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, > server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* > 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, > 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, > 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, > 28.2-android, 28.2-jre > * *Vulnerable Library Version:* org.apache.hive.hcatalog : > hive-hcatalog-core : 1.2.1 *CVE ID:* > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] > *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, > 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, > 3.1.1, 3.1.2 > * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 > *CVE ID:* > [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], > [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], > [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], > > [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] > *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 > * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE > ID:* > [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], > [CVE-2019-17196|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196] > *Import Path:* assembly/pom.xml, source-kafka/pom.xml, kylin-it/pom.xml, > stream-source-kafka/pom.xml, metrics-reporter-kafka/pom.xml *Suggested Safe > Versions:* 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0 > * *Vulnerable Library Version:* org.apache.hive : hive-jdbc : 1.2.1 *CVE > ID:* > [CVE-2016-3083|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3083], > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521], >
[jira] [Resolved] (KYLIN-4495) Support custom date formats for partition date column
[ https://issues.apache.org/jira/browse/KYLIN-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4495. -- Resolution: Fixed > Support custom date formats for partition date column > - > > Key: KYLIN-4495 > URL: https://issues.apache.org/jira/browse/KYLIN-4495 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v3.1.0 > > > Support custom date formats for partition date column. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1215: KYLIN-2971 Fix the wrong "Realization Names" and missing "Cuboid Ids" in logQuery when hit cache
shaofengshi merged pull request #1215: URL: https://github.com/apache/kylin/pull/1215 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114072#comment-17114072
]
ASF GitHub Bot commented on KYLIN-2971:
---
shaofengshi merged pull request #1215:
URL: https://github.com/apache/kylin/pull/1215
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhichao Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> # The value of "Realization Names" in logQuery is wrong when query two
> different sqls within the same thread and second sql hits cache:
> Example:
> 1). query Q1 hit project P1 and cube C1;
> 2). query Q2 hit project P2 and cube C2 in the same thread with Q1;
> 3). Q1 comes again and hits cache, it will show project P1 and
> {color:#f79232}cube C2{color}. However, it should be {color:#f79232}cube
> C1{color}.
> # Missing "Cuboid Ids" in logQuery when hit cache in a new thread which does
> not have OLAPContext;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114073#comment-17114073
]
ASF subversion and git services commented on KYLIN-2971:
Commit b8d55d08301779885dd6418651c979ccaaaf6165 in kylin's branch
refs/heads/master from Zhichao Zhang
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b8d55d0 ]
KYLIN-2971 Fix the wrong "Realization Names" and missing "Cuboid Ids" in
logQuery when hit cache
Problems:
1. The value of "Realization Names" in logQuery is wrong when query two
different sqls within the same thread and second sql hits cache:
Example:
1). query Q1 hit project P1 and cube C1;
2). query Q2 hit project P2 and cube C2 in the same thread with Q1;
3). Q1 comes again and hits cache, it will show project P1 and cube C2.
However, it should be cube C1.
2. Missing "Cuboid Ids" in logQuery when hit cache in a new thread which does
not have OLAPContext;
Solutions:
1. Call 'OLAPContext.clearThreadLocalContexts()' before a query starts;
2. Get "Cuboid Ids" from SQLResponse when
"OLAPContext.getThreadLocalContexts()" is null;
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhichao Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> # The value of "Realization Names" in logQuery is wrong when query two
> different sqls within the same thread and second sql hits cache:
> Example:
> 1). query Q1 hit project P1 and cube C1;
> 2). query Q2 hit project P2 and cube C2 in the same thread with Q1;
> 3). Q1 comes again and hits cache, it will show project P1 and
> {color:#f79232}cube C2{color}. However, it should be {color:#f79232}cube
> C1{color}.
> # Missing "Cuboid Ids" in logQuery when hit cache in a new thread which does
> not have OLAPContext;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] kyotoYaho commented on pull request #1215: KYLIN-2971 Fix the wrong "Realization Names" and missing "Cuboid Ids" in logQuery when hit cache
kyotoYaho commented on pull request #1215: URL: https://github.com/apache/kylin/pull/1215#issuecomment-632721375 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114090#comment-17114090
]
ASF GitHub Bot commented on KYLIN-2971:
---
kyotoYaho commented on pull request #1215:
URL: https://github.com/apache/kylin/pull/1215#issuecomment-632721375
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhichao Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> # The value of "Realization Names" in logQuery is wrong when query two
> different sqls within the same thread and second sql hits cache:
> Example:
> 1). query Q1 hit project P1 and cube C1;
> 2). query Q2 hit project P2 and cube C2 in the same thread with Q1;
> 3). Q1 comes again and hits cache, it will show project P1 and
> {color:#f79232}cube C2{color}. However, it should be {color:#f79232}cube
> C1{color}.
> # Missing "Cuboid Ids" in logQuery when hit cache in a new thread which does
> not have OLAPContext;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114095#comment-17114095 ] ASF GitHub Bot commented on KYLIN-4508: --- shaofengshi commented on pull request #1228: URL: https://github.com/apache/kylin/pull/1228#issuecomment-632722692 Merged by manual. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114096#comment-17114096 ] ASF GitHub Bot commented on KYLIN-4508: --- shaofengshi closed pull request #1228: URL: https://github.com/apache/kylin/pull/1228 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4508. -- Resolution: Fixed > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4508) Add more unit tests to improve test coverage
[ https://issues.apache.org/jira/browse/KYLIN-4508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi updated KYLIN-4508: - Component/s: Tools, Build and Test Fix Version/s: v3.1.0 > Add more unit tests to improve test coverage > > > Key: KYLIN-4508 > URL: https://issues.apache.org/jira/browse/KYLIN-4508 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong >Priority: Major > Fix For: v3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi closed pull request #1228: KYLIN-4508 Add more unit tests to improve test coverage
shaofengshi closed pull request #1228: URL: https://github.com/apache/kylin/pull/1228 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (KYLIN-4510) Automatically refresh the page after reload table
[ https://issues.apache.org/jira/browse/KYLIN-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4510. -- Resolution: Fixed > Automatically refresh the page after reload table > - > > Key: KYLIN-4510 > URL: https://issues.apache.org/jira/browse/KYLIN-4510 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v3.1.0 > > > Now after reload table, we need to manually refresh the page to get the > latest metadata information, which is unfriendly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4510) Automatically refresh the page after reload table
[ https://issues.apache.org/jira/browse/KYLIN-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114135#comment-17114135 ] ASF subversion and git services commented on KYLIN-4510: Commit fdf0d8f3e5d1574123f8c293f861a58bb35e0f06 in kylin's branch refs/heads/master from Guangxu Cheng [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=fdf0d8f ] KYLIN-4510 Automatically refresh the page after reload table > Automatically refresh the page after reload table > - > > Key: KYLIN-4510 > URL: https://issues.apache.org/jira/browse/KYLIN-4510 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v3.1.0 > > > Now after reload table, we need to manually refresh the page to get the > latest metadata information, which is unfriendly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4510) Automatically refresh the page after reload table
[ https://issues.apache.org/jira/browse/KYLIN-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114136#comment-17114136 ] ASF GitHub Bot commented on KYLIN-4510: --- shaofengshi merged pull request #1224: URL: https://github.com/apache/kylin/pull/1224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Automatically refresh the page after reload table > - > > Key: KYLIN-4510 > URL: https://issues.apache.org/jira/browse/KYLIN-4510 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v3.1.0 > > > Now after reload table, we need to manually refresh the page to get the > latest metadata information, which is unfriendly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1224: KYLIN-4510 Automatically refresh the page after reload table
shaofengshi merged pull request #1224: URL: https://github.com/apache/kylin/pull/1224 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-4510) Automatically refresh the page after reload table
[ https://issues.apache.org/jira/browse/KYLIN-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi updated KYLIN-4510: - Fix Version/s: v3.1.0 > Automatically refresh the page after reload table > - > > Key: KYLIN-4510 > URL: https://issues.apache.org/jira/browse/KYLIN-4510 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v3.1.0 > > > Now after reload table, we need to manually refresh the page to get the > latest metadata information, which is unfriendly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi commented on pull request #1210: KYLIN-4496: Metric data missing
shaofengshi commented on pull request #1210: URL: https://github.com/apache/kylin/pull/1210#issuecomment-632691337 +1 the FSDataOutputStream object should be closed on closing. @kyotoYaho yanghong, please review this as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4496) Metric data missing
[ https://issues.apache.org/jira/browse/KYLIN-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114088#comment-17114088 ] ASF GitHub Bot commented on KYLIN-4496: --- kyotoYaho commented on pull request #1210: URL: https://github.com/apache/kylin/pull/1210#issuecomment-632720447 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Metric data missing > --- > > Key: KYLIN-4496 > URL: https://issues.apache.org/jira/browse/KYLIN-4496 > Project: Kylin > Issue Type: Bug > Components: Metrics >Reporter: tianhui >Assignee: tianhui >Priority: Critical > Fix For: v3.1.0 > > > Metric data in memory is never flushed into file system when kylin closing, > So there is some metric data missing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] kyotoYaho commented on pull request #1210: KYLIN-4496: Metric data missing
kyotoYaho commented on pull request #1210: URL: https://github.com/apache/kylin/pull/1210#issuecomment-632720447 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4394) Upgrade dependency version for several CVEs
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114038#comment-17114038 ] ASF GitHub Bot commented on KYLIN-4394: --- shaofengshi commented on pull request #1148: URL: https://github.com/apache/kylin/pull/1148#issuecomment-632692480 merged manually, so close this pr. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade dependency version for several CVEs > --- > > Key: KYLIN-4394 > URL: https://issues.apache.org/jira/browse/KYLIN-4394 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.1.0 > > Attachments: apache-kylin_CVE-report.md > > > I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 > *CVE ID:* > [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] > *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 > *CVE ID:* > [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], > [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], > [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], > [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], > [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], > [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] > *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* > [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], > > [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] > *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* > 1.4.198, 1.4.199, 1.4.200 > * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* > [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] > *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, > server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* > 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, > 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, > 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, > 28.2-android, 28.2-jre > * *Vulnerable Library Version:* org.apache.hive.hcatalog : > hive-hcatalog-core : 1.2.1 *CVE ID:* > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] > *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, > 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, > 3.1.1, 3.1.2 > * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 > *CVE ID:* > [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], > [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], > [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], > > [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] > *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 > * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE > ID:* > [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], >
[jira] [Commented] (KYLIN-4394) Upgrade dependency version for several CVEs
[ https://issues.apache.org/jira/browse/KYLIN-4394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114037#comment-17114037 ] ASF GitHub Bot commented on KYLIN-4394: --- shaofengshi closed pull request #1148: URL: https://github.com/apache/kylin/pull/1148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade dependency version for several CVEs > --- > > Key: KYLIN-4394 > URL: https://issues.apache.org/jira/browse/KYLIN-4394 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: XuCongying >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.1.0 > > Attachments: apache-kylin_CVE-report.md > > > I noticed some of your libraries contained CVEs. I suggest updating their > versions to increase the security of your project. The following is a > detailed content. > * *Vulnerable Library Version:* org.scala-lang : scala-compiler : 2.11.0 > *CVE ID:* > [CVE-2017-15288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-15288] > *Import Path:* engine-flink/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml > *Suggested Safe Versions:* 2.11.12, 2.12.10, 2.12.4, 2.12.5, 2.12.6, 2.12.7, > 2.12.8, 2.12.9, 2.13.0, 2.13.0-M1, 2.13.0-M2, 2.13.0-M3, 2.13.0-M3-f73b161, > 2.13.0-M4, 2.13.0-M4-pre-20d3c21, 2.13.0-M5, 2.13.0-M5-1775dba, > 2.13.0-M5-5eef812, 2.13.0-M5-6e0cba7, 2.13.0-RC1, 2.13.0-RC2, 2.13.0-RC3, > 2.13.1 > * *Vulnerable Library Version:* org.apache.tomcat : tomcat-catalina : 7.0.91 > *CVE ID:* > [CVE-2016-8735|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8735], > [CVE-2019-0232|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-0232], > [CVE-2016-6794|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6794], > [CVE-2016-6816|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816], > [CVE-2016-8745|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-8745], > [CVE-2019-17563|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17563] > *Import Path:* tomcat-ext/pom.xml, server/pom.xml, server-base/pom.xml > *Suggested Safe Versions:* 10.0.0-M1, 7.0.100, 9.0.30, 9.0.31 > * *Vulnerable Library Version:* com.h2database : h2 : 1.4.196 *CVE ID:* > [CVE-2018-10054|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10054], > > [CVE-2018-14335|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-14335] > *Import Path:* server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, > source-hive/pom.xml, datasource-sdk/pom.xml *Suggested Safe Versions:* > 1.4.198, 1.4.199, 1.4.200 > * *Vulnerable Library Version:* com.google.guava : guava : 14.0 *CVE ID:* > [CVE-2018-10237|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-10237] > *Import Path:* core-storage/pom.xml, stream-receiver/pom.xml, > server/pom.xml, core-cube/pom.xml, core-metadata/pom.xml, jdbc/pom.xml, > tool-assembly/pom.xml, core-metrics/pom.xml *Suggested Safe Versions:* > 24.1.1-android, 24.1.1-jre, 25.0-android, 25.0-jre, 25.1-android, 25.1-jre, > 26.0-android, 26.0-jre, 27.0-android, 27.0-jre, 27.0.1-android, 27.0.1-jre, > 27.1-android, 27.1-jre, 28.0-android, 28.0-jre, 28.1-android, 28.1-jre, > 28.2-android, 28.2-jre > * *Vulnerable Library Version:* org.apache.hive.hcatalog : > hive-hcatalog-core : 1.2.1 *CVE ID:* > [CVE-2015-7521|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-7521] > *Import Path:* metrics-reporter-hive/pom.xml, assembly/pom.xml, > server/pom.xml, kylin-it/pom.xml, source-jdbc/pom.xml, source-hive/pom.xml, > server-base/pom.xml *Suggested Safe Versions:* 1.2.2, 2.0.0, 2.0.1, 2.1.0, > 2.1.1, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0.0, 3.1.0, > 3.1.1, 3.1.2 > * *Vulnerable Library Version:* org.apache.spark : spark-core_2.11 : 2.3.2 > *CVE ID:* > [CVE-2017-7678|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7678], > [CVE-2018-3826|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3826], > [CVE-2018-11770|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-11770], > > [CVE-2019-10099|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10099] > *Import Path:* server/pom.xml, kylin-it/pom.xml, engine-spark/pom.xml, > storage-hbase/pom.xml *Suggested Safe Versions:* 2.4.5 > * *Vulnerable Library Version:* org.apache.kafka : kafka_2.11 : 1.0.0 *CVE > ID:* > [CVE-2018-1288|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1288], > [CVE-2019-17196|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17196] > *Import Path:* assembly/pom.xml, source-kafka/pom.xml,
[GitHub] [kylin] shaofengshi closed pull request #1148: KYLIN-4394 Update kylin dependency
shaofengshi closed pull request #1148: URL: https://github.com/apache/kylin/pull/1148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] shaofengshi commented on pull request #1148: KYLIN-4394 Update kylin dependency
shaofengshi commented on pull request #1148: URL: https://github.com/apache/kylin/pull/1148#issuecomment-632692480 merged manually, so close this pr. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (KYLIN-2971) Fix the wrong "Realization Names" in logQuery when hit cache
[
https://issues.apache.org/jira/browse/KYLIN-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi resolved KYLIN-2971.
--
Resolution: Fixed
> Fix the wrong "Realization Names" in logQuery when hit cache
>
>
> Key: KYLIN-2971
> URL: https://issues.apache.org/jira/browse/KYLIN-2971
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhichao Zhang
>Priority: Minor
> Fix For: v3.1.0
>
> Attachments: APACHE-KYLIN-2971.patch
>
>
> # The value of "Realization Names" in logQuery is wrong when query two
> different sqls within the same thread and second sql hits cache:
> Example:
> 1). query Q1 hit project P1 and cube C1;
> 2). query Q2 hit project P2 and cube C2 in the same thread with Q1;
> 3). Q1 comes again and hits cache, it will show project P1 and
> {color:#f79232}cube C2{color}. However, it should be {color:#f79232}cube
> C1{color}.
> # Missing "Cuboid Ids" in logQuery when hit cache in a new thread which does
> not have OLAPContext;
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4432) duplicated queries with sytax error take unexpect long time when lazy query enabled
[
https://issues.apache.org/jira/browse/KYLIN-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114134#comment-17114134
]
ASF subversion and git services commented on KYLIN-4432:
Commit 6342bc5f191f88b9856f15c23b15d29548e0ecba in kylin's branch
refs/heads/master from Congling Xia
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6342bc5 ]
KYLIN-4432 duplicated queries with sytax error take unexpect long time when
lazy query enabled (#1167)
* KYLIN-4432 duplicated queries with sytax error take unexpect long time when
lazy query enabled
> duplicated queries with sytax error take unexpect long time when lazy query
> enabled
> ---
>
> Key: KYLIN-4432
> URL: https://issues.apache.org/jira/browse/KYLIN-4432
> Project: Kylin
> Issue Type: Bug
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.0
>
>
> Hi! Our Kylin server becomes unresponsive recently. All query threads were
> being TIMED_WAITING so that no more thread in the thread pool of Tomcat could
> response new requests. The server logs are full of "Duplicate SQL request"s,
> and many requests retrying for so long time:
> {code:java}
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head
> -3
> 2020-03-23 12:15:32,279 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,379 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,479 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail
> -3
> 2020-03-23 12:26:56,920 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,020 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,121 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> {code}
> This can be re-produced easily:
> * enable lazy query (KYLIN-2897) by setting environment variable
> kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
> * send the same query with syntax error many times in several minutes
> The first query will quickly response with error message, and the following
> queries will executing for long time and will repeatly output "Duplicated SQL
> request is running, waiting...".
> The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache
> indicates duplicated query will wait at most one minute with default setting
> of LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than
> that.
> {code:java}
> while (response.isRunning()) {
> // Wait at most one minute
> if (System.currentTimeMillis() - response.getLazyQueryStartTime() >=
> getConfig()
> .getLazyQueryWaitingTimeoutMilliSeconds()) {
> cache.evict(sqlRequest.getCacheKey());
> return null;
> }
> logger.info("Duplicated SQL request is running, waiting...");
> try {
> Thread.sleep(100L);
> } catch (InterruptedException e) {
> }
> wrapper = cache.get(sqlRequest.getCacheKey());
> if (wrapper == null) {
> return null;
> }
> response = (SQLResponse) wrapper.get();
> if (response == null) {
> return null;
> }
> }
> {code}
> After some work-around, I find that the dummy response status remains
> unchanged after execution with exception. So the 2nd query will wait
> LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the
> 1st query from cache, and put a new dummy response into the cache. And the
> 3rd query may use the new dummy response's start time to compute timeout, and
> will wait for another LazyQueryWaitingTimeoutMilliSeconds.
> Suppose we send _n_ identical bad queries to the server quickly. The last
> query may wait for nearly _(n-1)_ times of
> LazyQueryWaitingTimeoutMilliSeconds. That's why the Web container's thread
> pool was exhausted – user program uses bad queries to check availability of
> Kylin service and retries many times in a short time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4432) duplicated queries with sytax error take unexpect long time when lazy query enabled
[
https://issues.apache.org/jira/browse/KYLIN-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114133#comment-17114133
]
ASF subversion and git services commented on KYLIN-4432:
Commit 6342bc5f191f88b9856f15c23b15d29548e0ecba in kylin's branch
refs/heads/master from Congling Xia
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=6342bc5 ]
KYLIN-4432 duplicated queries with sytax error take unexpect long time when
lazy query enabled (#1167)
* KYLIN-4432 duplicated queries with sytax error take unexpect long time when
lazy query enabled
> duplicated queries with sytax error take unexpect long time when lazy query
> enabled
> ---
>
> Key: KYLIN-4432
> URL: https://issues.apache.org/jira/browse/KYLIN-4432
> Project: Kylin
> Issue Type: Bug
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.0
>
>
> Hi! Our Kylin server becomes unresponsive recently. All query threads were
> being TIMED_WAITING so that no more thread in the thread pool of Tomcat could
> response new requests. The server logs are full of "Duplicate SQL request"s,
> and many requests retrying for so long time:
> {code:java}
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head
> -3
> 2020-03-23 12:15:32,279 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,379 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,479 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail
> -3
> 2020-03-23 12:26:56,920 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,020 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,121 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> {code}
> This can be re-produced easily:
> * enable lazy query (KYLIN-2897) by setting environment variable
> kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
> * send the same query with syntax error many times in several minutes
> The first query will quickly response with error message, and the following
> queries will executing for long time and will repeatly output "Duplicated SQL
> request is running, waiting...".
> The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache
> indicates duplicated query will wait at most one minute with default setting
> of LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than
> that.
> {code:java}
> while (response.isRunning()) {
> // Wait at most one minute
> if (System.currentTimeMillis() - response.getLazyQueryStartTime() >=
> getConfig()
> .getLazyQueryWaitingTimeoutMilliSeconds()) {
> cache.evict(sqlRequest.getCacheKey());
> return null;
> }
> logger.info("Duplicated SQL request is running, waiting...");
> try {
> Thread.sleep(100L);
> } catch (InterruptedException e) {
> }
> wrapper = cache.get(sqlRequest.getCacheKey());
> if (wrapper == null) {
> return null;
> }
> response = (SQLResponse) wrapper.get();
> if (response == null) {
> return null;
> }
> }
> {code}
> After some work-around, I find that the dummy response status remains
> unchanged after execution with exception. So the 2nd query will wait
> LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the
> 1st query from cache, and put a new dummy response into the cache. And the
> 3rd query may use the new dummy response's start time to compute timeout, and
> will wait for another LazyQueryWaitingTimeoutMilliSeconds.
> Suppose we send _n_ identical bad queries to the server quickly. The last
> query may wait for nearly _(n-1)_ times of
> LazyQueryWaitingTimeoutMilliSeconds. That's why the Web container's thread
> pool was exhausted – user program uses bad queries to check availability of
> Kylin service and retries many times in a short time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4432) duplicated queries with sytax error take unexpect long time when lazy query enabled
[
https://issues.apache.org/jira/browse/KYLIN-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shao Feng Shi resolved KYLIN-4432.
--
Resolution: Fixed
> duplicated queries with sytax error take unexpect long time when lazy query
> enabled
> ---
>
> Key: KYLIN-4432
> URL: https://issues.apache.org/jira/browse/KYLIN-4432
> Project: Kylin
> Issue Type: Bug
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.0
>
>
> Hi! Our Kylin server becomes unresponsive recently. All query threads were
> being TIMED_WAITING so that no more thread in the thread pool of Tomcat could
> response new requests. The server logs are full of "Duplicate SQL request"s,
> and many requests retrying for so long time:
> {code:java}
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head
> -3
> 2020-03-23 12:15:32,279 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,379 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,479 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail
> -3
> 2020-03-23 12:26:56,920 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,020 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,121 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> {code}
> This can be re-produced easily:
> * enable lazy query (KYLIN-2897) by setting environment variable
> kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
> * send the same query with syntax error many times in several minutes
> The first query will quickly response with error message, and the following
> queries will executing for long time and will repeatly output "Duplicated SQL
> request is running, waiting...".
> The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache
> indicates duplicated query will wait at most one minute with default setting
> of LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than
> that.
> {code:java}
> while (response.isRunning()) {
> // Wait at most one minute
> if (System.currentTimeMillis() - response.getLazyQueryStartTime() >=
> getConfig()
> .getLazyQueryWaitingTimeoutMilliSeconds()) {
> cache.evict(sqlRequest.getCacheKey());
> return null;
> }
> logger.info("Duplicated SQL request is running, waiting...");
> try {
> Thread.sleep(100L);
> } catch (InterruptedException e) {
> }
> wrapper = cache.get(sqlRequest.getCacheKey());
> if (wrapper == null) {
> return null;
> }
> response = (SQLResponse) wrapper.get();
> if (response == null) {
> return null;
> }
> }
> {code}
> After some work-around, I find that the dummy response status remains
> unchanged after execution with exception. So the 2nd query will wait
> LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the
> 1st query from cache, and put a new dummy response into the cache. And the
> 3rd query may use the new dummy response's start time to compute timeout, and
> will wait for another LazyQueryWaitingTimeoutMilliSeconds.
> Suppose we send _n_ identical bad queries to the server quickly. The last
> query may wait for nearly _(n-1)_ times of
> LazyQueryWaitingTimeoutMilliSeconds. That's why the Web container's thread
> pool was exhausted – user program uses bad queries to check availability of
> Kylin service and retries many times in a short time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1167: KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled
shaofengshi merged pull request #1167: URL: https://github.com/apache/kylin/pull/1167 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4432) duplicated queries with sytax error take unexpect long time when lazy query enabled
[
https://issues.apache.org/jira/browse/KYLIN-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114132#comment-17114132
]
ASF GitHub Bot commented on KYLIN-4432:
---
shaofengshi merged pull request #1167:
URL: https://github.com/apache/kylin/pull/1167
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> duplicated queries with sytax error take unexpect long time when lazy query
> enabled
> ---
>
> Key: KYLIN-4432
> URL: https://issues.apache.org/jira/browse/KYLIN-4432
> Project: Kylin
> Issue Type: Bug
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.0
>
>
> Hi! Our Kylin server becomes unresponsive recently. All query threads were
> being TIMED_WAITING so that no more thread in the thread pool of Tomcat could
> response new requests. The server logs are full of "Duplicate SQL request"s,
> and many requests retrying for so long time:
> {code:java}
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | head
> -3
> 2020-03-23 12:15:32,279 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,379 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:15:32,479 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> $ cat kylin.log.25 | grep 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991 | tail
> -3
> 2020-03-23 12:26:56,920 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,020 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> 2020-03-23 12:26:57,121 INFO [Query
> 14db5fa0-78e0-4ca6-5d04-92995b2b8367-3133991] service.QueryService:571 :
> Duplicated SQL request is running, waiting...
> {code}
> This can be re-produced easily:
> * enable lazy query (KYLIN-2897) by setting environment variable
> kylin.query.cache-enabled=true and kylin.query.lazy-query-enabled=true
> * send the same query with syntax error many times in several minutes
> The first query will quickly response with error message, and the following
> queries will executing for long time and will repeatly output "Duplicated SQL
> request is running, waiting...".
> The code in org.apache.kylin.rest.service.QueryService#searchQueryInCache
> indicates duplicated query will wait at most one minute with default setting
> of LazyQueryWaitingTimeoutMilliSeconds but in fact it waits quite longer than
> that.
> {code:java}
> while (response.isRunning()) {
> // Wait at most one minute
> if (System.currentTimeMillis() - response.getLazyQueryStartTime() >=
> getConfig()
> .getLazyQueryWaitingTimeoutMilliSeconds()) {
> cache.evict(sqlRequest.getCacheKey());
> return null;
> }
> logger.info("Duplicated SQL request is running, waiting...");
> try {
> Thread.sleep(100L);
> } catch (InterruptedException e) {
> }
> wrapper = cache.get(sqlRequest.getCacheKey());
> if (wrapper == null) {
> return null;
> }
> response = (SQLResponse) wrapper.get();
> if (response == null) {
> return null;
> }
> }
> {code}
> After some work-around, I find that the dummy response status remains
> unchanged after execution with exception. So the 2nd query will wait
> LazyQueryWaitingTimeoutMilliSeconds, evict the dummy response created by the
> 1st query from cache, and put a new dummy response into the cache. And the
> 3rd query may use the new dummy response's start time to compute timeout, and
> will wait for another LazyQueryWaitingTimeoutMilliSeconds.
> Suppose we send _n_ identical bad queries to the server quickly. The last
> query may wait for nearly _(n-1)_ times of
> LazyQueryWaitingTimeoutMilliSeconds. That's why the Web container's thread
> pool was exhausted – user program uses bad queries to check availability of
> Kylin service and retries many times in a short time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] shaofengshi commented on pull request #1223: KYLIN-4509 get hive table rows from metadata when using beeline
shaofengshi commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632746333 Hi Guangxu, It seems this is very similar with https://github.com/apache/kylin/pull/1024 Could you please check that? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114140#comment-17114140 ] ASF GitHub Bot commented on KYLIN-4509: --- shaofengshi commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632746333 Hi Guangxu, It seems this is very similar with https://github.com/apache/kylin/pull/1024 Could you please check that? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Affects Versions: v2.6.5, v3.0.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4515) could not send mail on ssl port
[
https://issues.apache.org/jira/browse/KYLIN-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
CHEN Qing updated KYLIN-4515:
-
External issue URL: https://github.com/apache/kylin/pull/1214
Labels: pull-request-available (was: )
> could not send mail on ssl port
> ---
>
> Key: KYLIN-4515
> URL: https://issues.apache.org/jira/browse/KYLIN-4515
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v3.0.1
>Reporter: CHEN Qing
>Priority: Minor
> Labels: pull-request-available
> Fix For: v3.0.1
>
>
> Here is context of this
> problem：[https://www.mail-archive.com/user@kylin.apache.org/msg04009.html]
> When I set {{starttlsEnabled}} to true, and specify a ssl port as 465, the
> email could not be sent by this ssl port. It still use 25 default port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] shaofengshi commented on a change in pull request #1225: KYLIN-4511 Support project admin user to calculate column cardinality…
shaofengshi commented on a change in pull request #1225:
URL: https://github.com/apache/kylin/pull/1225#discussion_r429290527
##
File path: webapp/app/js/controllers/sourceMeta.js
##
@@ -131,6 +131,39 @@ KylinApp
});
};
+$scope.calCardinality = function (tableName) {
+ SweetAlert.swal({
+title: "",
+text: "Are you sure to recalculate column cardinality?",
+showCancelButton: true,
+confirmButtonColor: '#DD6B55',
+confirmButtonText: "Yes",
+cancelButtonText: "No",
+closeOnConfirm: true
+ }, function (isConfirm) {
+if (isConfirm) {
+ if (!$scope.projectModel.selectedProject) {
+SweetAlert.swal('', 'Please choose your project first!.', 'info');
+return;
+ }
+ loadingRequest.show();
+ TableService.genCardinality({tableName: tableName, pro:
$scope.projectModel.selectedProject}, {}, function () {
+loadingRequest.hide();
+MessageBox.successNotify('Cardinality job was calculated
successfully. Please wait a moment...');
Review comment:
Some suggestions on the sentence: 'Cardinality job has been submitted
successfully. Please wait a while to get the numbers.'
##
File path: webapp/app/js/controllers/sourceMeta.js
##
@@ -131,6 +131,39 @@ KylinApp
});
};
+$scope.calCardinality = function (tableName) {
+ SweetAlert.swal({
+title: "",
+text: "Are you sure to recalculate column cardinality?",
+showCancelButton: true,
+confirmButtonColor: '#DD6B55',
+confirmButtonText: "Yes",
+cancelButtonText: "No",
+closeOnConfirm: true
+ }, function (isConfirm) {
+if (isConfirm) {
+ if (!$scope.projectModel.selectedProject) {
+SweetAlert.swal('', 'Please choose your project first!.', 'info');
Review comment:
The message should be consistent with other parts in below: "Please
select a project." And, please avoid using "!" in the sentence as its mood is
too strong.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KYLIN-4511) Support project admin user to calculate column cardinality by web
[
https://issues.apache.org/jira/browse/KYLIN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114114#comment-17114114
]
ASF GitHub Bot commented on KYLIN-4511:
---
shaofengshi commented on a change in pull request #1225:
URL: https://github.com/apache/kylin/pull/1225#discussion_r429290527
##
File path: webapp/app/js/controllers/sourceMeta.js
##
@@ -131,6 +131,39 @@ KylinApp
});
};
+$scope.calCardinality = function (tableName) {
+ SweetAlert.swal({
+title: "",
+text: "Are you sure to recalculate column cardinality?",
+showCancelButton: true,
+confirmButtonColor: '#DD6B55',
+confirmButtonText: "Yes",
+cancelButtonText: "No",
+closeOnConfirm: true
+ }, function (isConfirm) {
+if (isConfirm) {
+ if (!$scope.projectModel.selectedProject) {
+SweetAlert.swal('', 'Please choose your project first!.', 'info');
+return;
+ }
+ loadingRequest.show();
+ TableService.genCardinality({tableName: tableName, pro:
$scope.projectModel.selectedProject}, {}, function () {
+loadingRequest.hide();
+MessageBox.successNotify('Cardinality job was calculated
successfully. Please wait a moment...');
Review comment:
Some suggestions on the sentence: 'Cardinality job has been submitted
successfully. Please wait a while to get the numbers.'
##
File path: webapp/app/js/controllers/sourceMeta.js
##
@@ -131,6 +131,39 @@ KylinApp
});
};
+$scope.calCardinality = function (tableName) {
+ SweetAlert.swal({
+title: "",
+text: "Are you sure to recalculate column cardinality?",
+showCancelButton: true,
+confirmButtonColor: '#DD6B55',
+confirmButtonText: "Yes",
+cancelButtonText: "No",
+closeOnConfirm: true
+ }, function (isConfirm) {
+if (isConfirm) {
+ if (!$scope.projectModel.selectedProject) {
+SweetAlert.swal('', 'Please choose your project first!.', 'info');
Review comment:
The message should be consistent with other parts in below: "Please
select a project." And, please avoid using "!" in the sentence as its mood is
too strong.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support project admin user to calculate column cardinality by web
> -
>
> Key: KYLIN-4511
> URL: https://issues.apache.org/jira/browse/KYLIN-4511
> Project: Kylin
> Issue Type: Improvement
> Components: Web
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: image-2020-05-21-13-56-19-962.png,
> image-2020-05-21-13-57-29-350.png
>
>
> Now only the administrator of kylin can recalculate the cardinality of the
> hive table through the admin web, but the project administrator cannot.
> Add an entry to calculate cardinality on the table detail web
> # add calculate cardinality button
> !image-2020-05-21-13-56-19-962.png|width=902,height=303!
> # confirm !image-2020-05-21-13-57-29-350.png|width=901,height=280!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] guangxuCheng commented on pull request #1223: KYLIN-4509 get hive table rows from metadata when using beeline
guangxuCheng commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632762551 @shaofengshi yes，It is a duplicate of #1024，I'm sorry I didn't notice this PR before. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4315) Use metadata numRows in beeline client for quick row counting
[ https://issues.apache.org/jira/browse/KYLIN-4315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114469#comment-17114469 ] ASF subversion and git services commented on KYLIN-4315: Commit 24f0063daacb6732aa06f5abbe6d198f570ecf95 in kylin's branch refs/heads/master from xiacongling [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=24f0063 ] KYLIN-4315 use metadata numRows in beeline client for quick row counting > Use metadata numRows in beeline client for quick row counting > - > > Key: KYLIN-4315 > URL: https://issues.apache.org/jira/browse/KYLIN-4315 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Congling Xia >Assignee: Congling Xia >Priority: Major > Fix For: v3.1.0 > > > Hi, I find that in `BeelineHiveClient`, method `getHiveTableRows` uses > "select count(*) from " for table row counting. The method is > invoked in flat intermediate table redistribution step in cube building. > This stats can be loaded in metastore. It costs much less time than scanning > all rows in Hive table. Since intermediate tables are created and inserted by > Kylin, statistics will be automatically calculated and stored in metastore > when > `[hive.stats.autogather|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.stats.autogather]` > is enabled (which is the default setting for Hive). > ref Hive wiki for more detail about `numRows` stats: > [https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables%E2%80%93ANALYZE] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi closed pull request #1223: KYLIN-4509 get hive table rows from metadata when using beeline
shaofengshi closed pull request #1223: URL: https://github.com/apache/kylin/pull/1223 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (KYLIN-4315) Use metadata numRows in beeline client for quick row counting
[ https://issues.apache.org/jira/browse/KYLIN-4315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4315. -- Resolution: Fixed > Use metadata numRows in beeline client for quick row counting > - > > Key: KYLIN-4315 > URL: https://issues.apache.org/jira/browse/KYLIN-4315 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Congling Xia >Assignee: Congling Xia >Priority: Major > Fix For: v3.1.0 > > > Hi, I find that in `BeelineHiveClient`, method `getHiveTableRows` uses > "select count(*) from " for table row counting. The method is > invoked in flat intermediate table redistribution step in cube building. > This stats can be loaded in metastore. It costs much less time than scanning > all rows in Hive table. Since intermediate tables are created and inserted by > Kylin, statistics will be automatically calculated and stored in metastore > when > `[hive.stats.autogather|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.stats.autogather]` > is enabled (which is the default setting for Hive). > ref Hive wiki for more detail about `numRows` stats: > [https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables%E2%80%93ANALYZE] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114471#comment-17114471 ] ASF GitHub Bot commented on KYLIN-4509: --- shaofengshi closed pull request #1223: URL: https://github.com/apache/kylin/pull/1223 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Affects Versions: v2.6.5, v3.0.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114470#comment-17114470 ] ASF GitHub Bot commented on KYLIN-4509: --- shaofengshi commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632961965 As it is similar with https://github.com/apache/kylin/pull/1024, I have merged #1024, and close this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Affects Versions: v2.6.5, v3.0.1 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] shaofengshi merged pull request #1024: KYLIN-4315 use metadata numRows in beeline client for quick row counting
shaofengshi merged pull request #1024: URL: https://github.com/apache/kylin/pull/1024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] shaofengshi commented on pull request #1223: KYLIN-4509 get hive table rows from metadata when using beeline
shaofengshi commented on pull request #1223: URL: https://github.com/apache/kylin/pull/1223#issuecomment-632961965 As it is similar with https://github.com/apache/kylin/pull/1024, I have merged #1024, and close this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4315) Use metadata numRows in beeline client for quick row counting
[ https://issues.apache.org/jira/browse/KYLIN-4315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114468#comment-17114468 ] ASF GitHub Bot commented on KYLIN-4315: --- shaofengshi merged pull request #1024: URL: https://github.com/apache/kylin/pull/1024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Use metadata numRows in beeline client for quick row counting > - > > Key: KYLIN-4315 > URL: https://issues.apache.org/jira/browse/KYLIN-4315 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Congling Xia >Assignee: Congling Xia >Priority: Major > Fix For: v3.1.0 > > > Hi, I find that in `BeelineHiveClient`, method `getHiveTableRows` uses > "select count(*) from " for table row counting. The method is > invoked in flat intermediate table redistribution step in cube building. > This stats can be loaded in metastore. It costs much less time than scanning > all rows in Hive table. Since intermediate tables are created and inserted by > Kylin, statistics will be automatically calculated and stored in metastore > when > `[hive.stats.autogather|https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.stats.autogather]` > is enabled (which is the default setting for Hive). > ref Hive wiki for more detail about `numRows` stats: > [https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables%E2%80%93ANALYZE] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi resolved KYLIN-4509. -- Resolution: Duplicate > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi closed KYLIN-4509. Duplicate with KYLIN-4315 > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4509) get hive table rows from metadata when using beeline
[ https://issues.apache.org/jira/browse/KYLIN-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shao Feng Shi updated KYLIN-4509: - Affects Version/s: (was: v3.0.1) (was: v2.6.5) > get hive table rows from metadata when using beeline > > > Key: KYLIN-4509 > URL: https://issues.apache.org/jira/browse/KYLIN-4509 > Project: Kylin > Issue Type: Improvement >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > > Now by running SQL "select count(*) from table" to get the number of the hive > table records, the performance is relatively poor and will consume a lot of > resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4511) Support project admin user to calculate column cardinality by web
[ https://issues.apache.org/jira/browse/KYLIN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114478#comment-17114478 ] ASF GitHub Bot commented on KYLIN-4511: --- coveralls edited a comment on pull request #1225: URL: https://github.com/apache/kylin/pull/1225#issuecomment-631864012 ## Pull Request Test Coverage Report for [Build 5936](https://coveralls.io/builds/30981977) * **0** of **0** changed or added relevant lines in **0** files are covered. * **945** unchanged lines in **22** files lost coverage. * Overall coverage increased (+**1.1%**) to **28.438%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/CalculateStatsFromBaseCuboidMapper.java](https://coveralls.io/builds/30981977/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FCalculateStatsFromBaseCuboidMapper.java#L150) | 1 | 95.35% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/30981977/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 3 | 78.42% | | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveTableMeta.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveTableMeta.java#L70) | 3 | 0% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarMemoryStorePersister.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FColumnarMemoryStorePersister.java#L112) | 10 | 94.92% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/SegmentMemoryStore.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FSegmentMemoryStore.java#L167) | 11 | 88.61% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEvent.java](https://coveralls.io/builds/30981977/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FRecordEvent.java#L119) | 14 | 76.34% | | [source-hive/src/main/java/org/apache/kylin/source/hive/CLIHiveClient.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FCLIHiveClient.java#L112) | 19 | 0% | | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveTableMetaBuilder.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveTableMetaBuilder.java#L39) | 19 | 0% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/BlockingReservoir.java](https://coveralls.io/builds/30981977/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FBlockingReservoir.java#L89) | 22 | 60.0% | | Totals | [](https://coveralls.io/builds/30981977) | | :-- | --: | | Change from base [Build 5911](https://coveralls.io/builds/30861301): | 1.1% | | Covered Lines: | 25440 | | Relevant Lines: | 89459 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support project admin user to calculate column cardinality by web > - > > Key: KYLIN-4511 > URL: https://issues.apache.org/jira/browse/KYLIN-4511 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v3.1.0 > > Attachments: image-2020-05-21-13-56-19-962.png, > image-2020-05-21-13-57-29-350.png > > > Now only the administrator of kylin can recalculate the cardinality of the > hive table through the admin web, but the project administrator cannot. > Add an entry to calculate cardinality on the table detail web > # add calculate cardinality button >
[GitHub] [kylin] coveralls edited a comment on pull request #1225: KYLIN-4511 Support project admin user to calculate column cardinality…
coveralls edited a comment on pull request #1225: URL: https://github.com/apache/kylin/pull/1225#issuecomment-631864012 ## Pull Request Test Coverage Report for [Build 5936](https://coveralls.io/builds/30981977) * **0** of **0** changed or added relevant lines in **0** files are covered. * **945** unchanged lines in **22** files lost coverage. * Overall coverage increased (+**1.1%**) to **28.438%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/CalculateStatsFromBaseCuboidMapper.java](https://coveralls.io/builds/30981977/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FCalculateStatsFromBaseCuboidMapper.java#L150) | 1 | 95.35% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/30981977/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 3 | 78.42% | | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveTableMeta.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveTableMeta.java#L70) | 3 | 0% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarMemoryStorePersister.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FColumnarMemoryStorePersister.java#L112) | 10 | 94.92% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/SegmentMemoryStore.java](https://coveralls.io/builds/30981977/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FSegmentMemoryStore.java#L167) | 11 | 88.61% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEvent.java](https://coveralls.io/builds/30981977/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FRecordEvent.java#L119) | 14 | 76.34% | | [source-hive/src/main/java/org/apache/kylin/source/hive/CLIHiveClient.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FCLIHiveClient.java#L112) | 19 | 0% | | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveTableMetaBuilder.java](https://coveralls.io/builds/30981977/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveTableMetaBuilder.java#L39) | 19 | 0% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/BlockingReservoir.java](https://coveralls.io/builds/30981977/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FBlockingReservoir.java#L89) | 22 | 60.0% | | Totals | [](https://coveralls.io/builds/30981977) | | :-- | --: | | Change from base [Build 5911](https://coveralls.io/builds/30861301): | 1.1% | | Covered Lines: | 25440 | | Relevant Lines: | 89459 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] guangxuCheng commented on a change in pull request #1225: KYLIN-4511 Support project admin user to calculate column cardinality…
guangxuCheng commented on a change in pull request #1225:
URL: https://github.com/apache/kylin/pull/1225#discussion_r429333754
##
File path: webapp/app/js/controllers/sourceMeta.js
##
@@ -131,6 +131,39 @@ KylinApp
});
};
+$scope.calCardinality = function (tableName) {
+ SweetAlert.swal({
+title: "",
+text: "Are you sure to recalculate column cardinality?",
+showCancelButton: true,
+confirmButtonColor: '#DD6B55',
+confirmButtonText: "Yes",
+cancelButtonText: "No",
+closeOnConfirm: true
+ }, function (isConfirm) {
+if (isConfirm) {
+ if (!$scope.projectModel.selectedProject) {
+SweetAlert.swal('', 'Please choose your project first!.', 'info');
+return;
+ }
+ loadingRequest.show();
+ TableService.genCardinality({tableName: tableName, pro:
$scope.projectModel.selectedProject}, {}, function () {
+loadingRequest.hide();
+MessageBox.successNotify('Cardinality job was calculated
successfully. Please wait a moment...');
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kylin] chenqingspring commented on pull request #1214: [Bugfix] fix error with sending mail on ssl
chenqingspring commented on pull request #1214: URL: https://github.com/apache/kylin/pull/1214#issuecomment-632772676 @shaofengshi Hi xiaofeng, what do you mean by associating github commit? I'm trying to rename this issue start with KYLIN-4515, but it seems not editable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
