[
https://issues.apache.org/jira/browse/KYLIN-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320204#comment-17320204
] ASF GitHub Bot commented on KYLIN-4941: --------------------------------------- codecov-io commented on pull request #1631: URL: https://github.com/apache/kylin/pull/1631#issuecomment-818775584 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#1631](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (528ad97) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0250081) will **decrease** coverage by `0.11%`. > The diff coverage is `5.21%`. [](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #1631 +/- ## ============================================ - Coverage 25.42% 25.31% -0.12% + Complexity 6765 6762 -3 ============================================ Files 1508 1509 +1 Lines 93918 94358 +440 Branches 13158 13230 +72 ============================================ + Hits 23877 23883 +6 - Misses 67662 68092 +430 - Partials 2379 2383 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.40% <0.00%> (-0.07%)` | `51.00 <0.00> (ø)` | | | [...in/java/org/apache/kylin/cube/CubeDescManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVEZXNjTWFuYWdlci5qYXZh) | `32.41% <0.00%> (-1.16%)` | `12.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `35.83% <0.00%> (-0.81%)` | `44.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `41.21% <0.00%> (-1.07%)` | `65.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `57.02% <0.00%> (-1.34%)` | `182.00 <0.00> (ø)` | | | [...n/java/org/apache/kylin/cube/model/RowKeyDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL1Jvd0tleURlc2MuamF2YQ==) | `71.42% <0.00%> (-11.91%)` | `17.00 <0.00> (ø)` | | | [...a/org/apache/kylin/job/common/ShellExecutable.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9jb21tb24vU2hlbGxFeGVjdXRhYmxlLmphdmE=) | `0.00% <ø> (ø)` | `0.00 <0.00> (ø)` | | | [...g/apache/kylin/job/execution/BrokenExecutable.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vQnJva2VuRXhlY3V0YWJsZS5qYXZh) | `60.00% <ø> (ø)` | `2.00 <0.00> (ø)` | | | [...ylin/job/impl/threadpool/DistributedScheduler.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGlzdHJpYnV0ZWRTY2hlZHVsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...g/apache/kylin/job/impl/threadpool/IJobRunner.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvSUpvYlJ1bm5lci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | ... and [72 more](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | | ------ [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [69368b0...528ad97](https://codecov.io/gh/apache/kylin/pull/1631?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] > Support encoding raw data to base cuboid column-by-column > --------------------------------------------------------- > > Key: KYLIN-4941 > URL: https://issues.apache.org/jira/browse/KYLIN-4941 > Project: Kylin > Issue Type: Improvement > Components: Job Engine > Affects Versions: v3.1.1 > Reporter: ShengJun Zheng > Assignee: ShengJun Zheng > Priority: Major > Fix For: v3.2.0 > > > When building with spark engine, the first step is to encode hive table's row > to base cuboid data. > The existing implementation is encoding row by row. If the cube has several > dictionary encoded measures, it has to use all dictionaries at the same time > to encode a single row. This causes heavy memory usage, and low cache hit > ratio of dictionary cache. > We optimized this case by encoding column by column, and it did bring > significant improvement over cubes with several high cardinality > dictionaries-encoded measures. > We will refine the implementation based on KYLIN3.x and share it out. -- This message was sent by Atlassian Jira (v8.3.4#803005)
