[jira] [Commented] (KYLIN-2578) Refactor DistributedLock
[
https://issues.apache.org/jira/browse/KYLIN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701484#comment-16701484
]
ASF GitHub Bot commented on KYLIN-2578:
---
coveralls commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442347902
## Pull Request Test Coverage Report for [Build
3898](https://coveralls.io/builds/20330244)
* **0** of **0** changed or added relevant lines in **0** files are
covered.
* **1** unchanged line in **1** file lost coverage.
* Overall coverage increased (+**0.001%**) to **25.949%**
---
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20330244/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
| 1 | 81.08% |
| Totals | [](https://coveralls.io/builds/20330244)
|
| :-- | --: |
| Change from base [Build 3894](https://coveralls.io/builds/20327415): |
0.001% |
| Covered Lines: | 17651 |
| Relevant Lines: | 68023 |
---
# 💛 - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor DistributedLock
>
>
> Key: KYLIN-2578
> URL: https://issues.apache.org/jira/browse/KYLIN-2578
> Project: Kylin
> Issue Type: Improvement
>Reporter: liyang
>Assignee: liyang
>Priority: Major
> Fix For: v2.1.0
>
>
> The current {{DistributedLock}} could use some improvement:
> - A {{lockClient}} is unnecessarily required.
> - The {{watchPath}} is actually an on-unlock listener and the current name
> failed to make it clear.
> - Could add a blocking version of {{lockPath}} and that will ease use cases
> like KYLIN-2557 and {{GlobalDictionaryBuilder.lock()}}
> - Should add more javadoc on the interface
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3623) Convert cuboid to Parquet in MR
[ https://issues.apache.org/jira/browse/KYLIN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3623: Component/s: Storage - Parquet > Convert cuboid to Parquet in MR > --- > > Key: KYLIN-3623 > URL: https://issues.apache.org/jira/browse/KYLIN-3623 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine, Storage - Parquet >Reporter: Shaofeng SHI >Assignee: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3625) Query engine for Parquet
[ https://issues.apache.org/jira/browse/KYLIN-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3625: Component/s: Storage - Parquet > Query engine for Parquet > > > Key: KYLIN-3625 > URL: https://issues.apache.org/jira/browse/KYLIN-3625 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine, Storage - Parquet >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3626) Allow customization for Parquet path
[ https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3626: Fix Version/s: v2.6.0 > Allow customization for Parquet path > > > Key: KYLIN-3626 > URL: https://issues.apache.org/jira/browse/KYLIN-3626 > Project: Kylin > Issue Type: Sub-task > Components: Metadata >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.0 > > > Due to the difference of HDFS and object store like S3/blob store, Kylin > needs to allow different path patterns for the files when putting to the > underlying storage. > > For example, to HDFS, this pattern is good: > /kylin/metadata_name/cube_name/segment_name/file1.parquet > > But, the above pattern is not good for S3, since S3 uses the first couple > bytes of the path to do the hashing, so we'd better add salt ahead of the > original pathname: > /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet > or do a reverse order: > /segment_name/cube_name/metadata_name/kylin/file1.parquet > > To get the flexibility, we should make this extensible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3624) Convert cuboid to Parquet in Spark
[ https://issues.apache.org/jira/browse/KYLIN-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3624: --- Assignee: Yichen Zhou Fix Version/s: v2.6.0 Component/s: Spark Engine Job Engine > Convert cuboid to Parquet in Spark > -- > > Key: KYLIN-3624 > URL: https://issues.apache.org/jira/browse/KYLIN-3624 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine, Spark Engine >Reporter: Shaofeng SHI >Assignee: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3621) Parquet as Cube storage
[ https://issues.apache.org/jira/browse/KYLIN-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3621: Fix Version/s: v2.6.0 > Parquet as Cube storage > --- > > Key: KYLIN-3621 > URL: https://issues.apache.org/jira/browse/KYLIN-3621 > Project: Kylin > Issue Type: New Feature > Components: Query Engine, Storage - Parquet >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > Attachments: Kylin_on_Parquet_v1.pdf > > > As discussed in dev@kylin, we plan to make a new storage engine based on > Apache Parquet and Apache Spark. The discussion is: > https://lists.apache.org/thread.html/8ab22f9c1bbb88397445bad236d734c7f09819100973cbc9bdab6f04@%3Cdev.kylin.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3624) Convert cuboid to Parquet in Spark
[ https://issues.apache.org/jira/browse/KYLIN-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3624: Component/s: Storage - Parquet > Convert cuboid to Parquet in Spark > -- > > Key: KYLIN-3624 > URL: https://issues.apache.org/jira/browse/KYLIN-3624 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine, Spark Engine, Storage - Parquet >Reporter: Shaofeng SHI >Assignee: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3621) Parquet as Cube storage
[ https://issues.apache.org/jira/browse/KYLIN-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3621: Component/s: (was: Storage - HBase) Storage - Parquet > Parquet as Cube storage > --- > > Key: KYLIN-3621 > URL: https://issues.apache.org/jira/browse/KYLIN-3621 > Project: Kylin > Issue Type: New Feature > Components: Query Engine, Storage - Parquet >Reporter: Shaofeng SHI >Priority: Major > Attachments: Kylin_on_Parquet_v1.pdf > > > As discussed in dev@kylin, we plan to make a new storage engine based on > Apache Parquet and Apache Spark. The discussion is: > https://lists.apache.org/thread.html/8ab22f9c1bbb88397445bad236d734c7f09819100973cbc9bdab6f04@%3Cdev.kylin.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3626) Allow customization for Parquet path
[ https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3626: Component/s: Storage - Parquet Job Engine > Allow customization for Parquet path > > > Key: KYLIN-3626 > URL: https://issues.apache.org/jira/browse/KYLIN-3626 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine, Metadata, Storage - Parquet >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.0 > > > Due to the difference of HDFS and object store like S3/blob store, Kylin > needs to allow different path patterns for the files when putting to the > underlying storage. > > For example, to HDFS, this pattern is good: > /kylin/metadata_name/cube_name/segment_name/file1.parquet > > But, the above pattern is not good for S3, since S3 uses the first couple > bytes of the path to do the hashing, so we'd better add salt ahead of the > original pathname: > /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet > or do a reverse order: > /segment_name/cube_name/metadata_name/kylin/file1.parquet > > To get the flexibility, we should make this extensible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3622) Cube layout in Parquet
[ https://issues.apache.org/jira/browse/KYLIN-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3622: Component/s: Storage - Parquet > Cube layout in Parquet > -- > > Key: KYLIN-3622 > URL: https://issues.apache.org/jira/browse/KYLIN-3622 > Project: Kylin > Issue Type: Sub-task > Components: Storage - Parquet >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > > The file and schema layout for Cubes in Parquet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3622) Cube layout in Parquet
[ https://issues.apache.org/jira/browse/KYLIN-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3622: Fix Version/s: v2.6.0 > Cube layout in Parquet > -- > > Key: KYLIN-3622 > URL: https://issues.apache.org/jira/browse/KYLIN-3622 > Project: Kylin > Issue Type: Sub-task >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > > The file and schema layout for Cubes in Parquet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3622) Cube layout in Parquet
[ https://issues.apache.org/jira/browse/KYLIN-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3622: --- Assignee: Shaofeng SHI > Cube layout in Parquet > -- > > Key: KYLIN-3622 > URL: https://issues.apache.org/jira/browse/KYLIN-3622 > Project: Kylin > Issue Type: Sub-task >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > > The file and schema layout for Cubes in Parquet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3626) Allow customization for Parquet path
[ https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3626: --- Assignee: Chao Long Component/s: Metadata > Allow customization for Parquet path > > > Key: KYLIN-3626 > URL: https://issues.apache.org/jira/browse/KYLIN-3626 > Project: Kylin > Issue Type: Sub-task > Components: Metadata >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.0 > > > Due to the difference of HDFS and object store like S3/blob store, Kylin > needs to allow different path patterns for the files when putting to the > underlying storage. > > For example, to HDFS, this pattern is good: > /kylin/metadata_name/cube_name/segment_name/file1.parquet > > But, the above pattern is not good for S3, since S3 uses the first couple > bytes of the path to do the hashing, so we'd better add salt ahead of the > original pathname: > /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet > or do a reverse order: > /segment_name/cube_name/metadata_name/kylin/file1.parquet > > To get the flexibility, we should make this extensible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3625) Query engine for Parquet
[ https://issues.apache.org/jira/browse/KYLIN-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3625: --- Assignee: Chao Long Fix Version/s: v2.6.0 Component/s: Query Engine > Query engine for Parquet > > > Key: KYLIN-3625 > URL: https://issues.apache.org/jira/browse/KYLIN-3625 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Shaofeng SHI >Assignee: Chao Long >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3623) Convert cuboid to Parquet in MR
[ https://issues.apache.org/jira/browse/KYLIN-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3623: --- Assignee: Yichen Zhou Fix Version/s: v2.6.0 > Convert cuboid to Parquet in MR > --- > > Key: KYLIN-3623 > URL: https://issues.apache.org/jira/browse/KYLIN-3623 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2578) Refactor DistributedLock
[
https://issues.apache.org/jira/browse/KYLIN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701483#comment-16701483
]
ASF GitHub Bot commented on KYLIN-2578:
---
codecov-io commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442347708
# [Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=h1) Report
> Merging [#365](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc)
will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #365 +/- ##
+ Coverage 23.63% 23.63% +<.01%
+ Complexity 4828 4827 -1
Files 1130 1130
Lines 6802368023
Branches 9657 9657
+ Hits 1607916080 +1
+ Misses5037750376 -1
Partials 1567 1567
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...ylin/job/impl/threadpool/DistributedScheduler.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGlzdHJpYnV0ZWRTY2hlZHVsZXIuamF2YQ==)
| `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
|
[...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==)
| `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | |
|
[...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh)
| `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | |
|
[...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=)
| `72.29% <0%> (+1.35%)` | `8% <0%> (ø)` | :arrow_down: |
|
[...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==)
| `40.47% <0%> (+4.76%)` | `6% <0%> (+1%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=footer). Last
update
[97ee98b...09117c0](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor DistributedLock
>
>
> Key: KYLIN-2578
> URL: https://issues.apache.org/jira/browse/KYLIN-2578
> Project: Kylin
> Issue Type: Improvement
>Reporter: liyang
>Assignee: liyang
>Priority: Major
> Fix For: v2.1.0
>
>
> The current {{DistributedLock}} could use some improvement:
> - A {{lockClient}} is unnecessarily required.
> - The {{watchPath}} is actually an on-unlock listener and the current name
> failed to make it clear.
> - Could add a blocking version of {{lockPath}} and that will ease use cases
> like KYLIN-2557 and {{GlobalDictionaryBuilder.lock()}}
> - Should add more javadoc on the interface
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] coveralls commented on issue #365: KYLIN-2578, clean deprecated config("kylin.job.lock") usage in the code.
coveralls commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442347902
## Pull Request Test Coverage Report for [Build
3898](https://coveralls.io/builds/20330244)
* **0** of **0** changed or added relevant lines in **0** files are
covered.
* **1** unchanged line in **1** file lost coverage.
* Overall coverage increased (+**0.001%**) to **25.949%**
---
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20330244/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
| 1 | 81.08% |
| Totals | [](https://coveralls.io/builds/20330244)
|
| :-- | --: |
| Change from base [Build 3894](https://coveralls.io/builds/20327415): |
0.001% |
| Covered Lines: | 17651 |
| Relevant Lines: | 68023 |
---
# 💛 - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io commented on issue #365: KYLIN-2578, clean deprecated config("kylin.job.lock") usage in the code.
codecov-io commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442347708
# [Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=h1) Report
> Merging [#365](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc)
will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #365 +/- ##
+ Coverage 23.63% 23.63% +<.01%
+ Complexity 4828 4827 -1
Files 1130 1130
Lines 6802368023
Branches 9657 9657
+ Hits 1607916080 +1
+ Misses5037750376 -1
Partials 1567 1567
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...ylin/job/impl/threadpool/DistributedScheduler.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGlzdHJpYnV0ZWRTY2hlZHVsZXIuamF2YQ==)
| `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
|
[...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==)
| `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | |
|
[...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh)
| `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | |
|
[...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=)
| `72.29% <0%> (+1.35%)` | `8% <0%> (ø)` | :arrow_down: |
|
[...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/365/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==)
| `40.47% <0%> (+4.76%)` | `6% <0%> (+1%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=footer). Last
update
[97ee98b...09117c0](https://codecov.io/gh/apache/kylin/pull/365?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[
https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701476#comment-16701476
]
ASF subversion and git services commented on KYLIN-3700:
Commit 4158d7b3b56cbeb3c10ec5c091c23334131616a5 in kylin's branch
refs/heads/master from ZhengshuaiPENG
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4158d7b ]
KYLIN-3700 Quote sql identities when creating flat table
> Quote sql identities when creating flat table
> -
>
> Key: KYLIN-3700
> URL: https://issues.apache.org/jira/browse/KYLIN-3700
> Project: Kylin
> Issue Type: Improvement
> Components: RDBMS Source
>Affects Versions: v2.5.1
>Reporter: XiaoXiang Yu
>Assignee: XiaoXiang Yu
>Priority: Major
> Fix For: v2.6.0
>
>
> *When kyin operate with different data source, DDL/DML may be broken because
> datasource has some different specific reserved keyword.*
>
> Sql identity should be quoted when:
> 1) Create flat table.
> 2) Push down to different datasource (different RDBMS has different reserved
> keyword) .
>
> {code:java}
> 2018-10-05 03:05:04,143 ERROR [pool-25-thread-2] conv.SqlConverter : Failed
> to default convert sql, will use the input: SELECT COUNT(TIME_KEY), TIME_KEY
> FROM CUBE.OPERATIONS_TIME WHERE TIME_KEY IS NOT NULL GROUP BY TIME_KEY ORDER
> BY COUNT(TIME_KEY) DESC LIMIT 10
> org.apache.calcite.sql.parser.SqlParseException: Encountered "FROM CUBE" at
> line 1, column 34.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "FROM" ...
> "FROM" ...
> "FROM" ...
> "FROM" ...
> "FROM" "LATERAL" ...
> "FROM" "(" ...
> "FROM" "UNNEST" ...
> "FROM" "TABLE" ...
> "," ...
> "AS" ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
> "(" ...
>
> at
> org.apache.calcite.sql.parser.impl.SqlParserImpl.convertException(SqlParserImpl.java:349)
> at
> org.apache.calcite.sql.parser.impl.SqlParserImpl.normalizeException(SqlParserImpl.java:130){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[
https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701475#comment-16701475
]
ASF GitHub Bot commented on KYLIN-3700:
---
shaofengshi closed pull request #362: KYLIN-3700 Quote sql identities when
creating flat table
URL: https://github.com/apache/kylin/pull/362
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
b/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
index 707b3f33c9..8a99906e3e 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
@@ -68,7 +68,8 @@ public String build() {
case CLI:
buf.append("hive -e \"");
for (String statement : statements) {
-buf.append(statement).append("\n");
+//in bash need escape " and ` by using \
+buf.append(statement.replaceAll("`", "`")).append("\n");
}
buf.append("\"");
buf.append(parseProps());
@@ -79,7 +80,7 @@ public String build() {
try {
tmpHqlPath = "/tmp/" + UUID.randomUUID().toString() + ".hql";
for (String statement : statements) {
-hql.append(statement);
+hql.append(statement.replaceAll("`", "`"));
hql.append("\n");
}
String createFileCmd = String.format(Locale.ROOT,
CREATE_HQL_TMP_FILE_TEMPLATE, tmpHqlPath, hql);
diff --git
a/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
b/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
index ecc8961e33..8c852c1197 100644
---
a/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
+++
b/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
@@ -61,12 +61,12 @@ public void testHiveCLI() {
hivePropsOverwrite.put("hive.execution.engine", "tez");
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW\n TABLES;");
hiveCmdBuilder.setHiveConfProps(hiveProps);
hiveCmdBuilder.overwriteHiveProps(hivePropsOverwrite);
assertEquals(
-"hive -e \"USE default;\nDROP TABLE test;\nSHOW\n TABLES;\n\"
--hiveconf hive.execution.engine=tez",
+"hive -e \"USE default;\nDROP TABLE \\`test\\`;\nSHOW\n
TABLES;\n\" --hiveconf hive.execution.engine=tez",
hiveCmdBuilder.build());
}
@@ -80,7 +80,7 @@ public void testBeeline() throws IOException {
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW TABLES;");
String cmd = hiveCmdBuilder.build();
@@ -91,7 +91,7 @@ public void testBeeline() throws IOException {
Pair execute =
cliCommandExecutor.execute(createFileCmd);
String hqlStatement = FileUtils.readFileToString(new File(hqlFile),
Charset.defaultCharset());
assertEquals(
-"USE default;" + lineSeparator + "DROP TABLE test;" +
lineSeparator + "SHOW TABLES;" + lineSeparator,
+"USE default;" + lineSeparator + "DROP TABLE `test`;" +
lineSeparator + "SHOW TABLES;" + lineSeparator,
hqlStatement);
assertBeelineCmd(cmd);
FileUtils.forceDelete(new File(hqlFile));
@@ -105,7 +105,7 @@ public void testSparkSqlForTableOps() throws IOException {
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW TABLES;");
String cmd = hiveCmdBuilder.build();
assertBeelineCmd(cmd);
diff --git a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
index ff48244687..d7e3b72e28 100644
--- a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
+++ b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
@@ -41,6 +41,9 @@
import org.apac
[GitHub] shaofengshi closed pull request #362: KYLIN-3700 Quote sql identities when creating flat table
shaofengshi closed pull request #362: KYLIN-3700 Quote sql identities when
creating flat table
URL: https://github.com/apache/kylin/pull/362
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
b/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
index 707b3f33c9..8a99906e3e 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/HiveCmdBuilder.java
@@ -68,7 +68,8 @@ public String build() {
case CLI:
buf.append("hive -e \"");
for (String statement : statements) {
-buf.append(statement).append("\n");
+//in bash need escape " and ` by using \
+buf.append(statement.replaceAll("`", "`")).append("\n");
}
buf.append("\"");
buf.append(parseProps());
@@ -79,7 +80,7 @@ public String build() {
try {
tmpHqlPath = "/tmp/" + UUID.randomUUID().toString() + ".hql";
for (String statement : statements) {
-hql.append(statement);
+hql.append(statement.replaceAll("`", "`"));
hql.append("\n");
}
String createFileCmd = String.format(Locale.ROOT,
CREATE_HQL_TMP_FILE_TEMPLATE, tmpHqlPath, hql);
diff --git
a/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
b/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
index ecc8961e33..8c852c1197 100644
---
a/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
+++
b/core-common/src/test/java/org/apache/kylin/common/util/HiveCmdBuilderTest.java
@@ -61,12 +61,12 @@ public void testHiveCLI() {
hivePropsOverwrite.put("hive.execution.engine", "tez");
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW\n TABLES;");
hiveCmdBuilder.setHiveConfProps(hiveProps);
hiveCmdBuilder.overwriteHiveProps(hivePropsOverwrite);
assertEquals(
-"hive -e \"USE default;\nDROP TABLE test;\nSHOW\n TABLES;\n\"
--hiveconf hive.execution.engine=tez",
+"hive -e \"USE default;\nDROP TABLE \\`test\\`;\nSHOW\n
TABLES;\n\" --hiveconf hive.execution.engine=tez",
hiveCmdBuilder.build());
}
@@ -80,7 +80,7 @@ public void testBeeline() throws IOException {
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW TABLES;");
String cmd = hiveCmdBuilder.build();
@@ -91,7 +91,7 @@ public void testBeeline() throws IOException {
Pair execute =
cliCommandExecutor.execute(createFileCmd);
String hqlStatement = FileUtils.readFileToString(new File(hqlFile),
Charset.defaultCharset());
assertEquals(
-"USE default;" + lineSeparator + "DROP TABLE test;" +
lineSeparator + "SHOW TABLES;" + lineSeparator,
+"USE default;" + lineSeparator + "DROP TABLE `test`;" +
lineSeparator + "SHOW TABLES;" + lineSeparator,
hqlStatement);
assertBeelineCmd(cmd);
FileUtils.forceDelete(new File(hqlFile));
@@ -105,7 +105,7 @@ public void testSparkSqlForTableOps() throws IOException {
HiveCmdBuilder hiveCmdBuilder = new HiveCmdBuilder();
hiveCmdBuilder.addStatement("USE default;");
-hiveCmdBuilder.addStatement("DROP TABLE test;");
+hiveCmdBuilder.addStatement("DROP TABLE `test`;");
hiveCmdBuilder.addStatement("SHOW TABLES;");
String cmd = hiveCmdBuilder.build();
assertBeelineCmd(cmd);
diff --git a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
index ff48244687..d7e3b72e28 100644
--- a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
+++ b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
@@ -41,6 +41,9 @@
import org.apache.kylin.metadata.model.TableRef;
import org.apache.kylin.metadata.model.TblColRef;
+import static org.apache.kylin.job.util.FlatTableSqlQuoteUtils.quote;
+import static
org.apache.kylin.job.util.FlatTableSqlQuoteUtils.quoteIdentifierInSqlExpr;
+
impor
[jira] [Commented] (KYLIN-2578) Refactor DistributedLock
[
https://issues.apache.org/jira/browse/KYLIN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701470#comment-16701470
]
ASF GitHub Bot commented on KYLIN-2578:
---
Aaron commented on a change in pull request #365: KYLIN-2578, clean
deprecated config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#discussion_r236961076
##
File path:
kylin-it/src/test/java/org/apache/kylin/job/BaseTestDistributedScheduler.java
##
@@ -69,7 +69,6 @@
@BeforeClass
public static void setup() throws Exception {
staticCreateTestMetadata();
-System.setProperty("kylin.job.lock",
"org.apache.kylin.storage.hbase.util.ZookeeperDistributedJobLock");
Review comment:
In fact, class "ZookeeperDistributedJobLock" already has been renamed to
"ZookeeperDistributedLock"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor DistributedLock
>
>
> Key: KYLIN-2578
> URL: https://issues.apache.org/jira/browse/KYLIN-2578
> Project: Kylin
> Issue Type: Improvement
>Reporter: liyang
>Assignee: liyang
>Priority: Major
> Fix For: v2.1.0
>
>
> The current {{DistributedLock}} could use some improvement:
> - A {{lockClient}} is unnecessarily required.
> - The {{watchPath}} is actually an on-unlock listener and the current name
> failed to make it clear.
> - Could add a blocking version of {{lockPath}} and that will ease use cases
> like KYLIN-2557 and {{GlobalDictionaryBuilder.lock()}}
> - Should add more javadoc on the interface
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] Aaaaaaron commented on a change in pull request #365: KYLIN-2578, clean deprecated config("kylin.job.lock") usage in the code.
Aaron commented on a change in pull request #365: KYLIN-2578, clean
deprecated config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#discussion_r236961076
##
File path:
kylin-it/src/test/java/org/apache/kylin/job/BaseTestDistributedScheduler.java
##
@@ -69,7 +69,6 @@
@BeforeClass
public static void setup() throws Exception {
staticCreateTestMetadata();
-System.setProperty("kylin.job.lock",
"org.apache.kylin.storage.hbase.util.ZookeeperDistributedJobLock");
Review comment:
In fact, class "ZookeeperDistributedJobLock" already has been renamed to
"ZookeeperDistributedLock"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-2578) Refactor DistributedLock
[
https://issues.apache.org/jira/browse/KYLIN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701468#comment-16701468
]
ASF GitHub Bot commented on KYLIN-2578:
---
asfgit commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442342755
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor DistributedLock
>
>
> Key: KYLIN-2578
> URL: https://issues.apache.org/jira/browse/KYLIN-2578
> Project: Kylin
> Issue Type: Improvement
>Reporter: liyang
>Assignee: liyang
>Priority: Major
> Fix For: v2.1.0
>
>
> The current {{DistributedLock}} could use some improvement:
> - A {{lockClient}} is unnecessarily required.
> - The {{watchPath}} is actually an on-unlock listener and the current name
> failed to make it clear.
> - Could add a blocking version of {{lockPath}} and that will ease use cases
> like KYLIN-2557 and {{GlobalDictionaryBuilder.lock()}}
> - Should add more javadoc on the interface
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-2578) Refactor DistributedLock
[
https://issues.apache.org/jira/browse/KYLIN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701467#comment-16701467
]
ASF GitHub Bot commented on KYLIN-2578:
---
Aaron opened a new pull request #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor DistributedLock
>
>
> Key: KYLIN-2578
> URL: https://issues.apache.org/jira/browse/KYLIN-2578
> Project: Kylin
> Issue Type: Improvement
>Reporter: liyang
>Assignee: liyang
>Priority: Major
> Fix For: v2.1.0
>
>
> The current {{DistributedLock}} could use some improvement:
> - A {{lockClient}} is unnecessarily required.
> - The {{watchPath}} is actually an on-unlock listener and the current name
> failed to make it clear.
> - Could add a blocking version of {{lockPath}} and that will ease use cases
> like KYLIN-2557 and {{GlobalDictionaryBuilder.lock()}}
> - Should add more javadoc on the interface
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] asfgit commented on issue #365: KYLIN-2578, clean deprecated config("kylin.job.lock") usage in the code.
asfgit commented on issue #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365#issuecomment-442342755
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] Aaaaaaron opened a new pull request #365: KYLIN-2578, clean deprecated config("kylin.job.lock") usage in the code.
Aaron opened a new pull request #365: KYLIN-2578, clean deprecated
config("kylin.job.lock") usage in the code.
URL: https://github.com/apache/kylin/pull/365
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[ https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701442#comment-16701442 ] ASF GitHub Bot commented on KYLIN-3700: --- coveralls edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-442330731 ## Pull Request Test Coverage Report for [Build 3897](https://coveralls.io/builds/20329685) * **72** of **244** **(29.51%)** changed or added relevant lines in **15** files are covered. * **6** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.006%**) to **25.941%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableRef.java](https://coveralls.io/builds/20329685/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableRef.java#L72) | 0 | 1 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/JdbcConnector.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2FJdbcConnector.java#L89) | 2 | 3 | 66.67% | [source-hive/src/main/java/org/apache/kylin/source/hive/GarbageCollectionStep.java](https://coveralls.io/builds/20329685/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FGarbageCollectionStep.java#L67) | 0 | 1 | 0.0% | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveInputBase.java](https://coveralls.io/builds/20329685/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveInputBase.java#L121) | 4 | 5 | 80.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/SqlConverter.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FSqlConverter.java#L75) | 7 | 9 | 77.78% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/DefaultConfiguer.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FDefaultConfiguer.java#L132) | 6 | 9 | 66.67% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableDesc.java](https://coveralls.io/builds/20329685/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableDesc.java#L223) | 0 | 6 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/ConvSqlWriter.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FConvSqlWriter.java#L156) | 5 | 12 | 41.67% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java](https://coveralls.io/builds/20329685/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveMRInput.java#L160) | 3 | 11 | 27.27% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FAbstractJdbcAdaptor.java#L300) | 4 | 18 | 22.22% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20329685/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/20329685/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L235) | 2 | 0.0% | | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FDefaultAdaptor.java#L145) | 3 | 35.51% | | Totals | [](https://coveralls.io/builds/20329685) | | :-- | --: | | Change from base [Build 3894](https://coveralls.io/builds/20327415): | -0.006% | | Covered Lines: | 17690 | | Relevant Lines: | 68193 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[ https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701445#comment-16701445 ] ASF GitHub Bot commented on KYLIN-3700: --- codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-441728994 # [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=h1) Report > Merging [#362](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `27.86%`. [](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #362 +/- ## - Coverage 23.63% 23.63% -0.01% - Complexity 4828 4837 +9 Files 1130 1131 +1 Lines 6802368193 +170 Branches 9657 9684 +27 + Hits 1607916119 +40 - Misses5037750507 +130 Partials 1567 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...k/datasource/framework/SourceConnectorFactory.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2ZyYW1ld29yay9Tb3VyY2VDb25uZWN0b3JGYWN0b3J5LmphdmE=) | `57.89% <ø> (ø)` | `1 <0> (ø)` | :arrow_down: | | [...java/org/apache/kylin/metadata/model/TableRef.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVSZWYuamF2YQ==) | `51.16% <0%> (-1.22%)` | `11 <0> (ø)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/kylin/metadata/model/TableDesc.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVEZXNjLmphdmE=) | `46.01% <0%> (-1.76%)` | `27 <0> (ø)` | | | [...pache/kylin/source/hive/GarbageCollectionStep.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0dhcmJhZ2VDb2xsZWN0aW9uU3RlcC5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...a/org/apache/kylin/common/util/HiveCmdBuilder.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0hpdmVDbWRCdWlsZGVyLmphdmE=) | `83.33% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../kylin/source/jdbc/extensible/JdbcHiveMRInput.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVNUklucHV0LmphdmE=) | `89.79% <100%> (+0.9%)` | `3 <0> (ø)` | :arrow_down: | | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `32.71% <19.35%> (-3.38%)` | `12 <1> (ø)` | | | [.../apache/kylin/job/util/FlatTableSqlQuoteUtils.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0ZsYXRUYWJsZVNxbFF1b3RlVXRpbHMuamF2YQ==) | `21% <21%> (ø)` | `8 <8> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `51.04% <22.22%> (-3.18%)` | `14 <2> (+1)` | | | ... and [10 more](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ =
[GitHub] codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table
codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-441728994 # [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=h1) Report > Merging [#362](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `27.86%`. [](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #362 +/- ## - Coverage 23.63% 23.63% -0.01% - Complexity 4828 4837 +9 Files 1130 1131 +1 Lines 6802368193 +170 Branches 9657 9684 +27 + Hits 1607916119 +40 - Misses5037750507 +130 Partials 1567 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...k/datasource/framework/SourceConnectorFactory.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2ZyYW1ld29yay9Tb3VyY2VDb25uZWN0b3JGYWN0b3J5LmphdmE=) | `57.89% <ø> (ø)` | `1 <0> (ø)` | :arrow_down: | | [...java/org/apache/kylin/metadata/model/TableRef.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVSZWYuamF2YQ==) | `51.16% <0%> (-1.22%)` | `11 <0> (ø)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/kylin/metadata/model/TableDesc.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVEZXNjLmphdmE=) | `46.01% <0%> (-1.76%)` | `27 <0> (ø)` | | | [...pache/kylin/source/hive/GarbageCollectionStep.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0dhcmJhZ2VDb2xsZWN0aW9uU3RlcC5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...a/org/apache/kylin/common/util/HiveCmdBuilder.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0hpdmVDbWRCdWlsZGVyLmphdmE=) | `83.33% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../kylin/source/jdbc/extensible/JdbcHiveMRInput.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVNUklucHV0LmphdmE=) | `89.79% <100%> (+0.9%)` | `3 <0> (ø)` | :arrow_down: | | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `32.71% <19.35%> (-3.38%)` | `12 <1> (ø)` | | | [.../apache/kylin/job/util/FlatTableSqlQuoteUtils.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0ZsYXRUYWJsZVNxbFF1b3RlVXRpbHMuamF2YQ==) | `21% <21%> (ø)` | `8 <8> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `51.04% <22.22%> (-3.18%)` | `14 <2> (+1)` | | | ... and [10 more](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=footer). Last update [97ee98b...f981778](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=lastupdated). Rea
[GitHub] coveralls edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table
coveralls edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-442330731 ## Pull Request Test Coverage Report for [Build 3897](https://coveralls.io/builds/20329685) * **72** of **244** **(29.51%)** changed or added relevant lines in **15** files are covered. * **6** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.006%**) to **25.941%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableRef.java](https://coveralls.io/builds/20329685/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableRef.java#L72) | 0 | 1 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/JdbcConnector.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2FJdbcConnector.java#L89) | 2 | 3 | 66.67% | [source-hive/src/main/java/org/apache/kylin/source/hive/GarbageCollectionStep.java](https://coveralls.io/builds/20329685/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FGarbageCollectionStep.java#L67) | 0 | 1 | 0.0% | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveInputBase.java](https://coveralls.io/builds/20329685/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveInputBase.java#L121) | 4 | 5 | 80.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/SqlConverter.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FSqlConverter.java#L75) | 7 | 9 | 77.78% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/DefaultConfiguer.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FDefaultConfiguer.java#L132) | 6 | 9 | 66.67% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableDesc.java](https://coveralls.io/builds/20329685/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableDesc.java#L223) | 0 | 6 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/ConvSqlWriter.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FConvSqlWriter.java#L156) | 5 | 12 | 41.67% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java](https://coveralls.io/builds/20329685/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveMRInput.java#L160) | 3 | 11 | 27.27% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FAbstractJdbcAdaptor.java#L300) | 4 | 18 | 22.22% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/20329685/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/20329685/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L235) | 2 | 0.0% | | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://coveralls.io/builds/20329685/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FDefaultAdaptor.java#L145) | 3 | 35.51% | | Totals | [](https://coveralls.io/builds/20329685) | | :-- | --: | | Change from base [Build 3894](https://coveralls.io/builds/20327415): | -0.006% | | Covered Lines: | 17690 | | Relevant Lines: | 68193 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3705) Segment Pruner mis-functions when the source data has Chinese characters
[ https://issues.apache.org/jira/browse/KYLIN-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3705: Affects Version/s: v2.5.0 Component/s: Query Engine Metadata > Segment Pruner mis-functions when the source data has Chinese characters > > > Key: KYLIN-3705 > URL: https://issues.apache.org/jira/browse/KYLIN-3705 > Project: Kylin > Issue Type: Bug > Components: Metadata, Query Engine >Affects Versions: v2.5.0, v2.5.1 >Reporter: jiezouSH >Assignee: jiezouSH >Priority: Critical > Fix For: v2.6.0 > > > Source Data: > "合肥市"，“西安市”，“北京市” > > sql: > select * from ZYZ.TEST_ZYZ_3 where name='合肥市' > > Actual Result: > empty result. > > Root Cause: > Dimension range is computed In factDistinctColumnsReducer, but unfortunately > the dimension range is not written in UTF-character. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3705) Segment Pruner mis-functions when the source data has Chinese characters
[ https://issues.apache.org/jira/browse/KYLIN-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3705: --- Assignee: jiezouSH > Segment Pruner mis-functions when the source data has Chinese characters > > > Key: KYLIN-3705 > URL: https://issues.apache.org/jira/browse/KYLIN-3705 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: jiezouSH >Assignee: jiezouSH >Priority: Critical > Fix For: v2.6.0 > > > Source Data: > "合肥市"，“西安市”，“北京市” > > sql: > select * from ZYZ.TEST_ZYZ_3 where name='合肥市' > > Actual Result: > empty result. > > Root Cause: > Dimension range is computed In factDistinctColumnsReducer, but unfortunately > the dimension range is not written in UTF-character. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3705) Segment Pruner mis-functions when the source data has Chinese characters
[ https://issues.apache.org/jira/browse/KYLIN-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3705: Fix Version/s: v2.6.0 > Segment Pruner mis-functions when the source data has Chinese characters > > > Key: KYLIN-3705 > URL: https://issues.apache.org/jira/browse/KYLIN-3705 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: jiezouSH >Priority: Critical > Fix For: v2.6.0 > > > Source Data: > "合肥市"，“西安市”，“北京市” > > sql: > select * from ZYZ.TEST_ZYZ_3 where name='合肥市' > > Actual Result: > empty result. > > Root Cause: > Dimension range is computed In factDistinctColumnsReducer, but unfortunately > the dimension range is not written in UTF-character. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[ https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701428#comment-16701428 ] ASF GitHub Bot commented on KYLIN-3700: --- coveralls commented on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-442330731 ## Pull Request Test Coverage Report for [Build 3896](https://coveralls.io/builds/20329464) * **72** of **244** **(29.51%)** changed or added relevant lines in **15** files are covered. * **6** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.006%**) to **25.941%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableRef.java](https://coveralls.io/builds/20329464/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableRef.java#L72) | 0 | 1 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/JdbcConnector.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2FJdbcConnector.java#L89) | 2 | 3 | 66.67% | [source-hive/src/main/java/org/apache/kylin/source/hive/GarbageCollectionStep.java](https://coveralls.io/builds/20329464/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FGarbageCollectionStep.java#L67) | 0 | 1 | 0.0% | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveInputBase.java](https://coveralls.io/builds/20329464/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveInputBase.java#L121) | 4 | 5 | 80.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/SqlConverter.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FSqlConverter.java#L75) | 7 | 9 | 77.78% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/DefaultConfiguer.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FDefaultConfiguer.java#L132) | 6 | 9 | 66.67% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableDesc.java](https://coveralls.io/builds/20329464/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableDesc.java#L223) | 0 | 6 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/ConvSqlWriter.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FConvSqlWriter.java#L156) | 5 | 12 | 41.67% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java](https://coveralls.io/builds/20329464/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveMRInput.java#L160) | 3 | 11 | 27.27% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FAbstractJdbcAdaptor.java#L300) | 4 | 18 | 22.22% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/20329464/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/20329464/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L235) | 2 | 0.0% | | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FDefaultAdaptor.java#L145) | 3 | 35.51% | | Totals | [](https://coveralls.io/builds/20329464) | | :-- | --: | | Change from base [Build 3894](https://coveralls.io/builds/20327415): | -0.006% | | Covered Lines: | 17690 | | Relevant Lines: | 68193 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please lo
[jira] [Commented] (KYLIN-3700) Quote sql identities when creating flat table
[ https://issues.apache.org/jira/browse/KYLIN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701429#comment-16701429 ] ASF GitHub Bot commented on KYLIN-3700: --- codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-441728994 # [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=h1) Report > Merging [#362](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `27.86%`. [](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #362 +/- ## - Coverage 23.63% 23.63% -0.01% - Complexity 4828 4838 +10 Files 1130 1131 +1 Lines 6802368193 +170 Branches 9657 9684 +27 + Hits 1607916119 +40 - Misses5037750507 +130 Partials 1567 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...k/datasource/framework/SourceConnectorFactory.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2ZyYW1ld29yay9Tb3VyY2VDb25uZWN0b3JGYWN0b3J5LmphdmE=) | `57.89% <ø> (ø)` | `1 <0> (ø)` | :arrow_down: | | [...java/org/apache/kylin/metadata/model/TableRef.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVSZWYuamF2YQ==) | `51.16% <0%> (-1.22%)` | `11 <0> (ø)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/kylin/metadata/model/TableDesc.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVEZXNjLmphdmE=) | `46.01% <0%> (-1.76%)` | `27 <0> (ø)` | | | [...pache/kylin/source/hive/GarbageCollectionStep.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0dhcmJhZ2VDb2xsZWN0aW9uU3RlcC5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...a/org/apache/kylin/common/util/HiveCmdBuilder.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0hpdmVDbWRCdWlsZGVyLmphdmE=) | `83.33% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../kylin/source/jdbc/extensible/JdbcHiveMRInput.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVNUklucHV0LmphdmE=) | `89.79% <100%> (+0.9%)` | `3 <0> (ø)` | :arrow_down: | | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `32.71% <19.35%> (-3.38%)` | `12 <1> (ø)` | | | [.../apache/kylin/job/util/FlatTableSqlQuoteUtils.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0ZsYXRUYWJsZVNxbFF1b3RlVXRpbHMuamF2YQ==) | `21% <21%> (ø)` | `8 <8> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `51.04% <22.22%> (-3.18%)` | `14 <2> (+1)` | | | ... and [10 more](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ =
[GitHub] codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table
codecov-io edited a comment on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-441728994 # [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=h1) Report > Merging [#362](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/97ee98bfbd93510b578db93998f2ab488199ef40?src=pr&el=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `27.86%`. [](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #362 +/- ## - Coverage 23.63% 23.63% -0.01% - Complexity 4828 4838 +10 Files 1130 1131 +1 Lines 6802368193 +170 Branches 9657 9684 +27 + Hits 1607916119 +40 - Misses5037750507 +130 Partials 1567 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...k/datasource/framework/SourceConnectorFactory.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2ZyYW1ld29yay9Tb3VyY2VDb25uZWN0b3JGYWN0b3J5LmphdmE=) | `57.89% <ø> (ø)` | `1 <0> (ø)` | :arrow_down: | | [...java/org/apache/kylin/metadata/model/TableRef.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVSZWYuamF2YQ==) | `51.16% <0%> (-1.22%)` | `11 <0> (ø)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ava/org/apache/kylin/metadata/model/TableDesc.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvVGFibGVEZXNjLmphdmE=) | `46.01% <0%> (-1.76%)` | `27 <0> (ø)` | | | [...pache/kylin/source/hive/GarbageCollectionStep.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0dhcmJhZ2VDb2xsZWN0aW9uU3RlcC5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...a/org/apache/kylin/common/util/HiveCmdBuilder.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0hpdmVDbWRCdWlsZGVyLmphdmE=) | `83.33% <100%> (ø)` | `12 <0> (ø)` | :arrow_down: | | [.../kylin/source/jdbc/extensible/JdbcHiveMRInput.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVNUklucHV0LmphdmE=) | `89.79% <100%> (+0.9%)` | `3 <0> (ø)` | :arrow_down: | | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `32.71% <19.35%> (-3.38%)` | `12 <1> (ø)` | | | [.../apache/kylin/job/util/FlatTableSqlQuoteUtils.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0ZsYXRUYWJsZVNxbFF1b3RlVXRpbHMuamF2YQ==) | `21% <21%> (ø)` | `8 <8> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `51.04% <22.22%> (-3.18%)` | `14 <2> (+1)` | | | ... and [10 more](https://codecov.io/gh/apache/kylin/pull/362/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=footer). Last update [97ee98b...708163d](https://codecov.io/gh/apache/kylin/pull/362?src=pr&el=lastupdated). Rea
[GitHub] coveralls commented on issue #362: KYLIN-3700 Quote sql identities when creating flat table
coveralls commented on issue #362: KYLIN-3700 Quote sql identities when creating flat table URL: https://github.com/apache/kylin/pull/362#issuecomment-442330731 ## Pull Request Test Coverage Report for [Build 3896](https://coveralls.io/builds/20329464) * **72** of **244** **(29.51%)** changed or added relevant lines in **15** files are covered. * **6** unchanged lines in **3** files lost coverage. * Overall coverage decreased (**-0.006%**) to **25.941%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableRef.java](https://coveralls.io/builds/20329464/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableRef.java#L72) | 0 | 1 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/JdbcConnector.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2FJdbcConnector.java#L89) | 2 | 3 | 66.67% | [source-hive/src/main/java/org/apache/kylin/source/hive/GarbageCollectionStep.java](https://coveralls.io/builds/20329464/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FGarbageCollectionStep.java#L67) | 0 | 1 | 0.0% | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveInputBase.java](https://coveralls.io/builds/20329464/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveInputBase.java#L121) | 4 | 5 | 80.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/SqlConverter.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FSqlConverter.java#L75) | 7 | 9 | 77.78% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/DefaultConfiguer.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FDefaultConfiguer.java#L132) | 6 | 9 | 66.67% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TableDesc.java](https://coveralls.io/builds/20329464/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTableDesc.java#L223) | 0 | 6 | 0.0% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/framework/conv/ConvSqlWriter.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fframework%2Fconv%2FConvSqlWriter.java#L156) | 5 | 12 | 41.67% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveMRInput.java](https://coveralls.io/builds/20329464/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveMRInput.java#L160) | 3 | 11 | 27.27% | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FAbstractJdbcAdaptor.java#L300) | 4 | 18 | 22.22% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/20329464/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/20329464/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L235) | 2 | 0.0% | | [datasource-sdk/src/main/java/org/apache/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://coveralls.io/builds/20329464/source?filename=datasource-sdk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsdk%2Fdatasource%2Fadaptor%2FDefaultAdaptor.java#L145) | 3 | 35.51% | | Totals | [](https://coveralls.io/builds/20329464) | | :-- | --: | | Change from base [Build 3894](https://coveralls.io/builds/20327415): | -0.006% | | Covered Lines: | 17690 | | Relevant Lines: | 68193 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3705) Segment Pruner mis-functions when the source data has Chinese characters
jiezouSH created KYLIN-3705: --- Summary: Segment Pruner mis-functions when the source data has Chinese characters Key: KYLIN-3705 URL: https://issues.apache.org/jira/browse/KYLIN-3705 Project: Kylin Issue Type: Bug Affects Versions: v2.5.1 Reporter: jiezouSH Source Data: "合肥市"，“西安市”，“北京市” sql: select * from ZYZ.TEST_ZYZ_3 where name='合肥市' Actual Result: empty result. Root Cause: Dimension range is computed In factDistinctColumnsReducer, but unfortunately the dimension range is not written in UTF-character. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701336#comment-16701336
]
ASF subversion and git services commented on KYLIN-3669:
Commit 48a2bec01caa7b764d55bfa74b3da5a4936c598d in kylin's branch
refs/heads/2.5.x-hadoop3.1 from [~Aron.tao]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=48a2bec ]
KYLIN-3669, add log to indicate the case when using GTStreamAggregateScanner.
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
> Fix For: v2.5.2
>
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3517) Couldn't update coprocessor on HBase 2.0
[
https://issues.apache.org/jira/browse/KYLIN-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701341#comment-16701341
]
ASF subversion and git services commented on KYLIN-3517:
Commit f849925b87b972f23f6309f1ff9641b827497cdc in kylin's branch
refs/heads/2.5.x-hadoop3.1 from [~caolijun1166]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=f849925 ]
KYLIN-3517 Upadate coprocessor on HBase2.0 is avaliable.
Signed-off-by: shaofengshi
> Couldn't update coprocessor on HBase 2.0
>
>
> Key: KYLIN-3517
> URL: https://issues.apache.org/jira/browse/KYLIN-3517
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Shaofeng SHI
>Assignee: Lijun Cao
>Priority: Major
> Fix For: v2.5.0
>
>
> On HDP 3.0, run update coprocessor, got this error:
>
> {code:java}
> 2018-08-28 00:24:26,683 ERROR [pool-7-thread-1] util.DeployCoprocessorCLI:383
> : Error processing KYLIN_O9JRT8XOQ9
> java.lang.UnsupportedOperationException: HTableDescriptor is read-only
> at
> org.apache.hadoop.hbase.client.ImmutableHTableDescriptor.getDelegateeForModification(ImmutableHTableDescriptor.java:59)
> at
> org.apache.hadoop.hbase.HTableDescriptor.removeCoprocessor(HTableDescriptor.java:768)
> at
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:300)
> at
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI$ResetCoprocessorWorker.run(DeployCoprocessorCLI.java:375)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3518) Coprocessor reports NPE when execute a query on HBase 2.0
[
https://issues.apache.org/jira/browse/KYLIN-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701340#comment-16701340
]
ASF subversion and git services commented on KYLIN-3518:
Commit 4f7988f0189b1fe2f157a78813bb0cbe3a310961 in kylin's branch
refs/heads/2.5.x-hadoop3.1 from [~caolijun1166]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4f7988f ]
KYLIN-3518 Fix Coprocessor NPE problem on hbase 2
> Coprocessor reports NPE when execute a query on HBase 2.0
> -
>
> Key: KYLIN-3518
> URL: https://issues.apache.org/jira/browse/KYLIN-3518
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Reporter: Shaofeng SHI
>Assignee: Lijun Cao
>Priority: Major
> Fix For: v2.5.0
>
>
> On HDP 3.0, build a cube and then run a simple count query, NPE occurred:
>
> {code:java}
> 2018-08-28 01:30:16,969 DEBUG [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> v2.CubeHBaseRPC:315 : hbase.rpc.timeout = 9 ms, use 81000 ms as timeout
> for coprocessor
> 2018-08-28 01:30:16,983 INFO [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> v2.CubeHBaseEndpointRPC:141 : Serialized scanRequestBytes 522 bytes,
> rawScanBytesString 44 bytes
> 2018-08-28 01:30:16,984 INFO [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> v2.CubeHBaseEndpointRPC:143 : The scan 67b41fc6 for segment
> kylin_sales_cube_clone[2012010100_2013010100] is as below with 1
> separate raw scans, shard part of start/end key is set to 0
> 2018-08-28 01:30:16,991 INFO [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> v2.CubeHBaseRPC:288 : Visiting hbase table KYLIN_5Q088VO5I0: cuboid require
> post aggregation, from 0 to 16384 Start:
> \x00\x00\x00\x00\x00\x00\x00\x00\x40\x00\x00\x00\x00
> (\x00\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00) Stop:
> \x00\x00\x00\x00\x00\x00\x00\x00\x40\x00\xFF\xFF\xFF\x00
> (\x00\x00\x00\x00\x00\x00\x00\x00@\x00\xFF\xFF\xFF\x00), No Fuzzy Key
> 2018-08-28 01:30:16,991 DEBUG [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> v2.CubeHBaseEndpointRPC:148 : Submitting rpc to 1 shards starting from shard
> 0, scan range count 1
> 2018-08-28 01:30:17,010 INFO [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> common.KylinConfig:332 : Loading kylin-defaults.properties from
> file:/root/shaofengshi/apache-kylin-2.5.0-SNAPSHOT-bin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0-SNAPSHOT.jar!/kylin-defaults.properties
> 2018-08-28 01:30:17,033 DEBUG [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> common.KylinConfig:291 : KYLIN_CONF property was not set, will seek
> KYLIN_HOME env variable
> 2018-08-28 01:30:17,051 INFO [pool-14-thread-1] hbase.HBaseConnection:110 :
> Creating coprocessor thread pool with max of 2048, core of 2048
> 2018-08-28 01:30:17,094 INFO [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> gtrecord.SequentialCubeTupleIterator:73 : Using SortedIteratorMergerWithLimit
> to merge segment results
> 2018-08-28 01:30:17,097 DEBUG [Query f7bf8004-b516-e372-18df-0d507075d471-71]
> enumerator.OLAPEnumerator:117 : return TupleIterator...
> 2018-08-28 01:30:21,607 INFO [kylin-coproc--pool9-t1]
> client.RpcRetryingCallerImpl:134 : Call exception, tries=6, retries=6,
> started=4410 ms ago, cancelled=false, msg=java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:468)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:253)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:)
> at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:8032)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2426)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2408)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42010)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> ... 3 more
> , details=row '' on table 'KYLIN_5Q088VO5I0' at
> region=KYLIN_5Q088VO5I0,,1535417272444.27b82cb4702db4557a98b9a7e60b7692.,
> hostname=ignite03.com,16020,1534313612401, seqNum=2
> 2018-08-28 01:30:25,633 INFO [kylin-coproc--pool9-t1]
> client.RpcRetryingCallerImpl:134 : Call exception, tries=7, retries=7,
> started=8436 ms ago, cancelled=false, msg=java.io.IOException
> at org.apache.hadoop.hb
[jira] [Commented] (KYLIN-3693) TopN, Count distinct incorrect in Spark engine
[
https://issues.apache.org/jira/browse/KYLIN-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701334#comment-16701334
]
ASF subversion and git services commented on KYLIN-3693:
Commit 1a7789cb5f01b22d6ca81fc28fc16e8d27c8e8c4 in kylin's branch
refs/heads/2.5.x-hadoop3.1 from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=1a7789c ]
KYLIN-3693 TopN incorrect in Spark engine
> TopN, Count distinct incorrect in Spark engine
> --
>
> Key: KYLIN-3693
> URL: https://issues.apache.org/jira/browse/KYLIN-3693
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Affects Versions: v2.5.0, v2.5.1
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.5.2
>
>
> Long Chao firstly reported this bug; And then we found the currect
> integration test didn't cover this count distinct, topn measures with Spark
> engine. In "ITKylinQueryTest", some queries are not tested in the "left" join
> case, which means they were not covered the spark case. If we manually change
> to built the "left join" cube with spark, IT will report wrong result:
>
>
> {code:java}
> [ERROR] Failures:
> [ERROR]
> ITCombinationTest>ITKylinQueryTest.testPreciselyDistinctCountQuery:284->KylinTestBase.execAndCompQuery:544->KylinTestBase.execAndCompQuery:647->KylinTestBase.assertTableEquals:691
> value (table=query result of query01, row=0, col=USER_COUNT)
> expected:<8[46]> but was:<8[52]>
> [ERROR]
> ITCombinationTest>ITKylinQueryTest.testTopNQuery:277->KylinTestBase.execAndCompQuery:544->KylinTestBase.execAndCompQuery:647->KylinTestBase.assertTableEquals:691
> value (table=query result of query82, row=0, col=CAL_DT)
> expected:<2012-01-[28]> but was:<2012-01-[05]>
> [INFO]
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3704) Upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701337#comment-16701337 ] ASF subversion and git services commented on KYLIN-3704: Commit 4a205c37f96784b05537a52606844c5abdea2edd in kylin's branch refs/heads/2.5.x-hadoop3.1 from Yifei.Wu [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4a205c3 ] KYLIN-3704 upgrade calcite version to 1.16.0-kylin-r2 version > Upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v2.5.2 > > > we have updated the customized Calcite version, which rebase the > changes of Apache Calcite 1.16.0. And in this version, we did some changes > to adapt the Apache Calcite, Such as the change the Grouping function > calculate method and adjust the EXTRACT function. > As a result，this issue is also resolved. > https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3666) Mege cube step 2: Update dictionary throws IllegalStateException
[
https://issues.apache.org/jira/browse/KYLIN-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701335#comment-16701335
]
ASF subversion and git services commented on KYLIN-3666:
Commit b4280a13469865f847b6c985d19a98dcd9c59b44 in kylin's branch
refs/heads/2.5.x-hadoop3.1 from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=b4280a1 ]
KYLIN-3666 HDFS metadata url not be recognized
> Mege cube step 2: Update dictionary throws IllegalStateException
>
>
> Key: KYLIN-3666
> URL: https://issues.apache.org/jira/browse/KYLIN-3666
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.5.0
> Environment: cdh5.11.0
>Reporter: Xi Chen
>Assignee: Chao Long
>Priority: Major
> Fix For: v2.5.2
>
>
> MERGE CUBE job failed in Step 2, throwing such exception:
>
> {code:java}
> java.lang.IllegalStateException: Metadata uri :
> kylin_metadata_my_service@hdfs,path=viewfs://hadoop-footest/kylin/kylin_metadata_my_service/kylin-9583f16b-93cb-9461-a58c-43c848007972/kylin_sales_cube_clone_spark/metadata
> is not recognized
> at org.apache.kylin.common.KylinConfig.decideUriType(KylinConfig.java:172)
> at
> org.apache.kylin.common.KylinConfig.createInstanceFromUri(KylinConfig.java:182)
> at
> org.apache.kylin.engine.mr.steps.UpdateDictionaryStep.doWork(UpdateDictionaryStep.java:70)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: Metadata uri :
> kylin_metadata_my_service@hdfs,path=viewfs://hadoop-footest/kylin/kylin_metadata_my_service/kylin-9583f16b-93cb-9461-a58c-43c848007972/kylin_sales_cube_clone_spark/metadata
> looks like a file but it's neither a file nor a directory
> at org.apache.kylin.common.KylinConfig.decideUriType(KylinConfig.java:162)
> ... 9 more
> {code}
> The metadataUrl is on hdfs, but decideUriType() method does not recognize it.
> Thus throwing such exception.
> Both MR and Spark engine are affected.
> Would [~Wayne0101] please have a look? I see it's related to KYLIN-3471
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-2565) Support Hadoop 3.0
[ https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701342#comment-16701342 ] ASF subversion and git services commented on KYLIN-2565: Commit 4c76ed7571b80e04776549e77cc70d176db09c80 in kylin's branch refs/heads/2.5.x-hadoop3.1 from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4c76ed7 ] KYLIN-2565 Add cdh60 profile > Support Hadoop 3.0 > -- > > Key: KYLIN-2565 > URL: https://issues.apache.org/jira/browse/KYLIN-2565 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Wang Cheng >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below > is the Hadoop3.0 components requirements: > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2565) Support Hadoop 3.0
[ https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701338#comment-16701338 ] ASF subversion and git services commented on KYLIN-2565: Commit 12e1a2e51b742b7d50acabe842709facfdff5260 in kylin's branch refs/heads/2.5.x-hadoop3.1 from [~blue20080] [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=12e1a2e ] KYLIN-2565, upgrade to Hadoop3.0 > Support Hadoop 3.0 > -- > > Key: KYLIN-2565 > URL: https://issues.apache.org/jira/browse/KYLIN-2565 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Wang Cheng >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below > is the Hadoop3.0 components requirements: > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2565) Support Hadoop 3.0
[ https://issues.apache.org/jira/browse/KYLIN-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701339#comment-16701339 ] ASF subversion and git services commented on KYLIN-2565: Commit 7f2debff902d79898c7d9aee26f68a42db348edf in kylin's branch refs/heads/2.5.x-hadoop3.1 from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7f2debf ] KYLIN-2565 upgrade to hadoop 3.0 hbase 2.0, pass UT > Support Hadoop 3.0 > -- > > Key: KYLIN-2565 > URL: https://issues.apache.org/jira/browse/KYLIN-2565 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: Wang Cheng >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Hadoop3.0-alpha is released, Kylin should also keep compatible with it. Below > is the Hadoop3.0 components requirements: > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0.0+release -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3696) TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
[
https://issues.apache.org/jira/browse/KYLIN-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701316#comment-16701316
]
yangwei commented on KYLIN-3696:
好的感谢，这么快就把原因找出。
> TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
>
>
> Key: KYLIN-3696
> URL: https://issues.apache.org/jira/browse/KYLIN-3696
> Project: Kylin
> Issue Type: Bug
> Components: Measure - TopN
>Affects Versions: v2.5.1
>Reporter: yangwei
>Priority: Major
> Attachments: image-2018-11-20-10-57-28-546.png,
> image-2018-11-20-11-01-25-120.png, image-2018-11-20-11-27-43-750.png
>
>
> 我使用的是v2.5.1，度量topN使用上出现不准的总量。
> 问题再现：
> 一，二个cube使用同一个模型就是同一张物理事实表。
> 二，二个cube同时包含相同的topN度量
> 三，二个cube状态都是Ready
> 目前我暂时的解决方法是在其中一个cube去掉一个topN度量
> 同一个sql在hive与kylin里查的的结果对不上相差很远,下面给出sql
> SELECT IP ,
> SUM(ACCESS_COUNT) c
> FROM API_ACCESS
> WHERE TAG_DATE = CAST('2018-11-19' AS DATE)
> group by ip
> ORDER BY
> c DESC
> LIMIT 10;
> 二个cube中的度量:
> cube1:
> !image-2018-11-20-10-57-28-546.png!
> cube2:
> !image-2018-11-20-11-01-25-120.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
[
https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701308#comment-16701308
]
XiaoXiang Yu edited comment on KYLIN-3695 at 11/28/18 2:15 AM:
---
[~gauravbrills] Thank you for your suggestion, I will check it as soon as
possible.
was (Author: hit_lacus):
Thank you for your suggestion, I will check it as soon as possible.
> Error while creating hive table through Kylin build cube with mysql imported
> tables
> ---
>
> Key: KYLIN-3695
> URL: https://issues.apache.org/jira/browse/KYLIN-3695
> Project: Kylin
> Issue Type: Bug
> Components: Integration
>Affects Versions: v2.5.0
>Reporter: Gaurav Rawat
>Assignee: XiaoXiang Yu
>Priority: Minor
>
> HI I am trying to build a cube with Kylin, the data gets souced fine from
> sqoop but the next step for creating hive tables fails . Looking at the
> command being fired it looks weird as the create statement looks good to me .
> I think the issue is with DOUBLE types as when I remove the same the create
> statement works fine . Can someone please help .
> I am using the stack in AWS EMR, kylin 2.5 hive 2.3.0
> The errors logs with commands as as below, the table is a msql table which
> had columns with DOUBLE type
> Command
> {code:java}
> hive -e "USE default; DROP TABLE IF EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368;
> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 (
> HOLDINGS_STOCK_INVESTOR_ID string ,STOCK_INVESTORS_CHANNEL string
> ,STOCK_STOCK_ID string ,STOCK_DOMICILE string ,STOCK_STOCK_NM string
> ,STOCK_APPROACH string ,STOCK_STOCK_TYP string ,INVESTOR_ID string
> ,INVESTOR_NM string ,INVESTOR_DOMICILE_CNTRY string ,CLIENT_NM string
> ,INVESTOR_HOLDINGS_GROSS_ASSETS_USD double(22)
> ,INVESTOR_HOLDINGS_NET_ASSETS_USD double(22) ) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY '|' STORED AS TEXTFILE LOCATION
> 's3://wfg1tst-models/kylin/kylin_metadata/kylin-4ae3b18b-831b-da66-eb8c-7318245c4448/kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368';
> ALTER TABLE
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 SET
> TBLPROPERTIES('auto.purge'='true'); " --hiveconf hive.merge.mapredfiles=false
> --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2
> --hiveconf hive.exec.compress.output=true --hiveconf
> hive.auto.convert.join.noconditionaltask=true --hiveconf
> mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false
> --hiveconf hive.auto.convert.join.noconditionaltask.size=1 --hiveconf
> hive.stats.autogather=true{code}
> Error is as below
> {code:java}
> OK Time taken: 1.315 seconds OK Time taken: 0.09 seconds
> MismatchedTokenException(334!=347) at
> org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
> at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at
> org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333) at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1316) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1456) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop
[jira] [Commented] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
[
https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701308#comment-16701308
]
XiaoXiang Yu commented on KYLIN-3695:
-
Thank you for your suggestion, I will check it as soon as possible.
> Error while creating hive table through Kylin build cube with mysql imported
> tables
> ---
>
> Key: KYLIN-3695
> URL: https://issues.apache.org/jira/browse/KYLIN-3695
> Project: Kylin
> Issue Type: Bug
> Components: Integration
>Affects Versions: v2.5.0
>Reporter: Gaurav Rawat
>Assignee: XiaoXiang Yu
>Priority: Minor
>
> HI I am trying to build a cube with Kylin, the data gets souced fine from
> sqoop but the next step for creating hive tables fails . Looking at the
> command being fired it looks weird as the create statement looks good to me .
> I think the issue is with DOUBLE types as when I remove the same the create
> statement works fine . Can someone please help .
> I am using the stack in AWS EMR, kylin 2.5 hive 2.3.0
> The errors logs with commands as as below, the table is a msql table which
> had columns with DOUBLE type
> Command
> {code:java}
> hive -e "USE default; DROP TABLE IF EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368;
> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 (
> HOLDINGS_STOCK_INVESTOR_ID string ,STOCK_INVESTORS_CHANNEL string
> ,STOCK_STOCK_ID string ,STOCK_DOMICILE string ,STOCK_STOCK_NM string
> ,STOCK_APPROACH string ,STOCK_STOCK_TYP string ,INVESTOR_ID string
> ,INVESTOR_NM string ,INVESTOR_DOMICILE_CNTRY string ,CLIENT_NM string
> ,INVESTOR_HOLDINGS_GROSS_ASSETS_USD double(22)
> ,INVESTOR_HOLDINGS_NET_ASSETS_USD double(22) ) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY '|' STORED AS TEXTFILE LOCATION
> 's3://wfg1tst-models/kylin/kylin_metadata/kylin-4ae3b18b-831b-da66-eb8c-7318245c4448/kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368';
> ALTER TABLE
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 SET
> TBLPROPERTIES('auto.purge'='true'); " --hiveconf hive.merge.mapredfiles=false
> --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2
> --hiveconf hive.exec.compress.output=true --hiveconf
> hive.auto.convert.join.noconditionaltask=true --hiveconf
> mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false
> --hiveconf hive.auto.convert.join.noconditionaltask.size=1 --hiveconf
> hive.stats.autogather=true{code}
> Error is as below
> {code:java}
> OK Time taken: 1.315 seconds OK Time taken: 0.09 seconds
> MismatchedTokenException(334!=347) at
> org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
> at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at
> org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333) at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1316) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1456) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.util.RunJar.run(RunJar.java:221) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:136) FAILED: ParseException
> line 15:42 mismatched input '(' expecting ) n
[jira] [Resolved] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Liang resolved KYLIN-3665. --- Resolution: Fixed > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > Attachments: > 0001-KYLIN-3665-Partition-time-column-may-never-be-added.patch > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Liang updated KYLIN-3665: -- Flags: Patch Component/s: Metadata > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > Attachments: > 0001-KYLIN-3665-Partition-time-column-may-never-be-added.patch > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Liang updated KYLIN-3665: -- Attachment: 0001-KYLIN-3665-Partition-time-column-may-never-be-added.patch > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > Attachments: > 0001-KYLIN-3665-Partition-time-column-may-never-be-added.patch > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701236#comment-16701236 ] ASF subversion and git services commented on KYLIN-3559: Commit 97ee98bfbd93510b578db93998f2ab488199ef40 in kylin's branch refs/heads/master from Wu Bin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=97ee98b ] KYLIN-3559 Use Splitter for splitting String (#364) * KYLIN-3559 Use Splitter for splitting String * KYLIN-3559 Resolve UT issues * KYLIN-3559 fix IT issues > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[
https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701233#comment-16701233
]
ASF GitHub Bot commented on KYLIN-3559:
---
shaofengshi closed pull request #364: KYLIN-3559 Use Splitter for splitting
String
URL: https://github.com/apache/kylin/pull/364
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
b/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
index 84c1da5919..4fc37c9154 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
@@ -22,6 +22,9 @@
import java.util.Collection;
import java.util.Locale;
+
+import com.google.common.base.Splitter;
+import com.google.common.collect.Iterables;
import org.apache.commons.lang.StringUtils;
/**
@@ -159,14 +162,17 @@ public static void appendWithSeparator(StringBuilder src,
String append) {
}
public static String[] splitAndTrim(String str, String splitBy) {
-String[] split = str.split(splitBy);
-ArrayList r = new ArrayList<>(split.length);
-for (String s : split) {
-s = s.trim();
-if (!s.isEmpty())
-r.add(s);
-}
-return r.toArray(new String[r.size()]);
+Splitter splitterWithTrim =
Splitter.on(splitBy).trimResults().omitEmptyStrings();
+
+return Iterables.toArray(splitterWithTrim.split(str), String.class);
+}
+
+public static String[] split(String str, String splitBy) {
+return Iterables.toArray(Splitter.on(splitBy).split(str),
String.class);
+}
+
+public static String[] splitByComma(String str) {
+return split(str, ",");
}
// calculating length in UTF-8 of Java String without actually encoding it
diff --git
a/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
b/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
new file mode 100644
index 00..88ed9d84a8
--- /dev/null
+++ b/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.common.util;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class StringUtilTest {
+@Test
+public void splitTest() {
+String normalText = "Try to make the code better";
+String[] expected = new String[] { "Try", "to", "make", "the", "code",
"better" };
+Assert.assertArrayEquals(expected, StringUtil.split(normalText, " "));
+
+// case in http://errorprone.info/bugpattern/StringSplitter
+expected = new String[] { "" };
+Assert.assertArrayEquals(expected, StringUtil.split("", ":"));
+
+expected = new String[] { "", "" };
+Assert.assertArrayEquals(expected, StringUtil.split(":", ":"));
+
+expected = new String[] { "1", "2" };
+Assert.assertArrayEquals(expected, StringUtil.split("1<|>2", "<|>"));
+}
+
+@Test
+public void splitAndTrimTest() {
+String[] expected = new String[] { "foo", "bar" };
+Assert.assertArrayEquals(expected, StringUtil.splitAndTrim(" foo...
bar. ", "."));
+}
+
+@Test
+public void splitByCommaTest() {
+String[] expected = new String[] { "Hello", "Kylin" };
+Assert.assertArrayEquals(expected,
StringUtil.splitByComma("Hello,Kylin"));
+}
+}
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
b/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
index 2eaebb1a9a..b149f11dec 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
@@ -2
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701237#comment-16701237 ] ASF subversion and git services commented on KYLIN-3559: Commit 97ee98bfbd93510b578db93998f2ab488199ef40 in kylin's branch refs/heads/master from Wu Bin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=97ee98b ] KYLIN-3559 Use Splitter for splitting String (#364) * KYLIN-3559 Use Splitter for splitting String * KYLIN-3559 Resolve UT issues * KYLIN-3559 fix IT issues > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701235#comment-16701235 ] ASF subversion and git services commented on KYLIN-3559: Commit 97ee98bfbd93510b578db93998f2ab488199ef40 in kylin's branch refs/heads/master from Wu Bin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=97ee98b ] KYLIN-3559 Use Splitter for splitting String (#364) * KYLIN-3559 Use Splitter for splitting String * KYLIN-3559 Resolve UT issues * KYLIN-3559 fix IT issues > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701234#comment-16701234 ] ASF subversion and git services commented on KYLIN-3559: Commit 97ee98bfbd93510b578db93998f2ab488199ef40 in kylin's branch refs/heads/master from Wu Bin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=97ee98b ] KYLIN-3559 Use Splitter for splitting String (#364) * KYLIN-3559 Use Splitter for splitting String * KYLIN-3559 Resolve UT issues * KYLIN-3559 fix IT issues > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #364: KYLIN-3559 Use Splitter for splitting String
shaofengshi closed pull request #364: KYLIN-3559 Use Splitter for splitting
String
URL: https://github.com/apache/kylin/pull/364
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
b/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
index 84c1da5919..4fc37c9154 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/StringUtil.java
@@ -22,6 +22,9 @@
import java.util.Collection;
import java.util.Locale;
+
+import com.google.common.base.Splitter;
+import com.google.common.collect.Iterables;
import org.apache.commons.lang.StringUtils;
/**
@@ -159,14 +162,17 @@ public static void appendWithSeparator(StringBuilder src,
String append) {
}
public static String[] splitAndTrim(String str, String splitBy) {
-String[] split = str.split(splitBy);
-ArrayList r = new ArrayList<>(split.length);
-for (String s : split) {
-s = s.trim();
-if (!s.isEmpty())
-r.add(s);
-}
-return r.toArray(new String[r.size()]);
+Splitter splitterWithTrim =
Splitter.on(splitBy).trimResults().omitEmptyStrings();
+
+return Iterables.toArray(splitterWithTrim.split(str), String.class);
+}
+
+public static String[] split(String str, String splitBy) {
+return Iterables.toArray(Splitter.on(splitBy).split(str),
String.class);
+}
+
+public static String[] splitByComma(String str) {
+return split(str, ",");
}
// calculating length in UTF-8 of Java String without actually encoding it
diff --git
a/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
b/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
new file mode 100644
index 00..88ed9d84a8
--- /dev/null
+++ b/core-common/src/test/java/org/apache/kylin/common/util/StringUtilTest.java
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.common.util;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class StringUtilTest {
+@Test
+public void splitTest() {
+String normalText = "Try to make the code better";
+String[] expected = new String[] { "Try", "to", "make", "the", "code",
"better" };
+Assert.assertArrayEquals(expected, StringUtil.split(normalText, " "));
+
+// case in http://errorprone.info/bugpattern/StringSplitter
+expected = new String[] { "" };
+Assert.assertArrayEquals(expected, StringUtil.split("", ":"));
+
+expected = new String[] { "", "" };
+Assert.assertArrayEquals(expected, StringUtil.split(":", ":"));
+
+expected = new String[] { "1", "2" };
+Assert.assertArrayEquals(expected, StringUtil.split("1<|>2", "<|>"));
+}
+
+@Test
+public void splitAndTrimTest() {
+String[] expected = new String[] { "foo", "bar" };
+Assert.assertArrayEquals(expected, StringUtil.splitAndTrim(" foo...
bar. ", "."));
+}
+
+@Test
+public void splitByCommaTest() {
+String[] expected = new String[] { "Hello", "Kylin" };
+Assert.assertArrayEquals(expected,
StringUtil.splitByComma("Hello,Kylin"));
+}
+}
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
b/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
index 2eaebb1a9a..b149f11dec 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/cli/CubeSignatureRefresher.java
@@ -24,6 +24,7 @@
import org.apache.commons.lang.ArrayUtils;
import org.apache.kylin.common.KylinConfig;
import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.util.StringUtil;
import org.apache.kylin.cube.CubeDescManager;
[jira] [Commented] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
[
https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700673#comment-16700673
]
Gaurav Rawat commented on KYLIN-3695:
-
Thanks [~hit_lacus] I had observed the same hence mentioned the same in
stackoverflow . But the second question I had
??Also observer that if you use DECIMAL(10) it does not work unless it has a
precision while building mode for the same it gives error as tries to use
precision as undefined ie . DECIMAL(10,undefined)??
This looks like a bug as kylin seems to default it as undefined if you dont
have a precision in DECIMAL type while importing from mysql , i.e something
like DECIMAL(10)
> Error while creating hive table through Kylin build cube with mysql imported
> tables
> ---
>
> Key: KYLIN-3695
> URL: https://issues.apache.org/jira/browse/KYLIN-3695
> Project: Kylin
> Issue Type: Bug
> Components: Integration
>Affects Versions: v2.5.0
>Reporter: Gaurav Rawat
>Assignee: XiaoXiang Yu
>Priority: Minor
>
> HI I am trying to build a cube with Kylin, the data gets souced fine from
> sqoop but the next step for creating hive tables fails . Looking at the
> command being fired it looks weird as the create statement looks good to me .
> I think the issue is with DOUBLE types as when I remove the same the create
> statement works fine . Can someone please help .
> I am using the stack in AWS EMR, kylin 2.5 hive 2.3.0
> The errors logs with commands as as below, the table is a msql table which
> had columns with DOUBLE type
> Command
> {code:java}
> hive -e "USE default; DROP TABLE IF EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368;
> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 (
> HOLDINGS_STOCK_INVESTOR_ID string ,STOCK_INVESTORS_CHANNEL string
> ,STOCK_STOCK_ID string ,STOCK_DOMICILE string ,STOCK_STOCK_NM string
> ,STOCK_APPROACH string ,STOCK_STOCK_TYP string ,INVESTOR_ID string
> ,INVESTOR_NM string ,INVESTOR_DOMICILE_CNTRY string ,CLIENT_NM string
> ,INVESTOR_HOLDINGS_GROSS_ASSETS_USD double(22)
> ,INVESTOR_HOLDINGS_NET_ASSETS_USD double(22) ) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY '|' STORED AS TEXTFILE LOCATION
> 's3://wfg1tst-models/kylin/kylin_metadata/kylin-4ae3b18b-831b-da66-eb8c-7318245c4448/kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368';
> ALTER TABLE
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 SET
> TBLPROPERTIES('auto.purge'='true'); " --hiveconf hive.merge.mapredfiles=false
> --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2
> --hiveconf hive.exec.compress.output=true --hiveconf
> hive.auto.convert.join.noconditionaltask=true --hiveconf
> mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false
> --hiveconf hive.auto.convert.join.noconditionaltask.size=1 --hiveconf
> hive.stats.autogather=true{code}
> Error is as below
> {code:java}
> OK Time taken: 1.315 seconds OK Time taken: 0.09 seconds
> MismatchedTokenException(334!=347) at
> org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
> at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at
> org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
> at
> org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333) at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:204) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77) at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70) at
> org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at
> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1316) at
> org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1456) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236) at
> org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226) at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) at
> org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) at
> org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(N
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[
https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700523#comment-16700523
]
ASF GitHub Bot commented on KYLIN-3597:
---
shaofengshi closed pull request #363: KYLIN-3597 fix static code issues
URL: https://github.com/apache/kylin/pull/363
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
b/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
index 33fc31a987..a2679219f3 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
@@ -41,7 +41,6 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
-
import sun.misc.Unsafe;
/**
@@ -116,6 +115,7 @@
// JHat says BU is 56 bytes.
// SizeOf which uses java.lang.instrument says 24 bytes. (3 longs?)
public static final int ESTIMATED_HEAP_TAX = 16;
+public static final String LENGTH_MUST_BE_GREATER_THAN_0 = "length must be
greater than 0";
/**
* Returns length of the byte array, returning 0 if the array is null.
@@ -491,7 +491,7 @@ public static int putLong(byte[] bytes, int offset, long
val) {
* @return incremented offset
*/
public static int putLongUnsafe(byte[] bytes, int offset, long val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Long.reverseBytes(val);
}
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe.putLong(bytes,
@@ -652,7 +652,7 @@ public static int toInt(byte[] bytes, int offset, final int
length) {
* @return the int value
*/
public static int toIntUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Integer.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getInt(bytes, (long) offset
@@ -672,7 +672,7 @@ public static int toIntUnsafe(byte[] bytes, int offset) {
* @return the short value
*/
public static short toShortUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Short.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getShort(bytes, (long) offset
@@ -692,7 +692,7 @@ public static short toShortUnsafe(byte[] bytes, int offset)
{
* @return the long value
*/
public static long toLongUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Long.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getLong(bytes, (long) offset
@@ -763,7 +763,7 @@ public static int putInt(byte[] bytes, int offset, int val)
{
* @return incremented offset
*/
public static int putIntUnsafe(byte[] bytes, int offset, int val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Integer.reverseBytes(val);
}
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe.putInt(bytes,
@@ -880,7 +880,7 @@ public static int putShort(byte[] bytes, int offset, short
val) {
* @return incremented offset
*/
public static int putShortUnsafe(byte[] bytes, int offset, short val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Short.re
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700524#comment-16700524 ] ASF subversion and git services commented on KYLIN-3597: Commit 7bcc6987e5bbcfd57f320aa19c4fba67ed02aaea in kylin's branch refs/heads/master from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7bcc698 ] KYLIN-3597 fix static code issues fix UT > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #363: KYLIN-3597 fix static code issues
shaofengshi closed pull request #363: KYLIN-3597 fix static code issues
URL: https://github.com/apache/kylin/pull/363
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
b/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
index 33fc31a987..a2679219f3 100644
--- a/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
+++ b/core-common/src/main/java/org/apache/kylin/common/util/Bytes.java
@@ -41,7 +41,6 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
-
import sun.misc.Unsafe;
/**
@@ -116,6 +115,7 @@
// JHat says BU is 56 bytes.
// SizeOf which uses java.lang.instrument says 24 bytes. (3 longs?)
public static final int ESTIMATED_HEAP_TAX = 16;
+public static final String LENGTH_MUST_BE_GREATER_THAN_0 = "length must be
greater than 0";
/**
* Returns length of the byte array, returning 0 if the array is null.
@@ -491,7 +491,7 @@ public static int putLong(byte[] bytes, int offset, long
val) {
* @return incremented offset
*/
public static int putLongUnsafe(byte[] bytes, int offset, long val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Long.reverseBytes(val);
}
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe.putLong(bytes,
@@ -652,7 +652,7 @@ public static int toInt(byte[] bytes, int offset, final int
length) {
* @return the int value
*/
public static int toIntUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Integer.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getInt(bytes, (long) offset
@@ -672,7 +672,7 @@ public static int toIntUnsafe(byte[] bytes, int offset) {
* @return the short value
*/
public static short toShortUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Short.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getShort(bytes, (long) offset
@@ -692,7 +692,7 @@ public static short toShortUnsafe(byte[] bytes, int offset)
{
* @return the long value
*/
public static long toLongUnsafe(byte[] bytes, int offset) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
return Long.reverseBytes(
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe
.getLong(bytes, (long) offset
@@ -763,7 +763,7 @@ public static int putInt(byte[] bytes, int offset, int val)
{
* @return incremented offset
*/
public static int putIntUnsafe(byte[] bytes, int offset, int val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Integer.reverseBytes(val);
}
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe.putInt(bytes,
@@ -880,7 +880,7 @@ public static int putShort(byte[] bytes, int offset, short
val) {
* @return incremented offset
*/
public static int putShortUnsafe(byte[] bytes, int offset, short val) {
-if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.littleEndian)
{
+if
(org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.LITTLEENDIAN)
{
val = Short.reverseBytes(val);
}
org.apache.kylin.common.util.Bytes.LexicographicalComparerHolder.UnsafeComparer.theUnsafe.putShort(bytes,
@@ -1097,7 +1097,7 @@ public Object run() {
}
}
-static final boolean
[GitHub] codecov-io commented on issue #363: KYLIN-3597 fix static code issues
codecov-io commented on issue #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363#issuecomment-442078521 # [Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@2d8e0fc`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `40.59%`. [](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #363 +/- ## = Coverage ? 23.62% Complexity? 4824 = Files ? 1130 Lines ?68022 Branches ? 9657 = Hits ?16072 Misses?50383 Partials ? 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `36.08% <ø> (ø)` | `12 <0> (?)` | | | [...in/rest/broadcaster/BroadcasterReceiveServlet.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvYnJvYWRjYXN0ZXIvQnJvYWRjYXN0ZXJSZWNlaXZlU2VydmxldC5qYXZh) | `0% <ø> (ø)` | `0 <0> (?)` | | | [.../org/apache/kylin/engine/mr/ByteArrayWritable.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvQnl0ZUFycmF5V3JpdGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...e/kylin/source/kafka/util/KafkaSampleProducer.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-c291cmNlLWthZmthL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zb3VyY2Uva2Fma2EvdXRpbC9LYWZrYVNhbXBsZVByb2R1Y2VyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../main/java/org/apache/kylin/common/util/Bytes.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0J5dGVzLmphdmE=) | `1.01% <0%> (ø)` | `3 <0> (?)` | | | [...re/bitmap/BitmapIntersectDistinctCountAggFunc.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iaXRtYXAvQml0bWFwSW50ZXJzZWN0RGlzdGluY3RDb3VudEFnZ0Z1bmMuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...apache/kylin/job/execution/AbstractExecutable.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vQWJzdHJhY3RFeGVjdXRhYmxlLmphdmE=) | `51.4% <23.07%> (ø)` | `48 <3> (?)` | | | [...be/model/validation/rule/AggregationGroupRule.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL3ZhbGlkYXRpb24vcnVsZS9BZ2dyZWdhdGlvbkdyb3VwUnVsZS5qYXZh) | `54.63% <27.27%> (ø)` | `20 <17> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `54.21% <33.33%> (ø)` | `13 <0> (?)` | | | [...g/apache/kylin/metadata/cachesync/Broadcaster.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvY2FjaGVzeW5jL0Jyb2FkY2FzdGVyLmphdmE=) | `52.04% <75%> (ø)` | `21 <1> (?)` | | | ... and [1 more](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=footer). Last update [2d8e0fc...516e16c](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700501#comment-16700501 ] ASF GitHub Bot commented on KYLIN-3597: --- codecov-io commented on issue #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363#issuecomment-442078521 # [Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@2d8e0fc`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `40.59%`. [](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #363 +/- ## = Coverage ? 23.62% Complexity? 4824 = Files ? 1130 Lines ?68022 Branches ? 9657 = Hits ?16072 Misses?50383 Partials ? 1567 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/kylin/sdk/datasource/adaptor/DefaultAdaptor.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvRGVmYXVsdEFkYXB0b3IuamF2YQ==) | `36.08% <ø> (ø)` | `12 <0> (?)` | | | [...in/rest/broadcaster/BroadcasterReceiveServlet.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvYnJvYWRjYXN0ZXIvQnJvYWRjYXN0ZXJSZWNlaXZlU2VydmxldC5qYXZh) | `0% <ø> (ø)` | `0 <0> (?)` | | | [.../org/apache/kylin/engine/mr/ByteArrayWritable.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvQnl0ZUFycmF5V3JpdGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...e/kylin/source/kafka/util/KafkaSampleProducer.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-c291cmNlLWthZmthL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zb3VyY2Uva2Fma2EvdXRpbC9LYWZrYVNhbXBsZVByb2R1Y2VyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../main/java/org/apache/kylin/common/util/Bytes.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0J5dGVzLmphdmE=) | `1.01% <0%> (ø)` | `3 <0> (?)` | | | [...re/bitmap/BitmapIntersectDistinctCountAggFunc.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iaXRtYXAvQml0bWFwSW50ZXJzZWN0RGlzdGluY3RDb3VudEFnZ0Z1bmMuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...apache/kylin/job/execution/AbstractExecutable.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vQWJzdHJhY3RFeGVjdXRhYmxlLmphdmE=) | `51.4% <23.07%> (ø)` | `48 <3> (?)` | | | [...be/model/validation/rule/AggregationGroupRule.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL3ZhbGlkYXRpb24vcnVsZS9BZ2dyZWdhdGlvbkdyb3VwUnVsZS5qYXZh) | `54.63% <27.27%> (ø)` | `20 <17> (?)` | | | [...in/sdk/datasource/adaptor/AbstractJdbcAdaptor.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-ZGF0YXNvdXJjZS1zZGsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Nkay9kYXRhc291cmNlL2FkYXB0b3IvQWJzdHJhY3RKZGJjQWRhcHRvci5qYXZh) | `54.21% <33.33%> (ø)` | `13 <0> (?)` | | | [...g/apache/kylin/metadata/cachesync/Broadcaster.java](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvY2FjaGVzeW5jL0Jyb2FkY2FzdGVyLmphdmE=) | `52.04% <75%> (ø)` | `21 <1> (?)` | | | ... and [1 more](https://codecov.io/gh/apache/kylin/pull/363/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/363?src=pr&el=footer). Last
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700496#comment-16700496 ] ASF GitHub Bot commented on KYLIN-3559: --- whuwb opened a new pull request #364: KYLIN-3559 Use Splitter for splitting String URL: https://github.com/apache/kylin/pull/364 Issue: https://issues.apache.org/jira/browse/KYLIN-3559 Solution: Use Splitter for splitting String IT: [INFO] Reactor Summary for Apache Kylin 2.6.0-SNAPSHOT: [INFO] [INFO] Apache Kylin ... SUCCESS [ 6.852 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 16.358 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 55.479 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [05:24 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:35 min] [INFO] Apache Kylin - Core Metrics SUCCESS [ 2.747 s] [INFO] Apache Kylin - Core Job SUCCESS [ 45.493 s] [INFO] Apache Kylin - Core Storage SUCCESS [ 9.592 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 24.407 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 8.147 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 6.442 s] [INFO] Apache Kylin - DataSource SDK .. SUCCESS [ 11.310 s] [INFO] Apache Kylin - Jdbc Source . SUCCESS [ 24.863 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 6.304 s] [INFO] Apache Kylin - Cache ... SUCCESS [ 9.572 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 19.651 s] [INFO] Apache Kylin - Query ... SUCCESS [ 19.320 s] [INFO] Apache Kylin - Metrics Reporter Hive ... SUCCESS [ 2.167 s] [INFO] Apache Kylin - Metrics Reporter Kafka .. SUCCESS [ 1.122 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 27.527 s] [INFO] Apache Kylin - REST Server . SUCCESS [01:32 min] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 12.682 s] [INFO] Apache Kylin - Assembly SUCCESS [ 20.715 s] [INFO] Apache Kylin - Tool SUCCESS [ 14.791 s] [INFO] Apache Kylin - Tool Assembly ... SUCCESS [ 5.632 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 02:38 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 1.715 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 02:52 h [INFO] Finished at: 2018-11-17T22:57:50Z [INFO] This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700497#comment-16700497 ] ASF GitHub Bot commented on KYLIN-3559: --- asfgit commented on issue #364: KYLIN-3559 Use Splitter for splitting String URL: https://github.com/apache/kylin/pull/364#issuecomment-442076913 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: Wu Bin >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #364: KYLIN-3559 Use Splitter for splitting String
asfgit commented on issue #364: KYLIN-3559 Use Splitter for splitting String URL: https://github.com/apache/kylin/pull/364#issuecomment-442076913 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] whuwb opened a new pull request #364: KYLIN-3559 Use Splitter for splitting String
whuwb opened a new pull request #364: KYLIN-3559 Use Splitter for splitting String URL: https://github.com/apache/kylin/pull/364 Issue: https://issues.apache.org/jira/browse/KYLIN-3559 Solution: Use Splitter for splitting String IT: [INFO] Reactor Summary for Apache Kylin 2.6.0-SNAPSHOT: [INFO] [INFO] Apache Kylin ... SUCCESS [ 6.852 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 16.358 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 55.479 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [05:24 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:35 min] [INFO] Apache Kylin - Core Metrics SUCCESS [ 2.747 s] [INFO] Apache Kylin - Core Job SUCCESS [ 45.493 s] [INFO] Apache Kylin - Core Storage SUCCESS [ 9.592 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 24.407 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 8.147 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 6.442 s] [INFO] Apache Kylin - DataSource SDK .. SUCCESS [ 11.310 s] [INFO] Apache Kylin - Jdbc Source . SUCCESS [ 24.863 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 6.304 s] [INFO] Apache Kylin - Cache ... SUCCESS [ 9.572 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 19.651 s] [INFO] Apache Kylin - Query ... SUCCESS [ 19.320 s] [INFO] Apache Kylin - Metrics Reporter Hive ... SUCCESS [ 2.167 s] [INFO] Apache Kylin - Metrics Reporter Kafka .. SUCCESS [ 1.122 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 27.527 s] [INFO] Apache Kylin - REST Server . SUCCESS [01:32 min] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 12.682 s] [INFO] Apache Kylin - Assembly SUCCESS [ 20.715 s] [INFO] Apache Kylin - Tool SUCCESS [ 14.791 s] [INFO] Apache Kylin - Tool Assembly ... SUCCESS [ 5.632 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 02:38 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 1.715 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 02:52 h [INFO] Finished at: 2018-11-17T22:57:50Z [INFO] This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700481#comment-16700481 ] ASF subversion and git services commented on KYLIN-3665: Commit 4e6a69b7fa819918b33a5f1a5d25308ae316df97 in kylin's branch refs/heads/master from Feng Liang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=4e6a69b ] KYLIN-3665 Partition time column may never be added > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3665) Partition time column may never be added
[
https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700480#comment-16700480
]
ASF GitHub Bot commented on KYLIN-3665:
---
shaofengshi closed pull request #360: KYLIN-3665 Partition time column may
never be added
URL: https://github.com/apache/kylin/pull/360
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
b/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
index dcb37ecdad..56ededb995 100644
---
a/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
+++
b/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
@@ -200,8 +200,12 @@ public String buildDateRangeCondition(PartitionDesc
partDesc, ISegment seg, Segm
StringBuilder builder = new StringBuilder();
if (partDesc.partitionColumnIsYmdInt()) {
-buildSingleColumnRangeCondAsYmdInt(builder,
partitionDateColumn, startInclusive, endExclusive,
-partDesc.getPartitionDateFormat());
+if (partitionTimeColumn == null) {
+buildSingleColumnRangeCondAsYmdInt(builder,
partitionDateColumn, startInclusive, endExclusive,
partDesc.getPartitionDateFormat());
+} else {
+buildMultipleColumnRangeCondition(builder,
partitionDateColumn, partitionTimeColumn, startInclusive,
+endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat(), true);
+}
} else if (partDesc.partitionColumnIsTimeMillis()) {
buildSingleColumnRangeCondAsTimeMillis(builder,
partitionDateColumn, startInclusive, endExclusive);
} else if (partitionDateColumn != null && partitionTimeColumn ==
null) {
@@ -212,14 +216,14 @@ public String buildDateRangeCondition(PartitionDesc
partDesc, ISegment seg, Segm
partDesc.getPartitionTimeFormat());
} else if (partitionDateColumn != null && partitionTimeColumn !=
null) {
buildMultipleColumnRangeCondition(builder,
partitionDateColumn, partitionTimeColumn, startInclusive,
-endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat());
+endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat(), false);
}
return builder.toString();
}
private static void
buildSingleColumnRangeCondAsTimeMillis(StringBuilder builder, TblColRef
partitionColumn,
-long startInclusive, long endExclusive) {
+ long
startInclusive, long endExclusive) {
String partitionColumnName = partitionColumn.getIdentity();
builder.append(partitionColumnName + " >= " + startInclusive);
builder.append(" AND ");
@@ -227,7 +231,7 @@ private static void
buildSingleColumnRangeCondAsTimeMillis(StringBuilder builder
}
private static void buildSingleColumnRangeCondAsYmdInt(StringBuilder
builder, TblColRef partitionColumn,
-long startInclusive, long endExclusive, String
partitionColumnDateFormat) {
+ long
startInclusive, long endExclusive, String partitionColumnDateFormat) {
String partitionColumnName = partitionColumn.getIdentity();
builder.append(partitionColumnName + " >= "
+ DateFormat.formatToDateStr(startInclusive,
partitionColumnDateFormat));
@@ -237,7 +241,7 @@ private static void
buildSingleColumnRangeCondAsYmdInt(StringBuilder builder, Tb
}
private static void buildSingleColumnRangeCondition(StringBuilder
builder, TblColRef partitionColumn,
-long startInclusive, long endExclusive, String
partitionColumnDateFormat) {
+long
startInclusive, long endExclusive, String partitionColumnDateFormat) {
String partitionColumnName = partitionColumn.getIdentity();
if (endExclusive <= startInclusive) {
@@ -261,36 +265,37 @@ private static void
buildSingleColumnRangeCondition(StringBuilder builder, TblCo
}
private static void buildMultipleColumnRangeCondition(StringBuilder
builder, TblColRef partitionDateColumn,
-TblColRef partitionTimeColumn, long startInclusive, long
endExclusive, String partitionColumnDateFormat,
-
[GitHub] shaofengshi closed pull request #360: KYLIN-3665 Partition time column may never be added
shaofengshi closed pull request #360: KYLIN-3665 Partition time column may
never be added
URL: https://github.com/apache/kylin/pull/360
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
b/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
index dcb37ecdad..56ededb995 100644
---
a/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
+++
b/core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java
@@ -200,8 +200,12 @@ public String buildDateRangeCondition(PartitionDesc
partDesc, ISegment seg, Segm
StringBuilder builder = new StringBuilder();
if (partDesc.partitionColumnIsYmdInt()) {
-buildSingleColumnRangeCondAsYmdInt(builder,
partitionDateColumn, startInclusive, endExclusive,
-partDesc.getPartitionDateFormat());
+if (partitionTimeColumn == null) {
+buildSingleColumnRangeCondAsYmdInt(builder,
partitionDateColumn, startInclusive, endExclusive,
partDesc.getPartitionDateFormat());
+} else {
+buildMultipleColumnRangeCondition(builder,
partitionDateColumn, partitionTimeColumn, startInclusive,
+endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat(), true);
+}
} else if (partDesc.partitionColumnIsTimeMillis()) {
buildSingleColumnRangeCondAsTimeMillis(builder,
partitionDateColumn, startInclusive, endExclusive);
} else if (partitionDateColumn != null && partitionTimeColumn ==
null) {
@@ -212,14 +216,14 @@ public String buildDateRangeCondition(PartitionDesc
partDesc, ISegment seg, Segm
partDesc.getPartitionTimeFormat());
} else if (partitionDateColumn != null && partitionTimeColumn !=
null) {
buildMultipleColumnRangeCondition(builder,
partitionDateColumn, partitionTimeColumn, startInclusive,
-endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat());
+endExclusive, partDesc.getPartitionDateFormat(),
partDesc.getPartitionTimeFormat(), false);
}
return builder.toString();
}
private static void
buildSingleColumnRangeCondAsTimeMillis(StringBuilder builder, TblColRef
partitionColumn,
-long startInclusive, long endExclusive) {
+ long
startInclusive, long endExclusive) {
String partitionColumnName = partitionColumn.getIdentity();
builder.append(partitionColumnName + " >= " + startInclusive);
builder.append(" AND ");
@@ -227,7 +231,7 @@ private static void
buildSingleColumnRangeCondAsTimeMillis(StringBuilder builder
}
private static void buildSingleColumnRangeCondAsYmdInt(StringBuilder
builder, TblColRef partitionColumn,
-long startInclusive, long endExclusive, String
partitionColumnDateFormat) {
+ long
startInclusive, long endExclusive, String partitionColumnDateFormat) {
String partitionColumnName = partitionColumn.getIdentity();
builder.append(partitionColumnName + " >= "
+ DateFormat.formatToDateStr(startInclusive,
partitionColumnDateFormat));
@@ -237,7 +241,7 @@ private static void
buildSingleColumnRangeCondAsYmdInt(StringBuilder builder, Tb
}
private static void buildSingleColumnRangeCondition(StringBuilder
builder, TblColRef partitionColumn,
-long startInclusive, long endExclusive, String
partitionColumnDateFormat) {
+long
startInclusive, long endExclusive, String partitionColumnDateFormat) {
String partitionColumnName = partitionColumn.getIdentity();
if (endExclusive <= startInclusive) {
@@ -261,36 +265,37 @@ private static void
buildSingleColumnRangeCondition(StringBuilder builder, TblCo
}
private static void buildMultipleColumnRangeCondition(StringBuilder
builder, TblColRef partitionDateColumn,
-TblColRef partitionTimeColumn, long startInclusive, long
endExclusive, String partitionColumnDateFormat,
-String partitionColumnTimeFormat) {
+ TblColRef
partitionTimeColumn, long startInclusive, long endExclusive, String
partitionColumnDateFormat,
+
[jira] [Comment Edited] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
[
https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700285#comment-16700285
]
XiaoXiang Yu edited comment on KYLIN-3695 at 11/27/18 11:53 AM:
Hi [~gauravbrills]
I installed the newest stable hive in my test env, which version is 2.3.4(
_Hive DDL doc says 2.2.0 support double with precision_). I found hive didn't
not support double with specific/user-defined precision.
e.g.
* [ERROR] create table haha(ha double 10);
* [ERROR] create table haha(ha double Precision 17);
* [ERROR] create table haha(ha double(22));
* [CORRECT] create table haha(ha double Precision);
{quote}hive> use ss;
OK
Time taken: 0.015 seconds
hive> create table haha(ha double 10);
FAILED: ParseException line 1:28 extraneous input '10' expecting ) near ''
hive> create table haha(ha double Precision 17);
FAILED: ParseException line 1:38 extraneous input '17' expecting ) near ''
hive> create table haha(ha double(22));
MismatchedTokenException(334!=347)
at
org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at
org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
at
org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:27 mismatched input '(' expecting ) near
'double' in create table statement
hive> create table haha(ha double Precision);
OK
Time taken: 0.625 seconds
{quote}
*So, as far as I can concerned, Hive didn't support such data type definition.
It can not be resolved by kylin dev.*
If you found any mistake, please let me know. And I found below links which
maybe helpful:
1.
[https://cwiki.apache.org/confluence/display/hive/LanguageManual+DDL#LanguageManualDDL-CreateTableCreate/Drop/TruncateTable]
2. https://issues.apache.org/jira/browse/HIVE-13556
was (Author: hit_lacus):
Hi [~gauravbrills]
I installed the newest stable hive in my test env, which version is 2.3.4.I
found hive didn't not support double with specific/user-defined precision.
e.g.
* [ERROR] create table haha(ha double 10);
* [ERROR] create table haha(ha double Precision 17);
* [ERROR] create table haha(ha double(22));
* [CORRECT] create table haha(ha double Precision);
{quote}hive> use ss;
OK
Time taken: 0.015 seconds
hive> create table haha(ha double 10);
FAILED: ParseException line 1:28 extraneous input '10' expecting ) near ''
hive> create table haha(ha double Precision 17);
FAILED: ParseException line 1:38 extraneous input '17' expecting ) near ''
hive> create table haha(ha double(22));
MismatchedTokenException(334!=347)
at
org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at
org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
at org.apache
[jira] [Commented] (KYLIN-3695) Error while creating hive table through Kylin build cube with mysql imported tables
[
https://issues.apache.org/jira/browse/KYLIN-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700285#comment-16700285
]
XiaoXiang Yu commented on KYLIN-3695:
-
Hi [~gauravbrills]
I installed the newest stable hive in my test env, which version is 2.3.4.I
found hive didn't not support double with specific/user-defined precision.
e.g.
* [ERROR] create table haha(ha double 10);
* [ERROR] create table haha(ha double Precision 17);
* [ERROR] create table haha(ha double(22));
* [CORRECT] create table haha(ha double Precision);
{quote}hive> use ss;
OK
Time taken: 0.015 seconds
hive> create table haha(ha double 10);
FAILED: ParseException line 1:28 extraneous input '10' expecting ) near ''
hive> create table haha(ha double Precision 17);
FAILED: ParseException line 1:38 extraneous input '17' expecting ) near ''
hive> create table haha(ha double(22));
MismatchedTokenException(334!=347)
at
org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at
org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:6179)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3808)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
FAILED: ParseException line 1:27 mismatched input '(' expecting ) near 'double'
in create table statement
hive> create table haha(ha double Precision);
OK
Time taken: 0.625 seconds
{quote}
And I found below links:
1.
https://cwiki.apache.org/confluence/display/hive/LanguageManual+DDL#LanguageManualDDL-CreateTableCreate/Drop/TruncateTable
2. https://issues.apache.org/jira/browse/HIVE-13556
> Error while creating hive table through Kylin build cube with mysql imported
> tables
> ---
>
> Key: KYLIN-3695
> URL: https://issues.apache.org/jira/browse/KYLIN-3695
> Project: Kylin
> Issue Type: Bug
> Components: Integration
>Affects Versions: v2.5.0
>Reporter: Gaurav Rawat
>Assignee: XiaoXiang Yu
>Priority: Minor
>
> HI I am trying to build a cube with Kylin, the data gets souced fine from
> sqoop but the next step for creating hive tables fails . Looking at the
> command being fired it looks weird as the create statement looks good to me .
> I think the issue is with DOUBLE types as when I remove the same the create
> statement works fine . Can someone please help .
> I am using the stack in AWS EMR, kylin 2.5 hive 2.3.0
> The errors logs with commands as as below, the table is a msql table which
> had columns with DOUBLE type
> Command
> {code:java}
> hive -e "USE default; DROP TABLE IF EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368;
> CREATE EXTERNAL TABLE IF NOT EXISTS
> kylin_intermediate_fm_inv_holdings_8a1c33df_d12b_3609_13ee_39e169169368 (
> HOLDINGS_STOCK_INVESTOR_ID string ,STOCK_INVESTORS_CHANNEL string
> ,STOCK_STOCK_ID string ,STOCK_DOMICILE string ,STOCK_STOCK_NM string
> ,STOCK_APPROACH string ,STOCK_STOCK_TYP string ,INVESTOR_ID string
> ,INVESTOR_NM string ,INVESTOR_DOMICILE_CNTRY string ,CLIENT_NM string
> ,INVESTOR_HOLDINGS_GROSS_ASSETS_USD double(22)
> ,INVESTOR_HOLDINGS_NET_ASSETS_USD double(22) ) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY '|' STORED AS TEXTFILE LOCATION
> 's
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700202#comment-16700202 ] ASF GitHub Bot commented on KYLIN-3597: --- shaofengshi opened a new pull request #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700203#comment-16700203 ] ASF GitHub Bot commented on KYLIN-3597: --- asfgit commented on issue #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363#issuecomment-442011938 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit commented on issue #363: KYLIN-3597 fix static code issues
asfgit commented on issue #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363#issuecomment-442011938 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] shaofengshi opened a new pull request #363: KYLIN-3597 fix static code issues
shaofengshi opened a new pull request #363: KYLIN-3597 fix static code issues URL: https://github.com/apache/kylin/pull/363 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3676) Update to custom calcite and remove the "atopcalcite" module
[ https://issues.apache.org/jira/browse/KYLIN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3676: Affects Version/s: (was: v2.4.0) Issue Type: Improvement (was: Bug) Summary: Update to custom calcite and remove the "atopcalcite" module (was: Update custom Calctie) > Update to custom calcite and remove the "atopcalcite" module > > > Key: KYLIN-3676 > URL: https://issues.apache.org/jira/browse/KYLIN-3676 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Lijun Cao >Assignee: Lijun Cao >Priority: Major > Fix For: v2.5.2 > > > The Tomcat version is upgraded to 8.5 in 2.5.x-hadoop3.1 branch. However, due > to the mechanism of the class load in Tomcat 8.5 is different from Tomcat 7, > current custom Calcite got some issues when working on Tomcat 8.5 env. So it > should be upgrade to adapt to Tomcat 8.5. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3696) TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
[
https://issues.apache.org/jira/browse/KYLIN-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700106#comment-16700106
]
Shaofeng SHI commented on KYLIN-3696:
-
[~yangwei] If a cube has little measure and they are in the same HBase column
family, Kylin will directly store the bytes to HBase KeyValue, this will avoid
one deserialize and serialization. The bug exists in TopN's serialization. So I
think this is why your first cube doesn't have this problem.
> TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
>
>
> Key: KYLIN-3696
> URL: https://issues.apache.org/jira/browse/KYLIN-3696
> Project: Kylin
> Issue Type: Bug
> Components: Measure - TopN
>Affects Versions: v2.5.1
>Reporter: yangwei
>Priority: Major
> Attachments: image-2018-11-20-10-57-28-546.png,
> image-2018-11-20-11-01-25-120.png, image-2018-11-20-11-27-43-750.png
>
>
> 我使用的是v2.5.1，度量topN使用上出现不准的总量。
> 问题再现：
> 一，二个cube使用同一个模型就是同一张物理事实表。
> 二，二个cube同时包含相同的topN度量
> 三，二个cube状态都是Ready
> 目前我暂时的解决方法是在其中一个cube去掉一个topN度量
> 同一个sql在hive与kylin里查的的结果对不上相差很远,下面给出sql
> SELECT IP ,
> SUM(ACCESS_COUNT) c
> FROM API_ACCESS
> WHERE TAG_DATE = CAST('2018-11-19' AS DATE)
> group by ip
> ORDER BY
> c DESC
> LIMIT 10;
> 二个cube中的度量:
> cube1:
> !image-2018-11-20-10-57-28-546.png!
> cube2:
> !image-2018-11-20-11-01-25-120.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700115#comment-16700115 ] ASF GitHub Bot commented on KYLIN-3665: --- FrancisLiang commented on issue #360: KYLIN-3665 Partition time column may never be added URL: https://github.com/apache/kylin/pull/360#issuecomment-441988794 > Hi Francis, the change looks good. While the commit message didn't comply with Apache's style. Could you please update the commit message to "KYLIN-3665 Partition time column may never be added" (start with JIRA ID, no [] needed). Thank you! Just did it. Thanks for the information! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] FrancisLiang commented on issue #360: KYLIN-3665 Partition time column may never be added
FrancisLiang commented on issue #360: KYLIN-3665 Partition time column may never be added URL: https://github.com/apache/kylin/pull/360#issuecomment-441988794 > Hi Francis, the change looks good. While the commit message didn't comply with Apache's style. Could you please update the commit message to "KYLIN-3665 Partition time column may never be added" (start with JIRA ID, no [] needed). Thank you! Just did it. Thanks for the information! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700110#comment-16700110 ] ASF GitHub Bot commented on KYLIN-3665: --- shaofengshi commented on issue #360: [KYLIN-3665]-Partition time column may never be added fixed. URL: https://github.com/apache/kylin/pull/360#issuecomment-441987516 Hi Francis, the change looks good. While the commit message didn't comply with Apache's style. Could you please update the commit message to "KYLIN-3665 Partition time column may never be added" (start with JIRA ID, no [] needed). Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi commented on issue #360: [KYLIN-3665]-Partition time column may never be added fixed.
shaofengshi commented on issue #360: [KYLIN-3665]-Partition time column may never be added fixed. URL: https://github.com/apache/kylin/pull/360#issuecomment-441987516 Hi Francis, the change looks good. While the commit message didn't comply with Apache's style. Could you please update the commit message to "KYLIN-3665 Partition time column may never be added" (start with JIRA ID, no [] needed). Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (KYLIN-3665) Partition time column may never be added
[ https://issues.apache.org/jira/browse/KYLIN-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3665: --- Assignee: Feng Liang (was: Chao Long) > Partition time column may never be added > > > Key: KYLIN-3665 > URL: https://issues.apache.org/jira/browse/KYLIN-3665 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Feng Liang >Priority: Major > Fix For: v2.6.0 > > > The partition time column will never be added to the WHERE clause if the > partition date column is type of int or bigint and the date format is > "MMdd", "MMddHH", "MMddHHmm" or "MMddHHmmss". > Let’s say I have a fact table with two date partition column: day as the > format of integer (MMdd) and minute as the format of string (HHmm). I > have models created based on it and set the Partition Date Column to be “day” > with the format of MMdd and the Partition Time Column to be “minute” with > the format of HHmm. When I build the cube built on top of that model, I set > the Start Date to be 2018-11-01 00:00:00 and the End Date to be 2018-11-01 > 01:00:00. The building process is supposed to retrieve the data from the fact > table in the range of “day = 20181101 and minute>=’’ and minute<’0100’”. > However, it retrieves the data in the range of “WHERE 1=1 AND (DAY >= > 20181101 AND DAY < 20181101), so no data are actually retrieved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3696) TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
[
https://issues.apache.org/jira/browse/KYLIN-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700052#comment-16700052
]
Shaofeng SHI commented on KYLIN-3696:
-
[~yangwei] I have cherry-pick that hot-fix to 2.5.x branch; You can make a new
binary package from there and then verify whether it fix the problem in your
cluster. Please share with us about your findings. Thank you!
> TOPN度量在同一个模型下2个cube同时开启统计值不准与真实值差得较多
>
>
> Key: KYLIN-3696
> URL: https://issues.apache.org/jira/browse/KYLIN-3696
> Project: Kylin
> Issue Type: Bug
> Components: Measure - TopN
>Affects Versions: v2.5.1
>Reporter: yangwei
>Priority: Major
> Attachments: image-2018-11-20-10-57-28-546.png,
> image-2018-11-20-11-01-25-120.png, image-2018-11-20-11-27-43-750.png
>
>
> 我使用的是v2.5.1，度量topN使用上出现不准的总量。
> 问题再现：
> 一，二个cube使用同一个模型就是同一张物理事实表。
> 二，二个cube同时包含相同的topN度量
> 三，二个cube状态都是Ready
> 目前我暂时的解决方法是在其中一个cube去掉一个topN度量
> 同一个sql在hive与kylin里查的的结果对不上相差很远,下面给出sql
> SELECT IP ,
> SUM(ACCESS_COUNT) c
> FROM API_ACCESS
> WHERE TAG_DATE = CAST('2018-11-19' AS DATE)
> group by ip
> ORDER BY
> c DESC
> LIMIT 10;
> 二个cube中的度量:
> cube1:
> !image-2018-11-20-10-57-28-546.png!
> cube2:
> !image-2018-11-20-11-01-25-120.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] shaofengshi closed pull request #355: detailed configurations could be override in the Cube-level
shaofengshi closed pull request #355: detailed configurations could be override
in the Cube-level
URL: https://github.com/apache/kylin/pull/355
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/website/_docs/install/configuration.cn.md
b/website/_docs/install/configuration.cn.md
index 4f298dc951..c833da4807 100644
--- a/website/_docs/install/configuration.cn.md
+++ b/website/_docs/install/configuration.cn.md
@@ -28,7 +28,7 @@ permalink: /cn/docs/install/configuration.html
- [Hive 客户端 & SparkSQL](#hive-client-and-sparksql)
- [配置 JDBC 数据源](#jdbc-datasource)
- [数据类型精度](#precision-config)
-- [Cube 设置](#cube-config)
+- [Cube 设计](#cube-config)
- [Cube 大小估计](#cube-estimate)
- [Cube 构建算法](#cube-algorithm)
- [自动合并](#auto-merge)
@@ -98,6 +98,20 @@ Kylin 会自动从环境中读取 Hadoop 配置（`core-site.xml`），Hive 配
在设计 Cube （**Cube Designer**）的 **Configuration Overwrites** 步骤可以添加配置项，进行 Cube
级别的配置重写，如下图所示：
+以下参数可以在 Cube 级别重写：
+
+- `kylin.cube.size-estimate*`
+- `kylin.cube.algorithm*`
+- `kylin.cube.aggrgroup*`
+- `kylin.metadata.dimension-encoding-max-length`
+- `kylin.cube.max-building-segments`
+- `kylin.cube.is-automerge-enabled`
+- `kylin.job.allow-empty-segment`
+- `kylin.job.sampling-percentage`
+- `kylin.source.hive.redistribute-flat-table`
+- `kylin.engine.spark*`
+- `kylin.query.skip-empty-segments`
+
### MapReduce 任务配置重写 {#mr-config-override}
@@ -194,7 +208,6 @@ export KYLIN_JVM_SETTINGS="-Xms1024M -Xmx4096M -Xss1024K
-XX`MaxPermSize=512M -v
### 元数据相关 {#metadata}
- `kylin.metadata.url`：指定元数据库路径，默认值为 kylin_metadata@hbase
-- `kylin.metadata.dimension-encoding-max-length`：指定维度作为 Rowkeys 时使用 fix_length
编码时的最大长度，默认值为 256
- `kylin.metadata.sync-retries`：指定元数据同步重试次数，默认值为 3
- `kylin.metadata.sync-error-handler`：默认值为 `DefaultSyncErrorHandler`
- `kylin.metadata.check-copy-on-write`：清除元数据缓存，默认值为 `FALSE`
@@ -265,7 +278,7 @@ export KYLIN_JVM_SETTINGS="-Xms1024M -Xmx4096M -Xss1024K
-XX`MaxPermSize=512M -v
-### Cube 设置 {#cube-config}
+### Cube 设计 {#cube-config}
- `kylin.cube.ignore-signature-inconsistency`：Cube desc 中的 signature 信息能保证
Cube 不被更改为损坏状态，默认值为 FALSE
- `kylin.cube.aggrgroup.max-combination`：指定一个 Cube 的聚合组 Cuboid 上限，默认值为 32768
@@ -273,6 +286,8 @@ export KYLIN_JVM_SETTINGS="-Xms1024M -Xmx4096M -Xss1024K
-XX`MaxPermSize=512M -v
- `kylin.cube.rowkey.max-size`：指定可以设置为 Rowkeys 的最大列数，默认值为 63
- `kylin.cube.allow-appear-in-multiple-projects`：是否允许一个 Cube 出现在多个项目中
- `kylin.cube.gtscanrequest-serialization-level`：默认值为 1
+- `kylin.metadata.dimension-encoding-max-length`：指定维度作为 Rowkeys 时使用 fix_length
编码时的最大长度，默认值为 256
+
diff --git a/website/_docs/install/configuration.md
b/website/_docs/install/configuration.md
index c8217405c8..cad8b967f9 100644
--- a/website/_docs/install/configuration.md
+++ b/website/_docs/install/configuration.md
@@ -88,16 +88,30 @@ Some configuration files in `$KYLIN_HOME/conf/` can be
overridden in the Web UI.
### Project-level Configuration Overriding {#project-config-override}
-Click *Manage Project* in the web UI interface, select a project, click *Edit*
-> *Project Config* -> *+ Property* to add configuration properties which could
override property values in configuration files, as the figure below shown:
+Click *Manage Project* in the web UI interface, select a project, click *Edit*
-> *Project Config* -> *+ Property* to add configuration properties which could
override property values in configuration files, as the figure below shown,
### Cube-level Configuration Overriding{#cube-config-override}
-In the *Configuration overrides* step of *Cube Designer*, user could rewrite
property values to override those in project level and configuration files, as
the figure below shown:
+In the *Configuration overrides* step of *Cube Designer*, user could rewrite
property values to override those in project level and configuration files, as
the figure below shown,
+The following configurations can be override in the Cube-level,
+
+- `kylin.cube.size-estimate*`
+- `kylin.cube.algorithm*`
+- `kylin.cube.aggrgroup*`
+- `kylin.metadata.dimension-encoding-max-length`
+- `kylin.cube.max-building-segments`
+- `kylin.cube.is-automerge-enabled`
+- `kylin.job.allow-empty-segment`
+- `kylin.job.sampling-percentage`
+- `kylin.source.hive.redistribute-flat-table`
+- `kylin.engine.spark*`
+- `kylin.query.skip-empty-segments`
+
### MapReduce Configuration Overriding {#mr-config-override}
This is an automated message from the Apache Git Service.
To respond to t
[GitHub] shaofengshi commented on issue #355: detailed configurations could be override in the Cube-level
shaofengshi commented on issue #355: detailed configurations could be override in the Cube-level URL: https://github.com/apache/kylin/pull/355#issuecomment-441965061 Merged with patch, thank you Dingqian! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
