[jira] [Commented] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932075#comment-16932075
]
ZhouKang commented on KYLIN-3973:
-
[~shaoxiaowei] If you are using HBase>2.0.0， you can fix it by
[https://github.com/apache/kylin/pull/837]
This a quick fix.
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932067#comment-16932067
]
shaoxiaowei commented on KYLIN-3973:
[~grzegorz_kolakowski] Today, I also encountered this problem. Excuse me, have
you solved it? Is there any good solution?
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4100: Fix Version/s: v3.0.0-beta > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4100. - Resolution: Fixed > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4103. - Resolution: Fixed > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4103: Fix Version/s: v2.6.4 v3.0.0-beta > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] codecov-io commented on issue #843: KYLIN-4169 remove useless listModels() in DataModelManager.curd.initE…
codecov-io commented on issue #843: KYLIN-4169 remove useless listModels() in DataModelManager.curd.initE… URL: https://github.com/apache/kylin/pull/843#issuecomment-532504822 # [Codecov](https://codecov.io/gh/apache/kylin/pull/843?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@199bc17`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `100%`. [](https://codecov.io/gh/apache/kylin/pull/843?src=pr=tree) ```diff @@Coverage Diff@@ ## master #843 +/- ## = Coverage ? 25.68% Complexity? 6120 = Files ? 1400 Lines ?83729 Branches ?11740 = Hits ?21507 Misses?60153 Partials ? 2069 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/843?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../apache/kylin/metadata/model/DataModelManager.java](https://codecov.io/gh/apache/kylin/pull/843/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvbW9kZWwvRGF0YU1vZGVsTWFuYWdlci5qYXZh) | `25.45% <100%> (ø)` | `9 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/843?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/843?src=pr=footer). Last update [199bc17...90417b3](https://codecov.io/gh/apache/kylin/pull/843?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4156.
-
Resolution: Fixed
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0-beta
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4156:
Fix Version/s: v3.0.0-beta
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0-beta
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen reopened KYLIN-4156:
-
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3392. - Resolution: Fixed > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3392: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4158) Query failed for GroupBy an expression of column with limit in SQL
[
https://issues.apache.org/jira/browse/KYLIN-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4158.
-
Resolution: Fixed
> Query failed for GroupBy an expression of column with limit in SQL
> --
>
> Key: KYLIN-4158
> URL: https://issues.apache.org/jira/browse/KYLIN-4158
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.6.2
>Reporter: PENG Zhengshuai
>Assignee: PENG Zhengshuai
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> When executing a sql query, the limit should not be pushdown to storage scan
> level if group by clause is an expression of columns.
> For example:
> {code:java}
> SELECT (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END) AS
> "LSTG_FORMAT_NAME__group_",
> SUM("TEST_KYLIN_FACT"."PRICE") AS "sum_PRICE_ok"
> FROM "TEST_KYLIN_FACT" "TEST_KYLIN_FACT"
> GROUP BY (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END)
> limit 4
> {code}
> In this SQL, group by clause is an expression, the limit should NOT be
> push-down to storage scan level, otherwise, the result is incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4154) Metadata inconsistency between multi Kylin server caused by Broadcaster closing
[ https://issues.apache.org/jira/browse/KYLIN-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4154: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Metadata inconsistency between multi Kylin server caused by Broadcaster > closing > > > Key: KYLIN-4154 > URL: https://issues.apache.org/jira/browse/KYLIN-4154 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0-beta > > > To avoid Broadcaster memory leak, KYLIN-4131 close the metadata sync thread > after receiving "Sync All" event, but there may be some events in the waiting > queue, which haven't been synced yet. So we should sync all events before > closing sync thread. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4158) Query failed for GroupBy an expression of column with limit in SQL
[
https://issues.apache.org/jira/browse/KYLIN-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4158:
Fix Version/s: v2.6.4
v3.0.0-beta
> Query failed for GroupBy an expression of column with limit in SQL
> --
>
> Key: KYLIN-4158
> URL: https://issues.apache.org/jira/browse/KYLIN-4158
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.6.2
>Reporter: PENG Zhengshuai
>Assignee: PENG Zhengshuai
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> When executing a sql query, the limit should not be pushdown to storage scan
> level if group by clause is an expression of columns.
> For example:
> {code:java}
> SELECT (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END) AS
> "LSTG_FORMAT_NAME__group_",
> SUM("TEST_KYLIN_FACT"."PRICE") AS "sum_PRICE_ok"
> FROM "TEST_KYLIN_FACT" "TEST_KYLIN_FACT"
> GROUP BY (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END)
> limit 4
> {code}
> In this SQL, group by clause is an expression, the limit should NOT be
> push-down to storage scan level, otherwise, the result is incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4154) Metadata inconsistency between multi Kylin server caused by Broadcaster closing
[ https://issues.apache.org/jira/browse/KYLIN-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4154. - Resolution: Fixed > Metadata inconsistency between multi Kylin server caused by Broadcaster > closing > > > Key: KYLIN-4154 > URL: https://issues.apache.org/jira/browse/KYLIN-4154 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0 > > > To avoid Broadcaster memory leak, KYLIN-4131 close the metadata sync thread > after receiving "Sync All" event, but there may be some events in the waiting > queue, which haven't been synced yet. So we should sync all events before > closing sync thread. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4157) When using PrepareStatement query, functions within WHERE will cause InternalErrorException
[ https://issues.apache.org/jira/browse/KYLIN-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4157: Fix Version/s: v3.0.0-beta > When using PrepareStatement query, functions within WHERE will cause > InternalErrorException > --- > > Key: KYLIN-4157 > URL: https://issues.apache.org/jira/browse/KYLIN-4157 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.6.3 >Reporter: Marc Wu >Assignee: Marc Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: image-2019-09-04-15-39-52-867.png, > image-2019-09-04-15-39-58-276.png, image-2019-09-04-20-57-46-121.png > > > Hi Kylin Team: > I found a bug when I'm using PreparedStatement query. > Let me use table KYLIN_SALES to explain the scenario. > There is a SQL like: > select LSTG_FORMAT_NAME, sum(PRICE) from KYLIN_SALES where > lower(LSTG_FORMAT_NAME) = 'fp-gtc' group by LSTG_FORMAT_NAME > In some cases, user doesn't know the LSTG_FORMAT_NAME is upper case or lower > case, or they just want to query data ignoring cases. > So assume they use lower(LSTG_FORMAT_NAME) = 'fp-gtc', it's a function > within the filter. > When I execute this SQL on Kylin web console, it can get the right result, > but when I tried to execute it by PreparedStatement query on Postman, it > threw InternalErrorException. !image-2019-09-04-15-39-58-276.png! > > !image-2019-09-04-15-39-52-867.png! > > > h2. Root Cause > I debugged Kylin to find the root cause, it turns out when the filter > contains function, compFilter.getColumn is null, and the > compFilter.getFunction contains lower(LSTG_FORMAT_NAME), so that > compFilter.getColumn().getType().isDateTimeFamily() will throw > NullPointerException, that's the root cause for this 500 error. > > h2. !image-2019-09-04-20-57-46-121.png! Solution > I suggest to check nullable first before using compFilter.getColumn. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4055. - Resolution: Fixed > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4055: Fix Version/s: v2.6.4 > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4055: Fix Version/s: v3.0.0-beta > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4169) too many logs while DataModelManager init, cause the first RESTful API hang for a long time
[
https://issues.apache.org/jira/browse/KYLIN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932005#comment-16932005
]
ASF GitHub Bot commented on KYLIN-4169:
---
zhoukangcn commented on pull request #843: KYLIN-4169 remove useless
listModels() in DataModelManager.curd.initE…
URL: https://github.com/apache/kylin/pull/843
…ntityAfterReload
See: https://issues.apache.org/jira/browse/KYLIN-4169
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> too many logs while DataModelManager init, cause the first RESTful API hang
> for a long time
> ---
>
> Key: KYLIN-4169
> URL: https://issues.apache.org/jira/browse/KYLIN-4169
> Project: Kylin
> Issue Type: Bug
>Reporter: ZhouKang
>Priority: Major
>
> Too many logs while DataModelManager init. It is will happend after startup
> or reload metastore.
> In our production enviroment，there are 120,000+ lines.
> {code:java}
> // code placeholder
> grep 'is missing or unloaded yet' kylin.log.8|wc -l
> 121861
> {code}
>
> which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] asf-ci commented on issue #843: KYLIN-4169 remove useless listModels() in DataModelManager.curd.initE…
asf-ci commented on issue #843: KYLIN-4169 remove useless listModels() in DataModelManager.curd.initE… URL: https://github.com/apache/kylin/pull/843#issuecomment-532500216 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-4159) The first step of build cube job will fail and throw "Column 'xx' in where clause is ambiguous" in jdbc datasource.
[ https://issues.apache.org/jira/browse/KYLIN-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4159. - Resolution: Fixed > The first step of build cube job will fail and throw "Column 'xx' in where > clause is ambiguous" in jdbc datasource. > --- > > Key: KYLIN-4159 > URL: https://issues.apache.org/jira/browse/KYLIN-4159 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Reappear: > In the model of kylin, If we have a column used as a partition column in the > fact table, and there is a column in lookup table has the same name. It can > be reappeared。 > RootCause: > In jdbc datasource , we use Sqoop option "--boundary-query sql" to split > data, the sql like "select max(partition_column), min(partition_column) from > flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" and there is more than one column > which have the same name of the "partition_column" in the flat_table. the sql > driver can not distinguish the table that partition_column belong to。 > How to resolve: > Alter the sql like: "select max(t.partition_column),min(t.partition_column) > from flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4159) The first step of build cube job will fail and throw "Column 'xx' in where clause is ambiguous" in jdbc datasource.
[ https://issues.apache.org/jira/browse/KYLIN-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4159: Fix Version/s: v2.6.4 v3.0.0-beta > The first step of build cube job will fail and throw "Column 'xx' in where > clause is ambiguous" in jdbc datasource. > --- > > Key: KYLIN-4159 > URL: https://issues.apache.org/jira/browse/KYLIN-4159 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Reappear: > In the model of kylin, If we have a column used as a partition column in the > fact table, and there is a column in lookup table has the same name. It can > be reappeared。 > RootCause: > In jdbc datasource , we use Sqoop option "--boundary-query sql" to split > data, the sql like "select max(partition_column), min(partition_column) from > flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" and there is more than one column > which have the same name of the "partition_column" in the flat_table. the sql > driver can not distinguish the table that partition_column belong to。 > How to resolve: > Alter the sql like: "select max(t.partition_column),min(t.partition_column) > from flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4160) Auto redirect to host:port/kylin when user only enter host:port in broswer
[ https://issues.apache.org/jira/browse/KYLIN-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4160. - Resolution: Fixed > Auto redirect to host:port/kylin when user only enter host:port in broswer > -- > > Key: KYLIN-4160 > URL: https://issues.apache.org/jira/browse/KYLIN-4160 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4160) Auto redirect to host:port/kylin when user only enter host:port in broswer
[ https://issues.apache.org/jira/browse/KYLIN-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4160: Fix Version/s: v2.6.4 v3.0.0-beta > Auto redirect to host:port/kylin when user only enter host:port in broswer > -- > > Key: KYLIN-4160 > URL: https://issues.apache.org/jira/browse/KYLIN-4160 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4143) truncate spark executable job output
[ https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4143. - Resolution: Fixed > truncate spark executable job output > - > > Key: KYLIN-4143 > URL: https://issues.apache.org/jira/browse/KYLIN-4143 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: ZhouKang >Priority: Major > Fix For: v3.0.0-beta > > Attachments: KYLIN-4143.master.001.patch > > > > truncate spark job's output when the job exec ret is not equal 0, which made > the execute output content too large. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4143) truncate spark executable job output
[ https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4143: Fix Version/s: v3.0.0-beta > truncate spark executable job output > - > > Key: KYLIN-4143 > URL: https://issues.apache.org/jira/browse/KYLIN-4143 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: ZhouKang >Priority: Major > Fix For: v3.0.0-beta > > Attachments: KYLIN-4143.master.001.patch > > > > truncate spark job's output when the job exec ret is not equal 0, which made > the execute output content too large. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4155) Cube status can not change immediately when executed disable or enable button in web
[ https://issues.apache.org/jira/browse/KYLIN-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4155. - Resolution: Fixed > Cube status can not change immediately when executed disable or enable button > in web > > > Key: KYLIN-4155 > URL: https://issues.apache.org/jira/browse/KYLIN-4155 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > When user execute disable or enable button in cube list dropdown menu , and > then api response was back successfully, cube status tag can not change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4034) The table should not display in Insight page when the user has no access to the table
[ https://issues.apache.org/jira/browse/KYLIN-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4034: Fix Version/s: v2.6.4 > The table should not display in Insight page when the user has no access to > the table > - > > Key: KYLIN-4034 > URL: https://issues.apache.org/jira/browse/KYLIN-4034 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.2 >Reporter: Na Zhai >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4155) Cube status can not change immediately when executed disable or enable button in web
[ https://issues.apache.org/jira/browse/KYLIN-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4155: Fix Version/s: v3.0.0-beta > Cube status can not change immediately when executed disable or enable button > in web > > > Key: KYLIN-4155 > URL: https://issues.apache.org/jira/browse/KYLIN-4155 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > When user execute disable or enable button in cube list dropdown menu , and > then api response was back successfully, cube status tag can not change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4148) Execute 'bin/kylin-port-replace-util.sh' to change port will cause the configuration of 'kylin.metadata.url' lost
[ https://issues.apache.org/jira/browse/KYLIN-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4148. - Resolution: Fixed > Execute 'bin/kylin-port-replace-util.sh' to change port will cause the > configuration of 'kylin.metadata.url' lost > -- > > Key: KYLIN-4148 > URL: https://issues.apache.org/jira/browse/KYLIN-4148 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4148) Execute 'bin/kylin-port-replace-util.sh' to change port will cause the configuration of 'kylin.metadata.url' lost
[ https://issues.apache.org/jira/browse/KYLIN-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4148: Fix Version/s: v3.0.0-beta > Execute 'bin/kylin-port-replace-util.sh' to change port will cause the > configuration of 'kylin.metadata.url' lost > -- > > Key: KYLIN-4148 > URL: https://issues.apache.org/jira/browse/KYLIN-4148 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4169) too many logs while DataModelManager init, cause the first RESTful API hang for a long time
ZhouKang created KYLIN-4169:
---
Summary: too many logs while DataModelManager init, cause the
first RESTful API hang for a long time
Key: KYLIN-4169
URL: https://issues.apache.org/jira/browse/KYLIN-4169
Project: Kylin
Issue Type: Bug
Reporter: ZhouKang
Too many logs while DataModelManager init. It is will happend after startup or
reload metastore.
In our production enviroment，there are 120,000+ lines.
{code:java}
// code placeholder
grep 'is missing or unloaded yet' kylin.log.8|wc -l
121861
{code}
which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4150) Improve docker for kylin instructions
[ https://issues.apache.org/jira/browse/KYLIN-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4150: Fix Version/s: v3.0.0-beta > Improve docker for kylin instructions > - > > Key: KYLIN-4150 > URL: https://issues.apache.org/jira/browse/KYLIN-4150 > Project: Kylin > Issue Type: Improvement >Reporter: weibin0516 >Assignee: weibin0516 >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4112) Add hdfs keberos token delegation in Spark to support HBase and MR use different HDFS clusters
[ https://issues.apache.org/jira/browse/KYLIN-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4112. - Resolution: Fixed > Add hdfs keberos token delegation in Spark to support HBase and MR use > different HDFS clusters > -- > > Key: KYLIN-4112 > URL: https://issues.apache.org/jira/browse/KYLIN-4112 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > Currently the SparkExecutable only delegate the token for yarn hdfs cluster, > not for the hdfs cluster used by the HBase cluster. > The spark job of Convert Cuboid Data to HFile will failed for kerberos issue. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4112) Add hdfs keberos token delegation in Spark to support HBase and MR use different HDFS clusters
[ https://issues.apache.org/jira/browse/KYLIN-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4112: Fix Version/s: v3.0.0-beta > Add hdfs keberos token delegation in Spark to support HBase and MR use > different HDFS clusters > -- > > Key: KYLIN-4112 > URL: https://issues.apache.org/jira/browse/KYLIN-4112 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > Currently the SparkExecutable only delegate the token for yarn hdfs cluster, > not for the hdfs cluster used by the HBase cluster. > The spark job of Convert Cuboid Data to HFile will failed for kerberos issue. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4114) Provided a self-contained docker image for Kylin
[ https://issues.apache.org/jira/browse/KYLIN-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4114: Fix Version/s: v3.0.0-beta > Provided a self-contained docker image for Kylin > > > Key: KYLIN-4114 > URL: https://issues.apache.org/jira/browse/KYLIN-4114 > Project: Kylin > Issue Type: Wish > Components: Integration >Reporter: Xiaoxiang Yu >Assignee: weibin0516 >Priority: Major > Fix For: v3.0.0-beta > > > Provided a self-contained docker image for Kylin will benifit to > Package/Integration Test/Demo purpose. > https://issues.apache.org/jira/browse/KYLIN-4040 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4095) Add RESOURCE_PATH_PREFIX option in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4095. - Resolution: Fixed > Add RESOURCE_PATH_PREFIX option in ResourceTool > --- > > Key: KYLIN-4095 > URL: https://issues.apache.org/jira/browse/KYLIN-4095 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > ResourceTool is very useful to fix the metadata with overlap segments. > But downloading and uploading entire metadata is too heavy. > It's better to have a RESOURCE_PATH_PREFIX option for downloading and > uploading cmds. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4095) Add RESOURCE_PATH_PREFIX option in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4095: Fix Version/s: v3.0.0-beta > Add RESOURCE_PATH_PREFIX option in ResourceTool > --- > > Key: KYLIN-4095 > URL: https://issues.apache.org/jira/browse/KYLIN-4095 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > ResourceTool is very useful to fix the metadata with overlap segments. > But downloading and uploading entire metadata is too heavy. > It's better to have a RESOURCE_PATH_PREFIX option for downloading and > uploading cmds. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4101) set hive and spark job name when building cube
[
https://issues.apache.org/jira/browse/KYLIN-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4101.
-
Resolution: Fixed
> set hive and spark job name when building cube
> --
>
> Key: KYLIN-4101
> URL: https://issues.apache.org/jira/browse/KYLIN-4101
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Minor
> Fix For: v3.0.0-beta
>
>
> Currently the job name of spark is
> {color:#22}org.apache.kylin.common.util.SparkEntry{color}, which is the
> main class name of spark . The mapreduce job name of hive sql is substring of
> the query, which is difficult to read.
> It's better to set a more readable name for the hive and spark jobs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-3901) Use multi threads to speed up the storage cleanup job
[ https://issues.apache.org/jira/browse/KYLIN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3901: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Use multi threads to speed up the storage cleanup job > - > > Key: KYLIN-3901 > URL: https://issues.apache.org/jira/browse/KYLIN-3901 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > Currently, the storage cleanup job only use one thread to clean up hbase > table, hive table, and hdfs dirs. > It''s better to use multi threads to speed it up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-3901) Use multi threads to speed up the storage cleanup job
[ https://issues.apache.org/jira/browse/KYLIN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3901. - Resolution: Fixed > Use multi threads to speed up the storage cleanup job > - > > Key: KYLIN-3901 > URL: https://issues.apache.org/jira/browse/KYLIN-3901 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: v3.0.0-beta > > > Currently, the storage cleanup job only use one thread to clean up hbase > table, hive table, and hdfs dirs. > It''s better to use multi threads to speed it up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4133) support override configuration in kafka job
[ https://issues.apache.org/jira/browse/KYLIN-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4133: Fix Version/s: (was: v2.6.4) > support override configuration in kafka job > --- > > Key: KYLIN-4133 > URL: https://issues.apache.org/jira/browse/KYLIN-4133 > Project: Kylin > Issue Type: Improvement > Components: NRT Streaming >Reporter: Zhixiong Chen >Assignee: Zhixiong Chen >Priority: Minor > Fix For: v3.0.0-beta > > > we can't override 'kafka.split.rows' config in kafka job. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4137) Accelerate metadata reloading
[ https://issues.apache.org/jira/browse/KYLIN-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4137: Fix Version/s: (was: v2.6.4) > Accelerate metadata reloading > - > > Key: KYLIN-4137 > URL: https://issues.apache.org/jira/browse/KYLIN-4137 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v2.6.2 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > Fix For: v3.0.0-beta > > > Now, org.apache.kylin.metadata.cachesync.CachedCrudAssist#reloadAll is using > an inappropriate method to deal with MySQLJdbcMetadata. > As the every call of reloadAt(path) will access the database, which is almost > fine to HBase but really unfriendly to the MySQL(RDBMS). > I think we should get all of the resource with single request instead of the > separate request to get every resource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4139) Compatible old user security xml config when user upgrate new kylin version
[ https://issues.apache.org/jira/browse/KYLIN-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4139: Fix Version/s: (was: v2.6.4) > Compatible old user security xml config when user upgrate new kylin version > > > Key: KYLIN-4139 > URL: https://issues.apache.org/jira/browse/KYLIN-4139 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > > Forward compatible > # when user config 'spring.profiles.active=testing', sync old version user > metadata with user config in kylinSecurity.xml > Other improvement > # Remove the logic of auto trans new user name to uppercase -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] codecov-io edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time
codecov-io edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time URL: https://github.com/apache/kylin/pull/804#issuecomment-522833971 # [Codecov](https://codecov.io/gh/apache/kylin/pull/804?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@199bc17`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `1.5%`. [](https://codecov.io/gh/apache/kylin/pull/804?src=pr=tree) ```diff @@Coverage Diff@@ ## master #804 +/- ## = Coverage ? 25.61% Complexity? 6123 = Files ? 1405 Lines ?83994 Branches ?11772 = Hits ?21519 Misses?60404 Partials ? 2071 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/804?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...e/kylin/stream/core/dict/LocalDictionaryStore.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL2RpY3QvTG9jYWxEaWN0aW9uYXJ5U3RvcmUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.76% <0%> (ø)` | `42 <0> (?)` | | | [.../kylin/stream/core/dict/RemoteDictionaryStore.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL2RpY3QvUmVtb3RlRGljdGlvbmFyeVN0b3JlLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...in/stream/core/dict/StreamingDictionaryClient.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL2RpY3QvU3RyZWFtaW5nRGljdGlvbmFyeUNsaWVudC5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ream/core/dict/StreamingDistributedDictionary.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL2RpY3QvU3RyZWFtaW5nRGlzdHJpYnV0ZWREaWN0aW9uYXJ5LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...in/stream/core/storage/IStreamingSegmentStore.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvSVN0cmVhbWluZ1NlZ21lbnRTdG9yZS5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...rg/apache/kylin/stream/server/StreamingServer.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLXJlY2VpdmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vc2VydmVyL1N0cmVhbWluZ1NlcnZlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ream/core/storage/columnar/SegmentMemoryStore.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvU2VnbWVudE1lbW9yeVN0b3JlLmphdmE=) | `83.66% <100%> (ø)` | `35 <1> (?)` | | | [...am/core/storage/columnar/ColumnarSegmentStore.java](https://codecov.io/gh/apache/kylin/pull/804/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTZWdtZW50U3RvcmUuamF2YQ==) | `58.11% <25%> (ø)` | `28 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/804?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/804?src=pr=footer). Last update [199bc17...22f9fd4](https://codecov.io/gh/apache/kylin/pull/804?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4122) Add kylin user and group manage modules
[ https://issues.apache.org/jira/browse/KYLIN-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4122: Fix Version/s: (was: v2.6.4) > Add kylin user and group manage modules > --- > > Key: KYLIN-4122 > URL: https://issues.apache.org/jira/browse/KYLIN-4122 > Project: Kylin > Issue Type: New Feature >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4130) Coordinator->StreamingBuildJobStatusChecker thread always hold a old CubeManager
[
https://issues.apache.org/jira/browse/KYLIN-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4130:
Fix Version/s: (was: v3.0.0)
v3.0.0-beta
> Coordinator->StreamingBuildJobStatusChecker thread always hold a old
> CubeManager
>
>
> Key: KYLIN-4130
> URL: https://issues.apache.org/jira/browse/KYLIN-4130
> Project: Kylin
> Issue Type: Improvement
> Components: Real-time Streaming
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> {code}
> private class StreamingBuildJobStatusChecker implements Runnable {
> private int maxJobTryCnt = 5;
> private CubeManager cubeManager =
> CubeManager.getInstance(KylinConfig.getInstanceFromEnv());
> private ConcurrentMap ConcurrentSkipListSet> segmentBuildJobMap = Maps
> .newConcurrentMap();
> private CopyOnWriteArrayList pendingCubeName =
> Lists.newCopyOnWriteArrayList();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] codecov-io edited a comment on issue #838: KYLIN-4167 Refactor streaming coordinator
codecov-io edited a comment on issue #838: KYLIN-4167 Refactor streaming coordinator URL: https://github.com/apache/kylin/pull/838#issuecomment-531585406 # [Codecov](https://codecov.io/gh/apache/kylin/pull/838?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@199bc17`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `14.56%`. [](https://codecov.io/gh/apache/kylin/pull/838?src=pr=tree) ```diff @@Coverage Diff@@ ## master #838 +/- ## = Coverage ? 25.63% Complexity? 6161 = Files ? 1406 Lines ?84642 Branches ?11867 = Hits ?21694 Misses?60859 Partials ? 2089 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/838?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...stream/core/consumer/StreamingConsumerChannel.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL2NvbnN1bWVyL1N0cmVhbWluZ0NvbnN1bWVyQ2hhbm5lbC5qYXZh) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...n/stream/coordinator/exception/StoreException.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvb3JkaW5hdG9yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vY29vcmRpbmF0b3IvZXhjZXB0aW9uL1N0b3JlRXhjZXB0aW9uLmphdmE=) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...dinator/exception/NotLeadCoordinatorException.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvb3JkaW5hdG9yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vY29vcmRpbmF0b3IvZXhjZXB0aW9uL05vdExlYWRDb29yZGluYXRvckV4Y2VwdGlvbi5qYXZh) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...m/coordinator/exception/ClusterStateException.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvb3JkaW5hdG9yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vY29vcmRpbmF0b3IvZXhjZXB0aW9uL0NsdXN0ZXJTdGF0ZUV4Y2VwdGlvbi5qYXZh) | `92.85% <ø> (ø)` | `3 <0> (?)` | | | [...eam/coordinator/exception/CoordinateException.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvb3JkaW5hdG9yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vY29vcmRpbmF0b3IvZXhjZXB0aW9uL0Nvb3JkaW5hdGVFeGNlcHRpb24uamF2YQ==) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...kylin/stream/core/model/stats/ReplicaSetState.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL21vZGVsL3N0YXRzL1JlcGxpY2FTZXRTdGF0ZS5qYXZh) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...am/core/storage/columnar/ColumnarSegmentStore.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTZWdtZW50U3RvcmUuamF2YQ==) | `58.69% <ø> (ø)` | `28 <0> (?)` | | | [...est/controller/StreamingCoordinatorController.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdDb29yZGluYXRvckNvbnRyb2xsZXIuamF2YQ==) | `0% <ø> (ø)` | `0 <0> (?)` | | | [.../apache/kylin/engine/mr/StreamingCubingEngine.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvU3RyZWFtaW5nQ3ViaW5nRW5naW5lLmphdmE=) | `0% <ø> (ø)` | `0 <0> (?)` | | | [...stream/server/rest/controller/AdminController.java](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree#diff-c3RyZWFtLXJlY2VpdmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vc2VydmVyL3Jlc3QvY29udHJvbGxlci9BZG1pbkNvbnRyb2xsZXIuamF2YQ==) | `0% <ø> (ø)` | `0 <0> (?)` | | | ... and [19 more](https://codecov.io/gh/apache/kylin/pull/838/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/838?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/838?src=pr=footer). Last update [199bc17...edc8da8](https://codecov.io/gh/apache/kylin/pull/838?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[GitHub] [kylin] asf-ci commented on issue #804: KYLIN-4141 Build Global Dictionary in no time
asf-ci commented on issue #804: KYLIN-4141 Build Global Dictionary in no time URL: https://github.com/apache/kylin/pull/804#issuecomment-532472822 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time
hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time URL: https://github.com/apache/kylin/pull/804#issuecomment-522820885 ### Need fix & check - [x] Retry logic need to added to `RemoteDictionaryStore` in case of unavailable of HBase cluster. - [x] Rocksdb need to be add more specific config. - [ ] Integer overflow - [ ] Correctness of count_distinct - [ ] Performance(ingest/rate) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time
hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time URL: https://github.com/apache/kylin/pull/804#issuecomment-522820885 ### Need fix & check - [x] Retry logic need to added to `RemoteDictionaryStore` in case of unavailable of HBase cluster. - [x] Rocksdb need to be add more specific config. - [ ] Integer overflow - [x] Correctness of count_distinct - [ ] Performance(ingest/rate) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time
hit-lacus edited a comment on issue #804: KYLIN-4141 Build Global Dictionary in no time URL: https://github.com/apache/kylin/pull/804#issuecomment-522820885 ### Need fix & check - [x] Retry logic need to added to `RemoteDictionaryStore` in case of unavailable of HBase cluster. - [ ] Rocksdb need to be add more specific config. - [ ] Integer overflow - [ ] Correctness of count_distinct - [ ] Performance(ingest/rate) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (KYLIN-2820) Query can't read window function's result from subquery
[
https://issues.apache.org/jira/browse/KYLIN-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-2820.
-
Resolution: Fixed
> Query can't read window function's result from subquery
> ---
>
> Key: KYLIN-2820
> URL: https://issues.apache.org/jira/browse/KYLIN-2820
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>Assignee: nichunen
>Priority: Major
> Labels: scope
> Fix For: v3.0.0-beta
>
>
> I executed a query like the follows:
> {code:sql}
> select first_page_name, count(*) as page_name_count from
> (
> select first_value(page_name) over(partition by session_id) as
> first_page_name from some_db.some_table
> ) group by first_page_name;
> {code}
> This query resulted in one single record with an empty string as the
> first_page_name, and a big number as the page_name_count.
> However, when I ran the subquery without the outer query, it resulted in
> multiple records with different first_page_name, which proved the query above
> was wrong.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-2820) Query can't read window function's result from subquery
[
https://issues.apache.org/jira/browse/KYLIN-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-2820:
Fix Version/s: v3.0.0-beta
> Query can't read window function's result from subquery
> ---
>
> Key: KYLIN-2820
> URL: https://issues.apache.org/jira/browse/KYLIN-2820
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>Assignee: nichunen
>Priority: Major
> Labels: scope
> Fix For: v3.0.0-beta
>
>
> I executed a query like the follows:
> {code:sql}
> select first_page_name, count(*) as page_name_count from
> (
> select first_value(page_name) over(partition by session_id) as
> first_page_name from some_db.some_table
> ) group by first_page_name;
> {code}
> This query resulted in one single record with an empty string as the
> first_page_name, and a big number as the page_name_count.
> However, when I ran the subquery without the outer query, it resulted in
> multiple records with different first_page_name, which proved the query above
> was wrong.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-4117) Intersect_count() return wrong result when column type is time
[ https://issues.apache.org/jira/browse/KYLIN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4117: Fix Version/s: (was: v3.0.0-alpha2) v3.0.0-beta > Intersect_count() return wrong result when column type is time > -- > > Key: KYLIN-4117 > URL: https://issues.apache.org/jira/browse/KYLIN-4117 > Project: Kylin > Issue Type: Bug >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0-beta > > > *Intersect_count()* return wrong result when column type is *time.* -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4108) Show slow query hit cube in slow query page
[ https://issues.apache.org/jira/browse/KYLIN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4108: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Show slow query hit cube in slow query page > --- > > Key: KYLIN-4108 > URL: https://issues.apache.org/jira/browse/KYLIN-4108 > Project: Kylin > Issue Type: Improvement > Components: Metrics >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-3878) NPE to run sonar analysis
[ https://issues.apache.org/jira/browse/KYLIN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3878: Fix Version/s: v3.0.0-beta > NPE to run sonar analysis > - > > Key: KYLIN-3878 > URL: https://issues.apache.org/jira/browse/KYLIN-3878 > Project: Kylin > Issue Type: Test > Components: Tools, Build and Test >Reporter: Shaofeng SHI >Priority: Major > Fix For: v3.0.0-beta > > > mvn sonar:sonar -Dsonar.host.url=https://sonarcloud.io > -Dsonar.organization=kylin -e > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:13 min > [INFO] Finished at: 2019-03-15T14:42:16Z > [INFO] > > [ERROR] Failed to execute goal > org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar > (default-cli) on project kylin: null: MojoExecutionException: > NullPointerException -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar > (default-cli) on project kylin: null > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:213) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:154) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:146) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) > at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke (Method.java:498) > at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced > (Launcher.java:289) > at org.codehaus.plexus.classworlds.launcher.Launcher.launch > (Launcher.java:229) > at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode > (Launcher.java:415) > at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) > Caused by: org.apache.maven.plugin.MojoExecutionException > at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute > (ScannerBootstrapper.java:67) > at org.sonarsource.scanner.maven.SonarQubeMojo.execute > (SonarQubeMojo.java:104) > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:208) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:154) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:146) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) > at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke
[jira] [Resolved] (KYLIN-3878) NPE to run sonar analysis
[ https://issues.apache.org/jira/browse/KYLIN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3878. - Resolution: Fixed > NPE to run sonar analysis > - > > Key: KYLIN-3878 > URL: https://issues.apache.org/jira/browse/KYLIN-3878 > Project: Kylin > Issue Type: Test > Components: Tools, Build and Test >Reporter: Shaofeng SHI >Priority: Major > Fix For: v3.0.0-beta > > > mvn sonar:sonar -Dsonar.host.url=https://sonarcloud.io > -Dsonar.organization=kylin -e > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:13 min > [INFO] Finished at: 2019-03-15T14:42:16Z > [INFO] > > [ERROR] Failed to execute goal > org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar > (default-cli) on project kylin: null: MojoExecutionException: > NullPointerException -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar > (default-cli) on project kylin: null > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:213) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:154) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:146) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) > at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke (Method.java:498) > at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced > (Launcher.java:289) > at org.codehaus.plexus.classworlds.launcher.Launcher.launch > (Launcher.java:229) > at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode > (Launcher.java:415) > at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356) > Caused by: org.apache.maven.plugin.MojoExecutionException > at org.sonarsource.scanner.maven.bootstrap.ScannerBootstrapper.execute > (ScannerBootstrapper.java:67) > at org.sonarsource.scanner.maven.SonarQubeMojo.execute > (SonarQubeMojo.java:104) > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:208) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:154) > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:146) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290) > at org.apache.maven.cli.MavenCli.main (MavenCli.java:194) > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke (Method.java:498) >
[jira] [Resolved] (KYLIN-4111) drop table failed with no valid privileges after KYLIN-3857
[
https://issues.apache.org/jira/browse/KYLIN-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4111.
-
Resolution: Fixed
> drop table failed with no valid privileges after KYLIN-3857
> ---
>
> Key: KYLIN-4111
> URL: https://issues.apache.org/jira/browse/KYLIN-4111
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> After KYLIN-3857, there will be quote ` around database and table.
> The drop table sql will be:
> {code:java}
> DROP TABLE IF EXISTS
> `kylin_onebox.kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`;{code}
> Hive (1.2)with sentry will throw exception:
> {code:java}
> Error: Error while compiling statement: FAILED: HiveAccessControlException No
> valid privileges
> Required privileges for this query:
> Server=server1->Db=`kylin_onebox->Table=kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`->action=drop;
> Query log:
> http://zjy-hadoop-prc-ct14.bj:18201/log?qid=898c7878-a961-443d-b120-cca0e2667d15_f486bd16-4bbd-4014-a0a7-c2ebfdbe6668
> (state=42000,code=4)
> {code}
> The reason is that hive identify the databse be `kylin_onebox and table be:
> kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`
> May be we can fix it in hive and sentry. Just create a jira to show this
> problem.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-4111) drop table failed with no valid privileges after KYLIN-3857
[
https://issues.apache.org/jira/browse/KYLIN-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4111:
Fix Version/s: v3.0.0-beta
> drop table failed with no valid privileges after KYLIN-3857
> ---
>
> Key: KYLIN-4111
> URL: https://issues.apache.org/jira/browse/KYLIN-4111
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> After KYLIN-3857, there will be quote ` around database and table.
> The drop table sql will be:
> {code:java}
> DROP TABLE IF EXISTS
> `kylin_onebox.kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`;{code}
> Hive (1.2)with sentry will throw exception:
> {code:java}
> Error: Error while compiling statement: FAILED: HiveAccessControlException No
> valid privileges
> Required privileges for this query:
> Server=server1->Db=`kylin_onebox->Table=kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`->action=drop;
> Query log:
> http://zjy-hadoop-prc-ct14.bj:18201/log?qid=898c7878-a961-443d-b120-cca0e2667d15_f486bd16-4bbd-4014-a0a7-c2ebfdbe6668
> (state=42000,code=4)
> {code}
> The reason is that hive identify the databse be `kylin_onebox and table be:
> kylin_intermediate_kylin_sales_cube_7be84be1_a153_07c4_3ce6_270e8d99ff85`
> May be we can fix it in hive and sentry. Just create a jira to show this
> problem.
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-4098) Add cube auto merge api
[ https://issues.apache.org/jira/browse/KYLIN-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4098: Fix Version/s: v3.0.0-beta > Add cube auto merge api > --- > > Key: KYLIN-4098 > URL: https://issues.apache.org/jira/browse/KYLIN-4098 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > > Currently the auto merging of cube is triggered by the event of new segment > is ready automatically. When the cluster restart, there may be too many > merging job. > It's better to have a rest api to trigger the merging and make it more > controllable. > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (KYLIN-4098) Add cube auto merge api
[ https://issues.apache.org/jira/browse/KYLIN-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4098. - Resolution: Fixed > Add cube auto merge api > --- > > Key: KYLIN-4098 > URL: https://issues.apache.org/jira/browse/KYLIN-4098 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > > Currently the auto merging of cube is triggered by the event of new segment > is ready automatically. When the cluster restart, there may be too many > merging job. > It's better to have a rest api to trigger the merging and make it more > controllable. > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4057) autoMerge job can not stop
[ https://issues.apache.org/jira/browse/KYLIN-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4057: Fix Version/s: (was: v3.0.0-alpha2) v3.0.0-beta > autoMerge job can not stop > -- > > Key: KYLIN-4057 > URL: https://issues.apache.org/jira/browse/KYLIN-4057 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.2 >Reporter: chenchen >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: 5031561517158_.pic_hd.jpg > > > In this version，I used DistributedScheduler。 > In my case, A cube config autoMerge, But the job that automatically merges > segments is wrong ，I can't fix this error, I can only discard and drop the > job, but Kylin's monitoring mechanism creates the same new job. > I want to ask if there is any way to fix this problem without banning cube. > The error is reported as follows. > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (KYLIN-4091) support fast mode and simple mode for running CI
[ https://issues.apache.org/jira/browse/KYLIN-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4091. - Resolution: Fixed > support fast mode and simple mode for running CI > > > Key: KYLIN-4091 > URL: https://issues.apache.org/jira/browse/KYLIN-4091 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > -DfastBuildMode = true // build or merge job in concurrent queue > -DsimpleBuildMode = true // skip merge job and just build only one segment -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4091) support fast mode and simple mode for running CI
[ https://issues.apache.org/jira/browse/KYLIN-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4091: Fix Version/s: v3.0.0-beta > support fast mode and simple mode for running CI > > > Key: KYLIN-4091 > URL: https://issues.apache.org/jira/browse/KYLIN-4091 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > -DfastBuildMode = true // build or merge job in concurrent queue > -DsimpleBuildMode = true // skip merge job and just build only one segment -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-1856) Kylin shows old error in job step output after resume - specifically in #4 Step Name: Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-1856: Fix Version/s: (was: v3.0.0-alpha2) v3.0.0-beta > Kylin shows old error in job step output after resume - specifically in #4 > Step Name: Build Dimension Dictionary > > > Key: KYLIN-1856 > URL: https://issues.apache.org/jira/browse/KYLIN-1856 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.2, v1.5.2.1 >Reporter: Richard Calaba >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0-beta > > > I have realized that if my job stops with error and I try to recover the > error and resume the job - then the latest step starts again from scratch. > This is fine but in my opinion the log of the Step should clear as well - now > it is showing the error from my previous attempt. > Specifically observed in #4 Step Name: Build Dimension Dictionary - but is > probbaly generic issue. > To correct this: clear the log of the Build Step after the job Step is > resumed. Already when the job step is restarted, not after it is completed. > (if Kylin fails i.e. for out of memory - it silently dies and analyzing the > step log shows wrong error (from previous run) - if it would be empty -> I > would know that most probable cause was that Kylin died) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4106) Illegal partition for SelfDefineSortableKey when “Extract Fact Table Distinct Columns”
[
https://issues.apache.org/jira/browse/KYLIN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4106:
Fix Version/s: v3.0.0-beta
> Illegal partition for SelfDefineSortableKey when “Extract Fact Table Distinct
> Columns”
> --
>
> Key: KYLIN-4106
> URL: https://issues.apache.org/jira/browse/KYLIN-4106
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.6.1, v2.6.2
>Reporter: langdamao
>Assignee: langdamao
>Priority: Critical
> Labels: easyfix
> Fix For: v3.0.0-beta, v2.6.4
>
>
> We got this error when Extract Fact Table Distinct Columns @kylin 2.6.1
>
> {code:java}
> Error: java.io.IOException: Illegal partition for
> org.apache.kylin.engine.mr.steps.SelfDefineSortableKey@6b69761b (254)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1096)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:727)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper.writeFieldValue(FactDistinctColumnsMapper.java:
> 281) at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper.doMap(FactDistinctColumnsMapper.java:186)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> I've found the problem in the follow code in
> *FactDistinctColumnsReducerMapping.java – engine-mr*
> {code:java}
> public int getReducerIdForCol(int colId, Object fieldValue) {
> int begin = colIdToReducerBeginId[colId];
> int span = colIdToReducerBeginId[colId + 1] - begin;
>
> if (span == 1)
> return begin;
>
> int hash = fieldValue == null ? 0 : fieldValue.hashCode();
> return begin + Math.abs(hash) % span;
> }
> {code}
> for the error rowkey it's begin=1, span=5 ，and we got hash=-2147483648
> ，meanwhile Math.abs(-2147483648) return -2147483648 ,so for the above code it
> return -2 ( which was 254 while unsigned).
> this will also cause problem bellow when Function getReduerIdForCol return
> -1 （when begin=1,span=3,hash= -2147483648) ,because value write to rowkey
> reducer is empty_text ， but No. -1 reducer need value text
> {code:java}
> Error: java.nio.BufferUnderflowException at
> java.nio.Buffer.nextGetIndex(Buffer.java:500)
> at java.nio.HeapByteBuffer.get(Heap.ByteBuffer.java:135)
> at org.apache.kylin.measure.hllc.HLLCounter.readRegisters(HLLCounter.java:327)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doReduce(FactDistinctColumnsReducer.java:145)
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doReduce(FactDistinctColumnsReducer.java:60)
> ...{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Reopened] (KYLIN-4039) ZookeeperDistributedLock may not release lock when unlock operation was interrupted
[ https://issues.apache.org/jira/browse/KYLIN-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4039: - > ZookeeperDistributedLock may not release lock when unlock operation was > interrupted > --- > > Key: KYLIN-4039 > URL: https://issues.apache.org/jira/browse/KYLIN-4039 > Project: Kylin > Issue Type: Bug >Reporter: PENG Zhengshuai >Assignee: PENG Zhengshuai >Priority: Major > Fix For: v3.0.0-alpha2 > > > ZookeeperDistributedLock may hold the lock and not release it when the unlock > operation was interrupted. > Because the unlock operation contains two steps: > 1. peekLock: get the owner of the lock > 2. purgeLock: purge the lock if the owner of the lock is the current client. > If the peekLock step is interrupted, the purgeLock step won't be executed. > Thus the lock won't be released. > Meanwhile, the lock operation should also consider the interrupt cases. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (KYLIN-4039) ZookeeperDistributedLock may not release lock when unlock operation was interrupted
[ https://issues.apache.org/jira/browse/KYLIN-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4039. - Resolution: Fixed > ZookeeperDistributedLock may not release lock when unlock operation was > interrupted > --- > > Key: KYLIN-4039 > URL: https://issues.apache.org/jira/browse/KYLIN-4039 > Project: Kylin > Issue Type: Bug >Reporter: PENG Zhengshuai >Assignee: PENG Zhengshuai >Priority: Major > Fix For: v3.0.0-beta > > > ZookeeperDistributedLock may hold the lock and not release it when the unlock > operation was interrupted. > Because the unlock operation contains two steps: > 1. peekLock: get the owner of the lock > 2. purgeLock: purge the lock if the owner of the lock is the current client. > If the peekLock step is interrupted, the purgeLock step won't be executed. > Thus the lock won't be released. > Meanwhile, the lock operation should also consider the interrupt cases. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4039) ZookeeperDistributedLock may not release lock when unlock operation was interrupted
[ https://issues.apache.org/jira/browse/KYLIN-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4039: Fix Version/s: (was: v3.0.0-alpha2) v3.0.0-beta > ZookeeperDistributedLock may not release lock when unlock operation was > interrupted > --- > > Key: KYLIN-4039 > URL: https://issues.apache.org/jira/browse/KYLIN-4039 > Project: Kylin > Issue Type: Bug >Reporter: PENG Zhengshuai >Assignee: PENG Zhengshuai >Priority: Major > Fix For: v3.0.0-beta > > > ZookeeperDistributedLock may hold the lock and not release it when the unlock > operation was interrupted. > Because the unlock operation contains two steps: > 1. peekLock: get the owner of the lock > 2. purgeLock: purge the lock if the owner of the lock is the current client. > If the peekLock step is interrupted, the purgeLock step won't be executed. > Thus the lock won't be released. > Meanwhile, the lock operation should also consider the interrupt cases. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (KYLIN-4066) No planner for not ROLE_ADMIN user on WebSite
[
https://issues.apache.org/jira/browse/KYLIN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4066.
-
Resolution: Fixed
> No planner for not ROLE_ADMIN user on WebSite
> -
>
> Key: KYLIN-4066
> URL: https://issues.apache.org/jira/browse/KYLIN-4066
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.6.1
> Environment: macOs
>Reporter: langdamao
>Priority: Major
> Labels: easyfix
> Fix For: v3.0.0-beta, v2.6.4
>
>
> There is no planner page for none ROLE_ADMIN user ,even though he has ADMIN
> access for the project.
> I found in webapp/app/partials/cubes/cube_detail.html line44
> {code:java}
> ng-if="(userService.hasRole('ROLE_ADMIN') || hasPermission(cube,
> permissions.ADMINISTRATION.mask)) && isShowCubeplanner"> ng-click="cube.visiblePage='planner';getCubePlanner(cube);">Planner
> {code}
> but other place the hasPermission function use as
> {code:java}
> hasPermission('cube',cube, permissions.ADMINISTRATION.mask)
> {code}
> the function hasPermission is defined in webapp/app/js/controllers/page.js
> line100
> {code:java}
> $scope.hasPermission = function (accessType, entity){
> ...
> if (accessType === 'cube') {
> project = entity.project
> } else if (accessType === 'project') {
> project = entity && entity.name || entity.selectedProject
> } else if (accessType === 'model') {
> project = ProjectModel.getProjectByCubeModel(entity.name)
> }
> ...
> }{code}
> I guess whether it's cause by no param of accessType.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-4066) No planner for not ROLE_ADMIN user on WebSite
[
https://issues.apache.org/jira/browse/KYLIN-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4066:
Fix Version/s: v3.0.0-beta
> No planner for not ROLE_ADMIN user on WebSite
> -
>
> Key: KYLIN-4066
> URL: https://issues.apache.org/jira/browse/KYLIN-4066
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.6.1
> Environment: macOs
>Reporter: langdamao
>Priority: Major
> Labels: easyfix
> Fix For: v3.0.0-beta, v2.6.4
>
>
> There is no planner page for none ROLE_ADMIN user ,even though he has ADMIN
> access for the project.
> I found in webapp/app/partials/cubes/cube_detail.html line44
> {code:java}
> ng-if="(userService.hasRole('ROLE_ADMIN') || hasPermission(cube,
> permissions.ADMINISTRATION.mask)) && isShowCubeplanner"> ng-click="cube.visiblePage='planner';getCubePlanner(cube);">Planner
> {code}
> but other place the hasPermission function use as
> {code:java}
> hasPermission('cube',cube, permissions.ADMINISTRATION.mask)
> {code}
> the function hasPermission is defined in webapp/app/js/controllers/page.js
> line100
> {code:java}
> $scope.hasPermission = function (accessType, entity){
> ...
> if (accessType === 'cube') {
> project = entity.project
> } else if (accessType === 'project') {
> project = entity && entity.name || entity.selectedProject
> } else if (accessType === 'model') {
> project = ProjectModel.getProjectByCubeModel(entity.name)
> }
> ...
> }{code}
> I guess whether it's cause by no param of accessType.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (KYLIN-3519) Upgrade Jacoco version to 0.8.2
[ https://issues.apache.org/jira/browse/KYLIN-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3519. - Resolution: Fixed > Upgrade Jacoco version to 0.8.2 > --- > > Key: KYLIN-3519 > URL: https://issues.apache.org/jira/browse/KYLIN-3519 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Assignee: weibin0516 >Priority: Minor > Fix For: v3.0.0-beta > > > Jacoco 0.8.2 adds Java 11 support: >https://github.com/jacoco/jacoco/releases/tag/v0.8.2 > Java 11 RC1 is out. > We should consider upgrading Jacoco. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-3519) Upgrade Jacoco version to 0.8.2
[ https://issues.apache.org/jira/browse/KYLIN-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3519: Fix Version/s: v3.0.0-beta > Upgrade Jacoco version to 0.8.2 > --- > > Key: KYLIN-3519 > URL: https://issues.apache.org/jira/browse/KYLIN-3519 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Assignee: weibin0516 >Priority: Minor > Fix For: v3.0.0-beta > > > Jacoco 0.8.2 adds Java 11 support: >https://github.com/jacoco/jacoco/releases/tag/v0.8.2 > Java 11 RC1 is out. > We should consider upgrading Jacoco. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (KYLIN-4099) Using no blocking RDD unpersist in spark cubing job
[
https://issues.apache.org/jira/browse/KYLIN-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4099.
-
Resolution: Fixed
> Using no blocking RDD unpersist in spark cubing job
>
>
> Key: KYLIN-4099
> URL: https://issues.apache.org/jira/browse/KYLIN-4099
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> By default, the unpersist operation of RDD in spark is blocking which may
> cost a lot time and
> some times it may failed for some spark executors lost.
> We can set blocking false to improve it.
> {code:java}
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:208)
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190)
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> scala.concurrent.Await$.result(package.scala:190)
> org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:81)
> org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:127)
> org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1709)
> org.apache.spark.rdd.RDD.unpersist(RDD.scala:216)
> org.apache.spark.api.java.JavaPairRDD.unpersist(JavaPairRDD.scala:73)
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:204)
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (KYLIN-4099) Using no blocking RDD unpersist in spark cubing job
[
https://issues.apache.org/jira/browse/KYLIN-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4099:
Fix Version/s: v3.0.0-beta
> Using no blocking RDD unpersist in spark cubing job
>
>
> Key: KYLIN-4099
> URL: https://issues.apache.org/jira/browse/KYLIN-4099
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> By default, the unpersist operation of RDD in spark is blocking which may
> cost a lot time and
> some times it may failed for some spark executors lost.
> We can set blocking false to improve it.
> {code:java}
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:208)
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:218)
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
> scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190)
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> scala.concurrent.Await$.result(package.scala:190)
> org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:81)
> org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:127)
> org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1709)
> org.apache.spark.rdd.RDD.unpersist(RDD.scala:216)
> org.apache.spark.api.java.JavaPairRDD.unpersist(JavaPairRDD.scala:73)
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:204)
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [kylin] nichunen opened a new pull request #842: Add some description info on home page
nichunen opened a new pull request #842: Add some description info on home page URL: https://github.com/apache/kylin/pull/842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asf-ci commented on issue #842: Add some description info on home page
asf-ci commented on issue #842: Add some description info on home page URL: https://github.com/apache/kylin/pull/842#issuecomment-532234756 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931233#comment-16931233 ] ASF subversion and git services commented on KYLIN-4100: Commit 199bc17e572652b8a8ec8c4f0c6aab1a2359c1ef in kylin's branch refs/heads/master from Liu Shaohui [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=199bc17 ] KYLIN-4100: Add overall job number statistics in monitor page > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931232#comment-16931232 ] ASF GitHub Bot commented on KYLIN-4100: --- luguosheng1314 commented on pull request #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] luguosheng1314 merged pull request #841: KYLIN-4100: Add overall job number statistics in monitor page
luguosheng1314 merged pull request #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #841: KYLIN-4100: Add overall job number statistics in monitor page
coveralls commented on issue #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841#issuecomment-532138215 ## Pull Request Test Coverage Report for [Build 5017](https://coveralls.io/builds/25755491) * **0** of **37** **(0.0%)** changed or added relevant lines in **4** files are covered. * **5** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.175%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/JobInfoConverter.java](https://coveralls.io/builds/25755491/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FJobInfoConverter.java#L240) | 0 | 1 | 0.0% | [core-job/src/main/java/org/apache/kylin/job/JobSearchResult.java](https://coveralls.io/builds/25755491/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJobSearchResult.java#L67) | 0 | 3 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/JobService.java](https://coveralls.io/builds/25755491/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FJobService.java#L1002) | 0 | 12 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/JobController.java](https://coveralls.io/builds/25755491/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FJobController.java#L104) | 0 | 21 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-job/src/main/java/org/apache/kylin/job/JobSearchResult.java](https://coveralls.io/builds/25755491/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJobSearchResult.java#L23) | 1 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/JobInfoConverter.java](https://coveralls.io/builds/25755491/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FJobInfoConverter.java#L241) | 1 | 35.16% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarStoreCache.java](https://coveralls.io/builds/25755491/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FColumnarStoreCache.java#L136) | 1 | 55.81% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/25755491/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | Totals | [](https://coveralls.io/builds/25755491) | | :-- | --: | | Change from base [Build 5007](https://coveralls.io/builds/25692587): | -0.01% | | Covered Lines: | 23591 | | Relevant Lines: | 83729 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4135) Real time streaming segment build task discard but can't be rebuilt
[ https://issues.apache.org/jira/browse/KYLIN-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4135: Fix Version/s: (was: Future) v3.0.0-beta > Real time streaming segment build task discard but can't be rebuilt > > > Key: KYLIN-4135 > URL: https://issues.apache.org/jira/browse/KYLIN-4135 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: Future >Reporter: wangxiaojing >Priority: Major > Fix For: v3.0.0-beta > > > Real time streaming segment build task discard in some case ,but now can't be > rebuilt, and cant't subbmit other new segment tsrange to build if it reach > the cube's max building number( > [https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-4134?filter=allopenissues]) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931216#comment-16931216
]
ASF subversion and git services commented on KYLIN-3973:
Commit df6e6a46759f335c199b3731fe6f566c7cbd7f6e in kylin's branch
refs/heads/master-hadoop3.1 from Zhou Kang
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=df6e6a4 ]
KYLIN-3973: InvalidProtocolBufferException: Protocol message was too large.
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931215#comment-16931215
]
ASF GitHub Bot commented on KYLIN-3973:
---
nichunen commented on pull request #837: KYLIN-3973:
InvalidProtocolBufferException
URL: https://github.com/apache/kylin/pull/837
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [kylin] nichunen merged pull request #837: KYLIN-3973: InvalidProtocolBufferException
nichunen merged pull request #837: KYLIN-3973: InvalidProtocolBufferException URL: https://github.com/apache/kylin/pull/837 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive
nichunen merged pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931209#comment-16931209 ] ASF GitHub Bot commented on KYLIN-4103: --- nichunen commented on pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931210#comment-16931210 ] ASF subversion and git services commented on KYLIN-4103: Commit 23846794fef1d6c4eae523869da7c4d2617b540c in kylin's branch refs/heads/master from Liu Shaohui [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=2384679 ] KYLIN-4103: Make the user string in granting operation of project is case insensitive > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] asf-ci commented on issue #841: KYLIN-4100: Add overall job number statistics in monitor page
asf-ci commented on issue #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841#issuecomment-532126806 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] lshmouse opened a new pull request #841: KYLIN-4100: Add overall job number statistics in monitor page
lshmouse opened a new pull request #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841 See: https://jira.apache.org/jira/browse/KYLIN-4100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931201#comment-16931201 ] Liu Shaohui commented on KYLIN-4100: You can see the demo image in the attachments. > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931202#comment-16931202 ] ASF GitHub Bot commented on KYLIN-4100: --- lshmouse commented on pull request #841: KYLIN-4100: Add overall job number statistics in monitor page URL: https://github.com/apache/kylin/pull/841 See: https://jira.apache.org/jira/browse/KYLIN-4100 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated KYLIN-4100: --- Attachment: x.png > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.2#803003)
