[
https://issues.apache.org/jira/browse/KYLIN-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708623#comment-16708623
]
Chao Long edited comment on KYLIN-3686 at 12/4/18 12:21 PM:
------------------------------------------------------------
Hi Vsevolod, The front-end will get the value of parameter
"kylin.storage.default"(default ID_SHARDED_HBASE=2) from the kylin.properties
and will use it as the cube's storage type. Can you check this parameter in
your kylin.properties?
Another situation is the front-end does not get this value for some unknown
reason, it will pass a null value to the back-end, and the back-end will use
ID_HBASE as the default value of storage type.
If the latter, KYLIN-3636 has fixed it and will release soon in kylin-2.5.2.
was (Author: wayne0101):
Hi Vsevolod, The front-end will get the value of parameter
"kylin.storage.default"(default ID_SHARDED_HBASE=2) from the kylin.properties
and will use it as the cube's storage type. Can you check this parameter in
your kylin.properties?
Another situation is the front-end does not get this value for some unknown
reason, it will pass a null value to the back-end, and the back-end will use
ID_HBASE as the default value of storage type.
If the latter, [KYLIN-3636|https://issues.apache.org/jira/browse/KYLIN-3636]
has fixed it and will release soon.
> Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the
> Web UI defaults to ID_HBASE and provides no safeguards against storage type
> mismatch
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-3686
> URL: https://issues.apache.org/jira/browse/KYLIN-3686
> Project: Kylin
> Issue Type: Improvement
> Components: Measure - TopN, Metadata, Web
> Affects Versions: v2.5.0
> Environment: HDP 2.5.6, Kylin 2.5
> Reporter: Vsevolod Ostapenko
> Assignee: Shaofeng SHI
> Priority: Major
> Fix For: v2.6.0
>
>
> When new cube is defined via Kylin 2.5 UI, the default cube storage type is
> set to 0 (ID_HBASE).
> Top_N metric support is currently hard coded to expect cube storage type 2
> (ID_SHARDED_HBASE), and it *_does not_* check if the cube storage type is the
> "sharded HBASE".
> UI provides no safeguards either to prevent a user from defining a cube with
> Top_N metric that would blow up on the cube building stage with a perplexing
> stack trace like the following:
> {quote}2018-10-22 16:15:50,388 ERROR [main]
> org.apache.kylin.engine.mr.KylinMapper:
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKeyInternal(NDCuboidBuilder.java:106)
> at
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKey(NDCuboidBuilder.java:71)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:112)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:47)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> {quote}
> Please, either:
> – modify Top_N code to support all cube storage types (not only
> ID_SHARDED_HBASE),
> or
> – modify Top_N code to perform explicit check for cube storage type and
> raise descriptive exception, when cube storage is not the one that is
> expected. Plus update the UI to prevent the user from creating cube
> definitions that are incompatible with the storage type compatible with Top_N
> measure
> PS: NDCCuboidBuilder,java contains the following line:
> {quote}int offset = RowConstants.ROWKEY_SHARDID_LEN +
> RowConstants.ROWKEY_CUBOIDID_LEN; // skip shard and cuboidId{quote}
> If cube storage type is not ID_SHARDED_HBASE, offset is calculated
> incorrectly, which leads to ArrayIndexOutOfBounds exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
