[jira] [Assigned] (KYLIN-3686) Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the Web UI defaults to ID_HBASE and provides no safeguards against storage type mismatch

2018-11-28 Thread Shaofeng SHI (JIRA) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shaofeng SHI reassigned KYLIN-3686:
---

Assignee: Shaofeng SHI

> Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the 
> Web UI defaults to ID_HBASE and provides no safeguards against storage type 
> mismatch
> -
>
> Key: KYLIN-3686
> URL: https://issues.apache.org/jira/browse/KYLIN-3686
> Project: Kylin
>  Issue Type: Improvement
>  Components: Measure - TopN, Metadata, Web 
>Affects Versions: v2.5.0
> Environment: HDP 2.5.6, Kylin 2.5
>Reporter: Vsevolod Ostapenko
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> When new cube is defined via Kylin 2.5 UI, the default cube storage type is 
> set to 0 (ID_HBASE).
>  Top_N metric support is currently hard coded to expect cube storage type 2 
> (ID_SHARDED_HBASE), and it *_does not_* check if the cube storage type is the 
> "sharded HBASE".
>  UI provides no safeguards either to prevent a user from defining a cube with 
> Top_N metric that would blow up on the cube building stage with a perplexing 
> stack trace like the following:
> {quote}2018-10-22 16:15:50,388 ERROR [main] 
> org.apache.kylin.engine.mr.KylinMapper:
>  java.lang.ArrayIndexOutOfBoundsException
>  at java.lang.System.arraycopy(Native Method)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKeyInternal(NDCuboidBuilder.java:106)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKey(NDCuboidBuilder.java:71)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:112)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:47)
>  at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> {quote}
> Please, either:
> – modify Top_N code to support all cube storage types (not only 
> ID_SHARDED_HBASE),
>  or 
>  – modify Top_N code to perform explicit check for cube storage type and 
> raise descriptive exception, when cube storage is not the one that is 
> expected. Plus update the UI to prevent the user from creating cube 
> definitions that are incompatible with the storage type compatible with Top_N 
> measure
> PS: NDCCuboidBuilder,java contains the following line:
> {quote}int offset = RowConstants.ROWKEY_SHARDID_LEN + 
> RowConstants.ROWKEY_CUBOIDID_LEN; // skip shard and cuboidId{quote}
> If cube storage type is not ID_SHARDED_HBASE, offset is calculated 
> incorrectly, which leads to ArrayIndexOutOfBounds exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3686) Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the Web UI defaults to ID_HBASE and provides no safeguards against storage type mismatch

2018-11-21 Thread XiaoXiang Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

XiaoXiang Yu reassigned KYLIN-3686:
---

Assignee: (was: XiaoXiang Yu)

> Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the 
> Web UI defaults to ID_HBASE and provides no safeguards against storage type 
> mismatch
> -
>
> Key: KYLIN-3686
> URL: https://issues.apache.org/jira/browse/KYLIN-3686
> Project: Kylin
>  Issue Type: Improvement
>  Components: Measure - TopN, Metadata, Web 
>Affects Versions: v2.5.0
> Environment: HDP 2.5.6, Kylin 2.5
>Reporter: Vsevolod Ostapenko
>Priority: Major
>
> When new cube is defined via Kylin 2.5 UI, the default cube storage type is 
> set to 0 (ID_HBASE).
>  Top_N metric support is currently hard coded to expect cube storage type 2 
> (ID_SHARDED_HBASE), and it *_does not_* check if the cube storage type is the 
> "sharded HBASE".
>  UI provides no safeguards either to prevent a user from defining a cube with 
> Top_N metric that would blow up on the cube building stage with a perplexing 
> stack trace like the following:
> {quote}2018-10-22 16:15:50,388 ERROR [main] 
> org.apache.kylin.engine.mr.KylinMapper:
>  java.lang.ArrayIndexOutOfBoundsException
>  at java.lang.System.arraycopy(Native Method)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKeyInternal(NDCuboidBuilder.java:106)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKey(NDCuboidBuilder.java:71)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:112)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:47)
>  at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> {quote}
> Please, either:
> – modify Top_N code to support all cube storage types (not only 
> ID_SHARDED_HBASE),
>  or 
>  – modify Top_N code to perform explicit check for cube storage type and 
> raise descriptive exception, when cube storage is not the one that is 
> expected. Plus update the UI to prevent the user from creating cube 
> definitions that are incompatible with the storage type compatible with Top_N 
> measure
> PS: NDCCuboidBuilder,java contains the following line:
> {quote}int offset = RowConstants.ROWKEY_SHARDID_LEN + 
> RowConstants.ROWKEY_CUBOIDID_LEN; // skip shard and cuboidId{quote}
> If cube storage type is not ID_SHARDED_HBASE, offset is calculated 
> incorrectly, which leads to ArrayIndexOutOfBounds exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (KYLIN-3686) Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the Web UI defaults to ID_HBASE and provides no safeguards against storage type mismatch

2018-11-15 Thread XiaoXiang Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

XiaoXiang Yu reassigned KYLIN-3686:
---

Assignee: XiaoXiang Yu

> Top_N metric code requires cube storage type to be ID_SHARDED_HBASE, but the 
> Web UI defaults to ID_HBASE and provides no safeguards against storage type 
> mismatch
> -
>
> Key: KYLIN-3686
> URL: https://issues.apache.org/jira/browse/KYLIN-3686
> Project: Kylin
>  Issue Type: Improvement
>  Components: Measure - TopN, Metadata, Web 
>Affects Versions: v2.5.0
> Environment: HDP 2.5.6, Kylin 2.5
>Reporter: Vsevolod Ostapenko
>Assignee: XiaoXiang Yu
>Priority: Major
>
> When new cube is defined via Kylin 2.5 UI, the default cube storage type is 
> set to 0 (ID_HBASE).
>  Top_N metric support is currently hard coded to expect cube storage type 2 
> (ID_SHARDED_HBASE), and it *_does not_* check if the cube storage type is the 
> "sharded HBASE".
>  UI provides no safeguards either to prevent a user from defining a cube with 
> Top_N metric that would blow up on the cube building stage with a perplexing 
> stack trace like the following:
> {quote}2018-10-22 16:15:50,388 ERROR [main] 
> org.apache.kylin.engine.mr.KylinMapper:
>  java.lang.ArrayIndexOutOfBoundsException
>  at java.lang.System.arraycopy(Native Method)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKeyInternal(NDCuboidBuilder.java:106)
>  at 
> org.apache.kylin.engine.mr.common.NDCuboidBuilder.buildKey(NDCuboidBuilder.java:71)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:112)
>  at 
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.doMap(NDCuboidMapper.java:47)
>  at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> {quote}
> Please, either:
> – modify Top_N code to support all cube storage types (not only 
> ID_SHARDED_HBASE),
>  or 
>  – modify Top_N code to perform explicit check for cube storage type and 
> raise descriptive exception, when cube storage is not the one that is 
> expected. Plus update the UI to prevent the user from creating cube 
> definitions that are incompatible with the storage type compatible with Top_N 
> measure
> PS: NDCCuboidBuilder,java contains the following line:
> {quote}int offset = RowConstants.ROWKEY_SHARDID_LEN + 
> RowConstants.ROWKEY_CUBOIDID_LEN; // skip shard and cuboidId{quote}
> If cube storage type is not ID_SHARDED_HBASE, offset is calculated 
> incorrectly, which leads to ArrayIndexOutOfBounds exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)