[jira] [Commented] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
[
https://issues.apache.org/jira/browse/KYLIN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503613#comment-15503613
]
Shaofeng SHI commented on KYLIN-2032:
-
hi chunen, it is related but different. I will check and may remove that code
block.
> Cube build failed when partition column isn't in dimension list
> ---
>
> Key: KYLIN-2032
> URL: https://issues.apache.org/jira/browse/KYLIN-2032
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.4
>Reporter: Shaofeng SHI
>
> Create a cube which use "LO_DATE" as the partition date column, while it
> wasn't in the dimension list; when build the cube, it got failed in the "Fact
> distinct" step, mappers reported such exception:
> {code}
> Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
> wasn't found on flat table. at
> org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
> at
> org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> If add it to dimension, then the build got success;
> In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in
> all case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
[
https://issues.apache.org/jira/browse/KYLIN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503238#comment-15503238
]
nichunen commented on KYLIN-2032:
-
Seems a related issue with KYLIN-2026
> Cube build failed when partition column isn't in dimension list
> ---
>
> Key: KYLIN-2032
> URL: https://issues.apache.org/jira/browse/KYLIN-2032
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.4
>Reporter: Shaofeng SHI
>
> Create a cube which use "LO_DATE" as the partition date column, while it
> wasn't in the dimension list; when build the cube, it got failed in the "Fact
> distinct" step, mappers reported such exception:
> {code}
> Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
> wasn't found on flat table. at
> org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
> at
> org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> If add it to dimension, then the build got success;
> In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in
> all case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
[
https://issues.apache.org/jira/browse/KYLIN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-2032:
Comment: was deleted
(was: Seems the same issue with KYLIN-2026.)
> Cube build failed when partition column isn't in dimension list
> ---
>
> Key: KYLIN-2032
> URL: https://issues.apache.org/jira/browse/KYLIN-2032
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.4
>Reporter: Shaofeng SHI
>
> Create a cube which use "LO_DATE" as the partition date column, while it
> wasn't in the dimension list; when build the cube, it got failed in the "Fact
> distinct" step, mappers reported such exception:
> {code}
> Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
> wasn't found on flat table. at
> org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
> at
> org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> If add it to dimension, then the build got success;
> In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in
> all case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
[
https://issues.apache.org/jira/browse/KYLIN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503221#comment-15503221
]
nichunen commented on KYLIN-2032:
-
Seems the same issue with KYLIN-2026.
> Cube build failed when partition column isn't in dimension list
> ---
>
> Key: KYLIN-2032
> URL: https://issues.apache.org/jira/browse/KYLIN-2032
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.4
>Reporter: Shaofeng SHI
>
> Create a cube which use "LO_DATE" as the partition date column, while it
> wasn't in the dimension list; when build the cube, it got failed in the "Fact
> distinct" step, mappers reported such exception:
> {code}
> Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
> wasn't found on flat table. at
> org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
> at
> org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> If add it to dimension, then the build got success;
> In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in
> all case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
[
https://issues.apache.org/jira/browse/KYLIN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2032:
Fix Version/s: (was: v1.6.0)
> Cube build failed when partition column isn't in dimension list
> ---
>
> Key: KYLIN-2032
> URL: https://issues.apache.org/jira/browse/KYLIN-2032
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.4
>Reporter: Shaofeng SHI
>
> Create a cube which use "LO_DATE" as the partition date column, while it
> wasn't in the dimension list; when build the cube, it got failed in the "Fact
> distinct" step, mappers reported such exception:
> {code}
> Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
> wasn't found on flat table. at
> org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
> at
> org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> If add it to dimension, then the build got success;
> In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in
> all case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-2032) Cube build failed when partition column isn't in dimension list
Shaofeng SHI created KYLIN-2032:
---
Summary: Cube build failed when partition column isn't in
dimension list
Key: KYLIN-2032
URL: https://issues.apache.org/jira/browse/KYLIN-2032
Project: Kylin
Issue Type: Bug
Affects Versions: v1.5.4
Reporter: Shaofeng SHI
Fix For: v1.6.0
Create a cube which use "LO_DATE" as the partition date column, while it wasn't
in the dimension list; when build the cube, it got failed in the "Fact
distinct" step, mappers reported such exception:
{code}
Error: java.lang.IllegalArgumentException: Column SSB.V_LINEORDER.LO_DATE
wasn't found on flat table. at
org.apache.kylin.cube.model.CubeJoinedFlatTableDesc.getColumnIndex(CubeJoinedFlatTableDesc.java:145)
at
org.apache.kylin.cube.model.CubeJoinedFlatTableEnrich.getColumnIndex(CubeJoinedFlatTableEnrich.java:122)
at
org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapperBase.setup(FactDistinctColumnsMapperBase.java:80)
at
org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.setup(FactDistinctHiveColumnsMapper.java:59)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{code}
If add it to dimension, then the build got success;
In CubeManager.getAllDictColumnsOnFact(), it said "add partition column in all
case"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-2031) some more DimensionEncoding
hongbin ma created KYLIN-2031: - Summary: some more DimensionEncoding Key: KYLIN-2031 URL: https://issues.apache.org/jira/browse/KYLIN-2031 Project: Kylin Issue Type: New Feature Reporter: hongbin ma Assignee: hongbin ma 1. for some use cases string value represent hash code is used. The string only consist of [0~9A~F] (hex values), so two characters can be squashed into one byte 2. The current IntegerDimEnc does not support negative values, need another IntegerDimEnc that supports negative values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2030) lookup table support group by primary key when no derived dimension
Zhong,Jason created KYLIN-2030: -- Summary: lookup table support group by primary key when no derived dimension Key: KYLIN-2030 URL: https://issues.apache.org/jira/browse/KYLIN-2030 Project: Kylin Issue Type: Improvement Components: Query Engine Affects Versions: v1.5.4 Reporter: Zhong,Jason Assignee: hongbin ma when we have derived dimension on lookup table, we can write sql like 'select pk from lookupTable group by pk', if no derived column on this lookup table. group by pk is not supported.need to add this feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2022) Kylin support change HBASE_HOME at runtime.
[ https://issues.apache.org/jira/browse/KYLIN-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guohui LI updated KYLIN-2022: - Fix Version/s: (was: v1.6.0) (was: Future) > Kylin support change HBASE_HOME at runtime. > --- > > Key: KYLIN-2022 > URL: https://issues.apache.org/jira/browse/KYLIN-2022 > Project: Kylin > Issue Type: New Feature >Reporter: Guohui LI > > In multi-tenancy system. each tenant can create one or more hbase cluster. > Different user may use different HBASE_HOME when using Kylin. So it's better > for user to point out HBASE_HOME when operating Cube or query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2022) Kylin support change HBASE_HOME at runtime.
[ https://issues.apache.org/jira/browse/KYLIN-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guohui LI updated KYLIN-2022: - Fix Version/s: Future > Kylin support change HBASE_HOME at runtime. > --- > > Key: KYLIN-2022 > URL: https://issues.apache.org/jira/browse/KYLIN-2022 > Project: Kylin > Issue Type: New Feature >Reporter: Guohui LI > > In multi-tenancy system. each tenant can create one or more hbase cluster. > Different user may use different HBASE_HOME when using Kylin. So it's better > for user to point out HBASE_HOME when operating Cube or query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2029) lookup table support count(distinct column)
Zhong,Jason created KYLIN-2029: -- Summary: lookup table support count(distinct column) Key: KYLIN-2029 URL: https://issues.apache.org/jira/browse/KYLIN-2029 Project: Kylin Issue Type: Improvement Components: Query Engine Affects Versions: v1.5.4 Reporter: Zhong,Jason Assignee: hongbin ma Now for dimension column on fact table, we can write sql like 'select count(distinct columnName)...', but on lookup table, it's not supported. need to add this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-2028) find-*-dependency script fail on Mac OS
[ https://issues.apache.org/jira/browse/KYLIN-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi reassigned KYLIN-2028: Assignee: Roger Shi > find-*-dependency script fail on Mac OS > --- > > Key: KYLIN-2028 > URL: https://issues.apache.org/jira/browse/KYLIN-2028 > Project: Kylin > Issue Type: Bug >Reporter: Roger Shi >Assignee: Roger Shi >Priority: Minor > > command "cut" doesn't support "--output-delimiter" on Mac -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2028) find-*-dependency script fail on Mac OS
Roger Shi created KYLIN-2028: Summary: find-*-dependency script fail on Mac OS Key: KYLIN-2028 URL: https://issues.apache.org/jira/browse/KYLIN-2028 Project: Kylin Issue Type: Bug Reporter: Roger Shi Priority: Minor command "cut" doesn't support "--output-delimiter" on Mac -- This message was sent by Atlassian JIRA (v6.3.4#6332)
