[jira] [Created] (KYLIN-4236) Can kylin support more than 64 dimensions?
zhao jintao created KYLIN-4236: -- Summary: Can kylin support more than 64 dimensions? Key: KYLIN-4236 URL: https://issues.apache.org/jira/browse/KYLIN-4236 Project: Kylin Issue Type: Wish Affects Versions: v2.6.4 Reporter: zhao jintao As we know: The default maximum size of kylin dimension is 63, which is specified by the kylin.cube.rowkey.max-size value in the configuration file. The default maximum cubic combination of kylin cuboids is 32768, which is specified by the kylin.cube.aggrgroup.max-combination value in the configuration file; We can use many various methods to reduce cuboid's combination In our system, we used more than 80 dimensions, the size combination is just 1000 combinations, is very fixed. After setting kylin.cube.rowkey.max-size to 128, if I save a cube which has more than 64 dimensions, kylin throws exception: "Cuboid -1 out of range 0-9223372036854775806". After analysis, the cuboid in kylin is identified by the N-bit binary number, and the cuboid is saved with long type. If a cube has more than 64 dimensions, the cuboid value exceeds the maximum value of the long type and the error is reported. Can kylin support this scene which has a limited combination of category dimensions (less than 1000) but has large number of dimensions (more than 80)? Can kylin save the cuboid value using BigInteger type? What risk if kylin save cuboid uses BigInteger type? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KYLIN-2978) add SQL's execution plan feature
[ https://issues.apache.org/jira/browse/KYLIN-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang reassigned KYLIN-2978: --- Assignee: Yaqian Zhang > add SQL's execution plan feature > > > Key: KYLIN-2978 > URL: https://issues.apache.org/jira/browse/KYLIN-2978 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v2.1.0 >Reporter: cnwangdp >Assignee: Yaqian Zhang >Priority: Minor > Labels: features > Fix For: Future > > > SQL's execution plan is import to performance analysis; -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] nichunen merged pull request #916: make sure calcite package is updated automatically
nichunen merged pull request #916: make sure calcite package is updated automatically URL: https://github.com/apache/kylin/pull/916 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4188) Parquet as Cube storage V2
[ https://issues.apache.org/jira/browse/KYLIN-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965647#comment-16965647 ] Jiatao Tao commented on KYLIN-4188: --- +1 > Parquet as Cube storage V2 > -- > > Key: KYLIN-4188 > URL: https://issues.apache.org/jira/browse/KYLIN-4188 > Project: Kylin > Issue Type: New Feature >Reporter: nichunen >Assignee: yiming.xu >Priority: Major > Fix For: Future > > > Branch "kylin-on-parquet" has implemented and it's ready for public review > and evaluation, the issue is https://issues.apache.org/jira/browse/KYLIN-3621. > However, as the discussion in > https://sematext.com/opensee/m/Kylin/8WImho86LC1RxadI?subj=Evaluate+Kylin+on+Parquet, > to make it more stable and with higher performance, some tasks of > improvement should be done. > The key tasks may be: > 1. Calculation of queries on Kylin should be all distributed > 2. Remove dictionaries for dimensions > 3. Store value with its data type(not binary) > 4. The new build engine with Spark-SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4188) Parquet as Cube storage V2
[ https://issues.apache.org/jira/browse/KYLIN-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiming.xu updated KYLIN-4188: - Component/s: (was: Storage - Parquet) > Parquet as Cube storage V2 > -- > > Key: KYLIN-4188 > URL: https://issues.apache.org/jira/browse/KYLIN-4188 > Project: Kylin > Issue Type: New Feature >Reporter: nichunen >Assignee: yiming.xu >Priority: Major > Fix For: Future > > > Branch "kylin-on-parquet" has implemented and it's ready for public review > and evaluation, the issue is https://issues.apache.org/jira/browse/KYLIN-3621. > However, as the discussion in > https://sematext.com/opensee/m/Kylin/8WImho86LC1RxadI?subj=Evaluate+Kylin+on+Parquet, > to make it more stable and with higher performance, some tasks of > improvement should be done. > The key tasks may be: > 1. Calculation of queries on Kylin should be all distributed > 2. Remove dictionaries for dimensions > 3. Store value with its data type(not binary) > 4. The new build engine with Spark-SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4145) compile failed due to incompatible version between scala and scala-maven-plugin
[
https://issues.apache.org/jira/browse/KYLIN-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng resolved KYLIN-4145.
--
Resolution: Won't Fix
> compile failed due to incompatible version between scala and
> scala-maven-plugin
> ---
>
> Key: KYLIN-4145
> URL: https://issues.apache.org/jira/browse/KYLIN-4145
> Project: Kylin
> Issue Type: Bug
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
>
> The version of scala-maven-plugin is not specified, and the latest
> version(4.1.1) will be used when compiling the project, which is incompatible
> with the version of scala(2.11.0).
> So, it would be better to specify the version number of scala-maven-plugin.
> {code:xml}
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:4.1.1:compile (scala-compile-first) on
> project kylin-engine-spark: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:4.1.1:compile failed: An API
> incompatibility was encountered while executing
> net.alchim31.maven:scala-maven-plugin:4.1.1:compile:
> java.lang.NoSuchMethodError:
> xsbt.ZincCompiler.classPath()Lscala/tools/nsc/util/ClassFileLookup;
> [ERROR] -
> [ERROR] realm =plugin>net.alchim31.maven:scala-maven-plugin:4.1.1
> [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
> [ERROR] urls[0] =
> file:/Users/cheng/.m2/repository/net/alchim31/maven/scala-maven-plugin/4.1.1/scala-maven-plugin-4.1.1.jar
> [ERROR] urls[1] =
> file:/Users/cheng/.m2/repository/org/apache/maven/maven-builder-support/3.3.9/maven-builder-support-3.3.9.jar
> [ERROR] urls[2] =
> file:/Users/cheng/.m2/repository/com/google/guava/guava/18.0/guava-18.0.jar
> [ERROR] urls[3] =
> file:/Users/cheng/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.jar
> [ERROR] urls[4] =
> file:/Users/cheng/.m2/repository/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar
> [ERROR] urls[5] =
> file:/Users/cheng/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2.jar
> [ERROR] urls[6] =
> file:/Users/cheng/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.6/plexus-component-annotations-1.6.jar
> [ERROR] urls[7] =
> file:/Users/cheng/.m2/repository/org/apache/maven/reporting/maven-reporting-api/3.0/maven-reporting-api-3.0.jar
> [ERROR] urls[8] =
> file:/Users/cheng/.m2/repository/org/eclipse/aether/aether-util/1.0.2.v20150114/aether-util-1.0.2.v20150114.jar
> [ERROR] urls[9] =
> file:/Users/cheng/.m2/repository/com/google/inject/guice/4.0/guice-4.0-no_aop.jar
> [ERROR] urls[10] =
> file:/Users/cheng/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar
> [ERROR] urls[11] =
> file:/Users/cheng/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar
> [ERROR] urls[12] =
> file:/Users/cheng/.m2/repository/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.jar
> [ERROR] urls[13] =
> file:/Users/cheng/.m2/repository/org/apache/commons/commons-lang3/3.4/commons-lang3-3.4.jar
> [ERROR] urls[14] =
> file:/Users/cheng/.m2/repository/org/apache/maven/shared/maven-dependency-tree/3.0.1/maven-dependency-tree-3.0.1.jar
> [ERROR] urls[15] =
> file:/Users/cheng/.m2/repository/org/apache/commons/commons-exec/1.3/commons-exec-1.3.jar
> [ERROR] urls[16] =
> file:/Users/cheng/.m2/repository/org/codehaus/plexus/plexus-utils/3.2.1/plexus-utils-3.2.1.jar
> [ERROR] urls[17] =
> file:/Users/cheng/.m2/repository/org/codehaus/plexus/plexus-archiver/4.1.0/plexus-archiver-4.1.0.jar
> [ERROR] urls[18] =
> file:/Users/cheng/.m2/repository/org/codehaus/plexus/plexus-io/3.1.1/plexus-io-3.1.1.jar
> [ERROR] urls[19] =
> file:/Users/cheng/.m2/repository/org/apache/commons/commons-compress/1.18/commons-compress-1.18.jar
> [ERROR] urls[20] =
> file:/Users/cheng/.m2/repository/org/iq80/snappy/snappy/0.4/snappy-0.4.jar
> [ERROR] urls[21] =
> file:/Users/cheng/.m2/repository/org/tukaani/xz/1.8/xz-1.8.jar
> [ERROR] urls[22] =
> file:/Users/cheng/.m2/repository/backport-util-concurrent/backport-util-concurrent/3.1/backport-util-concurrent-3.1.jar
> [ERROR] urls[23] =
> file:/Users/cheng/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
> [ERROR] urls[24] =
> file:/Users/cheng/.m2/repository/org/apache/maven/maven-archiver/3.4.0/maven-archiver-3.4.0.jar
> [ERROR] urls[25] =
> file:/Users/cheng/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.2.1/maven-shared-utils-3.2.1.jar
> [ERROR] urls[26] =
> file:/Users/cheng/.m2/repository/commons-io/commons-io/2.5/commons-io-2.5.jar
> [ERROR] urls[27] =
> file:/Users/cheng/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.9/doxia-sink-api-1.9.jar
> [ERROR] urls[28] =
