[jira] [Resolved] (KYLIN-2147) Move the creation of HTable after cube be built
[ https://issues.apache.org/jira/browse/KYLIN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-2147. - Resolution: Won't Fix There is dependency between this step and cube build; changing the sequence will cause error in building and query. Revert the change and mark this as "won't fix". > Move the creation of HTable after cube be built > --- > > Key: KYLIN-2147 > URL: https://issues.apache.org/jira/browse/KYLIN-2147 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > Fix For: v1.6.0 > > > In 1.5.x, the creation of HBase table is just after creating dictionaries and > before cube building; That is unnecessary as cube building doesn't operate > the table; It can be deferred to after the cube be built and before > converting to HFile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-2147) Move the creation of HTable after cube be built
[ https://issues.apache.org/jira/browse/KYLIN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-2147. --- Resolution: Won't Fix > Move the creation of HTable after cube be built > --- > > Key: KYLIN-2147 > URL: https://issues.apache.org/jira/browse/KYLIN-2147 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > > In 1.5.x, the creation of HBase table is just after creating dictionaries and > before cube building; That is unnecessary as cube building doesn't operate > the table; It can be deferred to after the cube be built and before > converting to HFile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-2147) Move the creation of HTable after cube be built
[ https://issues.apache.org/jira/browse/KYLIN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI closed KYLIN-2147. --- > Move the creation of HTable after cube be built > --- > > Key: KYLIN-2147 > URL: https://issues.apache.org/jira/browse/KYLIN-2147 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > > In 1.5.x, the creation of HBase table is just after creating dictionaries and > before cube building; That is unnecessary as cube building doesn't operate > the table; It can be deferred to after the cube be built and before > converting to HFile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KYLIN-2147) Move the creation of HTable after cube be built
[ https://issues.apache.org/jira/browse/KYLIN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reopened KYLIN-2147: - > Move the creation of HTable after cube be built > --- > > Key: KYLIN-2147 > URL: https://issues.apache.org/jira/browse/KYLIN-2147 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > Fix For: v1.6.0 > > > In 1.5.x, the creation of HBase table is just after creating dictionaries and > before cube building; That is unnecessary as cube building doesn't operate > the table; It can be deferred to after the cube be built and before > converting to HFile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2030) lookup table support group by primary key when no derived dimension
[ https://issues.apache.org/jira/browse/KYLIN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2030: -- Fix Version/s: v1.6.1 > lookup table support group by primary key when no derived dimension > --- > > Key: KYLIN-2030 > URL: https://issues.apache.org/jira/browse/KYLIN-2030 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v1.5.4 >Reporter: Zhong,Jason >Assignee: hongbin ma > Fix For: v1.6.1 > > > when we have derived dimension on lookup table, we can write sql like 'select > pk from lookupTable group by pk', if no derived column on this lookup table. > group by pk is not supported.need to add this feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2155) get-properties.sh doesn't support parameters starting with "-n"
[
https://issues.apache.org/jira/browse/KYLIN-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma updated KYLIN-2155:
--
Summary: get-properties.sh doesn't support parameters starting with "-n"
(was: get-properties.sh doesn't support parameters starting with hyphen)
> get-properties.sh doesn't support parameters starting with "-n"
> ---
>
> Key: KYLIN-2155
> URL: https://issues.apache.org/jira/browse/KYLIN-2155
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: Future, v1.6.1
>Reporter: Ricardo Jorge de Sousa Teixeira
>Assignee: hongbin ma
> Attachments: fix.patch
>
>
> The last echo in get-properties.sh silently ignores strings with {{-n}}.
> It should be double quoted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2155) get-properties.sh doesn't support parameters starting with hyphen
[
https://issues.apache.org/jira/browse/KYLIN-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631278#comment-15631278
]
hongbin ma commented on KYLIN-2155:
---
got it. then the patch will be suffice to deal with most cases.
I'm merging your patch to master
> get-properties.sh doesn't support parameters starting with hyphen
> -
>
> Key: KYLIN-2155
> URL: https://issues.apache.org/jira/browse/KYLIN-2155
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: Future, v1.6.1
>Reporter: Ricardo Jorge de Sousa Teixeira
>Assignee: hongbin ma
> Attachments: fix.patch
>
>
> The last echo in get-properties.sh silently ignores strings with {{-n}}.
> It should be double quoted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline
[
https://issues.apache.org/jira/browse/KYLIN-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631206#comment-15631206
]
Shaofeng SHI commented on KYLIN-1797:
-
You got it; User can modify the find-*-dependency.sh manually, or copy the
missing jars to $KYLIN_HOME/lib/.
> bin/find-hive-dependency.sh may not get correct local classpath when
> configured hive client is beeline
> --
>
> Key: KYLIN-1797
> URL: https://issues.apache.org/jira/browse/KYLIN-1797
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: v1.5.0, v1.3.0
>Reporter: Shaofeng SHI
> Labels: newbie
>
> in bin/find-hive-dependency.sh, if configured hive client is beeline, it will
> use beeline to execute "set" and then to find the classpath; I noticed the
> result doesn't have "hive-cli.jar", this will cause error in "load hive
> table" action, a NoClassDefFoundError error is thrown;
> If "hive -e set" will return all the needed hive jars.
> {code}
> SEVERE: Servlet.service() for servlet [kylin] in context with path [/kylin]
> threw exception [Handler processing failed; nested exception is
> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/cli/CliSessionState]
> with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.cli.CliSessionState
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
> at
> org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2155) get-properties.sh doesn't support parameters starting with hyphen
[
https://issues.apache.org/jira/browse/KYLIN-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631085#comment-15631085
]
Ricardo Jorge de Sousa Teixeira commented on KYLIN-2155:
For me it didn't work because I was using it to pass a username with {{-n
myuser}} to beeline.
I think it is counterintuitive that we have to be worried about the string
we put. Double quoting the string passed to {{echo}} is the normal approach.
Regards.
> get-properties.sh doesn't support parameters starting with hyphen
> -
>
> Key: KYLIN-2155
> URL: https://issues.apache.org/jira/browse/KYLIN-2155
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: Future, v1.6.1
>Reporter: Ricardo Jorge de Sousa Teixeira
>Assignee: hongbin ma
> Attachments: fix.patch
>
>
> The last echo in get-properties.sh silently ignores strings with {{-n}}.
> It should be double quoted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline
[
https://issues.apache.org/jira/browse/KYLIN-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630580#comment-15630580
]
Ricardo Jorge de Sousa Teixeira commented on KYLIN-1797:
As a quick fix I've changed {{bin/find-hive-dependency.sh}} to include it
{code}
hive_dependency=${hive_conf_path}:${hive_lib}:${hcatalog}:"/opt/cloudera/parcels/CDH/lib/hive/lib/hive-cli.jar"
{code}
> bin/find-hive-dependency.sh may not get correct local classpath when
> configured hive client is beeline
> --
>
> Key: KYLIN-1797
> URL: https://issues.apache.org/jira/browse/KYLIN-1797
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: v1.5.0, v1.3.0
>Reporter: Shaofeng SHI
> Labels: newbie
>
> in bin/find-hive-dependency.sh, if configured hive client is beeline, it will
> use beeline to execute "set" and then to find the classpath; I noticed the
> result doesn't have "hive-cli.jar", this will cause error in "load hive
> table" action, a NoClassDefFoundError error is thrown;
> If "hive -e set" will return all the needed hive jars.
> {code}
> SEVERE: Servlet.service() for servlet [kylin] in context with path [/kylin]
> threw exception [Handler processing failed; nested exception is
> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/cli/CliSessionState]
> with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.cli.CliSessionState
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
> at
> org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline
[
https://issues.apache.org/jira/browse/KYLIN-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630554#comment-15630554
]
Ricardo Jorge de Sousa Teixeira commented on KYLIN-1797:
Was this fix in anyway? I'm having the same problem.
> bin/find-hive-dependency.sh may not get correct local classpath when
> configured hive client is beeline
> --
>
> Key: KYLIN-1797
> URL: https://issues.apache.org/jira/browse/KYLIN-1797
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: v1.5.0, v1.3.0
>Reporter: Shaofeng SHI
> Labels: newbie
>
> in bin/find-hive-dependency.sh, if configured hive client is beeline, it will
> use beeline to execute "set" and then to find the classpath; I noticed the
> result doesn't have "hive-cli.jar", this will cause error in "load hive
> table" action, a NoClassDefFoundError error is thrown;
> If "hive -e set" will return all the needed hive jars.
> {code}
> SEVERE: Servlet.service() for servlet [kylin] in context with path [/kylin]
> threw exception [Handler processing failed; nested exception is
> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/cli/CliSessionState]
> with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.cli.CliSessionState
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
> at
> org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline
[
https://issues.apache.org/jira/browse/KYLIN-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630554#comment-15630554
]
Ricardo Jorge de Sousa Teixeira edited comment on KYLIN-1797 at 11/2/16 9:14 PM:
-
Was this fix in anyway or is there a workaround? I'm having the same problem.
was (Author: rjsteixeira):
Was this fix in anyway? I'm having the same problem.
> bin/find-hive-dependency.sh may not get correct local classpath when
> configured hive client is beeline
> --
>
> Key: KYLIN-1797
> URL: https://issues.apache.org/jira/browse/KYLIN-1797
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: v1.5.0, v1.3.0
>Reporter: Shaofeng SHI
> Labels: newbie
>
> in bin/find-hive-dependency.sh, if configured hive client is beeline, it will
> use beeline to execute "set" and then to find the classpath; I noticed the
> result doesn't have "hive-cli.jar", this will cause error in "load hive
> table" action, a NoClassDefFoundError error is thrown;
> If "hive -e set" will return all the needed hive jars.
> {code}
> SEVERE: Servlet.service() for servlet [kylin] in context with path [/kylin]
> threw exception [Handler processing failed; nested exception is
> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/cli/CliSessionState]
> with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.cli.CliSessionState
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
> at
> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
> at
> org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-2155) get-properties.sh doesn't support parameters starting with hyphen
[
https://issues.apache.org/jira/browse/KYLIN-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ricardo Jorge de Sousa Teixeira updated KYLIN-2155:
---
Description:
The last echo in get-properties.sh silently ignores strings with {{-n}}.
It should be double quoted.
was:
The last echo in get-properties.sh silently ignores strings with `-n`.
It should be double quoted.
> get-properties.sh doesn't support parameters starting with hyphen
> -
>
> Key: KYLIN-2155
> URL: https://issues.apache.org/jira/browse/KYLIN-2155
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: Future, v1.6.1
>Reporter: Ricardo Jorge de Sousa Teixeira
>Assignee: hongbin ma
> Attachments: fix.patch
>
>
> The last echo in get-properties.sh silently ignores strings with {{-n}}.
> It should be double quoted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-2155) get-properties.sh doesn't support parameters starting with hyphen
[
https://issues.apache.org/jira/browse/KYLIN-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ricardo Jorge de Sousa Teixeira updated KYLIN-2155:
---
Attachment: fix.patch
> get-properties.sh doesn't support parameters starting with hyphen
> -
>
> Key: KYLIN-2155
> URL: https://issues.apache.org/jira/browse/KYLIN-2155
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: Future, v1.6.1
>Reporter: Ricardo Jorge de Sousa Teixeira
>Assignee: hongbin ma
> Attachments: fix.patch
>
>
> The last echo in get-properties.sh silently ignores {-n}.
> It should be double quoted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-2154) source table rows will be skipped if TOPN's group column contains NULL values
hongbin ma created KYLIN-2154: - Summary: source table rows will be skipped if TOPN's group column contains NULL values Key: KYLIN-2154 URL: https://issues.apache.org/jira/browse/KYLIN-2154 Project: Kylin Issue Type: Bug Reporter: hongbin ma Assignee: hongbin ma topn: group by country and order by sum(price) if the country column contains NULL values: 2016-11-02 20:23:22,478 ERROR [main] org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Insane record: [20111209, \N, 1000122, 101, 9 java.lang.NullPointerException at org.apache.kylin.common.util.Bytes.toBytes(Bytes.java:351) at org.apache.kylin.measure.topn.TopNMeasureType$1.valueOf(TopNMeasureType.java:145) at org.apache.kylin.measure.topn.TopNMeasureType$1.valueOf(TopNMeasureType.java:116) at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValueOf(BaseCuboidMapperBase.java:186) at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValue(BaseCuboidMapperBase.java:157) at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMapperBase.java:203) at org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboidMapper.java:52) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2153) Allow user to skip the check in CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen updated KYLIN-2153:
--
Attachment: KYLIN-2153.patch
This is the patch.
> Allow user to skip the check in CubeMetaIngester
>
>
> Key: KYLIN-2153
> URL: https://issues.apache.org/jira/browse/KYLIN-2153
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
>Priority: Minor
> Attachments: KYLIN-2153.patch
>
>
> when the model has multiple cubes or the user want to overwrite the model or
> cube indeed, we should allow user to skip the check in {{CubeMetaIngester}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-2153) Allow user to skip the check in CubeMetaIngester
kangkaisen created KYLIN-2153:
-
Summary: Allow user to skip the check in CubeMetaIngester
Key: KYLIN-2153
URL: https://issues.apache.org/jira/browse/KYLIN-2153
Project: Kylin
Issue Type: Bug
Components: Tools, Build and Test
Affects Versions: v1.5.4.1
Reporter: kangkaisen
Assignee: kangkaisen
Priority: Minor
when the model has multiple cubes or the user want to overwrite the model or
cube indeed, we should allow user to skip the check in {{CubeMetaIngester}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (KYLIN-2141) Add include/exclude interface for ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang resolved KYLIN-2141. --- Resolution: Fixed Fix Version/s: v1.6.1 > Add include/exclude interface for ResourceTool > --- > > Key: KYLIN-2141 > URL: https://issues.apache.org/jira/browse/KYLIN-2141 > Project: Kylin > Issue Type: Improvement >Reporter: Billy(Yiming) Liu >Assignee: Billy(Yiming) Liu >Priority: Minor > Fix For: v1.6.1 > > Attachments: KYLIN-2141.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2135) Enlarge FactDistinctColumns reducer number
[
https://issues.apache.org/jira/browse/KYLIN-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen updated KYLIN-2135:
--
Attachment: KYLIN-2135.patch
minor update.
> Enlarge FactDistinctColumns reducer number
> --
>
> Key: KYLIN-2135
> URL: https://issues.apache.org/jira/browse/KYLIN-2135
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2135.patch, new.png, old.png
>
>
> When the hive table has billions of rows and use global dictionary for
> precise count distinct measures, the {{Extract Fact Table Distinct Columns}}
> job will run o long time.
> So we could use more reducer to deal with the one column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-2135) Enlarge FactDistinctColumns reducer number
[
https://issues.apache.org/jira/browse/KYLIN-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen updated KYLIN-2135:
--
Attachment: (was: KYLIN-2135.patch)
> Enlarge FactDistinctColumns reducer number
> --
>
> Key: KYLIN-2135
> URL: https://issues.apache.org/jira/browse/KYLIN-2135
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2135.patch, new.png, old.png
>
>
> When the hive table has billions of rows and use global dictionary for
> precise count distinct measures, the {{Extract Fact Table Distinct Columns}}
> job will run o long time.
> So we could use more reducer to deal with the one column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-2135) Enlarge FactDistinctColumns reducer number
[
https://issues.apache.org/jira/browse/KYLIN-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen updated KYLIN-2135:
--
Attachment: (was: KYLIN-2135.patch)
> Enlarge FactDistinctColumns reducer number
> --
>
> Key: KYLIN-2135
> URL: https://issues.apache.org/jira/browse/KYLIN-2135
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2135.patch, new.png, old.png
>
>
> When the hive table has billions of rows and use global dictionary for
> precise count distinct measures, the {{Extract Fact Table Distinct Columns}}
> job will run o long time.
> So we could use more reducer to deal with the one column.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2141) Add include/exclude interface for ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629071#comment-15629071 ] liyang commented on KYLIN-2141: --- Thanks Yiming. Merged. > Add include/exclude interface for ResourceTool > --- > > Key: KYLIN-2141 > URL: https://issues.apache.org/jira/browse/KYLIN-2141 > Project: Kylin > Issue Type: Improvement >Reporter: Billy(Yiming) Liu >Assignee: Billy(Yiming) Liu >Priority: Minor > Attachments: KYLIN-2141.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2132) Unable to pull data from Kylin Cube ( learn_kylin cube ) to Excel or Power BI for Visualization and some dimensions are not showing up.
[ https://issues.apache.org/jira/browse/KYLIN-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15628876#comment-15628876 ] liyang commented on KYLIN-2132: --- Thanks for reporting! > Unable to pull data from Kylin Cube ( learn_kylin cube ) to Excel or Power BI > for Visualization and some dimensions are not showing up. > --- > > Key: KYLIN-2132 > URL: https://issues.apache.org/jira/browse/KYLIN-2132 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Affects Versions: v1.5.4.1 > Environment: HDP 2.4, Kylin: 1.5.4.1 >Reporter: Bhanuprakash > > Hi all, > I am unable to load data from KYLIN CUbe to Excel / power BI, > i have installed all required drivers ( Microsoft Visual C++ 2012 > Redistributable & KYLIN ODBC drivers ). i am able to connect successfully to > KYLIN CUBE, but the data is not loading to excel / power BI because of below > issue and also some dimensions are not showing up in Excel. > Error Message: > Value was either too large or too small for an Int32. > Stack Trace: > Microsoft.Mashup.Evaluator.Interface.ErrorException: Value was either too > large or too small for an Int32. ---> > Microsoft.Mashup.Evaluator.Interface.ErrorException: Value was either too > large or too small for an Int32. ---> > Microsoft.Mashup.Evaluator.Interface.ErrorException: Value was either too > large or too small for an Int32. ---> > Microsoft.Mashup.Evaluator.Interface.ErrorException: Value was either too > large or too small for an Int32. ---> > Microsoft.Mashup.Evaluator.Interface.ErrorException: Value was either too > large or too small for an Int32. ---> System.OverflowException: Value was > either too large or too small for an Int32. ---> System.OverflowException: > Value was either too large or too small for an Int32. >at System.Convert.ToInt32(Int64 value) >at System.Int64.System.IConvertible.ToInt32(IFormatProvider provider) >at System.Convert.ToInt32(Object value, IFormatProvider provider) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcColumnInfoCollection.b__0(IOdbcConnection > connection) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcDataSource.<>c__DisplayClassf.b__e(IOdbcConnection > connection) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcDataSource.<>c__DisplayClassc`1.b__b() >at > Microsoft.Mashup.Engine1.Library.Common.DbExceptionHandler.RunWithRetryGuard[T](Int32 > maxRetryAttempts, Func`1 action) >at > Microsoft.Mashup.Engine1.Library.Common.DbExceptionHandler.Invoke[T](Int32 > maxRetry, Func`1 action) >at Microsoft.Mashup.Engine1.Library.Odbc.OdbcDataSource.Connect[T](String > currentCatalog, Func`2 func) >at Microsoft.Mashup.Engine1.Library.Odbc.OdbcDataSource.Connect(Action`1 > action) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcColumnInfoCollection.EnsureInitialized() >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcQueryDomain.NewQuery(OdbcTableInfo > table, List`1 fromItems) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcTableValue.OdbcTableValueBuilder.CreateTableValue(OdbcTableInfo > tableInfo) >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcHierarchicalNavigationTableValue.<>c__DisplayClass9.b__8() >at Microsoft.Mashup.Engine1.Runtime.DelayedValue.get_Value() >at > Microsoft.Mashup.Engine1.Library.Odbc.OdbcHierarchicalNavigationTableValue.OdbcTableLevelNavigationTableValue.<>c__DisplayClass1f.b__1d(Int32 > i) >at > Microsoft.Mashup.Engine1.Runtime.RecordValue.DemandRecordValue.get_Item(Int32 > index) >at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field) >at Microsoft.Mashup.Engine1.Language.FieldAccessExpression.Execute(Value > frame) >at Microsoft.Mashup.Engine1.Language.DebugExpression.Execute(Value frame) >at > Microsoft.Mashup.Engine1.Language.FunctionInvocationExpression2.Execute(Value > frame) >at Microsoft.Mashup.Engine1.Language.DebugExpression.Execute(Value frame) >at Microsoft.Mashup.Engine1.Language.RuntimeFunctionValue1.Invoke(Value > arg0) >at > Microsoft.Mashup.Engine1.Language.RecordExpression.RuntimeRecordValue.Force(Int32 > index) >at > Microsoft.Mashup.Engine1.Language.RecordExpression.RuntimeRecordValue.get_Item(Int32 > index) >at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field) >at Microsoft.Mashup.Engine1.Language.FieldAccessExpression.Execute(Value > frame) >at Microsoft.Mashup.Engine1.Language.RuntimeFunctionValue1.Invoke(Value > arg0) >at > Microsoft.Mashup.Engine1.Language.RecordExpression.RuntimeRecordValue.Force(Int32 > index) >at > Microsoft.Mashup.Engine1.Language.RecordExpression.RuntimeRecordValue.get_Item(Int32 > index) >at
[jira] [Commented] (KYLIN-2130) QueryMetrics concurrent bug fix
[ https://issues.apache.org/jira/browse/KYLIN-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15628842#comment-15628842 ] liyang commented on KYLIN-2130: --- +1 > QueryMetrics concurrent bug fix > --- > > Key: KYLIN-2130 > URL: https://issues.apache.org/jira/browse/KYLIN-2130 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.5.4, v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Attachments: KYLIN-2130.patch > > > Recently，I made a concurrent kylin query test and found a little bug in > QueryMetrics: > If the initial query to a cube or a project is concurrent, the QueryMetric > will register failed and throw a MetricsException. > The exception is like this: > "exception":"Metrics source kylin_test,sub=xxx already exists!" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2152) TopN group by column does not distinguish between NULL and ""
hongbin ma created KYLIN-2152: - Summary: TopN group by column does not distinguish between NULL and "" Key: KYLIN-2152 URL: https://issues.apache.org/jira/browse/KYLIN-2152 Project: Kylin Issue Type: Bug Reporter: hongbin ma Assignee: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2151) Use standard way to get Hive table signature
[ https://issues.apache.org/jira/browse/KYLIN-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2151: Assignee: (was: Dong Li) Labels: newbie (was: ) > Use standard way to get Hive table signature > > > Key: KYLIN-2151 > URL: https://issues.apache.org/jira/browse/KYLIN-2151 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI > Labels: newbie > > In HiveTable.getSignature(), kylin tries to read the HDFS info (size & last > modified time) of the table; This may fail when the Hive table is secured. > A better way is using some standard API or cli to get the info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2151) Use standard way to get Hive table signature
Shaofeng SHI created KYLIN-2151: --- Summary: Use standard way to get Hive table signature Key: KYLIN-2151 URL: https://issues.apache.org/jira/browse/KYLIN-2151 Project: Kylin Issue Type: Improvement Components: Job Engine Reporter: Shaofeng SHI Assignee: Dong Li In HiveTable.getSignature(), kylin tries to read the HDFS info (size & last modified time) of the table; This may fail when the Hive table is secured. A better way is using some standard API or cli to get the info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2150) detailed subquery + outer query filter will fail
[ https://issues.apache.org/jira/browse/KYLIN-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2150: -- Description: select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t where x = '1000121' group by x,y will succeed, select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from fact01_v ) t group by x,y will succeed too. However select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v ) t where x = '1000121' group by x,y will fail. error log is: error while executing SQL "select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v ) t where x = '1000121' group by x,y LIMIT 5": Can't find any realization. Please confirm with providers. SQL digest: fact table TEST_1031.FACT01_V,group by [TEST_1031.FACT01_V.SELLER_ID, TEST_1031.FACT01_V.TRANS_DATE_2],filter on [DEFAULT._KYLIN_TABLE.X],with aggregates[]. was: select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t where x = '1000121' group by x,y will succeed, select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from fact01_v ) t group by x,y will succeed too. However select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v ) t where x = '1000121' group by x,y will fail > detailed subquery + outer query filter will fail > > > Key: KYLIN-2150 > URL: https://issues.apache.org/jira/browse/KYLIN-2150 > Project: Kylin > Issue Type: Bug >Reporter: hongbin ma >Assignee: liyang > > select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y > from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t > where x = '1000121' group by x,y > will succeed, > select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from > fact01_v ) t group by x,y > will succeed too. However > select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y > from fact01_v ) t where x = '1000121' group by x,y > will fail. > error log is: > error while executing SQL "select x,y from (select cast(seller_id as > varchar(10)) x, trans_date_2 y from fact01_v ) t where x = '1000121' group by > x,y LIMIT 5": Can't find any realization. Please confirm with providers. > SQL digest: fact table TEST_1031.FACT01_V,group by > [TEST_1031.FACT01_V.SELLER_ID, TEST_1031.FACT01_V.TRANS_DATE_2],filter on > [DEFAULT._KYLIN_TABLE.X],with aggregates[]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2150) detailed subquery + outer query filter will fail
[ https://issues.apache.org/jira/browse/KYLIN-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2150: -- Description: select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t where x = '1000121' group by x,y will succeed, select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from fact01_v ) t group by x,y will succeed too. However select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v ) t where x = '1000121' group by x,y will fail was: select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t where x = '1000121' group by x,y will succeed, select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from fact01_v ) t group by x,y will succeed too. However select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group ) t where x = '1000121' group by x,y will fail > detailed subquery + outer query filter will fail > > > Key: KYLIN-2150 > URL: https://issues.apache.org/jira/browse/KYLIN-2150 > Project: Kylin > Issue Type: Bug >Reporter: hongbin ma >Assignee: liyang > > select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y > from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t > where x = '1000121' group by x,y > will succeed, > select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from > fact01_v ) t group by x,y > will succeed too. However > select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y > from fact01_v ) t where x = '1000121' group by x,y > will fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2150) detailed subquery + outer query filter will fail
hongbin ma created KYLIN-2150: - Summary: detailed subquery + outer query filter will fail Key: KYLIN-2150 URL: https://issues.apache.org/jira/browse/KYLIN-2150 Project: Kylin Issue Type: Bug Reporter: hongbin ma Assignee: liyang select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group by cast(seller_id as varchar(10)) , trans_date_2) t where x = '1000121' group by x,y will succeed, select x, y from (select cast(seller_id as varchar) x, trans_date_2 y from fact01_v ) t group by x,y will succeed too. However select x,y from (select cast(seller_id as varchar(10)) x, trans_date_2 y from fact01_v group ) t where x = '1000121' group by x,y will fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2149) support both joint and hierarchy
Zhong,Jason created KYLIN-2149: -- Summary: support both joint and hierarchy Key: KYLIN-2149 URL: https://issues.apache.org/jira/browse/KYLIN-2149 Project: Kylin Issue Type: Improvement Components: Metadata Affects Versions: v1.5.4.1 Reporter: Zhong,Jason Assignee: hongbin ma Say we have year, month,week,year_desc,month_desc,week_desc if we put year,month,week in hierarchy and build joint dimensions [year,year_desc] [month,month_desc] [week,week_desc] it's not supported now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2146) "Streaming Cluster" page should remove "Margin" inputbox
[ https://issues.apache.org/jira/browse/KYLIN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2146: Assignee: luguosheng (was: Zhong,Jason) > "Streaming Cluster" page should remove "Margin" inputbox > > > Key: KYLIN-2146 > URL: https://issues.apache.org/jira/browse/KYLIN-2146 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.6.0 >Reporter: Shaofeng SHI >Assignee: luguosheng >Priority: Minor > Fix For: v1.6.0 > > Attachments: Screen Shot 2016-11-02 at 10.26.59 AM.png > > > The "margin" is not needed in the new version of streaming; please remove all > its occurances from UI; besides, please move the "Parser Setting" above > "Advanced Setting". See attachement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2146) "Streaming Cluster" page should remove "Margin" inputbox
[ https://issues.apache.org/jira/browse/KYLIN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-2146. - Resolution: Fixed > "Streaming Cluster" page should remove "Margin" inputbox > > > Key: KYLIN-2146 > URL: https://issues.apache.org/jira/browse/KYLIN-2146 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.6.0 >Reporter: Shaofeng SHI >Assignee: Zhong,Jason >Priority: Minor > Fix For: v1.6.0 > > Attachments: Screen Shot 2016-11-02 at 10.26.59 AM.png > > > The "margin" is not needed in the new version of streaming; please remove all > its occurances from UI; besides, please move the "Parser Setting" above > "Advanced Setting". See attachement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2146) "Streaming Cluster" page should remove "Margin" inputbox
[ https://issues.apache.org/jira/browse/KYLIN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2146: Assignee: Zhong,Jason (was: Guohui LI) > "Streaming Cluster" page should remove "Margin" inputbox > > > Key: KYLIN-2146 > URL: https://issues.apache.org/jira/browse/KYLIN-2146 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.6.0 >Reporter: Shaofeng SHI >Assignee: Zhong,Jason >Priority: Minor > Fix For: v1.6.0 > > Attachments: Screen Shot 2016-11-02 at 10.26.59 AM.png > > > The "margin" is not needed in the new version of streaming; please remove all > its occurances from UI; besides, please move the "Parser Setting" above > "Advanced Setting". See attachement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2090) Allow updating cube level config even the cube is ready
[ https://issues.apache.org/jira/browse/KYLIN-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated KYLIN-2090: - Attachment: 0001-KYLIN-2090-Allow-updating-cube-level-config-even-the.patch I have completed this issue.Please checked it .Thanks! > Allow updating cube level config even the cube is ready > --- > > Key: KYLIN-2090 > URL: https://issues.apache.org/jira/browse/KYLIN-2090 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Shaofeng SHI >Assignee: Zhong,Jason > Fix For: Backlog > > Attachments: > 0001-KYLIN-2090-Allow-updating-cube-level-config-even-the.patch > > > Now once a Cube be built to "ready" state, user will not be able to update > it, including changing the cube level configurations, this is bad; as > sometimes user may want to add/change the config. > It should be designed as the "Notification List": user can change at anytime, > and it doesn't impact other things. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2147) Move the creation of HTable after cube be built
[ https://issues.apache.org/jira/browse/KYLIN-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-2147. - Resolution: Fixed > Move the creation of HTable after cube be built > --- > > Key: KYLIN-2147 > URL: https://issues.apache.org/jira/browse/KYLIN-2147 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > Fix For: v1.6.0 > > > In 1.5.x, the creation of HBase table is just after creating dictionaries and > before cube building; That is unnecessary as cube building doesn't operate > the table; It can be deferred to after the cube be built and before > converting to HFile. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2148) ActiveMq server not responding after some time
[ https://issues.apache.org/jira/browse/KYLIN-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627879#comment-15627879 ] Vidya Sagar D commented on KYLIN-2148: -- I am using ActiveMq5.13.2 and mysql as jdbc persistence. Producing and consuming messages fine but for finite period. After some time, suddenly the Server unable to produce the message, and unable to release the lock of table also. This causes our system to hang for some time, and when restarted it is working fine . Please help me, Thanks in advance. > ActiveMq server not responding after some time > -- > > Key: KYLIN-2148 > URL: https://issues.apache.org/jira/browse/KYLIN-2148 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: Vidya Sagar D >Assignee: Zhong,Jason > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2148) ActiveMq server not responding after some time
Vidya Sagar D created KYLIN-2148: Summary: ActiveMq server not responding after some time Key: KYLIN-2148 URL: https://issues.apache.org/jira/browse/KYLIN-2148 Project: Kylin Issue Type: Bug Components: Web Reporter: Vidya Sagar D Assignee: Zhong,Jason -- This message was sent by Atlassian JIRA (v6.3.4#6332)
