[jira] [Created] (KYLIN-1704) When load empty snapshot, NULL Pointer Exception occurs

2016-05-17 Thread Zhong Yanghong (JIRA) 
Zhong Yanghong created KYLIN-1704:
-

 Summary: When load empty snapshot, NULL Pointer Exception occurs
 Key: KYLIN-1704
 URL: https://issues.apache.org/jira/browse/KYLIN-1704
 Project: Kylin
  Issue Type: Bug
  Components: Metadata
Reporter: Zhong Yanghong
Assignee: Zhong Yanghong






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: expansion rate so high

2016-05-17 Thread hongbin ma 
I guess there is one or more dimensions with very high cardinality? have
you read http://kylin.apache.org/blog/2016/02/18/new-aggregation-group/?
there might be some inspirations there.

also, are the four fixed length dimensions all text? are there any
relationships between the dimensions? Why not using dict for them, are they
super-high cardinality columns?

On Mon, May 16, 2016 at 10:21 AM, Peng  wrote:

> Hi,
>
>Is it normal when the expansion rate is about 1600% ?
>
>My  cube : about one hundred million data;
>
> fact table has one lookup table;
>
> 5 dimensions , in which 4 dimensions' encoding are fixed length,
> separately
> the length are 9, 8, 8, 33 ;
>
> 2 measures,
>
>  finally the expansion rate is about 1600.
>
>
>
> Thanks
>
> Peng
>
>


-- 
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

Re: CubeInstance desc not found

2016-05-17 Thread hongbin ma 
Seems you metadata store is corrupted, did you perform any direct access to
hbase metadata store?

Dump you metadata store with `$KYLIN_HOME/bin/metastore.sh backup`, is
there a json named TERMINAL_MME_CUBE in cube_desc/ ?

You can also upload the dumped store so that the community can help to
analysis

On Tue, May 17, 2016 at 9:24 PM, 耳东 <775620...@qq.com> wrote:

> hi ALL:
>   I create a few cube in a project. When I refresh the web page,  some
> of the cubes will disappear, and the log shows below:
>
>
> 2016-05-17 21:23:36,330 ERROR [localhost-startStop-1] cube.CubeManager:856
> : Error during load cube instance /cube/terminal_mme_cube.json
> java.lang.IllegalStateException: CubeInstance desc not found
> 'TERMINAL_MME_CUBE', at /cube/terminal_mme_cube.json
> at
> org.apache.kylin.cube.CubeManager.reloadCubeLocalAt(CubeManager.java:829)
> at
> org.apache.kylin.cube.CubeManager.loadAllCubeInstance(CubeManager.java:814)
> at org.apache.kylin.cube.CubeManager.(CubeManager.java:127)
> at
> org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:96)
> at
> org.apache.kylin.rest.service.CacheService.initCubeChangeListener(CacheService.java:79)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
> at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
> at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
> at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609)
> at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:469)
> at
> org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
> at
> org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
> at
> org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
> at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
> at
> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
> at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
> at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
> at
> org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
> at
> org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>

Re: SQL Statements supported by Kylin 1.5

2016-05-17 Thread hongbin ma 
kylin uses Apache Calcite for sql parsing, so the supported sql will be a
subset of https://calcite.apache.org/docs/reference.html. However always
keep in mind metrics in kylin need pre-calculating, currently we still have
many unsupported metrics like standard deviation, percentile,etc.

On Wed, May 18, 2016 at 9:34 AM, Mars J  wrote:

> Hello all,
>  What SQL type is the valid sql supported by kylin 1.5 ?
>
>   We want to develop a front end for users quering to use kylin api.
> But we don't know which sqls is the valid . Is there something like
> specifications ?
>



-- 
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

Re: why can't I see the project which is created by other kylin server when I visit a kylin server ?

2016-05-17 Thread hongbin ma 
The update should be automatic. There might be some bug. What is your kylin
version?
Another question: does the auto update only fails at project creating? have
you also tried cube creating, create build,etc.?​

On Wed, May 18, 2016 at 10:45 AM, 杨海乐  wrote:

> The two kylin servers share same hbase metadata table.If I change metadata
> in
> a kylin server,I can see same metadata by click 'reload metadata' in
> another
> server. is there some option in kylin.properties to auto update metadata?
>
> --
> View this message in context:
> http://apache-kylin.74782.x6.nabble.com/why-can-t-I-see-the-project-which-is-created-by-other-kylin-server-when-I-visit-a-kylin-server-tp4575p4584.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Regards,

*Bin Mahone | 马洪宾*
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone

RE: why can't I see the project which is created by other kylin server when I visit a kylin server ?

2016-05-17 Thread 杨海乐 
The two kylin servers share same hbase metadata table.If I change metadata in
a kylin server,I can see same metadata by click 'reload metadata' in another
server. is there some option in kylin.properties to auto update metadata?

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/why-can-t-I-see-the-project-which-is-created-by-other-kylin-server-when-I-visit-a-kylin-server-tp4575p4584.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

[jira] [Created] (KYLIN-1703) The not-thread-safe ToolRunner.run() will cause concurrency issue in job engine

2016-05-17 Thread Dong Li (JIRA) 
Dong Li created KYLIN-1703:
--

 Summary: The not-thread-safe ToolRunner.run() will cause 
concurrency issue in job engine
 Key: KYLIN-1703
 URL: https://issues.apache.org/jira/browse/KYLIN-1703
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v1.5.1
Reporter: Dong Li
Assignee: Dong Li


Refer to 
http://stackoverflow.com/questions/22462665/is-hadoops-toorunner-thread-safe

ToolRunner.run() is not thread safe, if two MapReduceExecutable start MR Job at 
same time, there would be an NPE thrown from ToolRunner.run().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1702) The Key of the Snapshot to the related lookup table may be not informative

2016-05-17 Thread Zhong Yanghong (JIRA) 
Zhong Yanghong created KYLIN-1702:
-

 Summary: The Key of the Snapshot to the related lookup table may 
be not informative
 Key: KYLIN-1702
 URL: https://issues.apache.org/jira/browse/KYLIN-1702
 Project: Kylin
  Issue Type: Improvement
Reporter: Zhong Yanghong


Currently the key for the snapshot stored in hbase metadata file is as follows:
ResourceStore.SNAPSHOT_RESOURCE_ROOT + "/" + new 
File(signature.getPath()).getName() + "/" + uuid + ".snapshot"

However, some hive tables stored in hive may organized like 
dirName/tableName/00, dirName/tableName/01.

Based on current setting, the key will be ResourceStore.SNAPSHOT_RESOURCE_ROOT 
+ "/" + 00 + "/" + uuid + ".snapshot", which is lack of the table name 
information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

CubeInstance desc not found

2016-05-17 Thread ???? 
hi ALL:
  I create a few cube in a project. When I refresh the web page,  some of 
the cubes will disappear, and the log shows below:


2016-05-17 21:23:36,330 ERROR [localhost-startStop-1] cube.CubeManager:856 : 
Error during load cube instance /cube/terminal_mme_cube.json
java.lang.IllegalStateException: CubeInstance desc not found 
'TERMINAL_MME_CUBE', at /cube/terminal_mme_cube.json
at 
org.apache.kylin.cube.CubeManager.reloadCubeLocalAt(CubeManager.java:829)
at 
org.apache.kylin.cube.CubeManager.loadAllCubeInstance(CubeManager.java:814)
at org.apache.kylin.cube.CubeManager.(CubeManager.java:127)
at org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:96)
at 
org.apache.kylin.rest.service.CacheService.initCubeChangeListener(CacheService.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609)
at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:469)
at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at 
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at 
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at 
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at 
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at 
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at 
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

[jira] [Created] (KYLIN-1701) Cordova

2016-05-17 Thread akash (JIRA) 
akash created KYLIN-1701:


 Summary: Cordova
 Key: KYLIN-1701
 URL: https://issues.apache.org/jira/browse/KYLIN-1701
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Reporter: akash
Assignee: Zhong,Jason






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1700) Error When Start with Sample Cube

2016-05-17 Thread jingtao (JIRA) 
jingtao created KYLIN-1700:
--

 Summary: Error When Start with Sample Cube
 Key: KYLIN-1700
 URL: https://issues.apache.org/jira/browse/KYLIN-1700
 Project: Kylin
  Issue Type: Bug
Affects Versions: v1.5.1
 Environment: TERM = xterm
HBASE_CLASSPATH_PREFIX = 
./../tomcat/bin/bootstrap.jar:./../tomcat/bin/tomcat-juli.jar:./../tomcat/lib/*:
JAVA_HOME = /usr/java/latest
SHLVL = 2
HBASE_LOG_DIR = /var/log/hbase
HBASE_HOME = /opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hbase/bin/..
XFILESEARCHPATH = /usr/dt/app-defaults/%L/Dt
SSH_CLIENT = 10.230.142.234 58107 22
TMOUT = 
MAIL = /var/spool/mail/scapp
HOSTNAME = BigData08
PWD = /opt/apache-kylin-1.5.1-bin/bin
hbase_dependency = 
/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hbase/bin/../lib/hbase-common-1.0.0-cdh5.6.0.jar
SSH_TTY = /dev/pts/0
LOGNAME = scapp
CVS_RSH = ssh
G_BROKEN_FILENAMES = 1
HBASE_REST_OPTS = 
NLSPATH = /usr/dt/lib/nls/msg/%L/%N.cat
HADOOP_HOME = /opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/hadoop
ZOOKEEPER_HOME = 
/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/bin/../lib/zookeeper
LD_LIBRARY_PATH = 
:/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hbase/bin/../lib/native/Linux-amd64-64
MALLOC_ARENA_MAX = 4
tomcat_root = ./../tomcat
SSH_CONNECTION = 10.230.142.234 58107 10.248.14.95 22
SHELL = /bin/bash
HBASE_PID_DIR = /var/run/hbase
HISTSIZE = 1000
hive_dependency =

RE: why can't I see the project which is created by other kylin server when I visit a kylin server ?

2016-05-17 Thread Yapu Jia 
I think your kylin server not share one metadata. Maybe you kylin configuration 
have some problems.

-Original Message-
From: 杨海乐 [mailto:yangha...@letv.com] 
Sent: Tuesday, May 17, 2016 3:53 PM
To: dev@kylin.apache.org
Subject: why can't I see the project which is created by other kylin server 
when I visit a kylin server ?

Hello all,
   After creating a project in a kylin server,I visit other kylin serevr 
and can't see the project even if I clear browser cache。If I click "reload 
metadata",It work. I want to know why and how to auto update the metadata。
 thanks！

--
View this message in context: 
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fapache-kylin.74782.x6.nabble.com%2fwhy-can-t-I-see-the-project-which-is-created-by-other-kylin-server-when-I-visit-a-kylin-server-tp4575.html=01%7c01%7cyapujia%40064d.mgd.microsoft.com%7c66d2e46d2da94efa3b9708d37e2a52ba%7c72f988bf86f141af91ab2d7cd011db47%7c1=JKwP0IKDjlfw2RwugpTxQcrXp%2fcqc3eUPJaTGkFmTYE%3d
Sent from the Apache Kylin mailing list archive at Nabble.com.

[jira] [Created] (KYLIN-1699) The total number don't be updated in cubes page when filter by cube name

2016-05-17 Thread wangke (JIRA) 
wangke created KYLIN-1699:
-

 Summary: The total number don't be updated in cubes page when 
filter by cube name
 Key: KYLIN-1699
 URL: https://issues.apache.org/jira/browse/KYLIN-1699
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Affects Versions: v1.1, v1.1.1, v1.2, v1.3.0
 Environment: previous kylin-1.3.0
Reporter: wangke
Assignee: Zhong,Jason
Priority: Minor


1.  Select a project and log in.
2.  Filter by cube name on cubes page.
3.  Cubes that meet the conditions be displayed but the total number don't be 
updated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[Draft][REPORT] Apache Kylin - May 2016

2016-05-17 Thread Luke Han 
Hi dear community,
 I'm sorry to almost missed the report deadline. I have tried my best
to draft below one for review, please help to check and let me know if
there's any issue.

 Thanks.

Luke


## Description:
===
Apache Kylin is an open source Distributed Analytics Engine designed
to provide SQL interface and multi-dimensional analysis (OLAP) on
Hadoop supporting extremely large datasets.

## Issues:
==
- there are no issues requiring board attention at this time


## Activity:

- Mailing list, JIRA, and commit activity are at or above average
- Yang Li presented Kylin at Hadoop Summit EU in Dublin on 2016-04-13
- Shaofeng Shi presented Kylin at ITA2014 Big Data Event in Beijing
on 2016-04-22
- Apache Kylin meetup Beijing hosted on 2016-04-23 in Beijing,
engaged more than 200 participants, with 6 sessions from Kyligence,
JD, Meituan, Baidu and China Mobile.
- Luke Han presented Kylin at Apache Big Data 2016 NA in Vancouver
on 2016-05-09
- Luke Han presented Kylin community pracitices at ApacheCon 2016 NA
in Vancouver on 2016-05-13


## Community:
=
- 1 committers and 1 PMC members appointed after last report.
- Messages on the dev mailing list after last report: 1033
- Messages on the user mailing list after last report: 316
- 308 JIRA tickets created after last report
- 473 JIRA tickets closed/resolved after last report


## Releases:

- The next generation of Kylin, v1.5.0, released on 2016-03-12
- The latest release, v1.5.1, released on 2016-04-13