Keyword support in the sql queries

2016-06-07 Thread Vaibhav Taro 
It's not possible to run a query with a keyword in it, for ex.

select sum(sales_value) as sum from sales;

Above query gives an error (Encountered "as sum" at...), Is there any
workaround for this or any plan to support this in near future?

-- 
Regards,
VaibhaV

snapshot table not update

2016-06-07 Thread yubo-...@yolo24.com 
I define a loopup table in a cube,when data changed in lookup table,
snapshot not updated .

I can find logs as below:



2016-06-08 11:03:07,443 INFO  [pool-5-thread-7] lookup.SnapshotManager:181 :
Loading snapshotTable from /table_snapshot/siteidmapping/8bd
d3aba-0842-432b-be99-2ba2bb1a852d.snapshot, with loadData: true
2016-06-08 11:03:07,447 DEBUG [pool-5-thread-7] lookup.SnapshotManager:187 :
Loaded snapshot at /table_snapshot/siteidmapping/8bdd3aba-08
42-432b-be99-2ba2bb1a852d.snapshot
2016-06-08 11:03:07,447 INFO  [pool-5-thread-7] lookup.SnapshotManager:130 :
Identical snapshot content org.apache.kylin.dict.lookup.Snap
shotTable@9652a8b3, reuse existing snapshot at
/table_snapshot/siteidmapping/8bdd3aba-0842-432b-be99-2ba2bb1a852d.snapshot


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/snapshot-table-not-update-tp4854.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * from fact_table

2016-06-07 Thread Li Yang 
Yeah~~ Thanks Xiaoyu!

On Sun, Jun 5, 2016 at 8:47 PM, 王晓雨  wrote:

> I add a document about RAW measure.
> see:
> http://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/
>
> 在 2016年5月26日，11:08，王晓雨 > 写道：
>
> Comming soon!
>
> 在 2016年05月26日 11:02, Mars J 写道:
> I'm looking forward the documents too~
>
> 2016-05-26 9:48 GMT+08:00 赵天烁 >:
>
> Waiting for the release note~
>
> -
>
> 赵天烁
> Kevin Zhao
> Java工程师
> 研发中心-Flyme-大数据-平台研发
> + 86 18826908281 | zhaotians...@meizu.com
>
> 珠海市魅族科技有限公司
> MEIZU Technology Co., Ltd.
> 广东省珠海市科技创新海岸魅族科技楼
> MEIZU Tech Bldg., Technology & Innovation Coast
> Zhuhai, 519085, Guangdong, China
>
>
> meizu.com
>
>
>
> -邮件原件-
> 发件人: lidong [mailto:lid...@apache.org]
> 发送时间: 2016年5月26日 9:48
> 收件人: dev@kylin.apache.org
> 主题: Re: 答复: select * from fact_table
>
> It’s under voting currently.
>
>
> http://apache-kylin.74782.x6.nabble.com/VOTE-Release-apache-kylin-1-5-2-release-candidate-3-td4672.html
> If the vote got passed in 72 hrs, the binary release will roll out soon.
>
>
> Thanks,
> Dong
>
>
> Original Message
> Sender:赵天烁zhaotians...@meizu.com
> Recipient:dev@kylin.apache.org...@kylin.apache.org
> Date:Thursday, May 26, 2016 09:21
> Subject:答复: select * from fact_table
>
>
> Seems like already,
> https://github.com/apache/kylin/releases/tag/kylin-1.5.2
> - 赵天烁 Kevin Zhao
> Java工程师 研发中心-Flyme-大数据-平台研发 + 86 18826908281 | zhaotians...@meizu.com
> 珠海市魅族科技有限公司 MEIZU Technology Co., Ltd. 广东省珠海市科技创新海岸魅族科技楼 MEIZU Tech Bldg.,
> Technology  Innovation Coast Zhuhai, 519085, Guangdong, China meizu.com
> -邮件原件- 发件人: Mars J [mailto:xujiao.myc...@gmail.com] 发送时间:
> 2016年5月26日 9:16 收件人: dev@kylin.apache.org 主题: Re: select * from
> fact_table when 1.5.2 release ? 2016-05-22 13:28 GMT+08:00 Li Yang
> liy...@apache.org:  The coming 1.5.2 release has this fixed. Before that,
> please specify  the aggregation function or use group by clause.   On Wed,
> May 18, 2016 at 3:16 PM, Mars J xujiao.myc...@gmail.com wrote:Hello
> all,   after build a cube , using the sql select * from fact_table, it
>  will not return actual fact table data but rather the follow one,I   have
> define 3   measure include count/min/raw [image: 内嵌图片 1] DIMA_ID
> and DIMC_ID are dimension columns defined in model   dimensions  step.
>  Why 'select * from fact_table' doesn't return real fact data ? how   to
> query the all fact data ?
>
>
>
>

How to name derived dimensions & hierarchy dimensions in Chinese

2016-06-07 Thread 251469031 
Hi all:


I want to display the cube metadata by calling the RESTful API of 
kylinhost:7070/api/cubes/{cubeName} so that the  endusers can know the cube 
details, as the endusers' mother tongue is Chinese, so I want to set the 
CubeName, Dimension Name( including derived dimensions & hierarchy dimensions)  
and messures name can be in Chinese. 


But as we know, the derived dimensions & hierarchy dimensions is followed 
by hive columns and hive columns can not be set to Chinese, so would you tell 
me how to name derived dimensions & hierarchy dimensions in Chinese.

[Announce] Apache Kylin 1.5.2.1 released

2016-06-07 Thread ShaoFeng Shi 
The Apache Kylin team is pleased to announce the immediate availability of
the 1.5.2.1 release. The release note can be found here [1]; The source
code and binary package can be downloaded from Kylin's download page [2].

The Apache Kylin Team would like to hear from you and welcomes your
comments and contributions.

Thanks,
The Apache Kylin Team

[1] https://kylin.apache.org/docs15/release_notes.html
[2] https://kylin.apache.org/download/

-- 
Best regards,

Shaofeng Shi

Re: 答复: Timeout visiting cube!

2016-06-07 Thread ShaoFeng Shi 
Hi Gao, what's your Kylin version and HBase version?

2016-06-08 9:20 GMT+08:00 gaolv123...@163.com :

> 你好：
> 我并不是真的超时，而是kylin抛异常了，异常如下：
> 每次构建之后都会有这个问题，必须去$KYLIN_HOME/bin/kylin.sh
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI
> $KYLIN_HOME/lib/kylin-coprocessor-*.jar all才行
>
> Caused by: org.apache.hadoop.hbase.exceptions.UnknownProtocolException:
> org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered
> coprocessor service found for name CubeVisitService in region
> KYLIN_NDT1PYHI7P,,1465298389410.ebf41fc2bd7ac44fb2f0b14a2146ecae.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7457)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1891)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1873)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
>
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:325)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at
> org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:3861)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:362)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:358)
> at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1751)
> ... 4 more
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.UnknownProtocolException):
> org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered
> coprocessor service found for name CubeVisitService in region
> KYLIN_NDT1PYHI7P,,1465298389410.ebf41fc2bd7ac44fb2f0b14a2146ecae.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7457)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1891)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1873)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
>
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1235)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:222)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:323)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:32855)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1616)
> ... 13 more
>
>
>
> gaolv123...@163.com
>
> 发件人： 吴钰彬
> 发送时间： 2016-06-08 09:10
> 收件人： gaolv123...@163.com
> 主题： 答复: Timeout visiting cube!
>
> 我不是KYLIN的开发，一样是用户，我之前也碰到过这个问题
>
> 你可以尝试在kylin.profile的配置文件里面添加下面的参数，默认值是1，我这边是改成3倍
>
> kylin.query.cube.visit.timeout.times=3
> #default is 1
>
> -邮件原件-
> 发件人: gaolv123...@163.com

答复: a simple question about kylin input and output

2016-06-07 Thread yubo-...@yolo24.com 
I ask this question because we have a big dim table with 7000,000 records.

The dim table is defined as below

Id  分类1 分类2 分类3 品牌
Idcat1 cat2 cat3  brand

There are actually 4 dimetions in this table.

We have tested
1.
don’t join fact table and dim table before building the cube,
join tables in query phase , query time is about 10-40 seconds,

we define cube with a derived dimention ( id column) and other dimentions in 
the fact table.

I guess kylin store rowkey with column id , not cat1,cat2,cat3,brand ,am I 
right here?

when query by or group by cat1,cat2,cat3,
there are scan to the dim table and calculations for the measues.

Advantage: less dimentions, only cube
Disadvantage: query time is high

2. join tables before building the cube
No need to join in query phase. Query time is about 1-3 seconds.

Advantage: query time is low
Disadvantage: more dimentions, need to split one cube to more if nessesary


发件人: Yang [via Apache Kylin] [mailto:ml-node+s74782n4842...@n6.nabble.com]
发送时间: 2016年6月7日 17:53
收件人: yubo-ds1(于渤.大数据中心.大数据平台部)
主题: Re: a simple question about kylin input and output

I'm more curious about the purpose of such requirement. It's not obvious to
me.

On Thu, Jun 2, 2016 at 6:55 PM, lidong <[hidden 
email]> wrote:

> Does Hive view meet your need?
>
>
> Create a Hive view C base on A join B. And make C as input, then get C as
> output.
>
>
> Thanks,
> Dong
>
>
> Original Message
> Sender:[hidden email]-[hidden 
> email]
> Recipient:[hidden email]
> Date:Thursday, Jun 2, 2016 18:28
> Subject:a simple question about kylin input and output
>
>
> Hi all: We define a fact table and a dim table as input in a cube, after
> cube build finished, we can find a fact table and a dim table in the
> "insight" tab as output with same names as we defined. example: input 2
> table: A,B output 2 table: A,B My question is: Is there a way to produce
> only one table, which have joined the face table and the dim table?
> example: input 2 table :A,B output 1 table : C Thanks in advance for any
> hints. Yubo -- View this message in context:
> http://apache-kylin.74782.x6.nabble.com/a-simple-question-about-kylin-input-and-output-tp4784.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>


If you reply to this email, your message will be added to the discussion below:
http://apache-kylin.74782.x6.nabble.com/a-simple-question-about-kylin-input-and-output-tp4784p4842.html
To start a new topic under Apache Kylin, email ml-node+s74782n1...@n6.nabble.com
To unsubscribe from Apache Kylin, click 
here.
NAML


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/a-simple-question-about-kylin-input-and-output-tp4784p4848.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

回复: 答复: Timeout visiting cube!

2016-06-07 Thread gaolv123...@163.com 
你好：
我并不是真的超时，而是kylin抛异常了，异常如下：
每次构建之后都会有这个问题，必须去$KYLIN_HOME/bin/kylin.sh 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI 
$KYLIN_HOME/lib/kylin-coprocessor-*.jar all才行

Caused by: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: 
org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered 
coprocessor service found for name CubeVisitService in region 
KYLIN_NDT1PYHI7P,,1465298389410.ebf41fc2bd7ac44fb2f0b14a2146ecae.
at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7457)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1891)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1873)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:325)
at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1620)
at 
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
at 
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
at 
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
at 
org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
at 
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:3861)
at 
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:362)
at 
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:358)
at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1751)
... 4 more
Caused by: 
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.UnknownProtocolException):
 org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered 
coprocessor service found for name CubeVisitService in region 
KYLIN_NDT1PYHI7P,,1465298389410.ebf41fc2bd7ac44fb2f0b14a2146ecae.
at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7457)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1891)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1873)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1235)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:222)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:323)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:32855)
at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1616)
... 13 more



gaolv123...@163.com
 
发件人： 吴钰彬
发送时间： 2016-06-08 09:10
收件人： gaolv123...@163.com
主题： 答复: Timeout visiting cube!
 
我不是KYLIN的开发，一样是用户，我之前也碰到过这个问题
 
你可以尝试在kylin.profile的配置文件里面添加下面的参数，默认值是1，我这边是改成3倍
 
kylin.query.cube.visit.timeout.times=3
#default is 1
 
-邮件原件-
发件人: gaolv123...@163.com [mailto:gaolv123...@163.com] 
发送时间: 2016年6月7日 20:39
收件人: dev 
主题: Timeout visiting cube!
 
 
你好：
每次当我build完 cube之后，都必须去更新 Update HBase Coprocessor
否则查询的时候便会报错，如下：Error while executing SQL "SELECT * FROM 
YOOSHU_BID_REQUEST_VIEWS LIMIT 10": Timeout visiting cube!

[RESULT] [VOTE] Release apache-kylin-1.5.2.1 (release candidate 1)

2016-06-07 Thread ShaoFeng Shi 
Thanks to everyone who has tested the release candidate and given
their comments and votes.

The tally is as follows.

5 binding +1s:
Shaofeng Shi
Dong Li
Dayue Gao
Xiaoyu Wang
Luke Han

1 non-binding +1s:
Chunen Ni


No 0s or -1s.

Therefore I am delighted to announce that the proposal to release
Apache-Kylin-1.5.2.1 has passed.

-- 
Best regards,

Shaofeng Shi

[jira] [Created] (KYLIN-1773) Model should not be editable if used by cubes

2016-06-07 Thread Dong Li (JIRA) 
Dong Li created KYLIN-1773:
--

 Summary: Model should not be editable if used by cubes
 Key: KYLIN-1773
 URL: https://issues.apache.org/jira/browse/KYLIN-1773
 Project: Kylin
  Issue Type: Bug
Affects Versions: v1.5.2
Reporter: Dong Li
Assignee: Dong Li
Priority: Minor


With sample data
1. build cube kylin_sales_cube.
2. edit model kylin_sales_model
3. change fact table to other tables
4. save

Actual: model saved, and cube/model loading will fail
Expect: Model cannot be saved, with warn message like "Cannot edit model 
because there's cube references."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Timeout visiting cube!

2016-06-07 Thread gaolv123...@163.com 

你好：
每次当我build完 cube之后，都必须去更新 Update HBase Coprocessor
否则查询的时候便会报错，如下：Error while executing SQL "SELECT * FROM 
YOOSHU_BID_REQUEST_VIEWS LIMIT 10": Timeout visiting cube!

当执行完$KYLIN_HOME/bin/kylin.sh 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI 
$KYLIN_HOME/lib/kylin-coprocessor-*.jar all
之后，便可以正常查询。
但是下次build之后，又会出现这个问题。

应该如何解决的啊？




gaolv123...@163.com

Re: a simple question about kylin input and output

2016-06-07 Thread Li Yang 
I'm more curious about the purpose of such requirement. It's not obvious to
me.

On Thu, Jun 2, 2016 at 6:55 PM, lidong  wrote:

> Does Hive view meet your need?
>
>
> Create a Hive view C base on A join B. And make C as input, then get C as
> output.
>
>
> Thanks,
> Dong
>
>
> Original Message
> Sender:yubo-ds1@yolo24.comyubo-...@yolo24.com
> Recipient:dev...@kylin.apache.org
> Date:Thursday, Jun 2, 2016 18:28
> Subject:a simple question about kylin input and output
>
>
> Hi all: We define a fact table and a dim table as input in a cube, after
> cube build finished, we can find a fact table and a dim table in the
> "insight" tab as output with same names as we defined. example: input 2
> table: A,B output 2 table: A,B My question is: Is there a way to produce
> only one table, which have joined the face table and the dim table?
> example: input 2 table :A,B output 1 table : C Thanks in advance for any
> hints. Yubo -- View this message in context:
> http://apache-kylin.74782.x6.nabble.com/a-simple-question-about-kylin-input-and-output-tp4784.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: Extract Fact Table Distinct Columns step taking more time

2016-06-07 Thread Li Yang 
Need to identify why "step-2" is slow first.

Maybe start by checking if the mapper splits are even.

On Thu, Jun 2, 2016 at 5:25 PM, Vaibhav Taro 
wrote:

> Hey, ShaoFeng thanks for the reply. Yes, my Kylin version is 1.5.2.
>
> I have consistently observed that step-2 takes more time compared to other
> steps in cube build process. For example with 1m, 5m, 10m and even with
> 498m records in Hive. For 498m records in Hive, Cube build process took 56
> minutes and step-2 took around 16 minutes which was more than the time
> taken by other steps in the cube build.
>
> Is there any indicator in UI to know which cubing algorithm is being used
> for the cube build job? I can see, in my cube build(1m) that first two
> step, cube build step and convert cuboid data to Hfile step are the
> map-reduce jobs. This job is using "fast cubing" as I can see in the logs,
> however, statistics on the UI indicates that step-2 is taking more time.
>
> Is there any other tuning that I can do to minimize the time taken by cube
> build process? I can increase memory and parallelism.
>
>
>
> On Wed, Jun 1, 2016 at 8:07 PM, ShaoFeng Shi 
> wrote:
>
> > I guess you're running with 1.5.x which can build the cube with the "fast
> > cubing" algorithm: the cube build can be finished in one round of MR, and
> > if the data can be fit into memory the calculation is fast ;
> >
> > While the step 2 ("fetch distinct values of dimensions") should be a more
> > light-weighted step comparing with the cube build, in my experience it
> > should take less time than the cube build step; But as 1 million is such
> a
> > small data set, one or two rounds of testing may not reflect the real
> case;
> > Suggest you make more tests, and using a bigger data set; Please share us
> > with your findings, thanks!
> >
> > 2016-05-31 15:10 GMT+08:00 Vaibhav Taro :
> >
> > > I am building a cube with 1 million rows in Hive, where Hive table is
> in
> > > ORC format, partitioned by date and clustered into 8 buckets.
> > >
> > > Cube has 12 dimensions and 2 measures and Cube build job takes around 5
> > > minutes to complete.
> > >
> > > Extract fact table distinct columns step is taking around 3 minutes
> which
> > > is more compared to the time taken by other steps. The Map-reduce job
> is
> > > spawning 6 mappers and 10 reducers for the same.
> > >
> > > What are the factors affecting the time taken by this step?
> > > I am using Kylin version 1.5.2 with default settings.
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
> >
>
>
>
> --
> Regards,
> VaibhaV
>

question to TOP-N feature

2016-06-07 Thread Jie Tao 
is it possible to use this feature to show the last_N records, i.e. let 
Kylin sort in ascending order rather than descending order?


Cheers,

Jie

[jira] [Created] (KYLIN-1771) Run merge cube for old segments and build cube for new segment simultaneously

2016-06-07 Thread Vaibhav Taro (JIRA) 
Vaibhav Taro created KYLIN-1771:
---

 Summary: Run merge cube for old segments and build cube for new 
segment simultaneously
 Key: KYLIN-1771
 URL: https://issues.apache.org/jira/browse/KYLIN-1771
 Project: Kylin
  Issue Type: Improvement
Reporter: Vaibhav Taro


Right now with Kylin 1.5.2, it is not possible to run Cube merge for old 
segments and Cube build job for a new segment simultaneously. (I am not able to 
run cube merge if a cube build is already in progress)

This will be helpful if we want to run cube build job continuously to reflect 
new data and can run merge cube for old segments in parallel without stopping 
the new cube build job.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Merge cube and build cube simultaneously

2016-06-07 Thread Li Yang 
This is valid requirement, could you open a JIRA?

On Wed, Jun 1, 2016 at 6:47 PM, Vaibhav Taro 
wrote:

> Right now with Kylin 1.5.2, it is not possible to run Cube merge for old
> segments and Cube build job for a new segment simultaneously. (I am not
> able to run cube merge if a cube build is already in progress)
>
> Is there any plan to support this in near future?
>
> This will be helpful if we want to run cube build job continuously to
> reflect new data and can run merge cube for old segments in parallel
> without stopping cube build job.
>
> I have not explored auto cube merge feature, though. Does the auto cube
> merge happens in the background? and is it possible to run the cube build
> in parallel?
>
> --
> Regards,
> VaibhaV
>

[REPORT] Apache Kylin - Jun 2016

2016-06-07 Thread Luke Han 
## Description:
===
Apache Kylin is an open source Distributed Analytics Engine designed
to provide SQL interface and multi-dimensional analysis (OLAP) on
Hadoop supporting extremely large datasets.

## Issues:
==
- there are no issues requiring board attention at this time

## Activity:

- Mailing list, JIRA, and commit activity are at or above average
- Yang Li presented Kylin new archiecture at Hadoop Summit EU
in Dublin on 2016-04-13
- Shaofeng Shi presented Kylin deployment practices at ITA2014
Big Data Event in Beijing
on 2016-04-22
- Apache Kylin meetup Beijing hosted on 2016-04-23 in Beijing,
engaged more than 200 participants, with 6 sessions from Luke Han,
Xiaoyu Wang, Yerui Sun, Dong Wang, Lei Zhao and Shaofeng Shi.
- Luke Han presented Kylin at Apache Big Data 2016 NA in Vancouver
on 2016-05-09
- Luke Han presented Kylin community pracitices at ApacheCon 2016
NA in Vancouver on 2016-05-13
- Hongbin Ma presented performance topic at HBaseCon 2016 in San
Francisco on 2016-05-24

## Community:
=
- 1 committers and 1 PMC members appointed after last report.
- Messages on the dev mailing list after last report: 1234
- Messages on the user mailing list after last report: 335
- 365 JIRA tickets created after last report
- 510 JIRA tickets closed/resolved after last report

## Releases:

- The next generation of Kylin, v1.5.0, released on 2016-03-12
- v1.5.1, released on 2016-04-13
- The latest release, v1.5.2, released on 2016-05-26

Re: [VOTE] Release apache-kylin-1.5.2.1 (release candidate 1)

2016-06-07 Thread Luke Han 
+1 (binding)

mvn test passed
signature verified



Best Regards!
-

Luke Han

On Tue, Jun 7, 2016 at 11:22 AM, nichunen  wrote:

> +1(no binding)
>
> build success
> mvn clean test passed
>
> md5 verified
>
>
>
>
>   George/倪春恩
>
> Mobile:+86-13501723787| WeChat:nceecn
>
> 北京明略软件系统有限公司（MiningLamp.COM
> ）
>
>
>
>
>
> 上海市浦东新区晨晖路258号G座iDream张江科创中心C125
>
>
>
>
> Room C125#,Intelligent Industrial Park Building G,258#Chenhui Road, Pudong
> District,Shanghai,201203
>
>
>
>
> > On Jun 5, 2016, at 2:13 PM, Dayue Gao  wrote:
> >
> >
> > +1 (binding)
> >
> >
> >
> >
> >
> > mvn clean test passed.
> > Regards,
> > Dayue
> >
> >
> >
> >
> >
> > At 2016-06-04 20:31:48, "ShaoFeng Shi"  wrote:
> >>
> >>
> >> Hi all,
> >>
> >>
> >> I have created a build for Apache Kylin 1.5.2.1, release candidate 1.
> >>
> >>
> >> This is a hot-fix version for v1.5.2, including two major bug fixes:
> >>
> >>
> >> [KYLIN-1758] - createLookupHiveViewMaterializationStep will create
> >> intermediate table for fact table
> >> [KYLIN-1739] - kylin_job_conf_inmem.xml can impact non-inmem MR job
> >>
> >>
> >> Thanks to everyone who has contributed to this release. Here’s release
> >> notes:
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12335953
> >>
> >>
> >> The commit to be voted upon:
> >>
> >>
> >>
> https://github.com/apache/kylin/commit/71a847bbe97ec4e6f23b1bf271132d6ff9955a96
> >>
> >>
> >> Its hash is 71a847bbe97ec4e6f23b1bf271132d6ff9955a96.
> >>
> >>
> >> The artifacts to be voted on are located here:
> >> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-1.5.2.1-rc1/
> >>
> >>
> >> The hashes of the artifacts are as follows:
> >> apache-kylin-1.5.2.1-src.tar.gz.md5 502f853e541fb2eaddb3db4774f29046
> >> apache-kylin-1.5.2.1-src.tar.gz.sha1
> >> 6dc0b973f117ae57661364721457d6734bdefb01
> >>
> >>
> >> A staged Maven repository is available for review at:
> >> https://repository.apache.org/content/repositories/orgapachekylin-1030/
> >>
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/shaofengshi.asc
> >>
> >>
> >> Please vote on releasing this package as Apache Kylin 1.5.2.1.
> >>
> >>
> >> The vote is open for the next 72 hours and passes if a majority of
> >> at least three +1 PPMC votes are cast.
> >>
> >>
> >> [ ] +1 Release this package as Apache Kylin 1.5.2.1
> >> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> >> [ ] -1 Do not release this package because...
> >>
> >>
> >>
> >>
> >>
> >> Here is my vote:
> >>
> >>
> >> +1 (binding)
> >>
> >>
> >> Shaofeng Shi
> >>
>

Re: 回复：failed to start kylin server

2016-06-07 Thread Li Yang 
@耳东, according to the log, Kylin failed to read metadata during startup.
Please check HBase is on and healthy.


Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
after attempts=6, exceptions:
Sun May 29 02:02:19 CST 2016, null, java.io.InterruptedIOException: Origin:
InterruptedException
at
org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:270)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:203)
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:57)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at
org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:294)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:269)
at
org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:141)
at
org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:136)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:886)
at
org.apache.kylin.storage.hbase.HBaseResourceStore.visitFolder(HBaseResourceStore.java:136)
at
org.apache.kylin.storage.hbase.HBaseResourceStore.listResourcesImpl(HBaseResourceStore.java:106)

On Wed, Jun 1, 2016 at 3:37 PM, Yapu Jia  wrote:

> What is the mean of no web access address? The web address is
> http://yourip:7070/kylin.
> It's no need to configure the web address.
>
> -Original Message-
> From: bingclouds [mailto:bingclo...@163.com]
> Sent: Monday, May 30, 2016 1:37 AM
> To: 耳东 <775620...@qq.com>
> Cc: 耳东 
> Subject: Re: 回复：failed to start kylin server
>
> 请问在安装kylin 1.5.2的时候，kylin已经启动，但是却没有web访问地址，提示please visit
> http::7070/kylin，在哪里配置呢？又是什么原因呢
>
>
> 来自 魅族 MX5
>
>  原始邮件 
> 发件人：耳东 <775620...@qq.com>
> 时间：周日 5月29日 02:03
> 收件人：耳东 
> 主题：回复：failed to start kylin server
>
> >and when I stop the kylin sever, the kylin.log shows as follows:
> >
> >
> >2016-05-29 02:02:19,183 ERROR [localhost-startStop-1]
> context.ContextLoader:307 : Context initialization failed
> >org.springframework.beans.factory.BeanCreationException: Error creating
> bean with name 'cacheService': Invocation of init method failed; nested
> exception is java.lang.IllegalStateException: Failed to init CubeManager
> from kylin_metadata@hbase
> >   at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:135)
> >   at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
> >   at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448)
> >   at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
> >   at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
> >   at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
> >   at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
> >   at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
> >   at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
> >   at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609)
> >   at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
> >   at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:469)
> >   at
> org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
> >   at
> org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
> >   at
> org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
> >   at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
> >   at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
> >   at
> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
> >   at
>

[jira] [Created] (KYLIN-1770) Can't use PreparedStatement with "between and" expression

2016-06-07 Thread Dayue Gao (JIRA) 
Dayue Gao created KYLIN-1770:


 Summary: Can't use PreparedStatement with "between and" expression
 Key: KYLIN-1770
 URL: https://issues.apache.org/jira/browse/KYLIN-1770
 Project: Kylin
  Issue Type: Bug
  Components: Driver - JDBC
Affects Versions: v1.5.2, v1.5.1
Reporter: Dayue Gao


Sample code to reproduce:

{code:java}
final String sql = "select count(*) from kylin_sales where LSTG_SITE_ID 
between ? and ?";

try (PreparedStatement stmt = conn.prepareStatement(sql)) {
stmt.setInt(1, 0);
stmt.setInt(2, 5);

try (ResultSet rs = stmt.executeQuery()) {
printResultSet(rs);
}
}
{code}

Exception stack trace from server log:
{noformat}
java.sql.SQLException: Error while preparing statement [select count(*) from 
kylin_sales where LSTG_SITE_ID between ? and ?]
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:203)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:184)
at 
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:85)
at 
org.apache.calcite.avatica.AvaticaConnection.prepareStatement(AvaticaConnection.java:153)
at 
org.apache.kylin.rest.service.QueryService.execute(QueryService.java:353)
at 
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:274)
at 
org.apache.kylin.rest.service.QueryService.query(QueryService.java:120)
at 
org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
at 
org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$8610374f.query()
at 
org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
at 
org.apache.kylin.rest.controller.QueryController.prepareQuery(QueryController.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at 
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at 
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at 
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at

Re: question to building cube from kafka

2016-06-07 Thread ShaoFeng Shi 
Hi Jie, it need the json data format be flat (no embeded attribute);

2016-06-07 14:56 GMT+08:00 Jie Tao :

> It is a nice feature to build cube directly from kafka. From the example
> on your docs I see that the table schema is extracted from the input JSON.
> The question is: do your support recursive JSON structure, i.e., a JSON
> attribute is an object containing other attributes? Like:
>
> {
> "foo": {
> "attr1": 70,
> "att2: "blabla"
> },
> "fa":
> }
>
> Cheers,
>
> Jie
>



-- 
Best regards,

Shaofeng Shi

Re: kylin JDBC分页查询

2016-06-07 Thread Dayue 
Let's say you want to show 100 rows per page,

1st page: limit 100 offset 0
2nd page: limit 100 offset 100
3rd page: limit 100 offset 200
etc.

Best Regards,
Dayue


> 在 2016年6月7日，下午2:12，gaolv123...@163.com 写道：
> 
> 
> 请问下 KYLIN使用JDBC 如何分页查询啊，并不能使用 limit 0,10   这个语法。
> 
> 
> 
> gaolv123...@163.com

question to building cube from kafka

2016-06-07 Thread Jie Tao 
It is a nice feature to build cube directly from kafka. From the example 
on your docs I see that the table schema is extracted from the input 
JSON. The question is: do your support recursive JSON structure, i.e., a 
JSON attribute is an object containing other attributes? Like:


{
"foo": {
"attr1": 70,
"att2: "blabla"
},
"fa":
}

Cheers,

Jie

kylin JDBC分页查询

2016-06-07 Thread gaolv123...@163.com 

请问下 KYLIN使用JDBC 如何分页查询啊，并不能使用 limit 0,10   这个语法。



gaolv123...@163.com