Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread JiaTao Tao 
+1

ShaoFeng Shi  于2019年12月10日周二 下午2:07写道：

> Hi all,
>
> I have created a build for Apache Kylin 3.0.0, release candidate 1.
>
> Changes highlights:
> [KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
> [KYLIN-4167] - Refactor streaming coordinator
> [KYLIN-4273] - Make cube planner works for real-time streaming job
> [KYLIN-4187] - Building dimension dictionary using spark
> [KYLIN-4098] - Add cube auto-merge API
>
> Thanks to everyone who has contributed to this release.
> Here are the release notes:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
>
> The commit to being voted upon:
>
> https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
>
> Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
>
> The artifacts to be voted on, including the source package and two
> pre-compiled binary packages are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
>
> The hash of the artifacts are as follows:
> apache-kylin-3.0.0-source-release.zip.sha256
> 9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
> apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
> bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
> apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
> c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
> apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
> 116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
> apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
> 22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachekylin-1070/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/shaofengshi.asc
>
> Please vote on releasing this package as Apache Kylin 3.0.0.
>
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Kylin 3.0.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofeng...@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscr...@kylin.apache.org
> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>


-- 

Regards!

Aron Tao

Kylin with Spark: java.lang.NoSuchMethodError: org.apache.kylin.common.persistence.ResourceStore.getResource

2019-12-10 Thread Lingaraj 
I'm trying to build a cube in Kylin. It has around 10 steps among which Spark
code runs on step 7. At this step, it's throwing the following error:

Running org.apache.kylin.engine.spark.SparkCubingByLayer -hiveTable
default.kylin_intermediate_table -output
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/cuboid/
-input
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/kylin_intermediate_table
-segmentId 9047b119-14ad-ecb8-bb87-c76d255907df -metaUrl
kylin_metadata@hdfs,path=gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/metadata
-cubename cube
*Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.kylin.common.persistence.ResourceStore.getResource(Ljava/lang/String;Ljava/lang/Class;Lorg/apache/kylin/common/persistence/Serializer;)Lorg/apache/kylin/common/persistence/RootPersistentEntity;*
at
org.apache.kylin.cube.CubeManager.loadCubeInstance(CubeManager.java:682)
at
org.apache.kylin.cube.CubeManager.loadAllCubeInstance(CubeManager.java:671)
at org.apache.kylin.cube.CubeManager.(CubeManager.java:128)
at org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:97)
at
org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:137)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
The command is: 
export HADOOP_CONF_DIR=/etc/hadoop/conf && /usr/lib/spark/bin/spark-submit
--class org.apache.kylin.common.util.SparkEntry  --conf
spark.executor.instances=40  --conf spark.yarn.queue=default  --conf
spark.history.fs.logDirectory=hdfs:///kylin/spark-history  --conf
spark.io.compression.codec=org.apache.spark.io.SnappyCompressionCodec 
--conf spark.master=yarn  --conf
spark.hadoop.yarn.timeline-service.enabled=false  --conf
spark.executor.memory=4G  --conf spark.eventLog.enabled=true  --conf
spark.eventLog.dir=hdfs:///kylin/spark-history  --conf
spark.yarn.executor.memoryOverhead=1024  --conf spark.driver.memory=2G 
--conf spark.shuffle.service.enabled=true --jars
/etc/kylin/lib/kylin-job-2.6.4.jar /etc/kylin/lib/kylin-job-2.6.4.jar
-className org.apache.kylin.engine.spark.SparkCubingByLayer -hiveTable
default.kylin_intermediate_sol_v3_new_9047b119_14ad_ecb8_bb87_c76d255907df
-output
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/cuboid/
-input
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/kylin_intermediate_table
-segmentId 9047b119-14ad-ecb8-bb87-c76d255907df -metaUrl
kylin_metadata@hdfs,path=gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/metadata
-cubename cube
at
org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:347)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


I checked the *kylin-core-common-2.6.4.jar* file that I placed in the
/usr/lib/spark/jars path. The source code has the getResource() method
implementation in the ResourceStore class:

public final  T getResource(final String
resPath, final Serializer serializer) throws IOException {
return this.getResource(resPath,
(org.apache.kylin.common.persistence.ContentReader)new
ContentReader((Serializer)serializer));
}

public final  T getResource(String resPath,
final ContentReader reader) throws IOException {
resPath = this.norm(resPath);
final RawResource res = this.getResourceWithRetry(resPath);
if (res == null) {
return null;
}
return (T)reader.readContent(res);
}

public final RawResource getResource(

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Temple Zhou 
+1

Best regards,
Temple Zhou

On Wed, Dec 11, 2019 at 11:16 AM Wang rupeng  wrote:

> +1
>
> mvn test passed
>
>
>
> ---
>
> Best wishes,
>
> Rupeng Wang
>
>
>
>
>
>
>
> *发件人**: *"codingfor...@126.com" 
> *答复**: *
> *日期**: *2019年12月11日 星期三 10:44
> *收件人**: *
> *主题**: *Re: [VOTE] Release apache-kylin-3.0.0 (RC1)
>
>
>
> +1
> mvn test passed
>
> [image: cid:7D3EAE76-AF2E-4EFA-97B5-6220180DA6F6@hz.ali.com]
>
>
>
> 在 2019年12月11日，10:26，nichunen  写道：
>
>
>
> +1
>
>
>
> Best regards,
>
>
>
> Ni Chunen / George
>
>
>
> On 12/11/2019 10:26，Xiaoyuan Gu wrote：
> +1
> mvn test passed
>
> Bests，
> Xiaoyuan Gu
>
>
>
> At 2019-12-10 14:07:00, "ShaoFeng Shi"  wrote:
> Hi all,
>
> I have created a build for Apache Kylin 3.0.0, release candidate 1.
>
> Changes highlights:
> [KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
> [KYLIN-4167] - Refactor streaming coordinator
> [KYLIN-4273] - Make cube planner works for real-time streaming job
> [KYLIN-4187] - Building dimension dictionary using spark
> [KYLIN-4098] - Add cube auto-merge API
>
> Thanks to everyone who has contributed to this release.
> Here are the release notes:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
>
> The commit to being voted upon:
>
> https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
>
> Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
>
> The artifacts to be voted on, including the source package and two
> pre-compiled binary packages are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
>
> The hash of the artifacts are as follows:
> apache-kylin-3.0.0-source-release.zip.sha256
> 9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
> apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
> bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
> apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
> c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
> apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
> 116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
> apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
> 22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachekylin-1070/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/shaofengshi.asc
>
> Please vote on releasing this package as Apache Kylin 3.0.0.
>
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Kylin 3.0.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofeng...@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscr...@kylin.apache.org
> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>
>
>
>

How kylin can log in without password

2019-12-10 Thread wangdongd...@bidcc.cn 
Dear developer, due to the demand problem, we need to do password free login to 
use kylin system, but we have tried to modify it many times and found that it 
can't succeed. Therefore, I'd like to ask you how kylin can achieve password 
free login and what is the configuration?

This is my modified file kylinsecurity.xml. I don't know if it is this file. If 
you have time to help me, I will be very grateful. Thank you.



王栋栋  java研发工程师

中国科学院计算机网络信息中心
北京北龙云海网络数据科技有限责任公司

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Wang rupeng 
+1
mvn test passed

---
Best wishes,
Rupeng Wang



发件人: "codingfor...@126.com" 
答复: 
日期: 2019年12月11日 星期三 10:44
收件人: 
主题: Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

+1
mvn test passed
[cid:7D3EAE76-AF2E-4EFA-97B5-6220180DA6F6@hz.ali.com]


在 2019年12月11日，10:26，nichunen mailto:n...@apache.org>> 写道：

+1



Best regards,



Ni Chunen / George



On 12/11/2019 10:26，Xiaoyuan 
Gumailto:njguxiaoy...@163.com>> wrote：
+1
mvn test passed

Bests，
Xiaoyuan Gu



At 2019-12-10 14:07:00, "ShaoFeng Shi" 
mailto:shaofeng...@apache.org>> wrote:
Hi all,

I have created a build for Apache Kylin 3.0.0, release candidate 1.

Changes highlights:
[KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
[KYLIN-4167] - Refactor streaming coordinator
[KYLIN-4273] - Make cube planner works for real-time streaming job
[KYLIN-4187] - Building dimension dictionary using spark
[KYLIN-4098] - Add cube auto-merge API

Thanks to everyone who has contributed to this release.
Here are the release notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121

The commit to being voted upon:
https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc

Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.

The artifacts to be voted on, including the source package and two
pre-compiled binary packages are located here:
https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/

The hash of the artifacts are as follows:
apache-kylin-3.0.0-source-release.zip.sha256
9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachekylin-1070/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/shaofengshi.asc

Please vote on releasing this package as Apache Kylin 3.0.0.

The vote is open for the next 72 hours and passes if a majority of
at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Kylin 3.0.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...


Here is my vote:

+1 (binding)

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread codingfor...@126.com 
+1
mvn test passed


> 在 2019年12月11日，10:26，nichunen  写道：
> 
> +1 
> 
> 
> 
> Best regards,
> 
> 
> 
> Ni Chunen / George
> 
> 
> 
> On 12/11/2019 10:26，Xiaoyuan Gu wrote：
> +1
> mvn test passed
> 
> Bests，
> Xiaoyuan Gu
> 
> 
> 
> At 2019-12-10 14:07:00, "ShaoFeng Shi"  wrote:
> Hi all,
> 
> I have created a build for Apache Kylin 3.0.0, release candidate 1.
> 
> Changes highlights:
> [KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
> [KYLIN-4167] - Refactor streaming coordinator
> [KYLIN-4273] - Make cube planner works for real-time streaming job
> [KYLIN-4187] - Building dimension dictionary using spark
> [KYLIN-4098] - Add cube auto-merge API
> 
> Thanks to everyone who has contributed to this release.
> Here are the release notes:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
> 
> The commit to being voted upon:
> https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
> 
> Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
> 
> The artifacts to be voted on, including the source package and two
> pre-compiled binary packages are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
> 
> The hash of the artifacts are as follows:
> apache-kylin-3.0.0-source-release.zip.sha256
> 9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
> apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
> bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
> apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
> c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
> apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
> 116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
> apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
> 22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
> 
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachekylin-1070/
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/shaofengshi.asc
> 
> Please vote on releasing this package as Apache Kylin 3.0.0.
> 
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PMC votes are cast.
> 
> [ ] +1 Release this package as Apache Kylin 3.0.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
> 
> 
> Here is my vote:
> 
> +1 (binding)
> 
> Best regards,
> 
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofeng...@apache.org
> 
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscr...@kylin.apache.org
> Join Kylin dev mail group: dev-subscr...@kylin.apache.org

[jira] [Created] (KYLIN-4294) Add http api for metrics

2019-12-10 Thread xiang zhang (Jira) 
xiang zhang created KYLIN-4294:
--

 Summary: Add http api for metrics 
 Key: KYLIN-4294
 URL: https://issues.apache.org/jira/browse/KYLIN-4294
 Project: Kylin
  Issue Type: Improvement
  Components: REST Service
Affects Versions: v2.6.0
Reporter: xiang zhang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re:[VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Xiaoyuan Gu 
+1
mvn test passed

Bests，
Xiaoyuan Gu



At 2019-12-10 14:07:00, "ShaoFeng Shi"  wrote:
>Hi all,
>
>I have created a build for Apache Kylin 3.0.0, release candidate 1.
>
>Changes highlights:
>[KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
>[KYLIN-4167] - Refactor streaming coordinator
>[KYLIN-4273] - Make cube planner works for real-time streaming job
>[KYLIN-4187] - Building dimension dictionary using spark
>[KYLIN-4098] - Add cube auto-merge API
>
>Thanks to everyone who has contributed to this release.
>Here are the release notes:
>https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
>
>The commit to being voted upon:
>https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
>
>Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
>
>The artifacts to be voted on, including the source package and two
>pre-compiled binary packages are located here:
>https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
>
>The hash of the artifacts are as follows:
>apache-kylin-3.0.0-source-release.zip.sha256
>9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
>apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
>bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
>apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
>c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
>apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
>116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
>apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
>22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
>
>A staged Maven repository is available for review at:
>https://repository.apache.org/content/repositories/orgapachekylin-1070/
>
>Release artifacts are signed with the following key:
>https://people.apache.org/keys/committer/shaofengshi.asc
>
>Please vote on releasing this package as Apache Kylin 3.0.0.
>
>The vote is open for the next 72 hours and passes if a majority of
>at least three +1 PMC votes are cast.
>
>[ ] +1 Release this package as Apache Kylin 3.0.0
>[ ]  0 I don't feel strongly about it, but I'm okay with the release
>[ ] -1 Do not release this package because...
>
>
>Here is my vote:
>
>+1 (binding)
>
>Best regards,
>
>Shaofeng Shi 史少锋
>Apache Kylin PMC
>Email: shaofeng...@apache.org
>
>Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>Join Kylin user mail group: user-subscr...@kylin.apache.org
>Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re:[VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread nichunen 
+1 



Best regards,

 

Ni Chunen / George



On 12/11/2019 10:26，Xiaoyuan Gu wrote：
+1
mvn test passed

Bests，
Xiaoyuan Gu



At 2019-12-10 14:07:00, "ShaoFeng Shi"  wrote:
Hi all,

I have created a build for Apache Kylin 3.0.0, release candidate 1.

Changes highlights:
[KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
[KYLIN-4167] - Refactor streaming coordinator
[KYLIN-4273] - Make cube planner works for real-time streaming job
[KYLIN-4187] - Building dimension dictionary using spark
[KYLIN-4098] - Add cube auto-merge API

Thanks to everyone who has contributed to this release.
Here are the release notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121

The commit to being voted upon:
https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc

Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.

The artifacts to be voted on, including the source package and two
pre-compiled binary packages are located here:
https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/

The hash of the artifacts are as follows:
apache-kylin-3.0.0-source-release.zip.sha256
9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachekylin-1070/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/shaofengshi.asc

Please vote on releasing this package as Apache Kylin 3.0.0.

The vote is open for the next 72 hours and passes if a majority of
at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Kylin 3.0.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...


Here is my vote:

+1 (binding)

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org

[jira] [Created] (KYLIN-4293) Backport HBASE-22887 to Kylin HFileOutputFormat3

2019-12-10 Thread Shao Feng Shi (Jira) 
Shao Feng Shi created KYLIN-4293:


 Summary: Backport HBASE-22887 to Kylin HFileOutputFormat3
 Key: KYLIN-4293
 URL: https://issues.apache.org/jira/browse/KYLIN-4293
 Project: Kylin
  Issue Type: Improvement
  Components: Job Engine
Reporter: Shao Feng Shi






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KYLIN-4292) Use HFileOutputFormat3 in all places to replace HFileOutputFormat2

2019-12-10 Thread Shao Feng Shi (Jira) 
Shao Feng Shi created KYLIN-4292:


 Summary: Use HFileOutputFormat3 in all places to replace 
HFileOutputFormat2
 Key: KYLIN-4292
 URL: https://issues.apache.org/jira/browse/KYLIN-4292
 Project: Kylin
  Issue Type: Improvement
  Components: Job Engine
Reporter: Shao Feng Shi






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KYLIN-4291) Parallel segment building may causes WriteConflictException

2019-12-10 Thread Liang Geng (Jira) 
Liang Geng created KYLIN-4291:
-

 Summary: Parallel segment building may causes 
WriteConflictException
 Key: KYLIN-4291
 URL: https://issues.apache.org/jira/browse/KYLIN-4291
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v2.6.4
Reporter: Liang Geng


We noticed a segment refresh job at ERROR status at "*2019-12-10 02:27:15*". 
After a deep dive into kylin.log, we found that the WriteConflictException was 
raised when trying to update the last modified time for the dict.

 

(Sensitive content are consered)
{code:java}
2019-12-10 02:27:15,576 ERROR [Scheduler 1186524190 Job 
f76fb12e-477f-b404-e021-adaf520d04bb-321] common.HadoopShellExecutable:65 : 
error execute HadoopShellExecutable{id=f76fb12e-477f-b404-e021-adaf520d04bb-03, 
name=Build Dimension Dictionary, state=RUNNING}
org.apache.kylin.common.persistence.WriteConflictException: Overwriting 
conflict 
/dict/THIS_IS_CUBE_NAME/THIS_IS_COLUMN_NAME/010dbf72-52a9-e759-08a7-ed7cde0c6e0d.dict,
 expect old TS 1575916011421, but it is 1575916035573
at 
org.apache.kylin.storage.hbase.HBaseResourceStore.updateTimestampImpl(HBaseResourceStore.java:372)
at 
org.apache.kylin.common.persistence.ResourceStore.lambda$updateTimestampWithRetry$4(ResourceStore.java:443)
at 
org.apache.kylin.common.persistence.ExponentialBackoffRetry.doWithRetry(ExponentialBackoffRetry.java:52)
at 
org.apache.kylin.common.persistence.ResourceStore.updateTimestampWithRetry(ResourceStore.java:442)
at 
org.apache.kylin.common.persistence.ResourceStore.updateTimestampCheckPoint(ResourceStore.java:437)
at 
org.apache.kylin.common.persistence.ResourceStore.updateTimestamp(ResourceStore.java:432)
at 
org.apache.kylin.dict.DictionaryManager.updateExistingDictLastModifiedTime(DictionaryManager.java:197)
at 
org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:157)
at 
org.apache.kylin.dict.DictionaryManager.saveDictionary(DictionaryManager.java:339)
at 
org.apache.kylin.cube.CubeManager$DictionaryAssist.saveDictionary(CubeManager.java:1145)
at org.apache.kylin.cube.CubeManager.saveDictionary(CubeManager.java:1107)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:100)
at 
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
at 
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
at 
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
We have a glance at the codebase (Kylin-2.6.4), we found the last modified time 
of the dict (which has been built) will be updated at the step "Build Dimension 
Dictionary". The log blow shows the same dict is processed many times.
{code:java}
Liangs-MacBook-Pro:~ pwrliang$ cat 12-10.log |grep "has already been built, 
save it"|grep "THIS_IS_COLUMN_NAME"
2019-12-10 02:07:10,103 DEBUG [Scheduler 1186524190 Job 
5c4b2ee9-c8aa-9a69-ffb0-3a6811ad22a9-372] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:11,036 DEBUG [Scheduler 1186524190 Job 
593979b8-e80b-590f-335b-526c2fc49080-295] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:20,455 DEBUG [Scheduler 1186524190 Job 
160952f3-fef2-4fdf-c08a-77d49f830805-246] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:20,888 DEBUG [Scheduler 1186524190 Job 
69304c1c-cc02-3bbe-67d4-efdd3858e88a-234] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:21,634 DEBUG [Scheduler 1186524190 Job 
345b688c-7992-f2cf-ae06-835675f545ea-117] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:23,040 DEBUG [Scheduler 1186524190 Job 
0ea711ed-71b5-ae47-a34a-f0a27fd9d7cc-303] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been built, save it
2019-12-10 02:07:36,358 DEBUG [Scheduler 1186524190 Job 
6daf351f-5073-128f-8754-f499e345db54-308] cli.DictionaryGeneratorCLI:99 : Dict 
for 'THIS_IS_COLUMN_NAME' has already been buil

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Yaqian Zhang 
+1
mvn test passed

> 在 2019年12月10日，19:48，Chao Long  写道：
> 
> +1 real-time OLAP is coming!
> 
> mvn test passed
> 
> On Tue, Dec 10, 2019 at 3:26 PM yuzhang  wrote:
> 
>> expect it  +1
>> 
>> 
>> | |
>> yuzhang
>> |
>> |
>> Email:shifengdefan...@163.com
>> |
>> 
>> Signature is customized by Netease Mail Master
>> 
>> On 12/10/2019 14:07, ShaoFeng Shi wrote:
>> Hi all,
>> 
>> I have created a build for Apache Kylin 3.0.0, release candidate 1.
>> 
>> Changes highlights:
>> [KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
>> [KYLIN-4167] - Refactor streaming coordinator
>> [KYLIN-4273] - Make cube planner works for real-time streaming job
>> [KYLIN-4187] - Building dimension dictionary using spark
>> [KYLIN-4098] - Add cube auto-merge API
>> 
>> Thanks to everyone who has contributed to this release.
>> Here are the release notes:
>> 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
>> 
>> The commit to being voted upon:
>> 
>> https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
>> 
>> Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
>> 
>> The artifacts to be voted on, including the source package and two
>> pre-compiled binary packages are located here:
>> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
>> 
>> The hash of the artifacts are as follows:
>> apache-kylin-3.0.0-source-release.zip.sha256
>> 9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
>> apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
>> bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
>> apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
>> c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
>> apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
>> 116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
>> apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
>> 22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
>> 
>> A staged Maven repository is available for review at:
>> https://repository.apache.org/content/repositories/orgapachekylin-1070/
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/shaofengshi.asc
>> 
>> Please vote on releasing this package as Apache Kylin 3.0.0.
>> 
>> The vote is open for the next 72 hours and passes if a majority of
>> at least three +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Kylin 3.0.0
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not release this package because...
>> 
>> 
>> Here is my vote:
>> 
>> +1 (binding)
>> 
>> Best regards,
>> 
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofeng...@apache.org
>> 
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscr...@kylin.apache.org
>> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>>

[jira] [Created] (KYLIN-4290) Add file lock to kylin startup script to avoid starting multiple instances on one node

2019-12-10 Thread Zheng Yang (Jira) 
Zheng Yang created KYLIN-4290:
-

 Summary: Add file lock to kylin startup script to avoid starting 
multiple instances on one node
 Key: KYLIN-4290
 URL: https://issues.apache.org/jira/browse/KYLIN-4290
 Project: Kylin
  Issue Type: Improvement
  Components: Tools, Build and Test
Affects Versions: v3.0.0, v2.6.5
Reporter: Zheng Yang
 Fix For: v2.6.0


When we restart the Kylin instance, due to the slow start of kylin, some 
monitoring scripts fail to detect the surviving kylin, and will also start the 
kylin instance. As a result, there are multiple kylin instances on a node.
By adding a file lock to the startup script (kylin.sh), it is guaranteed that 
only one kylin instance will be started when concurrent startup on a node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Chao Long 
+1 real-time OLAP is coming!

mvn test passed

On Tue, Dec 10, 2019 at 3:26 PM yuzhang  wrote:

> expect it  +1
>
>
> | |
> yuzhang
> |
> |
> Email:shifengdefan...@163.com
> |
>
> Signature is customized by Netease Mail Master
>
> On 12/10/2019 14:07, ShaoFeng Shi wrote:
> Hi all,
>
> I have created a build for Apache Kylin 3.0.0, release candidate 1.
>
> Changes highlights:
> [KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
> [KYLIN-4167] - Refactor streaming coordinator
> [KYLIN-4273] - Make cube planner works for real-time streaming job
> [KYLIN-4187] - Building dimension dictionary using spark
> [KYLIN-4098] - Add cube auto-merge API
>
> Thanks to everyone who has contributed to this release.
> Here are the release notes:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005&styleName=&projectId=12316121
>
> The commit to being voted upon:
>
> https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc
>
> Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.
>
> The artifacts to be voted on, including the source package and two
> pre-compiled binary packages are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/
>
> The hash of the artifacts are as follows:
> apache-kylin-3.0.0-source-release.zip.sha256
> 9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
> apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
> bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
> apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
> c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
> apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
> 116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
> apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
> 22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachekylin-1070/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/shaofengshi.asc
>
> Please vote on releasing this package as Apache Kylin 3.0.0.
>
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Kylin 3.0.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofeng...@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscr...@kylin.apache.org
> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>

[jira] [Created] (KYLIN-4289) java.lang.NoSuchMethodError: com.google.common.hash.Hasher.putString

2019-12-10 Thread Kanivel (Jira) 
Kanivel created KYLIN-4289:
--

 Summary: java.lang.NoSuchMethodError: 
com.google.common.hash.Hasher.putString
 Key: KYLIN-4289
 URL: https://issues.apache.org/jira/browse/KYLIN-4289
 Project: Kylin
  Issue Type: Bug
Affects Versions: v2.6.4
Reporter: Kanivel
 Attachments: Screenshot from 2019-12-05 12-55-22.png, Screenshot from 
2019-12-05 12-55-47.png

Hi Kylin Developers,

Kylin Version:2.6.4
Hadoop Version:3.1

   I got mapreduce exception when building cube at step 
3.When I check Job history server logs,it shows below error.


2019-12-10 09:42:54,655 INFO [main] org.apache.kylin.engine.mr.KylinMapper: Do 
cleanup, available memory: 616m
2019-12-10 09:42:54,656 FATAL [Thread-13] 
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
Thread[Thread-13,5,main] threw an Error.  Shutting down now...
java.lang.NoSuchMethodError: 
com.google.common.hash.Hasher.putString(Ljava/lang/CharSequence;)Lcom/google/common/hash/Hasher;
at 
org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper$CuboidStatCalculator.putRowKeyToHLLNew(FactDistinctColumnsMapper.java:385)
at 
org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper$CuboidStatCalculator.run(FactDistinctColumnsMapper.java:419)
at java.lang.Thread.run(Thread.java:748)
2019-12-10 09:42:54,660 INFO [Thread-13] org.apache.hadoop.util.ExitUtil: 
Exiting with status -1: ExitException




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Cube not allowed for parallel Building

2019-12-10 Thread ShaoFeng Shi 
"[xyz[FULL_BUILD],"

"FULL_BUILD" means this is a full build, so only 1 segment is created, so
no need for parallel building (for this cube).

If you want to build multiple segments, please select a "partition date
column" when you define the data model. Then your cube can have multiple
partitions, they can be built in parallel.

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org




codingfor...@126.com  于2019年12月10日周二 下午6:26写道：

> Hi, Akashi_07, When *kylin.cube.cubeplanner.enabled* is true (default is
> true), when the cube has a segment that has not yet completed the build, it
> is not allowed to build another segment.
>
> The only way to bypass this check is to set kylin.cube.cubeplanner.enabled
> to false. Please decide whether to set this parameter to false if you know
> the role of cube planner. Please check:
> http://kylin.apache.org/docs23/howto/howto_use_cube_planner.html
>
>
> 在 2019年12月10日，18:07，Akashi_07  写道：
>
> Hello,
>
> I have created a data model and a cube based on it. But when I try to build
> the cube I get the following error.
>
> Lets assume my cube name is xyz.
>
> error is as follows
>
> The cube xyz hads segments [xyz[FULL_BUILD], but none of them is ready.
> It's
> not allowed for parallel building.
>
> How do I solve this? When do we get this error? And also what does "Not
> allowed for parallel building" actually mean.
>
> This is the first instance of cube build for this, so there should be not
> be
> any segments related to it if i'm not wrong. But i still get this error.
> Please help me.
>
> --
> Sent from: http://apache-kylin.74782.x6.nabble.com/
>
>
>

Re: Cube not allowed for parallel Building

2019-12-10 Thread codingfor...@126.com 
Hi, Akashi_07, When kylin.cube.cubeplanner.enabled is true (default is true), 
when the cube has a segment that has not yet completed the build, it is not 
allowed to build another segment.

The only way to bypass this check is to set kylin.cube.cubeplanner.enabled to 
false. Please decide whether to set this parameter to false if you know the 
role of cube planner. Please check: 
http://kylin.apache.org/docs23/howto/howto_use_cube_planner.html 




> 在 2019年12月10日，18:07，Akashi_07  写道：
> 
> Hello,
> 
> I have created a data model and a cube based on it. But when I try to build
> the cube I get the following error.
> 
> Lets assume my cube name is xyz.
> 
> error is as follows
> 
> The cube xyz hads segments [xyz[FULL_BUILD], but none of them is ready. It's
> not allowed for parallel building. 
> 
> How do I solve this? When do we get this error? And also what does "Not
> allowed for parallel building" actually mean. 
> 
> This is the first instance of cube build for this, so there should be not be
> any segments related to it if i'm not wrong. But i still get this error.
> Please help me. 
> 
> --
> Sent from: http://apache-kylin.74782.x6.nabble.com/

Cube not allowed for parallel Building

2019-12-10 Thread Akashi_07 
Hello,

I have created a data model and a cube based on it. But when I try to build
the cube I get the following error.

Lets assume my cube name is xyz.

error is as follows
 
The cube xyz hads segments [xyz[FULL_BUILD], but none of them is ready. It's
not allowed for parallel building. 

How do I solve this? When do we get this error? And also what does "Not
allowed for parallel building" actually mean. 

This is the first instance of cube build for this, so there should be not be
any segments related to it if i'm not wrong. But i still get this error.
Please help me. 

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

[jira] [Created] (KYLIN-4288) 查询样例数据抛出异常

2019-12-10 Thread zjp (Jira) 
zjp created KYLIN-4288:
--

 Summary: 查询样例数据抛出异常
 Key: KYLIN-4288
 URL: https://issues.apache.org/jira/browse/KYLIN-4288
 Project: Kylin
  Issue Type: Bug
  Components: Query Engine
Affects Versions: v2.6.4, v2.5.2
 Environment: centOS7.2  
jdk-8u191-linux-x64.tar.gz
zookeeper-3.4.5.tar.gz
hadoop-2.7.7.tar.gz
hbase-1.1.1-bin.tar.gz , hbase-1.5.0-bin.tar.gz
apache-hive-1.2.1-bin.tar.gz
apache-kylin-2.5.2-bin-hbase1x.tar.gz , apache-kylin-2.6.4-bin-hbase1x.tar.gz
Reporter: zjp
 Attachments: kylin_conf.txt, kylin_environment.txt, kylin_error1.png, 
kylin_error2.png, kylin_error3.png, kylin_error_log.txt

感谢您百忙之中路过此地,请帮忙看一下:):
cube构建完成后,运行任意语句,查询出现:
No registered coprocessor service found for name CubeVisitService in region 
KYLIN_QLF9A6GPM7,,1575951985092.4ff8b4d4f0ddcfef105bf518e25a30cb.

从网上查到需要更新coprocessor:
$KYLIN_HOME/bin/kylin.sh 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI 
$KYLIN_HOME/lib/kylin-coprocessor-*.jar all

原来是用的kylin-2.5.2运行报错:
Exception in thread "main" java.io.FileNotFoundException: /root/all (No such 
file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.uploadCoprocessorJar(DeployCoprocessorCLI.java:455)
at 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:139)

从网上查到这可能是2.5.2的bug,所以更换kylin版本2.6.4,更换之后,查询依然报错,再次运行
 $KYLIN_HOME/bin/kylin.sh 
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI 
$KYLIN_HOME/lib/kylin-coprocessor-*.jar all

2019-12-10 15:16:35,611 INFO  [main] util.DeployCoprocessorCLI:136 : Processed 
time: 104
2019-12-10 15:16:35,611 INFO  [main] util.DeployCoprocessorCLI:137 : Processed 
tables count: 0
2019-12-10 15:16:35,612 INFO  [main] util.DeployCoprocessorCLI:138 : Processed 
tables: []
2019-12-10 15:16:35,612 ERROR [main] util.DeployCoprocessorCLI:139 : Failed 
tables count: 0
2019-12-10 15:16:35,612 ERROR [main] util.DeployCoprocessorCLI:140 : Failed 
tables : []
这个不知道是不是运行成功了,但是查询依然还是报错,尝试过重启,也尝试过更换hbase版本,
hbase1.1.1更换hbase1.5.0后错误依旧存在
No registered coprocessor service found for name CubeVisitService in region 
KYLIN_QLF9A6GPM7,,1575951985092.4ff8b4d4f0ddcfef105bf518e25a30cb.


 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KYLIN-4287) SegmentPruner cannot prune segment with "IN" or "OR" CompareTupleFilter

2019-12-10 Thread yangzhang (Jira) 
yangzhang created KYLIN-4287:


 Summary: SegmentPruner cannot prune segment with "IN" or "OR" 
CompareTupleFilter
 Key: KYLIN-4287
 URL: https://issues.apache.org/jira/browse/KYLIN-4287
 Project: Kylin
  Issue Type: Improvement
  Components: Query Engine
Reporter: yangzhang
Assignee: yangzhang


repro steps：

build 2 segs in a cube, [2012-01-01 - 2012-01-03]  And [2012-01-03 - 2012-01-05]

query: select sum(PRICE) from KYLIN_SALES
where "KYLIN_SALES"."PART_DT" = DATE '2012-01-01'
 OR "KYLIN_SALES"."PART_DT" = DATE '2020-07-02'

Expected results：segment[2012-01-03 - 2012-01-05] should not be scaned, should 
be pruned by SegmentPruner

 

but i found in method 'org.apache.kylin.cube.common.SegmentPruner#satisfy'

when filter's operator is "In", when satisfy segment[2012-01-03 - 2012-01-05] 
,the function return true



--
This message was sent by Atlassian Jira
(v8.3.4#803005)