答复: [VOTE] Release apache-kylin-4.0.0-beta (RC2)

2021-02-03 Thread Wang rupeng 
+1
Mvn clean test passed  in my environment



---
Best wishes,
Rupeng Wang



发件人: Yaqian Zhang 
日期: 星期四, 2021年2月4日 14:05
收件人: dev@kylin.apache.org 
主题: Re: [VOTE] Release apache-kylin-4.0.0-beta (RC2)
+1
Mvn clean test passed.
Happy path on EMR5.x passed.

> 在 2021年2月3日，21:31，Xiaoxiang Yu  写道：
>
> Hi all,
>
>
>
>
> I have created a build for Apache Kylin 4.0.0-beta, release candidate 2. 
> Please note that this release is built on kylin-on-parquet-v2 branch.
>
>
>
>
> Changes highlights:
>
>
>
>
> [KYLIN-4857] - Refactor system cube for kylin4
>
> [KYLIN-4842] - Supports grouping sets function for Kylin 4
>
> [KYLIN-4829] - Support to use thread-level SparkSession to execute query
>
> [KYLIN-4813] - Refine spark logger for Kylin 4 build engine
>
> [KYLIN-4858] - Support Kylin4 deployment on CDH 6.X
>
> [KYLIN-4818] - Calculate cuboid statistics in Kylin 4
>
> [KYLIN-4817] - Refine Cube Migration Tool for Kylin4
>
>
>
>
> Thanks to everyone who has contributed to this release.
>
> Here are the release notes:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12348723
>
>
>
>
> The commit to being voted upon:
>
> https://github.com/apache/kylin/commit/546381392c55136310f591969e5a3f3db6074988
>
> Its hash is 546381392c55136310f591969e5a3f3db6074988.
>
>
>
>
> The artifacts to be voted on, including the source package and one 
> pre-compiled binary packages are located here:
>
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-4.0.0-beta-rc2/
>
>
>
>
> The hash of the artifacts are as follows:
>
> apache-kylin-4.0.0-beta-source-release.zip.sha256
>
> d5837ed106c4e10f71a6b7a6324520afdd536c10f5afd86bf6c98d734a3e64a8
>
>
>
>
> apache-kylin-4.0.0-beta-bin.tar.gz.sha256
>
> e86ab855533104c2eac8fae53dc067d425ab263cebf558c666260a3ceb336b16
>
>
>
>
>
>
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/content/repositories/orgapachekylin-1087/
>
>
>
>
>
>
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/xxyu.asc
>
>
>
>
> Please vote on releasing this package as Apache Kylin 4.0.0-beta.
>
>
>
>
> The vote is open for the next 72 hours and passes if a majority of
>
> at least three +1 binding votes are cast.
>
>
>
>
> [ ] +1 Release this package as Apache Kylin 4.0.0-beta
>
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>
> [ ] -1 Do not release this package because...
>
>
>
>
> Here is my vote:
>
> +1 (binding)
> --
> Best wishes to you !
> From ：Xiaoxiang Yu

答复: [VOTE] Release apache-kylin-4.0.0-beta (RC1)

2021-01-31 Thread Wang rupeng 
+1  from my side
mvn test passed


---
Best wishes,
Rupeng Wang



发件人: Yaqian Zhang 
日期: 星期日, 2021年1月31日 12:53
收件人: dev@kylin.apache.org 
主题: Re: [VOTE] Release apache-kylin-4.0.0-beta (RC1)
+1
Mvn clean test passed.
Happy path passed on CDH6, EMR5, EMR6.


> 在 2021年1月31日，10:03，Zhichao Zhang <441586...@qq.com> 写道：
>
> +1 from my side.
>
> Test passed on HDP2, CDH 5.
>
>
> Best regards,
> Zhichao Zhang

答复: [ANNOUNCE] new committer: Xiao Chu

2020-11-05 Thread Wang rupeng 
Congratulation, Xiao Chu!

---
Best wishes,
Rupeng Wang



发件人: Rui Zou 
日期: 星期四, 2020年11月5日 17:46
收件人: dev@kylin.apache.org , crow...@163.com 

主题: Re: [ANNOUNCE] new committer: Xiao Chu
Congratulations, Xiao!
在 2020/11/4 下午9:45，“ShaoFeng Shi” 写入:

Hello Kylin developers,

The Project Management Committee (PMC) for Apache Kylin
has invited Xiao Chu (初晓, email: crow...@163.com) to become a committer and
we are pleased
to announce that he has accepted.

In the past years, Xiao has contributed 21 commits in the master branch
with an additional 1 in the
document branch, resolved 21 JIRA issues. Besides, he published 5 articles
on Kylin and gave 1
speech in the community.

Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.

Let's warmly welcome Xiao as a Kylin committer!

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: [VOTE] Release apache-kylin-3.1.1 (RC1)

2020-10-14 Thread Wang rupeng 
+1 
mvn test passed 

> 2020年10月15日 10:01，Yaqian Zhang  写道：
> 
> +1
> Happy path passed on CDH57.
> 
>> 在 2020年10月14日，23:21，Zhichao Zhang <441586...@qq.com> 写道：
>> 
>> +1 from my side.
>> Happy path passed on HDP3.
>> 
>> Best regards,
>> Zhichao Zhang
>

Re: [VOTE] Release apache-kylin-3.0.1 (RC2)

2020-02-16 Thread Wang rupeng 
+1 
Mvn test passed


---
Best wishes,
Rupeng Wang
 


在 2020/2/16 13:45，“Yaqian Zhang” 写入:

+1

mvn test passed

> 在 2020年2月16日，10:45，Xiaoyuan Gu  写道：
> 
> +1
> mvn test passed
> 
> 
> 
> 
> Bests,
> Xiaoyuan Gu
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> At 2020-02-14 21:12:19, "George Ni"  wrote:
>> Hi all,
>> 
>> 
>> 
>> I have created a build for Apache Kylin 3.0.1, release candidate 2.
>> 
>> 
>> 
>> Changes highlights:
>> 
>> [KYLIN-4374] - Fix security issues reported by code analysis platform 
LGTM
>> 
>> [KYLIN-4333] - Memory leak on job server
>> 
>> [KYLIN-4297] - Build cube throw NPE error when partition column is not 
set
>> in JDBC Data Source
>> 
>> 
>> 
>> Thanks to everyone who has contributed to this release.
>> 
>> Here are the release notes:
>> 
>> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12346958==12316121
>> 
>> 
>> 
>> The commit to being voted upon:
>> 
>> 
https://github.com/apache/kylin/commit/818c21cc9c2a995a58d85497a8649ebcf11a975c
>> 
>> 
>> 
>> Its hash is 818c21cc9c2a995a58d85497a8649ebcf11a975c.
>> 
>> 
>> 
>> The artifacts to be voted on, including the source package and four
>> 
>> pre-compiled binary packages are located here:
>> 
>> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.1-rc2/
>> 
>> 
>> 
>> The hash of the artifacts are as follows:
>> 
>> apache-kylin-3.0.1-source-release.zip.sha256
>> 
>> 78cceaba517b7c62f0dfa6c4610144d7913051588d8a99083a89cf74e37a724f
>> 
>> apache-kylin-3.0.1-bin-hbase1x.tar.gz.sha256
>> 
>> 9bf9db5e9152755928f08ebe9fa2dc0d2647feb56604ceb06343e95fd362dc50
>> 
>> apache-kylin-3.0.1-bin-cdh57.tar.gz.sha256
>> 
>> f1f60cbc525942c7c044aee212455eaabb092da0e99bede11501920b8258ba42
>> 
>> apache-kylin-3.0.1-bin-hadoop3.tar.gz.sha256
>> 
>> b405c2dd0cd93400c18c46bbb36f601cf58cb5e48cc760f422daa4bcc7899084
>> 
>> apache-kylin-3.0.1-bin-cdh60.tar.gz.sha256
>> 
>> 5ef70897392072ee74d0909971a197413a64e677f89952f1b23abf6969c3443d
>> 
>> 
>> 
>> A staged Maven repository is available for review at:
>> 
>> https://repository.apache.org/content/repositories/orgapachekylin-1074/
>> 
>> 
>> 
>> Release artifacts are signed with the following key:
>> 
>> https://people.apache.org/keys/committer/nic.asc
>> 
>> 
>> 
>> Please vote on releasing this package as Apache Kylin 3.0.1.
>> 
>> 
>> 
>> The vote is open for the next 72 hours and passes if a majority of
>> 
>> at least three +1 PMC votes are cast.
>> 
>> 
>> 
>> [ ] +1 Release this package as Apache Kylin 3.0.1
>> 
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> 
>> [ ] -1 Do not release this package because...
>> 
>> 
>> 
>> 
>> 
>> Here is my vote:
>> 
>> 
>> 
>> +1 (binding)
>> 
>> -- 
>> 
>> -
>> 
>> Best regards,
>> 
>> 
>> 
>> Ni Chunen / George

Re: [VOTE] Release apache-kylin-2.6.5 (RC2)

2020-02-16 Thread Wang rupeng 
+1  mvn test passed

---
Best wishes,
Rupeng Wang
 


在 2020/2/16 13:45，“Yaqian Zhang” 写入:

+1
mvn test passed

> 在 2020年2月16日，10:45，Xiaoyuan Gu  写道：
> 
> +1
> mvn test passed
> 
> 
> 
> 
> Bests,
> Xiaoyuan Gu
> 
> 
> 
> 
> 
> At 2020-02-14 21:21:30, "George Ni"  wrote:
>> Hi all,
>> 
>> 
>> 
>> I have created a build for Apache Kylin 2.6.5, release candidate 2.
>> 
>> 
>> 
>> Changes highlights:
>> 
>> [KYLIN-4374] - Fix security issues reported by code analysis platform 
LGTM
>> 
>> [KYLIN-4291] - Parallel segment building may causes 
WriteConflictException
>> 
>> [KYLIN-4263] - Inappropriate exception handling causes job stuck on 
running
>> status
>> 
>> [KYLIN-4169] - Too many logs during DataModelManager’s initiation, cause
>> the first RESTful API to hang for a long time
>> 
>> 
>> 
>> Thanks to everyone who has contributed to this release.
>> 
>> Here are the release notes:
>> 
>> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12346281==12316121
>> 
>> 
>> 
>> The commit to being voted upon:
>> 
>> 
https://github.com/apache/kylin/commit/73d42edec5f6492b3d3ffc222c26dce4bdfe7263
>> 
>> 
>> Its hash is 73d42edec5f6492b3d3ffc222c26dce4bdfe7263.
>> 
>> 
>> 
>> The artifacts to be voted on, including the source package and four
>> 
>> pre-compiled binary packages are located here:
>> 
>> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.6.5-rc2/
>> 
>> 
>> 
>> The hash of the artifacts are as follows:
>> 
>> apache-kylin-2.6.5-source-release.zip.sha256
>> 
>> 32fb722a58ed49318dc5f72da429506238ee2089170236c742f22240d93080a3
>> 
>> apache-kylin-2.6.5-bin-hbase1x.tar.gz.sha256
>> 
>> f8e04fafa7e63cdeb28aa4e8c35bc2f54456888bef37d4cca060c424be722e1e
>> 
>> apache-kylin-2.6.5-bin-cdh57.tar.gz.sha256
>> 
>> 71deb9bc84e5b75320e2c8ca358449f808ef2decbee11679e7f4339abeeb3d3f
>> 
>> apache-kylin-2.6.5-bin-hadoop3.tar.gz.sha256
>> 
>> e77218204800a7b6be42e90e0fcb03e6e8458d600125e44bc112a8a359291fc4
>> 
>> apache-kylin-2.6.5-bin-cdh60.tar.gz.sha256
>> 
>> 2f7003eba8b528198aa7f886be04ecb75e86ece9e2e67dba4c15a89fb8817f7a
>> 
>> 
>> 
>> A staged Maven repository is available for review at:
>> 
>> https://repository.apache.org/content/repositories/orgapachekylin-1075/
>> 
>> 
>> 
>> Release artifacts are signed with the following key:
>> 
>> https://people.apache.org/keys/committer/nic.asc
>> 
>> 
>> 
>> Please vote on releasing this package as Apache Kylin 2.6.5
>> 
>> 
>> 
>> The vote is open for the next 72 hours and passes if a majority of
>> 
>> at least three +1 PMC votes are cast.
>> 
>> 
>> 
>> [ ] +1 Release this package as Apache Kylin 2.6.5
>> 
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> 
>> [ ] -1 Do not release this package because...
>> 
>> 
>> 
>> Here is my vote:
>> 
>> 
>> 
>> +1 (binding)
>> 
>> -- 
>> 
>> -
>> 
>> Best regards,
>> 
>> 
>> 
>> Ni Chunen / George

Re: RESTful API文档咨询

2020-01-15 Thread Wang rupeng 
Hi,
Agreed. 
As far as I know, if we set one column to ‘derived’, it means that the 
derived column corresponds to the primary key of the lookup table and it can 
reduce the number of the cuboids. You can get more information from the 
fllowing article.
http://kylin.apache.org/docs/howto/howto_optimize_cubes.html

By the way, it seems like the pictures are sent failed. 
---
Best wishes,
Rupeng Wang

在 2020/1/15 17:44，“Yaqian Zhang” 写入:

Hi Yi Chen:

Here is my understanding:

The cube desc in kylin official website comes from the earlier version of 
kylin, which is different from the cube that can be designed by the current 
version of kylin. I will modify it.

In the current version, 3 and 4 should not be the case.

The first question is that when a dimension is designed as a derived 
dimension, it means that the column value can be derived from the primary key, 
so the dimension will not be added to the calculation in fact, and only the 
normal type dimension will appear in the column.

The second question is that the name of the dimension can be customized, so 
it can be independent of the actual column name in the derived array.

> 在 2020年1月15日，11:34，Yi Chen  写道：
> 
> hi
> 我们公司正在尝试在Kylin上构建支持多维分析的BI系统，其中需要从Kylin的cube_desc 
API接口中，获得cube的维度信息，与我们自己元数据中管理的统一维度进行关联，供后续的分析使用。
> 但是在使用 cube_desc这个API的时候遇到了一些问题：
> 1. 我们在Kylin中将维表中的维度设置成Derived，这时候该维度在 
cube_desc的dimensions字段中，column会被设置成null，同时derived字段的数组中会出现该维度的名称。这个字段值的逻辑具体是什么？
> 2. 我在官网文档http://kylin.apache.org/docs/howto/howto_use_restapi.html 
中，看到了如下示例，derived数组中的值与维度name没有关系，这种情况会在什么情况下出现？
> 
>3. 
derived是个数组，官网示例中也出现了derived数组中有多个值的情况，在Kylin系统中我没法把这个情况配置出来。请问在什么情况下，derived数组中会有不止一个值出现？
> 
> 4. 官网中column字段也出现了数组类型，里面存在多个值，这个是在什么情况下会出现的？
> 
>   
> 谢谢。

Re: [Discuss] Reposition Kylin as "Analytical Data warehouse for big data"

2020-01-12 Thread Wang rupeng 
+1
Aggreed. Kylin has come a long way and now plays more than just an OLAP engine.

在 2020/1/13 12:35，“Yaqian Zhang” 写入:

Agreed.

With the Kylin becoming more and more powerful, it has the ability of data 
warehouse.
More and more users use it as a data warehouse.

> 在 2020年1月13日，11:06，Xiaoyuan Gu  写道：
> 
> +1
> Agreed. As Kylin has been equipped with plenty of "new" functionalities 
and is capable to suit various roles in analyzing data in large scale, a proper 
tag will definitely be much helpful not only to new users who are seeking for a 
suitable analyzing tool, but also to old users who want to explore 
possibilities in handling new scenarios with Kylin.
> 
> 
> Bests,
> Xiaoyuan Gu
> 
> 
> 
> At 2020-01-12 20:32:12, "ShaoFeng Shi"  wrote:
>> Hello, Kylin developers and users, HAPPY NEW YEAR 2020!
>> 
>> In last month, we released Kylin 3.0, with the new Real-time streaming
>> feature and a Lambda architecture. This allows our users to host only one
>> system for both batch and real-time analytics, and then can query batch 
and
>> streaming data together.
>> 
>> If you look at Kylin's home page, its slogan is still the "OLAP Engine 
for
>> Big data", which was made 5 years ago when it was born. While today,
>> Kylin's capability has been verified beyond an "OLAP engine". I visited
>> many Kylin users in China, US, Euro in last year, and have got many
>> different scenarios:
>> 
>> 1. eBay initiated the Kylin project to offload analytical workloads from
>> Teradata to Hadoop; Kylin serves the online queries with high performance
>> and high availability. Till today, Kylin serves millions of queries every
>> day, most are in < 1 seconds;
>> 2. China Unionpay and CPIC use Kylin to replace IBM Cognos cubes. One 
Kylin
>> cube replaced more than 100 Cognos cubes, with better building 
performance
>> and query performance.
>> 3. China Construction Bank uses Hadoop + Kylin to offload the Greenplum.
>> Some systems have been migrated to Kylin successfully.
>> 4. Yum (KFC) and several other users are using Kylin to replace Microsoft
>> SSAS.
>> 5. Meituan, Ctrip, JD, Didi, Xiao Mi, Huawei, OLX group, autohome.com.cn,
>> Xactly, and many others are using Kylin as the platform of their DaaS 
(Data
>> as a Service), providing data service to their thousands of internal
>> analysts and tens of thousands of external tenants.
>> 
>> Now let's look at the definition of Data warehouse [1]:
>> 
>> "*A data warehouse is a subject-oriented, integrated, time-variant and
>> non-volatile collection of data in support of management's 
decision-making
>> process.*"
>> 
>> In Kylin, each model/cube is created for a certain subject; Kylin
>> integrates well with Hive, Hadoop, Spark, Kafka, and other systems; Kylin
>> incremental loads the data by time, build the cube and then save as
>> segments (partitions), and they are non-volatile unless you refresh them;
>> During the analysis (roll-up, drill-down, etc), the data is always
>> consistent. Kylin provides SQL interface and JDBC/ODBC/HTTP API for you 
to
>> easily connect from BI/visualization tools like Tableau and others.
>> 
>> All in all, you can see that users are using Kylin not just as a SQL
>> engine, but also as an Analytical Data Warehouse, for very large scale 
data
>> (PB scale). In the world of big data, Kylin is unique. Its design is
>> elegant, its architecture is scalable and pluggable.  In order to give
>> Kylin more visibility and can be discovered by more people, I propose to
>> change Kylin's position/slogan from the "OLAP engine for big data" to
>> "Analytical Data warehouse for big data".
>> 
>> Please feel free to share your comments.
>> 
>> [1] 
https://www.1keydata.com/datawarehousing/data-warehouse-definition.html
>> 
>> Best regards,
>> 
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofeng...@apache.org
>> 
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscr...@kylin.apache.org
>> Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: [Announce] Apache Kylin 3.0.0 released

2019-12-23 Thread Wang rupeng 
Congratulations! 

---
Best wishes,
Rupeng Wang
 


在 2019/12/24 12:08，“Xiaoyuan Gu” 写入:

Big congrats! Looking forward to seeing Kylin embraces more 
state-of-the-art features. Kudos to all contributors!


Bests,
Xiaoyuan



At 2019-12-20 20:45:16, "ShaoFeng Shi"  wrote:
>The Apache Kylin team is pleased to announce the immediate availability of
>the 3.0.0 release.
>
>This is the GA release of Kylin’s next generation after 2.x, with the new
>real-time OLAP feature, Kylin can query streaming data with sub-second
>latency. All of the
> changes in this release can be found in:
>https://kylin.apache.org/docs/release_notes.html
>
>
>You can download the source release and binary packages from Apache Kylin's
>download page:https://kylin.apache.org/download/
>
>
>Apache Kylin is an open-source Distributed Analytics Engine designed to
>provide SQL interface and multi-dimensional analysis (OLAP) on Apache
>Hadoop, supporting extremely
> large datasets.
>
>
>Apache Kylin lets you query massive dataset at sub-second latency in 3
>steps:
>1. Identify a star schema or snowflake schema data set on Hadoop.
>2. Build Cube on Hadoop.
>3. Query data with ANSI-SQL and get results in sub-second, via ODBC, JDBC
>or RESTful API.
>
>
>Thanks to everyone who has contributed to this release.
>
>
>We welcome your help and feedback. For more information on how to report
>problems, and to get involved, visit the project website at
>https://kylin.apache.org/
>
>Best regards,
>
>Shaofeng Shi 史少锋
>Apache Kylin PMC
>Email: shaofeng...@apache.org
>
>Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>Join Kylin user mail group: user-subscr...@kylin.apache.org
>Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: Need help on building a cube with Spark

2019-12-11 Thread Wang rupeng 
Hi, 
What's the version of Kylin you are using? I try to locate the error 
position in the code, but I cannot find any function named loadCubeInstance() 
or loadAllCubeInstance() in CubeManager. Besides that, It seems like you are 
using google storage, we haven't test kylin in google cloud. Maybe you can tell 
us more environment details, like Hadoop version, etc.


---
Best wishes,
Rupeng Wang
 


在 2019/12/12 12:04，“Lingaraja A J” 写入:

Hi team,

Greetings for the day!

I'm using Kylin to build an OLAP cube system for my project. I have create 
a cube with engine type set to Spark and am trying to build it. It consists of 
11 steps among which Spark code runs on step 6. At this step, it's throwing the 
following error:

Running org.apache.kylin.engine.spark.SparkCubingByLayer -hiveTable 
default.kylin_intermediate_table -output 
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/cuboid/
 -input 
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/kylin_intermediate_table
 -segmentId 9047b119-14ad-ecb8-bb87-c76d255907df -metaUrl 
kylin_metadata@hdfs,path=gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/metadata
 -cubename cube
Exception in thread "main" java.lang.NoSuchMethodError: 
org.apache.kylin.common.persistence.ResourceStore.getResource(Ljava/lang/String;Ljava/lang/Class;Lorg/apache/kylin/common/persistence/Serializer;)Lorg/apache/kylin/common/persistence/RootPersistentEntity;
at 
org.apache.kylin.cube.CubeManager.loadCubeInstance(CubeManager.java:682)
at 
org.apache.kylin.cube.CubeManager.loadAllCubeInstance(CubeManager.java:671)
at org.apache.kylin.cube.CubeManager.(CubeManager.java:128)
at org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:97)
at 
org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:137)
at 
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
The command is:
export HADOOP_CONF_DIR=/etc/hadoop/conf && /usr/lib/spark/bin/spark-submit 
--class org.apache.kylin.common.util.SparkEntry  --conf 
spark.executor.instances=40  --conf spark.yarn.queue=default  --conf 
spark.history.fs.logDirectory=hdfs:///kylin/spark-history  --conf 
spark.io.compression.codec=org.apache.spark.io.SnappyCompressionCodec  --conf 
spark.master=yarn  --conf spark.hadoop.yarn.timeline-service.enabled=false  
--conf spark.executor.memory=4G  --conf spark.eventLog.enabled=true  --conf 
spark.eventLog.dir=hdfs:///kylin/spark-history  --conf 
spark.yarn.executor.memoryOverhead=1024  --conf spark.driver.memory=2G  --conf 
spark.shuffle.service.enabled=true --jars /etc/kylin/lib/kylin-job-2.6.4.jar 
/etc/kylin/lib/kylin-job-2.6.4.jar -className 
org.apache.kylin.engine.spark.SparkCubingByLayer -hiveTable 
default.kylin_intermediate_sol_v3_new_9047b119_14ad_ecb8_bb87_c76d255907df 
-output 
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/cuboid/
 -input 
gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/kylin_intermediate_table
 -segmentId 9047b119-14ad-ecb8-bb87-c76d255907df -metaUrl 
kylin_metadata@hdfs,path=gs://bucket/kylin/kylin_metadata/kylin-b52a579f-3084-1ced-3544-fdf5b3eccb8a/model/metadata
 -cubename cube
   at 
org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:347)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

2019-12-10 Thread Wang rupeng 
+1
mvn test passed

---
Best wishes,
Rupeng Wang



发件人: "codingfor...@126.com" 
答复: 
日期: 2019年12月11日 星期三 10:44
收件人: 
主题: Re: [VOTE] Release apache-kylin-3.0.0 (RC1)

+1
mvn test passed
[cid:7D3EAE76-AF2E-4EFA-97B5-6220180DA6F6@hz.ali.com]


在 2019年12月11日，10:26，nichunen mailto:n...@apache.org>> 写道：

+1



Best regards,



Ni Chunen / George



On 12/11/2019 10:26，Xiaoyuan 
Gumailto:njguxiaoy...@163.com>> wrote：
+1
mvn test passed

Bests，
Xiaoyuan Gu



At 2019-12-10 14:07:00, "ShaoFeng Shi" 
mailto:shaofeng...@apache.org>> wrote:
Hi all,

I have created a build for Apache Kylin 3.0.0, release candidate 1.

Changes highlights:
[KYLIN-4258] - Real-time OLAP may return an incorrect result for some case
[KYLIN-4167] - Refactor streaming coordinator
[KYLIN-4273] - Make cube planner works for real-time streaming job
[KYLIN-4187] - Building dimension dictionary using spark
[KYLIN-4098] - Add cube auto-merge API

Thanks to everyone who has contributed to this release.
Here are the release notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345005==12316121

The commit to being voted upon:
https://github.com/apache/kylin/commit/c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc

Its hash is c75242a9b55fd57a3a58d92a2dfa9f21cfe4eebc.

The artifacts to be voted on, including the source package and two
pre-compiled binary packages are located here:
https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-rc1/

The hash of the artifacts are as follows:
apache-kylin-3.0.0-source-release.zip.sha256
9224742a87750b8d127c5031c03f3716e3af732c9805a6d0c64871605704f6c0
apache-kylin-3.0.0-bin-hbase1x.tar.gz.sha256
bdeddee3eb453c139eabaa2ce7ebd5d14f72d5ac48e5a64636aba2ed7357dda9
apache-kylin-3.0.0-bin-cdh57.tar.gz.sha256
c2ae9498f61edbacb6dae5fc32e2c4ea14539ef6d906d53194492e042c80185f
apache-kylin-3.0.0-bin-hadoop3.tar.gz.sha256
116ba002d794058bd34bd05989da2c3a7ff87cf67d3647d2f1cc5b5717d445f6
apache-kylin-3.0.0-bin-cdh60.tar.gz.sha256
22a0701b5a03a8d40c8b1be4fe4acb1ff2550a18c52d509b592d59ef5a094f7e

A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachekylin-1070/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/shaofengshi.asc

Please vote on releasing this package as Apache Kylin 3.0.0.

The vote is open for the next 72 hours and passes if a majority of
at least three +1 PMC votes are cast.

[ ] +1 Release this package as Apache Kylin 3.0.0
[ ]  0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...


Here is my vote:

+1 (binding)

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Email: shaofeng...@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: kylin2.6.4流式构建不能与hdp3.1.4.0使用(issue3970)

2019-10-24 Thread Wang rupeng 
Does your error log like this KYLIN-3970 ? If yes, there're some ways to solve 
this problem, it may be helpful.


在 2019/10/24 17:30，“田家铮” 写入:

I followed your steps and found the error report again. Can you help me fix 
this problem?



2019-10-24 05:25:11,171 ERROR [pool-11-thread-2] 
threadpool.DefaultScheduler:116 : ExecuteException 
job:9722333e-789b-1f30-50e6-38e9905802c0
org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job 
job_1571906689528_0004Job Diagnostics:Task failed 
task_1571906689528_0004_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
[2019-10-24 05:25:00.454]Exception from container-launch.
Container id: container_e01_1571906689528_0004_01_05
Exit code: 255


[2019-10-24 05:25:00.456]Container exited with a non-zero exit code 255. 
Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger 
(org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.




[2019-10-24 05:25:00.457]Container exited with a non-zero exit code 255. 
Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger 
(org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.








at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job 
job_1571906689528_0004Job Diagnostics:Task failed 
task_1571906689528_0004_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
[2019-10-24 05:25:00.454]Exception from container-launch.
Container id: container_e01_1571906689528_0004_01_05
Exit code: 255


[2019-10-24 05:25:00.456]Container exited with a non-zero exit code 255. 
Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger 
(org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.




[2019-10-24 05:25:00.457]Container exited with a non-zero exit code 255. 
Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger 
(org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.








at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
... 4 more
Caused by: org.apache.kylin.engine.mr.exception.MapReduceException: no 
counters for job job_1571906689528_0004Job Diagnostics:Task failed 
task_1571906689528_0004_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
[2019-10-24 05:25:00.454]Exception from container-launch.
Container id: container_e01_1571906689528_0004_01_05
Exit code: 255


[2019-10-24 05:25:00.456]Container exited with a non-zero exit code 255. 
Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger 
(org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See

Re: kylin2.6.4流式构建不能与hdp3.1.4.0使用(issue3970)

2019-10-24 Thread Wang rupeng 
It seems like your map reduce job got a problem, please check mr job log from 
your yarn.

在 2019/10/24 14:24，“田家铮” 写入:

The error reported in the log is the same as the error reported in the 
issue, and the scheme provided by the issue has been tried, but it still cannot 
be solved, and the reason is unknown.
This is the logs , please take a look ,thanks 



2019-10-09 23:58:44,395 ERROR [pool-11-thread-1] 
threadpool.DefaultScheduler:116 : ExecuteException 
job:dab5ba1f-e1ed-c8fa-409c-29f71715e8e0
org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job 
job_1570617080701_0005Job Diagnostics:Task failed 
task_1570617080701_0005_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
Error: 
org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set;


at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kylin.job.exception.ExecuteException: 
org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job 
job_1570617080701_0005Job Diagnostics:Task failed 
task_1570617080701_0005_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
Error: 
org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set;


at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:182)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
... 4 more
Caused by: org.apache.kylin.engine.mr.exception.MapReduceException: no 
counters for job job_1570617080701_0005Job Diagnostics:Task failed 
task_1570617080701_0005_m_00
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 
killedReduces: 0
Failure task Diagnostics:
Error: 
org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set;


at 
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
... 6 more
2019-10-09 23:58:44,396 INFO  [FetcherRunner 1609136206-49] 
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual 
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others
2019-10-09 23:58:44,400 DEBUG [http-nio-7070-exec-7] 
cachesync.Broadcaster:246 : Broadcasting UPDATE, execute_output, 
dab5ba1f-e1ed-c8fa-409c-29f71715e8e0
2019-10-09 23:58:44,401 DEBUG [http-nio-7070-exec-7] 
cachesync.Broadcaster:280 : Done broadcasting UPDATE, execute_output, 
dab5ba1f-e1ed-c8fa-409c-29f71715e8e0
2019-10-09 23:59:12,827 INFO  [FetcherRunner 1609136206-49] 
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 0 actual 
running, 0 stopped, 0 ready, 0 already succeed, 1 error, 0 discarded, 0 others




 


-- Original --
From:  "Xiaoxiang Yu";
Date:  Thu, Oct 24, 2019 01:32 PM
To:  "dev@kylin.apache.org"; 
"tianjiazh...@tuzhanai.com"; 

Subject:  Re: kylin2.6.4流式构建不能与hdp3.1.4.0使用(issue3970)

 

Hi Sir,
   I cannot see your screenshot, could you please paste the stack trace (in 
kylin.log) or provided more description to us?
  And have you ever try use the workaround which was provided in KYLIN-3970 
to fix your problem?

Best wishes,
Xiaoxiang Yu


发件人: 田家铮 
答复: "dev@kylin.apache.org" 
日期: 2019年10月24日 星期四 11:44
收件人: dev 
主题: kylin2.6.4流式构建不能与hdp3.1.4.0使用(issue3970)

环境配置：
 kylin-2.6.4
 HDP-3.1.4。0

issue:
 3970
 
https://issues.apache.org/jira/browse/KYLIN-3970?jql=text%20~%20%22partition.assignment.strategy%22


问题：
流式构建读取kafka数据时，无法构建，但是hive构建没问题

[cid:E5BB65D6@3C8BF016.500DB15D]
[cid:1F6BECD4@A04E8373.500DB15D]

Re: [VOTE] Release apache-kylin-3.0.0-beta (RC2)

2019-10-17 Thread Wang rupeng 
+1
mvn test passed.

在 2019/10/18 10:00，“Yaqian Zhang” 写入:

+1
mvn test passed.

> 在 2019年10月18日，09:42，codingfor...@126.com 写道：
> 
> +1, mvn test passed.
> 
>> 在 2019年10月15日，16:59，ShaoFeng Shi  写道：
>> 
>> Hi all,
>> 
>> I have created a build for Apache Kylin 3.0.0-beta, release candidate 2.
>> 
>> Changes highlights:
>> [KYLIN-4122] - Add Kylin user and group management modules
>> [KYLIN-4167] - Refactor streaming coordinator
>> [KYLIN-4114] - Provided a self-contained docker image for Kylin
>> [KYLIN-4137] - Accelerate metadata reloading
>> 
>> Thanks to everyone who has contributed to this release.
>> Here’s the release notes:
>> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12345686
>> 
>> The commit to being voted upon:
>> 
https://github.com/apache/kylin/commit/9d523862f29920292ab195b31c231f2f57f18594
>> 
>> Its hash is 9d523862f29920292ab195b31c231f2f57f18594.
>> 
>> The artifacts to be voted on, including the source package and two
>> pre-compiled binary packages are located here:
>> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-beta-rc2/
>> 
>> The hash of the artifacts are as follows:
>> apache-kylin-3.0.0-beta-source-release.zip.sha256
>> 56c60b585a2033dd3867430fd053438e0c4ba3001e9b94a235118472a35e6fcf
>> apache-kylin-3.0.0-beta-bin-hbase1x.tar.gz.sha256
>> 2418466be24cd344c96c9bbbd86b08375134a68f9aabad83f8138fcf8136b879
>> apache-kylin-3.0.0-beta-bin-cdh57.tar.gz.sha256
>> 7a5b3d09229e559b27a5da45f6e3fbc6ee2d27277f55eed35c7c1f02907da328
>> apache-kylin-3.0.0-beta-bin-hadoop3.tar.gz.sha256
>> 4d16b8e7c3a8f47364b6c5946c817cbc855489cbd10c1f2f502997c887f8ca01
>> apache-kylin-3.0.0-beta-bin-cdh60.tar.gz.sha256
>> 0b8c1ff107ecae592812344e4a4fbb328628a3e51e74ee67109f918ba685b4c5
>> 
>> A staged Maven repository is available for review at:
>> https://repository.apache.org/content/repositories/orgapachekylin-1069/
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/shaofengshi.asc
>> 
>> Please vote on releasing this package as Apache Kylin 3.0.0-beta.
>> 
>> The vote is open for the next 72 hours and passes if a majority of
>> at least three +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Kylin 3.0.0-beta
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not release this package because...
>> 
>> Here is my vote:
>> 
>> +1 (binding)
>> 
>> Best regards,
>> 
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofeng...@apache.org
>> 
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscr...@kylin.apache.org
>> Join Kylin dev mail group: dev-subscr...@kylin.apache.org

Re: [VOTE] Release apache-kylin-2.6.4 (RC1)

2019-10-08 Thread Wang rupeng 

+1 mvn test passed
在 2019/10/8 20:04，“Yaqian Zhang” 写入:

+1
> mvn test passed.

> 在 2019年10月8日，18:23，Chao Long  写道：
> 
> +1
> mvn test passed.
> 
> On Tue, Oct 8, 2019 at 5:01 PM ShaoFeng Shi  
wrote:
> 
>> Hi all,
>> 
>> 
>> 
>> I have created a build for Apache Kylin 2.6.4, release candidate 1.
>> 
>> 
>> 
>> Changes highlights:
>> 
>> [KYLIN-3628] - Lookup table queries always use the latest snapshot
>> 
>> [KYLIN-3797] - Avoid out-of-memory error in Kylin server when flatting
>> query filters with too many OR conditions
>> 
>> [KYLIN-4121] - Cleanup hive view intermediate tables after job finished
>> 
>> [KYLIN-1856] - Fix the issue that Kylin jobs show outdated output
>> information when recovering from failures
>> 
>> [KYLIN-4034] - Fix the issue that Insight page shows tables to which 
users
>> have no access
>> 
>> [KYLIN-4066] - Allow users who are not ROLE_ADMIN to access Planner pages
>> 
>> [KYLIN-4131] - Fix memory leak issue within Broadcaster
>> 
>> [KYLIN-4153] - Fix inconsistency within the transaction of metadata
>> pushdown
>> 
>> [KYLIN-4157] - Fix InternalErrorException throwing issue if
>> users’PrepareStatement
>> queries contain functions within WHERE clause
>> 
>> [KYLIN-4158] - Fix wrong results caused by pushing down LIMIT condition
>> when queries have expressions of columns within GROUP BY clause
>> 
>> 
>> 
>> Thanks to everyone who has contributed to this release.
>> 
>> Here are the release notes:
>> 
>> 
>> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12345948
>> 
>> 
>> 
>> The commit to being voted upon:
>> 
>> 
>> 
https://github.com/apache/kylin/commit/7a8639f92f87b70dc712c6089d17120706fba87a
>> 
>> 
>> 
>> its hash is 7a8639f92f87b70dc712c6089d17120706fba87a
>> 
>> 
>> 
>> The artifacts to be voted on, including the source package and two
>> 
>> pre-compiled binary packages are located here:
>> 
>> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.6.4-rc1/
>> 
>> 
>> 
>> The hash of the artifacts are as follows:
>> 
>> apache-kylin-2.6.4-bin-cdh57.tar.gz.sha256
>> 
>> 52c4083bd459e20e5b64672eaa23d00ccaa5a03ca7ee04f0d07d7c672b8e1974
>> 
>> apache-kylin-2.6.4-bin-cdh60.tar.gz.sha256
>> 
>> 4b369073fb6aff2257d6c62abf1415917a46ab6ae6014889d037f4b52603dbf5
>> 
>> apache-kylin-2.6.4-bin-hadoop3.tar.gz.sha256
>> 
>> b3939e78bd11830b792a05b035e3027f8ea13f84178cfd718b9ec83204958f5f
>> 
>> apache-kylin-2.6.4-bin-hbase1x.tar.gz.sha256
>> 
>> adb763495f3ba132045842c1d2f8d99def8f6a8cf3aadff60087e8dc110ed674
>> 
>> apache-kylin-2.6.4-source-release.zip.sha256
>> 
>> 3faad44f24830efff6e1f799caf09db70409e9ed3d3501afcf68c96f237f978c
>> 
>> 
>> 
>> A staged Maven repository is available for review at:
>> 
>> https://repository.apache.org/content/repositories/orgapachekylin-1068/
>> 
>> 
>> 
>> Release artifacts are signed with the following key:
>> 
>> https://people.apache.org/keys/committer/shaofengshi.asc
>> 
>> 
>> 
>> Please vote on releasing this package as Apache Kylin 2.6.4.
>> 
>> 
>> 
>> The vote is open for the next 72 hours and passes if a majority of
>> 
>> at least three +1 PMC votes are cast.
>> 
>> 
>> 
>> [ ] +1 Release this package as Apache Kylin 2.6.4
>> 
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> 
>> [ ] -1 Do not release this package because...
>> 
>> 
>> 
>> Here is my vote:
>> 
>> 
>> 
>> +1 (binding)
>> 
>> 
>> Best regards,
>> 
>> Shaofeng Shi 史少锋
>> Apache Kylin PMC
>> Email: shaofeng...@apache.org
>> 
>> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
>> Join Kylin user mail group: user-subscr...@kylin.apache.org
>> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>>

Re: [VOTE] Release apache-kylin-3.0.0-beta (RC1)

2019-09-26 Thread Wang rupeng 
+1
---
Best wishes,
Rupeng Wang
 

在 2019/9/27 09:45，“Chao Long” 写入:

+1
mvn test passed

On Thu, Sep 26, 2019 at 8:44 PM Yaqian Zhang  wrote:

> +1
> mvn test passed
>
> > 在 2019年9月26日，20:10，nichunen  写道：
> >
> > +1
> >
> >
> >
> > Best regards,
> >
> >
> >
> > Ni Chunen / George
> >
> >
> >
> > On 09/26/2019 16:41，ShaoFeng Shi wrote：
> > Hi all,
> >
> > I have created a build for Apache Kylin 3.0.0-beta, release candidate 1.
> >
> > Changes highlights:
> > [KYLIN-4122] - Add Kylin user and group management modules
> > [KYLIN-4167] - Refactor streaming coordinator
> > [KYLIN-4114] - Provided a self-contained docker image for Kylin
> > [KYLIN-4137] - Accelerate metadata reloading
> >
> > Thanks to everyone who has contributed to this release.
> > Here’s the release notes:
> >
> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12345686
> >
> > The commit to being voted upon:
> >
> 
https://github.com/apache/kylin/commit/721be80866223fecad9a6231fa2427a847bc8f48
> >
> > Its hash is 721be80866223fecad9a6231fa2427a847bc8f48.
> >
> > The artifacts to be voted on, including the source package and two
> > pre-compiled binary packages, are located here:
> >
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-beta-rc1/
> >
> > The hash of the artifacts are as follows:
> > apache-kylin-3.0.0-beta-source-release.zip.sha256
> > 53547e8a94eb74cdcd329777ff03f1c79209020016c2f9a62351e8c73ac8e0bd
> > apache-kylin-3.0.0-beta-bin-hbase1x.tar.gz.sha256
> > 1d50348660899baa9005b78cf45243e0eb2495fa0403d6250b3439ff50bf1731
> > apache-kylin-3.0.0-beta-bin-cdh57.tar.gz.sha256
> > bc9e303154901d4061dbac3876157cb4be25f23307f4c709d083da70aa18524b
> > apache-kylin-3.0.0-beta-bin-hadoop3.tar.gz.sha256
> > 681452450248f56ebe107d278e3ccb1478e42137875a2dded953db8c03488f9a
> > apache-kylin-3.0.0-beta-bin-cdh60.tar.gz.sha256
> > 2f66497ed39d7d78ea5a634a8796ab408586dce369edc97ed9374ba90a88b03d
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachekylin-1066/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/shaofengshi.asc
> >
> > Please vote on releasing this package as Apache Kylin 3.0.0-beta.
> >
> > The vote is open for the next 72 hours and passes if a majority of
> > at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Kylin 3.0.0-beta
> > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> > Apache Kylin PMC
> > Email: shaofeng...@apache.org
> >
> > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> > Join Kylin user mail group: user-subscr...@kylin.apache.org
> > Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>
>

Re: Apache Kylin Meetup @Shenzhen, Sep. 7, 2019

2019-09-05 Thread Wang rupeng 
Awesome !

---
Best wishes,
Rupeng Wang
 

在 2019/9/4 13:00，“zjsy...@163.com 代表 nichunen” 写入:

Hello Kylin users,

 

There will be a Kylin Meetup the week after next (Sep. 7) in Shenzhen, 
China. Engineers from Tencent(腾讯), Alibaba Group (阿里), Ping An Group(平安) and 
Kyligence will share their use cases and experiences with Kylin & RocketMQ. 

 

Date: Sep. 7, 2019

Time: 12:30 PM - 16:40 PM

Location: Lecture Hall, 1st Floor, School of Computer and Software, South 
Campus, Shenzhen University, 3688 Nanhai Avenue, Nanshan District, Shenzhen, 
Guangdong, China

Language: Chinese

Fee: Free!

Link for Registration: https://www.huodongxing.com/event/3506680147611






Best regards,

 

Ni Chunen / George

Re: Unable to create cube in Spark Mode -Apache Kylin on Cloudera

2019-08-29 Thread Wang rupeng 
Hi, 
It seems the problem is following 
"60505 [dispatcher-event-loop-6] ERROR  
org.apache.spark.scheduler.cluster.YarnScheduler  - Lost executor 1 on 
*: Container marked as failed:"
It usually comes out with not enough memory for your yarn so that yarn 
container is closed because of lack of memory , you can go to yarn resource 
manager web page to see more details with yarn log. 
If it's the memory issue, you can try to allocate more memory for spark 
yarn executor by change the following configuration item in 
"$KYLIN_HOME/conf/kylin.properties"
kylin.engine.spark-conf.spark.yarn.executor.memoryOverhead=384
   

---
Best wishes,
Rupeng Wang
 

在 2019/8/29 14:57，“Gourav Gupta” 写入:

Hi Sir,

I have installed and configured Apache Kylin 2.4 on Cloudera Platform for
creating the Cube.

I have been able to create a cube in MapReduce mode but getting the
below-mentioned caveat while executes on spark mode. I have had followed
all the steps and tried many remedies for debugging the problem.



Please let me know how to resolve this bug. Thanks in Advance.





1091 [main] ERROR org.apache.spark.SparkContext  - Error adding jar
(java.lang.IllegalArgumentException: requirement failed: JAR
kylin-job-2.4.0.jar already registered.), was the --addJars option used?

[Stage 0:>  (0 + 0)
/ 2]
[Stage 0:>  (0 + 2)
/ 2]


60505 [dispatcher-event-loop-6] ERROR
org.apache.spark.scheduler.cluster.YarnScheduler  - Lost executor 1 on **
***: Container marked as failed:
container_e62_1566915974858_6628_01_03 on host: ***. Exit status:
50. Diagnostics: Exception from container-launch.
Container id: container_e62_1566915974858_6628_01_03
Exit code: 50
Stack trace: ExitCodeException exitCode=50:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:604)
at org.apache.hadoop.util.Shell.run(Shell.java:507)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789)
at

org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
at

org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at

org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 50

82664 [dispatcher-event-loop-5] ERROR
org.apache.spark.scheduler.cluster.YarnScheduler
 - Lost executor 2 on ***: Container marked as failed:
container_e62_1566915974858_6628_01_04 on host: ***. Exit status:
50. Diagnostics: Exception from container-launch.
Container id: container_e62_1566915974858_6628_01_04
Exit code: 50
Stack trace: ExitCodeException exitCode=50:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:604)
at org.apache.hadoop.util.Shell.run(Shell.java:507)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.
launchContainer(DefaultContainerExecutor.java:213)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.
launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.
launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 50


The command is:
export HADOOP_CONF_DIR=/etc/hadoop/conf && /usr/lib/spark/bin/spark-submit
--class org.apache.kylin.common.util.SparkEntry  --conf
spark.executor.instances=1  --conf spark.yarn.archive=hdfs://
namenode:8020/kylin/spark/spark-libs.jar  --conf spark.yarn.queue=default
 --conf spark.yarn.am.extraJavaOptions=-Dhdp.version=current  --conf
spark.history.fs.logDirectory=hdfs:///kylin/spark-history  --conf
spark.driver.extraJavaOptions=-Dhdp.version=current  --conf
spark.io.compression.codec=org.apache.spark.io.SnappyCompressionCodec
 --conf spark.master=yarn  --conf
spark.executor.extraJavaOptions=-Dhdp.version=current

Re: Kylin Service Starts Fine, Can't Reach Web UI

2019-08-01 Thread Wang rupeng 
Hi, Phil, it seems there're some problems with web server in your kylin. You 
can check the log if there's exception when you start up the kylin. The path of 
the log file is $KYLIN_HOME/log/kylin.log. Otherwise, you can share the log 
detail with us that we can help you to solve the problem.


---
Best wishes,
Rupeng Wang
 

在 2019/8/2 07:06，“Phil” 写入:

Hi, I've been stuck on this for a few weeks now.  I can start the Kylin
service, but get the "page not found" or "can't reach this page" error,
depending on the browser I use, when trying to get to the Kylin Web UI.

My Ambari works just fine at localhost:8080/, but localhost:7070 gives me
the above errors.  I've tried:
localhost:7070
localhost:7070/kylin
localhost:7070/kylin/login


I also tried the address that shows up in the command prompt when the
service starts:
http://sandbox.hortonworks.com:7070/kylin

No luck, I can't get to the Kylin Web UI.  

I'm running:

Kylin 2.6.3
HDP 2.5.0.0
HDFS 2.7.1.2.5
Hadoop 2.7.3.2.5.0.0-1245
Hive 1.2.1.2.5
HBase 1.1.2.2.5
Kafka 0.10.0.2.5

Port 7070 appears to be listening:
netstat -tulpn | grep 7070

tcp0  0 :::7070 :::*   
LISTEN  15184/java

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

Re: kylin2.6.3可以和CDH6.1.1集成么

2019-08-01 Thread Wang rupeng 
Before your install, you can check these two issues KYLIN0-3564 ,  KYLIN-4072 .
And download binary package apache-kylin-2.6.3-bin-cdh60.tar.gz from here 
http://kylin.apache.org/cn/download/ 


在 2019/8/1 11:19，“liulong”<253719...@qq.com> 写入:

kylin2.6.3可以和CDH6.1.1集成么

Re: kylin2.6.3可以和CDH6.1.1集成么

2019-07-31 Thread Wang rupeng 
你好，是可以的__

在 2019/8/1 11:21，“liulong”<253719...@qq.com> 写入:

kylin2.6.3可以和CDH6.1.1集成么

Re: [VOTE] Release apache-kylin-3.0.0-alpha2 (RC1)

2019-07-26 Thread Wang rupeng 
+1
mvn clean test passed

在 2019/7/26 13:47，“codingfor...@126.com” 写入:

+1
mvn clean test passed



> On Jul 26, 2019, at 13:43, Yichen Zhou  wrote:
> 
> +1
> mvn test passed
> 
> Regards,
> Yichen
> 
> On Fri, Jul 26, 2019 at 11:41 AM Chao Long  
wrote:
> 
>> +1
>> mvn test passed
>> 
>> On Fri, Jul 26, 2019 at 10:38 AM Temple Zhou  wrote:
>> 
>>> +1
>>> 
>>> On Fri, Jul 26, 2019 at 10:00 AM nichunen  wrote:
>>> 
 +1
 
 
 
 Best regards,
 
 
 
 Ni Chunen / George
 
 
 
 On 07/26/2019 09:48，JiaTao Tao wrote：
 +1
 
 --
 
 Regards!
 
 Aron Tao
 
 ShaoFeng Shi  于2019年7月25日周四 下午2:46写道：
 
 Hi all,
 
 
 
 I have created a build for Apache Kylin 3.0.0-alpha2, release candidate
>>> 1.
 
 
 
 Changes highlights:
 
 [KYLIN-3942] - Rea-time OLAP supports multi-level json event
 
 [KYLIN-4086] - Support connect Kylin with Tableau by JDBC
 
 [KYLIN-3841] - Build Global Dict by MR/Hive
 
 [KYLIN-4017] - Fix building engine failed to get zk lock and leads to
>> the
 building engine doesn't work
 
 [KYLIN-3843] - List kylin instances with their server mode on web
 
 [KYLIN-3997] - Add a health check job of Kylin
 
 [KYLIN-4028] - Speed up startup progress using cached dependency
 
 [KYLIN-4035] - Calculate column cardinality by using spark engine
 
 
 
 Thanks to everyone who has contributed to this release.
 
 Here’s the release notes:
 
 
 
 
>>> 
>> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12345840
 
 
 
 The commit to being voted upon:
 
 
 
 
 
 
>>> 
>> 
https://github.com/apache/kylin/commit/06f441cd04a98a988197f7b1d750608d6ee51cd8
 
 
 
 Its hash is 06f441cd04a98a988197f7b1d750608d6ee51cd8.
 
 
 
 The artifacts to be voted on, including the source package and two
 pre-compiled
 binary packages, are located here:
 
 
>>> 
>> 
https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-3.0.0-alpha2-rc1/
 
 
 
 The hash of the artifacts are as follows:
 
 apache-kylin-3.0.0-alpha2-source-release.zip.sha256
 
 dc206ab0527703271e4cc435368a4f0980247465947a199396567dfc776bf7f8
 
 apache-kylin-3.0.0-alpha2-bin-hbase1x.tar.gz.sha256
 
 406f2d7ea318df230d1e65600f4ffccb22accfeee47a7e2f85ee4f70bbc6bbdc
 
 apache-kylin-3.0.0-alpha2-bin-cdh57.tar.gz.sha256
 
 1dadf9910d07ed5af233477a92013a8b6090ae18b3f8d9b48c7ee915acbafe25
 
 apache-kylin-3.0.0-alpha2-bin-hadoop3.tar.gz.sha256
 
 f0d075b0e2bc3c0953bf17c7220ddae03ae611fae03456fad215b36f9393eb28
 
 apache-kylin-3.0.0-alpha2-bin-cdh60.tar.gz.sha256
 
 47faa1810f21fa8c4c109b82af9eb09d44c2f0d14d245a94655aaf783780a143
 
 
 
 
 
 A staged Maven repository is available for review at:
 
 
>> https://repository.apache.org/content/repositories/orgapachekylin-1065/
 
 
 
 Release artifacts are signed with the following key:
 
 https://people.apache.org/keys/committer/shaofengshi.asc
 
 
 
 Please vote on releasing this package as Apache Kylin 3.0.0-alpha2.
 
 
 
 The vote is open for the next 72 hours and passes if a majority of
 
 at least three +1 PMC votes are cast.
 
 
 
 [ ] +1 Release this package as Apache Kylin 3.0.0-alpha2
 
 [ ]  0 I don't feel strongly about it, but I'm okay with the release
 
 [ ] -1 Do not release this package because...
 
 
 
 
 
 Here is my vote:
 
 
 
 +1 (binding)
 
 
 Best regards,
 
 Shaofeng Shi 史少锋
 Apache Kylin PMC
 Email: shaofeng...@apache.org
 
 Apache Kylin FAQ:
>> https://kylin.apache.org/docs/gettingstarted/faq.html
 Join Kylin user mail group: user-subscr...@kylin.apache.org
 Join Kylin dev mail group: dev-subscr...@kylin.apache.org
 
 
>>> 
>>

Re: [VOTE] Release apache-kylin-2.6.3 (RC1)

2019-07-01 Thread Wang rupeng 
+1
mvn test passed

在 2019/7/1 下午2:16，“Chao Long” 写入:

+1
mvn test passed

On Mon, Jul 1, 2019 at 2:09 PM Cheng wang  wrote:

> ＋1(binding)
>
> Best regards,
> Cheng Wang
>
>
> On 7/1/19, 9:27 AM, "ShaoFeng Shi"  wrote:
>
> Hi all,
>
> I have created a build for Apache Kylin 2.6.3, release candidate 1.
>
> Changes highlights:
> - [KYLIN-4024] - Support pushdown to Presto
> - [KYLIN-3977] - Avoid mistaken deleting dicts by storage cleanup 
while
> building jobs are running
> - [KYLIN-4015] – Fix build cube error at the "Build UHC Dictionary"
> step
> - [KYLIN-4022] - Error with message "Unrecognized column type:
> DECIMAL(xx,xx)" happens when doing query pushdown
>
> Thanks to everyone who has contributed to this release.
> Here’s release notes:
>
> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121=12345582
>
> The commit to being voted upon:
>
>
> 
https://github.com/apache/kylin/commit/0d5f85b0a40c301134122de927204a0d17ad65fa
>
> Its hash is 0d5f85b0a40c301134122de927204a0d17ad65fa.
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.6.3-rc1/
>
> The hash of the artifact is as follows:
> apache-kylin-2.6.3-source-release.zip.sha256
> 50d1cad423f1a15a5e25f1c3c68748c7ce10e0116fd67fa9e38c1470a11d389c
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/content/repositories/orgapachekylin-1063/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/shaofengshi.asc
>
> Please vote on releasing this package as Apache Kylin 2.6.3.
>
> The vote is open for the next 72 hours and passes if a majority of
> at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Kylin 2.6.3
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
>
> Here is my vote:
>
> +1 (binding)
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofeng...@apache.org
>
> Apache Kylin FAQ:
> https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscr...@kylin.apache.org
> Join Kylin dev mail group: dev-subscr...@kylin.apache.org
>
>
>