Re: Apache Jenkins is very slow

2018-12-24 Thread ShaoFeng Shi
The Infra team has restarted the Jenkins master, now it is back to normal. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail

Re: [Tech blog] Why Meituan develop Kylin on Druid?

2018-12-24 Thread ShaoFeng Shi
The English version of the 1/2 part has been updated on Kylin website, for your reference: https://kylin.apache.org/blog/2018/12/12/why-did-meituan-develop-kylin-on-druid-part1-of-2/ Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https

Apache Jenkins is very slow

2018-12-24 Thread ShaoFeng Shi
of Apache Jenkins service. I have reported this to Infra team, but as it is the Christmas holiday, it may not be updated soon: https://issues.apache.org/jira/browse/INFRA-17497 So, before the Jenkins become healthy, let's use the Travis-CI's result as the check standard. Best regards, Shaofeng Shi 史少锋

Re: Kylin will not delete old hbase table when refresh the segment

2018-12-17 Thread ShaoFeng Shi
It can be improved to do the cleanup automatically. I heard some community user has implemented that, a patch is welcomed! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs

Evaluate Kylin on Parquet

2018-12-14 Thread ShaoFeng Shi
This is not the final version; there is room to improve in many aspects, parquet, spark, and Kylin. It can be used for PoC at this moment. Your comments are welcomed. Let's improve it together. [1] https://issues.apache.org/jira/browse/KYLIN-3621 [2] https://github.com/apache/kylin/tree/kylin-on-parquet B

Re: [Tech blog] Why Meituan develop Kylin on Druid?

2018-12-09 Thread ShaoFeng Shi
ache/kylin/tree/kylin-on-druid [2] https://github.com/apache/kylin/tree/kylin-on-druid/storage-druid Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html

Kylin 2.5 users please upgrade to v2.5.2

2018-12-09 Thread ShaoFeng Shi
a Chinese character); 2) Using Spark engine builds a Cube which has TopN or Count distinct measure; For detailed issue description, please check the JIRA in release notes. https://kylin.apache.org/docs/release_notes.html Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng

[Tech blog] Why Meituan develop Kylin on Druid?

2018-12-09 Thread ShaoFeng Shi
/ https://zhuanlan.zhihu.com/p/51722793 Thanks to the author Xiaoxiang Yu, and the reviewer Kaisen Kang! Enjoy! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted

Re: 3 Year Anniversary of Apache Kylin Graduation

2018-12-09 Thread ShaoFeng Shi
Luke; it is a good summary; Thank you! Now Kylin is on the way to be the next generation big data OLAP engine. It's very honored that so many users and friends are together with us. Looking forward to the next three years! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng

Re: [DISCUSS] Stop inserting git diffs to JIRA ticket

2018-12-04 Thread ShaoFeng Shi
Hi folks, thanks for your comment. Since there is no objection, I reported a JIRA to the infra team: https://issues.apache.org/jira/browse/INFRA-17362 Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ

[Announce] Apache Kylin 2.5.2 released

2018-12-04 Thread ShaoFeng Shi
release. We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kylin.apache.org/ Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache

[RESULT][VOTE] Release apache-kylin-2.5.2 (RC2)

2018-12-03 Thread ShaoFeng Shi
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 3 binding +1s: Shaofeng Shi Billy Liu Luke Han 8 non-binding +1s: Chao Long Na Zhai Jiatao Tao Yichen Zhou Xiaoxiang Yu Shaoxiong Zhan Rongchuan Jin Chunen(George) Ni No 0s

[Use case] How Xiaomi build one-stop OLAP platform with Apache Kylin

2018-12-03 Thread ShaoFeng Shi
languages. Hope it can help you to understand Kylin's value. [1] https://www.infoq.cn/article/bNLypvSJVpSHv-dLogsu Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs

[DISCUSS] Stop inserting git diffs to JIRA ticket

2018-12-02 Thread ShaoFeng Shi
input! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail group: dev-subscr...@kylin.apache.org

[VOTE] Release apache-kylin-2.5.2 (RC2)

2018-11-30 Thread ShaoFeng Shi
+1 (binding) Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev ma

[CANCELLED][VOTE] Release apache-kylin-2.5.2 (RC1)

2018-11-29 Thread ShaoFeng Shi
This vote is cancelled due to KYLIN-3705 <https://issues.apache.org/jira/browse/KYLIN-3705>. Will send out RC2 once it got fixed. Thanks for your attention. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache

Re: [VOTE] Release apache-kylin-2.5.2 (RC1)

2018-11-29 Thread ShaoFeng Shi
he RC2 soon. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join Kylin dev mail grou

[VOTE] Release apache-kylin-2.5.2 (RC1)

2018-11-27 Thread ShaoFeng Shi
feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (binding) Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://k

[Announce] Welcome new Apache Kylin committer: ChunEn Ni (倪春恩)

2018-11-27 Thread ShaoFeng Shi
The Project Management Committee (PMC) for Apache Kylin has invited ChunEn Ni(倪春恩) to become a committer and we are pleased to announce that he has accepted. Congratulations and welcome, ChunEn! Shaofeng Shi On behalf of the Apache Kylin PMC

[jira] [Created] (KYLIN-3702) Release 2.5.2

2018-11-25 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3702: --- Summary: Release 2.5.2 Key: KYLIN-3702 URL: https://issues.apache.org/jira/browse/KYLIN-3702 Project: Kylin Issue Type: Task Reporter: Shaofeng

Re: Redistribute intermediate table default not by rand()

2018-11-22 Thread ShaoFeng Shi
columns are all low cardinality or they are related, which may cause data skew. In this case, maybe distribute by random or by all column is better. Please feel free to report a JIRA on this, with your observations. Thank you! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaof

Re: apache kylin left join 数据查询混乱咨询

2018-11-21 Thread ShaoFeng Shi
Interesting finding; I guess it is a bug of Calcite. Kylin's calcite is a little old, we will upgrade it to a newer version, maybe it can be fixed with that. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin

Re: [DISCUSS] Kylin On Druid Storage

2018-11-21 Thread ShaoFeng Shi
ded column, etc; - Documentation We also welcome community users to try and refine this major feature. Please create PR to this branch before it merges to the master. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Ap

Re: Join Our Meetup/Webinar: Apache Kylin for Extreme OLAP and Big Data (Nov. 29th Thursday)

2018-11-20 Thread ShaoFeng Shi
Nice! I will join it from remote! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org Join

Re: [REPORT] Kylin - November 2018

2018-11-20 Thread ShaoFeng Shi
Wow, I can't believe we have so many activities in the second half year of 2018; Thanks to every Kylin community member! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs

Re: apache kylin left join 数据查询混乱咨询

2018-11-20 Thread ShaoFeng Shi
Hi Ling, Could you please provide the Kylin log when executing the first query? Thanks! Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin

Re: [DISCUSS] Kylin On Druid Storage

2018-11-18 Thread ShaoFeng Shi
please open a JIRA for this new feature? Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: user-subscr...@kylin.apache.org

[jira] [Created] (KYLIN-3693) TopN, Count distinct incorrect in Spark engine

2018-11-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3693: --- Summary: TopN, Count distinct incorrect in Spark engine Key: KYLIN-3693 URL: https://issues.apache.org/jira/browse/KYLIN-3693 Project: Kylin Issue Type: Bug

Re: apache-kylin-2.5.0-bin-hadoop3版使用中发现的问题

2018-11-13 Thread ShaoFeng Shi
ource runs. Just try to make your own binary package, it is not that difficult to do. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Work email: shaofeng@kyligence.io Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user

[jira] [Created] (KYLIN-3679) Fetch Kafka topic with Spark streaming

2018-11-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3679: --- Summary: Fetch Kafka topic with Spark streaming Key: KYLIN-3679 URL: https://issues.apache.org/jira/browse/KYLIN-3679 Project: Kylin Issue Type: New Feature

[jira] [Created] (KYLIN-3678) CacheStateChecker may remove a cache file that under building

2018-11-08 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3678: --- Summary: CacheStateChecker may remove a cache file that under building Key: KYLIN-3678 URL: https://issues.apache.org/jira/browse/KYLIN-3678 Project: Kylin

[Announce] Apache Kylin 2.5.1 released

2018-11-06 Thread ShaoFeng Shi
release. We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kylin.apache.org/ -- Best regards, Shaofeng Shi 史少锋

Re: doubt about measure of processedRowCount

2018-11-06 Thread ShaoFeng Shi
y statics. for example, The log is as following: >>>>> >>>>>"Processed rows for each storageContext: *valueA*". *valueA *is >>>>> processedRowCount. >>>>> >>>>>What I understand is processedRowCount is the record rows >>>>> numbers returned by hbase. >>>>> >>>>>Hbase corprocessor will log region stats, including: "*Total >>>>> scanned row*","Total filtered/aggred row". >>>>> >>>>> For one region, final records returned by hbase = *Total scanned >>>>> row - *Total filtered/aggred row; >>>>>Suppose this query need to scan 10 region in hbase, we can get >>>>> every region stats. we can get all records *valueB *returned by >>>>> hbase by >>>>>suming every final records in 10 region. >>>>> >>>>> In general, *valueA *is equal to * valueB*, but *valueB *is >>>>> much larger than *valueA* in sometimes. Why? >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> >>>> Regards! >>>> >>>> Aron Tao >>>> >>> >>> >>> -- >>> >>> >>> Regards! >>> >>> Aron Tao >>> >> >> >> -- >> >> >> Regards! >> >> Aron Tao >> > > > -- > > > Regards! > > Aron Tao > -- Best regards, Shaofeng Shi 史少锋

Re: kylin get error when run step 3

2018-11-06 Thread ShaoFeng Shi
t; hadoop log show : > > org.apache.hadoop.mapreduce.v2.app.MRAppMaster:Error starting > MRAppMaster > > java.lang.NoSuchMethodError:org.apache.hadoop.http.HttpConfig.setPolicy(Lorg/apache/hadoop/http/HttpConfig$Policy;)V > > May I ask what is the reason? > > > > >

Re: Understanding about region cut size for base

2018-11-06 Thread ShaoFeng Shi
o our preferred region numbers. Hope Kylin will update its > metadata accordingly in this case, without any issues or > ?Thanks,ke...@exponential.com Sent from my Samsung Galaxy smartphone. > Original message From: ShaoFeng Shi < > shaofeng...@apache.org> Date: 06

Re: Understanding about region cut size for base

2018-11-05 Thread ShaoFeng Shi
; 'kylin.storage.hbase.compression-codec=lz4’ > The number of regions need to be kept in control, for our Hbase cluster to > be performant. > > Please share the understanding, how this property works, and what can be > the possible reasons why it is not working as intended. > > Thanks, > Ketan@Exponential > > -- Best regards, Shaofeng Shi 史少锋

[RESULT][VOTE] Release apache-kylin-2.5.1 (RC1)

2018-11-05 Thread ShaoFeng Shi
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 4 binding +1s: Shaofeng Shi Billy Liu Yanghong Zhong Dong Li 9 non-binding +1s: Chao Long Shaoxiong Zhan Jiatao Tao Cheng Wang Na Zhai Yichen Zhou Xiaoxiang Yu Lijun Cao George

Re: How to increase split number for Fact distinct columns when using spark engine?(picture added)

2018-11-05 Thread ShaoFeng Shi
nk you for your reply. > I've checked the doc, it does not contain tuning for spark engine. > > -- > Best regards, > > Xi Chen > > -Original Message- > From: ShaoFeng Shi > Sent: Monday, November 5, 2018 3:30 PM > To: dev > Subject: Re: How to incre

Re: How to increase split number for Fact distinct columns when using spark engine?(picture added)

2018-11-04 Thread ShaoFeng Shi
he input sequence file is. > > I wonder how to increase the number of split for this step? > > I’m new to spark and any help will be great thanks! > > > > P.S. Spark job of #3 Step Name: Extract Fact Table Distinct Columns. > > -- > > Best regards, > > > > Xi Chen > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Apache Kylin online tutorials

2018-11-04 Thread ShaoFeng Shi
ocess simply. > > Asim Ali > *Software Developer* > > Email: a...@easyemployer.com > > Phone: 1300 855 642 <1300855642> > Website: www.easyemployer.com > > [image: easyemployer] <http://www.easyemployer.com/> > > > On Wed, 31 Oct 2018 at 20:04, ShaoFen

Re: Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-02 Thread ShaoFeng Shi
rent version is Kylin 2.1.0, hope it can be done before Nov.30, but > I cannot guarantee it, there is lots of other works to do. > > At 2018-11-01 15:08:12, "ShaoFeng Shi" wrote: > >Hi Gang, > > > >Thank you for the information, that is helpful for understanding t

[jira] [Created] (KYLIN-3664) Hive metrics reporter HiveProducer doesn't support multiple instances on one host

2018-11-02 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3664: --- Summary: Hive metrics reporter HiveProducer doesn't support multiple instances on one host Key: KYLIN-3664 URL: https://issues.apache.org/jira/browse/KYLIN-3664

[VOTE] Release apache-kylin-2.5.1 (RC1)

2018-11-02 Thread ShaoFeng Shi
vote: +1 (binding) -- Best regards, Shaofeng Shi 史少锋

Re: Redistribute intermediate table default not by rand()

2018-11-02 Thread ShaoFeng Shi
1:38,Chao Long 写道: > > > > Hi zhixin, > > Data may become not correct if use "distribute by rand()". > > https://issues.apache.org/jira/browse/KYLIN-3388 > > > > > > > > > > -- 原始邮件 ------ > > 发件

Re: Redistribute intermediate table default not by rand()

2018-11-01 Thread ShaoFeng Shi
a/browse/KYLIN-3388 > > > > > -- 原始邮件 -- > 发件人: "liuzhixin"; > 发送时间: 2018年11月2日(星期五) 中午12:53 > 收件人: "dev"; > 抄送: "ShaoFeng Shi"; > 主题: Re: Redistribute intermediate table default not by rand() > > > > Hi

Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-01 Thread ShaoFeng Shi
uot; > >主题: [DISCUSS] New Kylin Streaming Solution From eBay > > > >Hi all, > > > >eBay Kylin team has developed a new Kylin streaming solution, the basic > idea is to build a streaming cluster to ingest data from streaming > source(Kafka), and provide query for real-time data, the data preparation > latency is milliseconds, which means the data is queryable almost when it > is ingested, attach is the architecture design doc. > >We would like to contribute the feature to community, please let us know > if you have any concern. > > > >Thanks, > >Gang(Allen) Ma > > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Kylin2.5不支持Hive的SQL中的关键字识别

2018-10-31 Thread ShaoFeng Shi
Best wishes! -- Best regards, Shaofeng Shi 史少锋

Re: Apache Kylin Meetup @Hangzhou 26 Oct

2018-10-31 Thread ShaoFeng Shi
Table + Kylin practices in DingXiangYuan (in Chinese) https://www.slidestalk.com/s/TableauKylin69396 Apache Kylin in Dianjia (in Chinese) https://www.slidestalk.com/s/ApacheKylinInDianjia ShaoFeng Shi 于2018年10月17日周三 下午3:23写道: > Hi Shrikant, > > Yes, the slides will be uploaded. I'm not sur

Apache Kylin online tutorials

2018-10-31 Thread ShaoFeng Shi
://www.youtube.com/results?search_query=Apache+Kylin We will add the links to Kylin website after a while. If you have other materials, welcome to share! -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-10-30 Thread ShaoFeng Shi
a is queryable almost when it > is ingested, attach is the architecture design doc. > We would like to contribute the feature to community, please let us know > if you have any concern. > > Thanks, > Gang(Allen) Ma > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Merge Job in inconsistent state

2018-10-30 Thread ShaoFeng Shi
Kylin document is updated now, please refresh your web browser: https://kylin.apache.org/docs/howto/howto_backup_metadata.html ShaoFeng Shi 于2018年10月30日周二 上午11:35写道: > Hi Ketan, > > You're welcome. JIRA is created for this issue: > https://issues.apache.org/jira/brows

Re: ArrayIndexOutOfBoundsException for NDCuboidBuilder

2018-10-29 Thread ShaoFeng Shi
12.nextCur(Iterator.scala:434) > at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440) > at > org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:191) > at > org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) > at > org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) > at org.apache.spark.scheduler.Task.run(Task.scala:109) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > -- Best regards, Shaofeng Shi 史少锋

Re: Merge Job in inconsistent state

2018-10-29 Thread ShaoFeng Shi
Thanks,ke...@exponential.com Sent from my Samsung Galaxy smartphone. > Original message ----From: ShaoFeng Shi < > shaofeng...@apache.org> Date: 30/10/2018 7:49 am (GMT+05:30) To: dev < > dev@kylin.apache.org> Subject: Re: Merge Job in inconsistent state I'm > up

[jira] [Created] (KYLIN-3652) Segment auto-merge should not be triggered in some cases

2018-10-29 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3652: --- Summary: Segment auto-merge should not be triggered in some cases Key: KYLIN-3652 URL: https://issues.apache.org/jira/browse/KYLIN-3652 Project: Kylin Issue

Re: Merge Job in inconsistent state

2018-10-29 Thread ShaoFeng Shi
as we don't want to take > risk for 3 plus years (120+TB) data in Hbase for Kylin.Again,, really > appreciate the help provided.Thanks,Ketan Sent from my Samsung Galaxy > smartphone. > ---- Original message From: ShaoFeng Shi < > shaofeng...@apache.org> Date: 29/

Re: Merge Job in inconsistent state

2018-10-29 Thread ShaoFeng Shi
t; Region Count: 1 > Size: 164 MB > > Segment: 20181010150000_2018101016 > Start Time: 2018-10-10 15:00:00 > End Time: 2018-10-10 16:00:00 > Source Count: 723669 > HBase Table: KYLIN_RM0ICHV5EP > Region Count: 1 > Size: 155 MB > > Segment: 2018101016_2018101117 > Start Time: 2018-10-10 16:00:00 > End Time: 2018-10-11 17:00:00 > Source Count: 17476745 > HBase Table: KYLIN_Y8ZOSIWNJP > Region Count: 1 > Size: 941 MB > > Segment: 2018101117_2018101217 > Start Time: 2018-10-11 17:00:00 > End Time: 2018-10-12 17:00:00 > Source Count: 15485276 > HBase Table: KYLIN_RWTQZFY6J4 > Region Count: 1 > Size: 887 MB > > Thanks, > Ketan@Exponential -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-3651) JDBCResourceStore doesn't list all resources

2018-10-28 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3651: --- Summary: JDBCResourceStore doesn't list all resources Key: KYLIN-3651 URL: https://issues.apache.org/jira/browse/KYLIN-3651 Project: Kylin Issue Type: Bug

Re: Apache Kylin

2018-10-26 Thread ShaoFeng Shi
f the > message due to technical problems. PBZ d.d. reserves the right to supervise > and store both incoming and outgoing e-mail messages. > > > > -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-10-26 Thread ShaoFeng Shi
computation from Kylin to the Hadoop cluster. Except for Parquet, > Apache > > ORC is also a candidate. > > > > Now I raise this discussion to get your ideas about Kylin’s > next-generation > > storage engine. If you have good ideas or any related data, welcome > discuss in > > the community. > > > > Thank you! > > > > [1] Apache Kylin on HBase > > > > https://www.slideshare.net/ShiShaoFeng1/apache-kylin-on-hbase-extreme-olap-engine-for-big-data > > [2] Apache Kylin Plugin Architecture > > https://kylin.apache.org/development/plugin_arch.html > > [3] 基于Druid的Kylin存储引擎实践 > https://blog.bcmeng.com/post/kylin-on-druid.html-- > > Best regards, > > > > Shaofeng Shi 史少锋 > -- Best regards, Shaofeng Shi 史少锋

Re: does Apache Kylin need a Apache Derby or Mysql for run the sample cube

2018-10-26 Thread ShaoFeng Shi
gt; > spark-2.2.2-bin-without-hadoop > > apache-kylin-2.3.1-bin > > I will be grateful if someone in Help me with Kyle's installation and > configuration them. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-3648) Kylin may not add "kylin.storage.hbase.cluster-hdfs-config-file" to configuration properly

2018-10-24 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3648: --- Summary: Kylin may not add "kylin.storage.hbase.cluster-hdfs-config-file" to configuration properly Key: KYLIN-3648 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (KYLIN-3646) Add instruction about rowkey sequence in "Advanced setting" page

2018-10-24 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3646: --- Summary: Add instruction about rowkey sequence in "Advanced setting" page Key: KYLIN-3646 URL: https://issues.apache.org/jira/browse/KYLIN-3646 Proj

Re: Slow Query Performance With 'WHERE' Clause

2018-10-22 Thread ShaoFeng Shi
its not getting used > when query runs ( in case of multiple times also) . > > 4. Any best practises to tune the queries with WHERE clause? > > > Thank You, > Shrikant Bang. > > -- Best regards, Shaofeng Shi 史少锋

Re: NPE In QueryMetricsFacade

2018-10-22 Thread ShaoFeng Shi
There is already a JIRA for this NPE: https://issues.apache.org/jira/browse/KYLIN-3609 ShaoFeng Shi 于2018年10月22日周一 上午9:08写道: > It seems the result set is null, and there is a bug which didn't handle > this situation. Would you like to report a JIRA to Kylin? > > Regarding the das

[jira] [Created] (KYLIN-3641) Release 2.5.1

2018-10-21 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3641: --- Summary: Release 2.5.1 Key: KYLIN-3641 URL: https://issues.apache.org/jira/browse/KYLIN-3641 Project: Kylin Issue Type: Task Reporter: Shaofeng

Re: NPE In QueryMetricsFacade

2018-10-21 Thread ShaoFeng Shi
vlet.java:970) > at > org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:650) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) > > > Thank You, > Shrikant Bang. > -- Best regards, Shaofeng Shi 史少锋

Re: UNION ALL is not working with count()

2018-10-18 Thread ShaoFeng Shi
JIRA created for it: https://issues.apache.org/jira/browse/KYLIN-3637 ShaoFeng Shi 于2018年10月17日周三 下午11:04写道: > Hi Yi, > > This is a bug. Could you please report a JIRA to Kylin? Thank you! > > yiwang <756470...@qq.com> 于2018年10月15日周一 下午2:28写道: > >> Hi Kylin Team

[jira] [Created] (KYLIN-3637) Union All not working with count()

2018-10-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3637: --- Summary: Union All not working with count() Key: KYLIN-3637 URL: https://issues.apache.org/jira/browse/KYLIN-3637 Project: Kylin Issue Type: Bug

[jira] [Created] (KYLIN-3635) Percentile calculation on Spark engine is not thread safe

2018-10-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3635: --- Summary: Percentile calculation on Spark engine is not thread safe Key: KYLIN-3635 URL: https://issues.apache.org/jira/browse/KYLIN-3635 Project: Kylin Issue

Re: UNION ALL is not working with count()

2018-10-17 Thread ShaoFeng Shi
in) { return apply( > (Record1_0) acc, (Record2_1) in); } } , new > org.apache.calcite.linq4j.function.Function1() { public long > apply(Record1_0 > acc) { return acc.f0; } public Object apply(Object acc) { return apply( > (Record1_0) acc); } } ))).take(5); } public Class getElementType() { > return long.class; } while executing SQL: "SELECT count(TRANS_ID) as > TRANS_ID FROM KYLIN_SALES where TRANS_ID <> 1 union all select > count(TRANS_ID) as TRANS_ID FROM KYLIN_SALES LIMIT 5" > > Best regards > Yi > > > -- > Sent from: http://apache-kylin.74782.x6.nabble.com/ > -- Best regards, Shaofeng Shi 史少锋

Re: [Announce] Welcome new Apache Kylin committer :Allen Ma

2018-10-16 Thread ShaoFeng Shi
have him to be our new committer. > > > > Please join me to welcome Allen. > > > > Luke Han > > > > On behalf of the Apache Kylin PPMC > -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-10-16 Thread ShaoFeng Shi
Hi guys, I uploaded the initial design document to JIRA, please feel free to comment: https://issues.apache.org/jira/browse/KYLIN-3621 ShaoFeng Shi 于2018年10月12日周五 上午9:44写道: > JIRA and sub-tasks are created for this. Welcome to comment there: > https://issues.apache.org/jira/browse/KYLI

Apache Kylin Meetup @Hangzhou 26 Oct

2018-10-15 Thread ShaoFeng Shi
to join this event. Here is the event page: http://www.huodongxing.com/event/7461326621900 If you have any suggestion, welcome to give us feedback. -- Best regards, Shaofeng Shi 史少锋

Re: kylin 作为Grafana 支持的一个数据源

2018-10-15 Thread ShaoFeng Shi
? Currently, Grafana doesn't provide a plugin for Kylin. 潘博存 于2018年10月16日周二 上午11:27写道: > > hi,all >大数据可视化这一块,Grafana 是我们的一个重要展现工具,kylin 的快速查询 是大数据查询的利器,我们想在grafana > 上展示kylin的数据,不知道大家有没有这样使用的?是否有可以直接使用的grafana -kylin 插件.目前grafana > 是没有kylin的插件的, > -- Best regards, Shaofeng Shi 史少锋

Re: Some wrong with kylin2.5-hbase2.* for protobuf-java-3.1.0

2018-10-15 Thread ShaoFeng Shi
Hi Zx, The source code of 1.13.0-kylin-r4. is in Kyligence's fork: https://github.com/Kyligence/calcite Please provide the full kylin.log, maybe we can find new clue there. liuzhixin 于2018年10月15日周一 下午3:06写道: > Hi ShaoFeng Shi: > > This is the original error, > > When I build th

Re: Some wrong with kylin2.5-hbase2.* for protobuf-java

2018-10-15 Thread ShaoFeng Shi
e version 2.3.3 can work well with HDP 3. > > Perhaps you can test the Kylin with hive version 2.3.3. > > Maybe it’s other error. Thanks! > > Best wishes! > > > 在 2018年10月15日,下午1:24,ShaoFeng Shi 写道: > > Hi zhixin, > > I think the problem is how to run Hive 2

Re: Some wrong with kylin2.5-hbase2.* for protobuf-java

2018-10-14 Thread ShaoFeng Shi
上午11:25写道: > Hi ShaoFeng Shi > > Yes, the error from hive version 2.3.3, > > And Kylin need hive version 3.1.0. > > So how to solve the question? > > Best wishes! > > > 在 2018年10月15日,上午11:10,ShaoFeng Shi 写道: > > > > Hi Zhixin, > > > &g

Re: Some wrong with kylin2.5-hbase2.* for protobuf-java

2018-10-14 Thread ShaoFeng Shi
alCmd(CliDriver.java:233) > >>> at > org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > >>> at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > >>> at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336) > >>> at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787) > >>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > >>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>> at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>> at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>> at java.lang.reflect.Method.invoke(Method.java:498) > >>> at org.apache.hadoop.util.RunJar.run(RunJar.java:318) > >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:232) > >>> The command is: > >>> hive -e "USE default; > >> > >> > > > > -- Best regards, Shaofeng Shi 史少锋

Re: LDAP Sync issue - Empty filter; nested exception is javax.naming.directory.InvalidSearchFilterException

2018-10-13 Thread ShaoFeng Shi
Filter.doFilterInternal(HeaderWriterFilter.java:64) > at > > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at > > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > > org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) > > > -- > Sent from: http://apache-kylin.74782.x6.nabble.com/ > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-10-11 Thread ShaoFeng Shi
JIRA and sub-tasks are created for this. Welcome to comment there: https://issues.apache.org/jira/browse/KYLIN-3621 ShaoFeng Shi 于2018年10月8日周一 下午2:45写道: > I agree; the new storage should be Hadoop/HDFS compliant, and also need be > cloud storage (like S3, blob storage) friendly, a

[jira] [Created] (KYLIN-3626) Allow customization for Parquet path

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3626: --- Summary: Allow customization for Parquet path Key: KYLIN-3626 URL: https://issues.apache.org/jira/browse/KYLIN-3626 Project: Kylin Issue Type: Sub-task

[jira] [Created] (KYLIN-3624) Convert cuboid to Parquet in Spark

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3624: --- Summary: Convert cuboid to Parquet in Spark Key: KYLIN-3624 URL: https://issues.apache.org/jira/browse/KYLIN-3624 Project: Kylin Issue Type: Sub-task

[jira] [Created] (KYLIN-3625) Query engine for Parquet

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3625: --- Summary: Query engine for Parquet Key: KYLIN-3625 URL: https://issues.apache.org/jira/browse/KYLIN-3625 Project: Kylin Issue Type: Sub-task

[jira] [Created] (KYLIN-3623) Convert cuboid to Parquet in MR

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3623: --- Summary: Convert cuboid to Parquet in MR Key: KYLIN-3623 URL: https://issues.apache.org/jira/browse/KYLIN-3623 Project: Kylin Issue Type: Sub-task

[jira] [Created] (KYLIN-3622) Cube layout in Parquet

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3622: --- Summary: Cube layout in Parquet Key: KYLIN-3622 URL: https://issues.apache.org/jira/browse/KYLIN-3622 Project: Kylin Issue Type: Sub-task Reporter

[jira] [Created] (KYLIN-3621) Parquet as Cube storage

2018-10-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3621: --- Summary: Parquet as Cube storage Key: KYLIN-3621 URL: https://issues.apache.org/jira/browse/KYLIN-3621 Project: Kylin Issue Type: New Feature

Re: "--" should not be an annotation use between single quotes in SQL

2018-10-11 Thread ShaoFeng Shi
OM TABLEA WHERE NAME='Jason--Arthur' > > Seems like KYLIN use '--' as an annotation. > > Thanks > > -- Best regards, Shaofeng Shi 史少锋

re: [DISCUSS] Not sending Github PR notifications to dev@kylin

2018-10-09 Thread ShaoFeng Shi
and iss...@kylin.apache.org * > > > > The related JIRA to Apache Infra is > > https://issues.apache.org/jira/browse/INFRA-17073 > > > > Please +1 if you agree with the new rule, or -1 if you want to keep > as > > today. If no objection, we will move on with the new rule. > > > > -- > > Best regards, > > > > Shaofeng Shi 史少锋 > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-10-08 Thread ShaoFeng Shi
e 3 roles pretty well. > > > > In general, I prefer to choose from the state of the art instead of > > re-inventing. Indeed, Kylin is not a storage project. A new storage > format > > is not Kylin's mission. Any storage innovations we come across here would > > be mo

[jira] [Created] (KYLIN-3611) Upgrade Tomcat to 7.0.91, 8.5.34 or later

2018-10-03 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3611: --- Summary: Upgrade Tomcat to 7.0.91, 8.5.34 or later Key: KYLIN-3611 URL: https://issues.apache.org/jira/browse/KYLIN-3611 Project: Kylin Issue Type

[DISCUSS] Not sending Github PR notifications to dev@kylin

2018-10-01 Thread ShaoFeng Shi
ule, or -1 if you want to keep as today. If no objection, we will move on with the new rule. -- Best regards, Shaofeng Shi 史少锋

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-10-01 Thread ShaoFeng Shi
character: cloud-native storage support. > Quite a few users are using S3 on AWS, or Azure Data Lake Storage on > Azure. If new storage engine could be more cloud friendly, more user > could get benefits from it. > > With Warm regards > > Billy Liu > ShaoFeng Shi 于2018年9

[jira] [Created] (KYLIN-3606) Unstable KylinUserManagerTest

2018-09-29 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3606: --- Summary: Unstable KylinUserManagerTest Key: KYLIN-3606 URL: https://issues.apache.org/jira/browse/KYLIN-3606 Project: Kylin Issue Type: Bug Affects

[jira] [Created] (KYLIN-3605) Fix security issues reported by snyk.io

2018-09-29 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3605: --- Summary: Fix security issues reported by snyk.io Key: KYLIN-3605 URL: https://issues.apache.org/jira/browse/KYLIN-3605 Project: Kylin Issue Type: Improvement

Re: [DISCUSS] Columnar storage engine for Apache Kylin

2018-09-28 Thread ShaoFeng Shi
create a new file > format with Kylin specified requirements, like cuboid info. > > -- > Best regards, > Yanghong Zhong > > > On 9/28/18, 2:15 PM, "ShaoFeng Shi" wrote: > > Hi Kylin developers. > > HBase has been Kylin’s storage engine since t

Re: regarding creating and modifying cube

2018-09-28 Thread ShaoFeng Shi
n-cut-mb" on cube building performance. > > how to decide the optimum value of > "kylin.engine.spark.rdd-partition-cut-mb" for cube creation? > > -- Best regards, Shaofeng Shi 史少锋

[DISCUSS] Columnar storage engine for Apache Kylin

2018-09-28 Thread ShaoFeng Shi
ylin-on-druid.html-- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-3597) Fix sonar reported static code issues

2018-09-25 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-3597: --- Summary: Fix sonar reported static code issues Key: KYLIN-3597 URL: https://issues.apache.org/jira/browse/KYLIN-3597 Project: Kylin Issue Type: Improvement

Re: Re:Kylin concurrent issue for query engine;

2018-09-25 Thread ShaoFeng Shi
+1 to add this into Kylin documentation. huaicui <270922...@qq.com> 于2018年9月25日周二 下午5:22写道: > I want to enable this function in V2.5.0, but i don't how to use this > PreparedStatement. > > -- > Sent from: http://apache-kylin.74782.x6.nabble.com/ > -- Best regards, Shaofeng Shi 史少锋

Re: [Announce] Apache Kylin 2.5.0 released

2018-09-24 Thread ShaoFeng Shi
ty. > > With Warm regards > > Billy Liu > ShaoFeng Shi 于2018年9月19日周三 上午8:31写道: > > > > The Apache Kylin team is pleased to announce the immediate availability > of the 2.5.0 release. > > > > This is a major release after 2.4, with more than 100 enhancemen

<    1   2   3   4   5   6   7   8   9   10   >