Re: Bump KYLIN-976

2015-12-09 Thread Li Yang
uot;Adunuthula, Seshu" <sadunuth...@ebay.com> wrote: > > > > >This is awesome! > > > > > >On 12/8/15, 6:05 AM, "Shi, Shaofeng" <shao...@ebay.com> wrote: > > > > > >>This is another important refactor since making the bui

Re: [Request for comments] A client library to help automatic cube building/refreshing

2015-12-09 Thread Li Yang
+1 This is a pain point of many users. A similar request being some shell commands to trigger Kylin build. Rest API is fine but many scheduling and triggering comes down to shell. On Thu, Dec 10, 2015 at 1:55 PM, hongbin ma wrote: > Currently most users

Re: A problem,maybe a bug, when querying in kylin using not in

2015-12-10 Thread Li Yang
Which version of Kylin are you on? I cannot reproduce this on 2.x branch. On Wed, Dec 9, 2015 at 4:29 PM, DroopyHoo wrote: > Equal or more than 2 elements in NOT IN clause is also a bug in calcite ... > https://issues.apache.org/jira/browse/CALCITE-980 > .. > > > 在 15/12/9

Re: Xianbin volunteers to work on KYLIN-1079

2015-12-10 Thread Li Yang
Welcome Xianbin~ On Tue, Dec 8, 2015 at 3:25 PM, whenwin wrote: > hi all! > > thank for the warm welcome, and help from hongbin, xiaoyu, luke. > > big smile on face! > > regards > > -- > View this message in context: >

Re: Kylin ingnores startTime in cube build process

2015-12-10 Thread Li Yang
What Marek described is as designed (an early design though, and we can discuss if the design makes sense). Out of simplicity, Kylin 1.x requires all segment be continuous and there's no gap in between. So a new incremental segment always starts from where the last segment ends. I agree the

Re: Cassandra instead of HBase in Kylin

2015-12-10 Thread Li Yang
Not for the core team at least. Cassandra lacks of a CoProcessor equivalent (the last time I checked). However it is highly welcome if some one want to do a Proof of Concept in this direction. Not limited to Cassandra, consider Kudu and other K-V store options. If a PoC shows another storage can

Re: Bump KYLIN-976

2015-12-10 Thread Li Yang
i measure is > possible. > In another word, we could define the measure type in factory using > funcName and returnType, instead of only funcName for now. > > Do you think this make sense? Looking for your comment. > > > 在 2015年12月10日,14:57,Li Yang <liy...@apache.org> 写

Re: cut size for hbase region

2016-01-05 Thread Li Yang
Given v1.1 code. Just from the code, the only guess I could make is the "kylin.hbase.region.count.max" in kylin.properties is really more than 1000. To confirm, we need to see the reducer log of step "Calculate HTable Region Splits" if it's still there... On Tue, Jan 5, 2016 at 1:38 PM, Zhang,

Re: kylin sync hive table failed

2016-01-05 Thread Li Yang
Suggest attach the full kylin.log instead of saying no log. On Mon, Jan 4, 2016 at 10:24 AM, Jian Zhong wrote: > any more log? > > On Thu, Dec 31, 2015 at 2:33 PM, 和风 <363938...@qq.com> wrote: > > > run find-hive-dependency.sh can find jars; > > env: hadoop2.7.1,hive

Re: about the parameter 'acceptPartial'

2016-01-05 Thread Li Yang
You want it be "false" always. When "true", Kylin may choose to return incorrect partial result as purpose of preview. On Mon, Jan 4, 2016 at 6:16 PM, wangsh...@sinoaudit.cn < wangsh...@sinoaudit.cn> wrote: > Hi all: > Can anybody tell me what the query parameter 'acceptPartial' means? and I >

Re: Re: about the parameter 'acceptPartial'

2016-01-05 Thread Li Yang
You don't need to set it up in JDBC, it's false by default. On Tue, Jan 5, 2016 at 6:10 PM, wangsh...@sinoaudit.cn < wangsh...@sinoaudit.cn> wrote: > Yes, how can I do? > > > > wangsh...@sinoaudit.cn > > From: Li Yang > Date: 2016-01-05 17:29 > To: dev &

Re: Re: encouter Deserialization error when load hive table

2015-12-31 Thread Li Yang
> Task Id : attempt_1450856278246_0003_m_00_0, Status : FAILED So this is an error on a MR node. Likely the hive versions are not consistent across your cluster. A work around is let Kylin submit hive jars as part of MR job. See https://issues.apache.org/jira/browse/KYLIN-1021 On Wed, Dec

Re: suggest to revise kylin log preamble format

2016-01-06 Thread Li Yang
Still need month and date for copy and paste lines here and there. Others are great. On Thu, Jan 7, 2016 at 10:22 AM, hongbin ma wrote: > the reason to remove date info is that our kylin-server-log4j properties > uses DailyRollingFileAppender, putting date info in such

Re: Re: about apache kylin using problem!

2016-01-07 Thread Li Yang
Let me answer this in Chinese since a similar question has been answered in English. 麒麟Cube有Segment的概念,每个Segement对应一段时间。每次Build其实都是构建一段时间的Segment(这里假设Cube在创建时设置为Partitioned,并且设置了时间字段)。所以当数据变化,只需要刷新对应的Segment就可以了。 对于很古老的不再会更新的Segment,可以多个Segment合并起来,一般保持在一个月或者几个月一个Segment,这样总体的性能会比较好。 On Thu,

Re: Incremental builds assumptions and clarifications

2015-12-24 Thread Li Yang
Em.. don't think Luke has all the questions fully answered. My additions. >Is there a document explaining the assumptions for incremental builds. The only assumption (or requirement) is that there is date or timestamp column on the fact table that distinguishes the old from the new. >Do

Re: HBase 1.1 support

2015-12-27 Thread Li Yang
I'm merging 1.x-staging (which contains all latest stuff included in 1.2) to 1.x-HBase1.1.3 periodically. Drop a reminder in mailing list if you see 1.x-HBase1.1.3 is lag behind. Usually it will catch up pretty soon. On Thu, Dec 17, 2015 at 11:02 PM, Luke Han wrote: > here

Re: [ANNOUNCE] Apache Kylin 1.2 released

2015-12-24 Thread Li Yang
> nice, we long for the following 2 improvements for a long time: > [KYLIN-389] - Can't edit cube name for existing cubes > [KYLIN-1154] - Load job page is very slow when there are a lot of history job Ashamed that some basic function is finally fixed. But glad that we continue to improve. :-)

Re: Apache软件基金会宣布Apache Kylin成为顶级项目

2015-12-24 Thread Li Yang
用力鼓掌!!! 2015-12-13 13:30 GMT+08:00 Luke Han : > Here's Chinese translation of ASF announces Apache Kylin as a Top-Level > Project [1]. > 这是Apache基金会宣布Apache Kylin成为顶级项目新闻的中文翻译 > > Great thanks to Luwei Chen who did this. > 感谢陈露薇翻译这篇文章 > > [1]. >

Re: Incremental builds assumptions and clarifications

2015-12-30 Thread Li Yang
te, you need to refresh (or rebuild) the related segments. E.g. if 2 records on T2 were removed, you should refresh [T2, T3) segment. On Fri, Dec 25, 2015 at 4:36 PM, Abhilash L L <abhil...@infoworks.io> wrote: > Thanks for the clarification Luke, Li Yang. > > Please find my

Re: ApplicationNotFoundException

2015-12-30 Thread Li Yang
Seems a hadoop problem... On Mon, Dec 28, 2015 at 3:49 PM, 胡志华(万里通科技及数据中心商务智能团队数据分析组) < huzhihua...@pingan.com.cn> wrote: > Hi,all > > I stopped at step 14 “ Convert cuboid data to HFile” with > exception ApplicationNotFoundException, who can help me ? > > >

Re: encouter Deserialization error when load hive table

2015-12-29 Thread Li Yang
Try hive command from the Kylin node, does simple hive queries work? On Thu, Dec 24, 2015 at 2:03 PM, xianbin wang wrote: > anyone have a idea for such a error,when load hive table, error as follow: > > L4J [2015-12-24

Re: 答复: sample.sh running error

2015-12-29 Thread Li Yang
Need the full kylin.log to pin down the issue. Apparently Kylin is not accessing the hbase you think it should.

Re: 答复: sample.sh running error

2015-12-29 Thread Li Yang
Need the full kylin.log to pin down the issue. Apparently Kylin is not accessing the hbase you think it should. The console log produced by `kylin.sh start` may help too. Your cluster contains multiple hbase versions? On Tue, Dec 29, 2015 at 6:01 PM, Li Yang <liy...@apache.org> wrote:

Re: A problem,maybe a bug, when querying in kylin using not in

2016-01-12 Thread Li Yang
> FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY" > LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_AREA" "KYLIN_TV_DIM_AREA" > ON ("KYLIN_VIEW_TVAD_SUMMARY"."AREA" = > "KYLIN_TV_DIM_

Re: Re: encouter Deserialization error when load hive table

2016-01-12 Thread Li Yang
Then let's confirm your Kylin version and HDP version first. What are they? On Thu, Jan 7, 2016 at 4:03 PM, xianbin wang wrote: > hi all! > > I use HDP sandbox, there is only one node, table is loaded into hive by hdp > sandbox hive script, I don't think there will

Re: Cube build error - file kylin_job_meta does not exist

2016-01-12 Thread Li Yang
Seems your hadoop has a problem when copying distributed cache. However not sure of the best way to troubleshoot the problem. Maybe try a simpler MR job that involves distributed cache? On Fri, Jan 8, 2016 at 7:05 AM, Jiunn Jye Ng wrote: > Hi, > I am trying out Kylin 1.2.

Re: [Draft][REPORT] Apache Kylin - Jun 2016

2016-06-04 Thread Li Yang
Covers everything I know! :-) On Wed, Jun 1, 2016 at 11:23 PM, Luke Han wrote: > Dear community, > I have drafted below board report for review, please help to check and > let me know if there's any issue. > Feel free to reply here if there's more activities,

Re: kylin question

2016-06-02 Thread Li Yang
HBase API mismatch. What's your HBase version and Kylin version? Note HBase 0.98 / HBase 1.x / CDH HBase are all different in HBase API. There are different packages in Kylin download page. On Wed, May 25, 2016 at 11:37 AM, 水。。。海 <549066...@qq.com> wrote: > Run ${KYLIN_HOME}/bin/sample.sh ;

Re: A error at cube build. @ #3 Step Name: Build Dimension Dictionary Duration: 0.03 mins

2016-06-02 Thread Li Yang
Please open a JIRA with the reproduce steps. Thanks. On Tue, May 24, 2016 at 10:30 AM, 陈佛林 <chenfo...@gmail.com> wrote: > Yeah, the problem reproduces > > 2016-05-22 9:55 GMT+08:00 Li Yang <liy...@apache.org>: > > > Source code at the line is > > > >

Re: Merge cube and build cube simultaneously

2016-06-07 Thread Li Yang
This is valid requirement, could you open a JIRA? On Wed, Jun 1, 2016 at 6:47 PM, Vaibhav Taro wrote: > Right now with Kylin 1.5.2, it is not possible to run Cube merge for old > segments and Cube build job for a new segment simultaneously. (I am not > able to run cube

Re: While executin query with joins to dimension getting error

2016-06-07 Thread Li Yang
What's the problematic SQL since this is a SQL syntax issue? On Wed, Jun 1, 2016 at 5:53 PM, Uma Maheshwar Kamuni wrote: > When i am trying to execute query with dimensions joins to fact_table in > Web Interface in Insight Page. > > I am getting below error : > >

Re: 回复:failed to start kylin server

2016-06-07 Thread Li Yang
@耳东, according to the log, Kylin failed to read metadata during startup. Please check HBase is on and healthy. Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=6, exceptions: Sun May 29 02:02:19 CST 2016, null, java.io.InterruptedIOException: Origin:

Re: Extract Fact Table Distinct Columns step taking more time

2016-06-07 Thread Li Yang
Need to identify why "step-2" is slow first. Maybe start by checking if the mapper splits are even. On Thu, Jun 2, 2016 at 5:25 PM, Vaibhav Taro wrote: > Hey, ShaoFeng thanks for the reply. Yes, my Kylin version is 1.5.2. > > I have consistently observed that step-2

Re: a simple question about kylin input and output

2016-06-07 Thread Li Yang
I'm more curious about the purpose of such requirement. It's not obvious to me. On Thu, Jun 2, 2016 at 6:55 PM, lidong wrote: > Does Hive view meet your need? > > > Create a Hive view C base on A join B. And make C as input, then get C as > output. > > > Thanks, > Dong > > >

Re: 答复: 答复: how to extend the threshold for kylin query?(from baixing.com)

2016-06-08 Thread Li Yang
.budget=64424509440 > #default is 3G > kylin.query.cube.visit.timeout.times=3 > #default is 1 > > > > B.R > Austin.Woo > > -邮件原件- > 发件人: Li Yang [mailto:liy...@apache.org] > 发送时间: 2016年6月7日 8:55 > 收件人: dev@kylin.apache.org > 抄送: 李欣 <li...@baixing.com&

Re: How to name derived dimensions & hierarchy dimensions in Chinese

2016-06-08 Thread Li Yang
You can have a name mapping outside of Kylin. On Wed, Jun 8, 2016 at 10:47 AM, 251469031 <251469...@qq.com> wrote: > Hi all: > > > I want to display the cube metadata by calling the RESTful API of > kylinhost:7070/api/cubes/{cubeName} so that the endusers can know the cube > details, as the

Re: select * from fact_table

2016-06-07 Thread Li Yang
赵天烁 Kevin Zhao > Java工程师 研发中心-Flyme-大数据-平台研发 + 86 18826908281 | zhaotians...@meizu.com > 珠海市魅族科技有限公司 MEIZU Technology Co., Ltd. 广东省珠海市科技创新海岸魅族科技楼 MEIZU Tech Bldg., > Technology Innovation Coast Zhuhai, 519085, Guangdong, China meizu.com > -邮件原件- 发件人: Mars J [mail

Re: How to name derived dimensions & hierarchy dimensions in Chinese

2016-06-11 Thread Li Yang
nsions in > the future or not? > > > > > > > ------ 原始邮件 -- > 发件人: "Li Yang";<liy...@apache.org>; > 发送时间: 2016年6月8日(星期三) 下午5:34 > 收件人: "dev"<dev@kylin.apache.org>; > > 主题: Re: How to name derived dimensions & hierarchy di

Re: snapshot table not update

2016-06-11 Thread Li Yang
To confirm understanding. You built a cube, updated the lookup table in hive, and built it again. And the second build didn't pick up the latest lookup table. Is that correct? On Wed, Jun 8, 2016 at 11:45 AM, yubo-...@yolo24.com wrote: > I define a loopup table in a

Re: mvn install error

2016-06-13 Thread Li Yang
Kylin requires JDK 7. Seems you are on JDK 6. On Sat, Jun 11, 2016 at 11:43 AM, TTS2沉默天使 <85546...@qq.com> wrote: > [INFO] --- exec-maven-plugin:1.4.0:exec (build_cube_with_engine) @ > kylin-it --- > Exception in thread "main" java.lang.UnsupportedClassVersionError: >

Re: kylin query get empty result

2016-06-13 Thread Li Yang
I tried to reproduce this problem but I cannot. If you are using 1.5.2, there is a diagnosis tool which can extract more logs that allows people at remote to help. On Wed, Jun 8, 2016 at 10:33 AM, zhangrongkun <563364...@qq.com> wrote: >

Re: Re: Timeout visiting cube!

2016-06-13 Thread Li Yang
kage。 > > What may be the problem? > > > > gaolv123...@163.com > > 发件人: Li Yang > 发送时间: 2016-06-08 17:24 > 收件人: dev > 抄送: 吴钰彬 > 主题: Re: 答复: Timeout visiting cube! > There must be something wrong during cube build. If you are using 1.5.2, > there is a diagnosis t

Re: kylin.war not deployed correctly with version 1.5

2016-06-13 Thread Li Yang
; INFORMATION: Starting service Catalina > >>> Jun 10, 2016 2:50:36 PM org.apache.catalina.core.StandardEngine > >>> startInternal > >>> INFORMATION: Starting Servlet Engine: Apache Tomcat/7.0.59 > >>> Jun 10, 2016 2:50:36 PM org.apache.catalina.start

Re: kylin.war not deployed correctly with version 1.5

2016-06-06 Thread Li Yang
The log is not related. What's in the kylin.out? On Mon, May 30, 2016 at 10:09 PM, Jie Tao wrote: > after starting Kylin this URL (http://localhost:7070/kylin/) keeps > connecting to local host but shows nothing. This happened with 1.5.0, 1.5.1 > and 1.5.2. In

Re: 答复: how to extend the threshold for kylin query?(from baixing.com)

2016-06-06 Thread Li Yang
Hi Austin, Note the image didn't get through mail list, thus was not displayed. So we didn't quite get you issue yet. Could you try describe again? You can use file hosting service to communicate attachments. Also it's always better to adopt the latest version. If you are early in pilot stage,

Re: RE: RE: RE: kylin question

2016-05-31 Thread Li Yang
水,could you give more details on the CDH version and correct Jackson version? Others can benefit from your work. On Mon, May 23, 2016 at 9:29 AM, 水。。。海 <549066...@qq.com> wrote: > I solve the exception, I replace the jackson* jars in the HBase > > > > > -- Original

Re: kylin intermediate tables in Hive

2016-06-17 Thread Li Yang
Woo... something new to me. Anybody knows? On Tue, Jun 14, 2016 at 6:57 PM, Jie Tao wrote: > Kylin actually drops useless intermediate tables after cube building, but > I still see one "kylin_intermediate_cubename_searchdata" table for each > cube building in Hive. Are

Re: Welcome new Apache Kylin committer: Yu Feng

2016-01-15 Thread Li Yang
Hey, welcome Feng Yu! On Mon, Jan 11, 2016 at 10:43 AM, Jian Zhong wrote: > Welcome! Feng Yu > > On Sun, Jan 10, 2016 at 4:56 PM, hongbin ma wrote: > > > welcome and congrats! > > > > On Sat, Jan 9, 2016 at 10:13 AM, Xiaoyu Wang

Re: Relationship between rowkey column length and cube size

2016-01-15 Thread Li Yang
Some misc thoughts on this line. - Could abstract a DimensionEncoder to encode dimension on rowkey. Currently there are two ways of encoding -- dictionary and fixed len. - For long text description, they could be stored in hbase value instead of rowkey. This will make them much slower to filter,

Re: Cube build error - file kylin_job_meta does not exist

2016-01-19 Thread Li Yang
, @liyang is our usage of hadoop distributed cache typical? it's not > easy to find documentations for it, is it a stable and sustainable function > of hadoop? > > On Tue, Jan 12, 2016 at 6:50 PM, Li Yang <liy...@apache.org> wrote: > > > Seems your hadoop has a problem when

Re: Re: encouter Deserialization error when load hive table

2016-01-18 Thread Li Yang
Kylin 2.0-rc still requires hbase 0.9x, it does not work with HBase 1.x. That likely be the problem. On Tue, Jan 12, 2016 at 5:55 PM, wangxianbin1...@gmail.com < wangxianbin1...@gmail.com> wrote: > HDP2.3.2, kylin 2.0-rc > > > > wangxianbin1...@gmail.com > > From: Li

Re: can we support adding mapping cube columns to hive table columns

2016-01-18 Thread Li Yang
Valid request. Creating new JIRA KYLIN-1332 . On Tue, Jan 12, 2016 at 5:17 PM, yu feng wrote: > My suggestion is adding this mapping by creating view, So, you can change > the column name in hive table, and recreate the

Re: Timeout visiting cube

2016-06-26 Thread Li Yang
select count(distinct xxx) from Query like this returns only one row, and only put very little pressure to hbase. It could take some time to calculate if the data set is huge, but shall never bring down hbase. I cannot reproduce the problem on sample cube. May find a bigger cube and try

Re: flush map output at step Build Base Cuboid Data

2016-06-26 Thread Li Yang
Don't have similar experience. If you have access to Hadoop node, maybe jstack debug the hanging process? On Mon, Jun 20, 2016 at 9:58 PM, Jie Tao wrote: > another guess: the two mappers need communication? I saw that both mappers > have a progress of 0.667 and then not

Re: update hbase coprocessor

2016-06-26 Thread Li Yang
Thanks for sharing the root cause. :-) On Fri, Jun 24, 2016 at 11:35 AM, 移动苏州研发中心-陈雷雷 <775620...@qq.com> wrote: > ignore this question, problem solved. reason: hbase - nproc in > /etc/security/limits.d/hbase.conf is too small. > > > > > -- 原始邮件 -- > 发件人:

Re: HBase Region Replication.

2016-06-26 Thread Li Yang
Sure, please open a JIRA to track the task. Contribution in this area is very appreciated. On Wed, Jun 22, 2016 at 7:11 PM, Joel Victor wrote: > Hi, > > HDP 2.2 offers "HBase Read HA" functionality. More details about this > are here[1]. > Currently in Kylin when the

Re: java.lang.NegativeArraySizeException

2016-06-26 Thread Li Yang
Kylin version? On Fri, Jun 24, 2016 at 2:01 PM, 倪成伟 <549066...@qq.com> wrote: > When I run cube to #3 Step Name: Build Dimension Dictionary ,I got the > following exceptions . > How to solve the exceptions? > > > java.lang.NegativeArraySizeException > at > >

Re: A error at cube build. @ #3 Step Name: Build Dimension Dictionary Duration: 0.03 mins

2016-06-26 Thread Li Yang
Limited effort on 1.3 branch at moment. Could you try 1.5? On Fri, Jun 24, 2016 at 2:58 PM, 倪成伟 <549066...@qq.com> wrote: > Did you solve this problem? > > -- > View this message in context: >

Re: about code formatting (KYLIN-1821)

2016-06-27 Thread Li Yang
Thanks Hongbin! Well done! We hold high bar about code quality and with tools to enforce it. On Sat, Jun 25, 2016 at 8:51 PM, hongbin ma wrote: > Hi guys > > The code reformat is done for all of the JAVA files. > > In order to make sure everyone is committing well

[VOTE] Release apache-kylin-2.0-alpha (release candidate 1)

2016-02-09 Thread Li Yang
Hi all, I have created a build for Apache Kylin 2.0-alpha, release candidate 1. It is alpha due to the big amount of new features and improvements accumulated and I want to be cautious. Yet still it is well tested. Cubes (hundreds of TB) have been rebuilt and compared with previous version to

Re: [VOTE] Release apache-kylin-2.0-alpha (release candidate 1)

2016-02-09 Thread Li Yang
ther one later. > > Thanks. > > > > > > Best Regards! > - > > Luke Han > > On Tue, Feb 9, 2016 at 11:03 PM, ShaoFeng Shi <shaofeng...@apache.org> > wrote: > > > +1 binding > > > > I checked the md5 hash, and verified the source

[RESULT][VOTE] Release apache-kylin-2.0-alpha (release candidate 1)

2016-02-09 Thread Li Yang
This candidate is cancelled due to two potential show stoppers in the web frontend. https://issues.apache.org/jira/browse/KYLIN-1413 https://issues.apache.org/jira/browse/KYLIN-1414 Yang

Re: re-use Hive and Hbase from another kylin cluster issue

2016-02-09 Thread Li Yang
Some big metadata (like dictionary) does not fit in HBase and in that case is stored in HDFS as a fallback. Might be something you want to check. Anyway, checkout the kylin.log is always the first step of troubleshooting. On Tue, Feb 9, 2016 at 4:14 PM, hongbin ma wrote:

[DISCUSS]Apache Kylin 2.0 Release Features & Criteria

2016-02-01 Thread Li Yang
> > We will be doing the community a huge disservice by pushing this out by > end of February. > > Regards > Seshu Adunuthula > > > On 1/31/16, 11:46 PM, "Li Yang" <liy...@apache.org> wrote: > > >Just to add more colors. > > > >The

[DISCUSS]Apache Kylin 2.0 Release Features & Criteria

2016-01-31 Thread Li Yang
Just to add more colors. The 2.0 rc1 has been stabilizing in the 2.0-rc branch for a few month. The 2.0 rc1 contains: - A plugin-able architecture, to allow alternative cube engine / storage engine / data source. - A better MR cubing algorithm, about 1.5 times faster than 1.x by comparing

Re: Consider submitting a talk to hbasecon2016

2016-01-29 Thread Li Yang
In the coming 2.x release, we started to do parallel scans with endpoint coprocessor. This benefits slow queries significantly compare to the 1.x observer coprocessor. Something we can share with others. On Saturday, January 30, 2016, Luke Han wrote: > Thank you to let our

Re: Hot swapping cube post build

2016-01-29 Thread Li Yang
Assuming the cube definition does not change, all you need is "refresh" an existing cube segment. The old cube segment will continue serving until the new build is complete. No down time during the whole process. Try "refresh" On Friday, January 29, 2016, hongbin ma

Re: Look for definitive books

2016-02-25 Thread Li Yang
Thanks for your interest XiangYun. There's no Kylin book at the moment. Your interest is a strong encouragement. We shall work on a book at some point of time. On Thu, Feb 25, 2016 at 4:39 PM, Xiang Yun RZ Zheng wrote: > > > Hi, > I am very interested in an amazing software

Re: Exception Building Cube 2nd Step (IncompatibleClassChangeError)

2016-01-20 Thread Li Yang
Your classpath contains hadoop v2.2 and hadoop v2.5 at the same time. That must be why things become creepy. /home/hadoop/hbase/hbase-0.98.15-hadoop2/lib/hadoop-mapreduce-client-app-2. 2.0.jar /home/hadoop/hadoop/hadoop-2.5.2/share/hadoop/mapreduce/* On Sat, Jan 16, 2016 at 4:03 AM, Eric

Re: Support for Hive on Tez or Hive on Spark, cube build automation and best practices

2016-01-21 Thread Li Yang
In principal, Kylin does not do any scheduling stuff. Because only upstream ETL knows when the data lands in hive. That's why Kylin provides Rest API for upstream to call when data is ready. On Sat, Jan 16, 2016 at 8:18 PM, hongbin ma wrote: > ​kylin invokes shell to

Re: From the Build Base Cuboid Data step to Build N-Dimension steps, Too much time is taken.

2016-01-22 Thread Li Yang
Reduce "kylin.job.mapreduce.default.reduce.input.mb" will give you more reducers and can speed up the MR if the bottleneck is in reducer and there are extra reducer slots in your cluster. However there are many other reasons why a MR is slow. E.g. data skew, where a certain mapper or reducer gets

Re: 关于 kylin 函数的问题

2016-01-22 Thread Li Yang
The question was how or does Kylin support TopN with window functions. We haven't tested that yet.. far as I know. Can someone give a try and report any findings? 2016-01-20 15:09 GMT+08:00 王琳 : > Dear > 关于kylin 函数的问题请教一下: > 业务场景: TopN查询 > >

Re: kylin job压缩支持的参数!

2016-01-21 Thread Li Yang
http://kylin.apache.org/docs/install/advance_settings.html Here you find settings about compression. On Tue, Jan 19, 2016 at 8:33 PM, hongbin ma wrote: > ​you can comment out all entries containing snappy in kylin_hive_conf.xml, > kylin_job_conf.xml and kylin.properties

Re: Kylin service crash easily while building cube in HDP sandbox

2016-01-21 Thread Li Yang
What's the HDP version? Kylin 1.2 only works with HDP 2.2.4. Make sure you are NOT running on latest HDP 2.3 On Tue, Jan 19, 2016 at 1:34 PM, 宋轶 wrote: > I remember we can config the service to be a job engine or a query engine. > > > From: mahong...@apache.org > > Date:

Re: A question about the sql generated by Tableau through Kylin ODBC

2016-01-20 Thread Li Yang
The "not in" issue will be fixed very soon. We are waiting for the next Calcite release v1.6. Kylin not yet work well with TPC-DS for two main reasons. - Kylin supports only star schema, while TPC-DS is snowflake. - Kylin supports a limited set of SQL functions. E.g. substr() is only supported

Re: Where review for patches should happen: Github Pull Requests or Reviewboard?

2016-01-20 Thread Li Yang
We prefer patch as stated in the "How to contribute" [1]. Still evaluating the review board tool. I think it's optional. Not all commits require such heavy review process. [1] http://kylin.apache.org/development/howto_contribute.html On Thu, Jan 14, 2016 at 10:46 AM, hongbin ma

Re: Re: Using apache reviewboard for reviewing patches

2016-01-20 Thread Li Yang
I see patch files and PR basically the same thing. Personally prefer patch file, but PR is fine too. On Thu, Jan 14, 2016 at 4:48 PM, hongbin ma wrote: > good point > > in this case we should think about trying out both review ways, and pick > whichever suits us:) > > On

Re: Do we have document for 2.x-staging?

2016-02-16 Thread Li Yang
Documents now maintained in the "document" branch. The "How to Document" describes this. http://kylin.apache.org/development/howto_docs.html On Tue, Feb 16, 2016 at 3:09 PM, Zhao, John wrote: > I can find documents in master branch under website/_dev/. > But there is

Re: How to use kylin with high cardinality dimensions.

2016-02-17 Thread Li Yang
Better support of UHC (ultra high cardinality) columns is on dev plan. I'm thinking add custom encoding for dimension. However, even with those done, filtering URL using like will be still very slow because Kylin cannot pre-process and get prepared for such filtering. Alternatively, I'd suggest

Re: kylin error in building cube step 12

2016-02-18 Thread Li Yang
So you use the binary package from http://kylin.apache.org/download/ ? - apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin.tar.gz According to the source code:

Re: How to use kylin with high cardinality dimensions.

2016-02-18 Thread Li Yang
user can increase the value > while query count(distinct) value. > > anything will be pleased If you have some suggestion. > > 2016-02-18 14:58 GMT+08:00 Li Yang <liy...@apache.org>: > > > Better support of UHC (ultra high cardinality) columns is on dev plan. > I'm

Re: how to format ScientificNotation(BigDecimal)

2016-02-22 Thread Li Yang
Check out calcite's SQL reference[1]. That's the SQL interface of Kylin. There you find all the supported SQL functions, like CEIL() and Anything beyond that, consider do the formatting in your own GUI tier. [1] http://calcite.apache.org/docs/reference.html On Mon, Feb 22, 2016 at 4:00 PM,

Re: Re: Back to one dev branch

2016-03-09 Thread Li Yang
10, 2016 13:57 > Subject:Re: Back to one dev branch > > > +1 pointing the latest dev branch is more intuitive for those who are not > that familiar with Kylin dev. On Thu, Mar 10, 2016 at 12:37 PM, Li Yang > liy...@apache.org wrote: Hi all With the settling of 1.3.0 release, &

Back to one dev branch

2016-03-09 Thread Li Yang
Hi all With the settling of 1.3.0 release, the development of 1.x-staging will come to a maintenance mode. New features have been and will go on in 2.x-staging, which is the main dev branch. So I'd like to take this chance to go back to a more common branch setup -- *let master be the main dev

[RESULT][VOTE] Release apache-kylin-1.5.0 (release candidate 1)

2016-03-15 Thread Li Yang
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 6 binding +1s: 4 non-binding +1s: No 0s or -1s. Therefore I am delighted to announce that the proposal to release Apache-Kylin-1.5.0 has passed. Yang

Re: [RESULT][VOTE] Release apache-kylin-1.5.0 (release candidate 1)

2016-03-15 Thread Li Yang
Amend the names of voters. The tally is as follows. 6 binding +1s: Yang Li Shaofeng Shi Qianhao Zhou Yerui Sun Hongbin Ma Luke Han 4 non-binding +1s: Dong Li Xiaoyu Wang Meng Liang Chunen Ni No 0s or -1s. On Wed, Mar 16, 2016 at 6:22 AM, Li Yang <

Re: ClassNotFoundException in apache-kylin-1.5.0

2016-03-15 Thread Li Yang
UpdateCubeInfoAfterIndex I cannot find this class in v1.5.0 source code, neither in v1.3.0 On Mon, Mar 14, 2016 at 2:20 PM, hongbin ma wrote: > what is the current version you're using? > > 1.5.0 is not release yet, and the metadata migration tool for 1.2.x => > 1.5.0

Re: [VOTE] Release apache-kylin-1.3.0 (release candidate 2)

2016-03-09 Thread Li Yang
+1 binding `mvn test` passed on java version "1.7.0_79", OpenJDK Runtime Environment (rhel-2.5.5.1.el6_6-x86_64 u79-b14) On Thu, Mar 10, 2016 at 11:19 AM, nichunen wrote: > +1(no binding) > build success > mvn test passed > > md5 verified > > > > George/倪春恩 > >

[VOTE] Release apache-kylin-1.5.0 (release candidate 1)

2016-03-12 Thread Li Yang
Hi all, I have created a build for Apache Kylin 1.5.0, release candidate 1. It is the first release from the master branch after the reorg. Significant changes have taken place in metadata and cube data, upgrade from v1.3 and before is difficult. Recommend build new cube from scratch with this

[ANNOUNCE] Apache Kylin 1.5.0 released

2016-03-18 Thread Li Yang
The Apache Kylin team is pleased to announce the immediate availability of the 1.5.0 release. The release note can be found here [1]; The source code and binary package can be downloaded from Kylin's download page [2]. The Apache Kylin Team would like to hear from you and welcomes your comments

Re: Empty result return in the Insight

2016-04-08 Thread Li Yang
Don't think 1.3 and 1.5 has any difference regarding loading data from hive. Anyway, glad 1.3 has worked. On Thu, Apr 7, 2016 at 1:07 PM, kevinchen wrote: > I checked as you said, the hive return data. > Rollback to 1.3, it works. > > -- > View this message in context: >

Re: Unusable cube with large measures

2016-04-08 Thread Li Yang
Pls try latest version 1.5.1 (voting now) or 1.3.0 armed with calcite 1.6.0. Suppose the bug has been fixed in calcite 1.6.0 On Wed, Apr 6, 2016 at 4:41 AM, vipul jhawar wrote: > Hi Zhong, Luke > > I wanted to bring your attention to >

Re: Sample cube giving error on step 2

2016-04-08 Thread Li Yang
The latest version 1.5.1 under voting will have a package for HBase 1.x. Stay tuned and give it a try. On Wed, Apr 6, 2016 at 8:38 PM, Yagyank Chadha wrote: > Hi, > > I am trying to run sample cube 2 but is stuck on step 2. My problem is same > as given here( > >

Re: RANK/DENSE_RANK on KYLIN

2016-04-09 Thread Li Yang
Isn't it the same as below? SELECT CST_KEY, AMT FROM FCT ORDER BY AMT DESC On Fri, Apr 8, 2016 at 11:29 AM, hongbin ma wrote: > it's not working for kylin > > On Thu, Apr 7, 2016 at 12:51 AM, sdangi wrote: > > > Does Kylin support these analytic

Re: RANK/DENSE_RANK on KYLIN

2016-04-09 Thread Li Yang
Sorry forget the group by and sum().. SELECT CST_KEY, sum(AMT) FROM FCT group by CST_KEY ORDER BY 2 DESC On Sat, Apr 9, 2016 at 2:16 PM, Li Yang <liy...@apache.org> wrote: > Isn't it the same as below? > > SELECT CST_KEY, AMT > FROM FCT > ORDER BY AMT DESC > > On F

Re: [VOTE] Release apache-kylin-1.5.1 (release candidate 1)

2016-04-09 Thread Li Yang
+1 binding mvn test pass java version "1.7.0_71" OpenJDK Runtime Environment (rhel-2.5.3.1.el6-x86_64 u71-b14) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) On Fri, Apr 8, 2016 at 2:43 PM, Dong Li wrote: > Hi all, > > > I have created a build for Apache Kylin

Re: [Review Request] Resolve KYLIN-1434 Kylin Job Monitor API: /kylin/api/jobs is too slow in large kylin deployment

2016-03-19 Thread Li Yang
Saw the update in JIRA. Will take a look. On Fri, Mar 18, 2016 at 3:17 PM, Hao Chen wrote: > Hi Team, > > I have finished a series of refactoring patches about Kylin metadata > persistence API to resolve following problem: > > http://issues.apache.org/jira/browse/KYLIN-1504:

Re: Failed to find metadata store by url: kylin_metadata@hbase

2016-03-26 Thread Li Yang
> Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Get.setCheckExistenceOnly(Z)V Are you sure HBase version is 0.99.0? According to the source code, the method is there.

Re: Can we choose layered cubing or im-memory cubing manually

2016-03-26 Thread Li Yang
Currently the choice of inmem/layer is a global parameter "kylin.cube.algorithm" -- its value can be "auto", "layer", or "inmem". The default is "auto". For "auto", the choice is decided at runtime by looking at each cube's stats. There's another global parameter

Re: [VOTE] Release apache-kylin-1.3 (release candidate 1)

2016-03-07 Thread Li Yang
-1 binding `mvn test ` fail on CentOS 6.6, java version "1.7.0_79", OpenJDK Runtime Environment (rhel-2.5.5.1.el6_6-x86_64 u79-b14). Same error as Shaofeng mentioned. On Tue, Mar 8, 2016 at 1:22 PM, ShaoFeng Shi wrote: > -1 binding > > Verified the signature, md5

  1   2   3   4   5   >