Re: Streaming Cube Bulkload

2017-01-18 Thread ShaoFeng Shi
.RpcRetryingCaller@39f0ffa8, > > org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase. > RegionTooBusyException: > > failed to get a lock in 6 ms. > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2409) Performance tunning for in-mem cubing

2017-01-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2409: --- Summary: Performance tunning for in-mem cubing Key: KYLIN-2409 URL: https://issues.apache.org/jira/browse/KYLIN-2409 Project: Kylin Issue Type: Improvement

[jira] [Created] (KYLIN-2404) Add "hive.merge.mapfiles" and "hive.merge.mapredfiles" to kylin_hive_conf.xml

2017-01-17 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2404: --- Summary: Add "hive.merge.mapfiles" and "hive.merge.mapredfiles" to kylin_hive_conf.xml Key: KYLIN-2404 URL: https://issues.apache.org/jira/browse/KYLIN-2404

Re: kylin1.6& HuaWei FusionInsight: load hive table on page. retrun error "java.lang.ClassNotFoundException:org.apache.hive.jdbc.HiveDriver"

2017-01-15 Thread ShaoFeng Shi
from > CLI beeline ) > > > 9. kylin.sh start (find: > /opt/huawei/Bigdata/FusionInsight_V100R002C60U10/ > FusionInsight-Hive-1.3.0/hive-1.3.0/bin/..//lib > :No such file or directory) > > > 10. Click "Model" on page. Click "load hive table" . Then retun error > message. > > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2393) Add "hive.auto.convert.join" and "hive.stats.autogather" to kylin_hive_conf.xml

2017-01-15 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2393: --- Summary: Add "hive.auto.convert.join" and "hive.stats.autogather" to kylin_hive_conf.xml Key: KYLIN-2393 URL: https://issues.apache.org/jira/browse/KYLIN-2393

Re: Cube build error @First Step : DFS Command incompatible with Ranger

2017-01-12 Thread ShaoFeng Shi
; nabble.com/Cube-build-error-First-Step-DFS-Command- > > incompatible-with-Ranger-tp6924.html > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

Re: Spatial Functions on top of Kylin

2017-01-11 Thread ShaoFeng Shi
n context: http://apache-kylin.74782.x6. > nabble.com/Spatial-Functions-on-top-of-Kylin-tp6902.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Error in kylin with standalone HBase cluster

2017-01-11 Thread ShaoFeng Shi
Good; enjoy! 2017-01-11 18:01 GMT+08:00 柯南 <zzm_b...@qq.com>: > Thank you! It works,and the file "yarn-site.xml" and "mapred-site.xml" > also should be replaced. > > > > > ------ 原始邮件 -- > 发件人: "ShaoFeng Shi&qu

Re: Consulting "EXTENDED_COLUMN"

2017-01-11 Thread ShaoFeng Shi
> > this article “https://issues.apache.org/jira/browse/KYLIN-1313”. > > > > What,s the means about parameters of “Host Column” and “Extended > Column”? > > > > Why use this expression,and what aspects of optimization that this > > > > expression solved? > > > > Can be combined with a SQL statement to explain? > > > > > > > > > > > > Thanks~ > > > > > > > > > > > > > > > -- > > With Warm regards > > > > Yiming Liu (刘一鸣) > -- Best regards, Shaofeng Shi 史少锋

Re: Error in kylin with standalone HBase cluster

2017-01-10 Thread ShaoFeng Shi
kylin.properties,when to use for? > If I need to dependent hbase and hive/mapreduce cluster's on diffent hdfs, > counld kylin support that? Or may be in the future? > > > -- 原始邮件 -- > 发件人: "ShaoFeng Shi";<shaofeng...@apache.org>; >

[jira] [Created] (KYLIN-2375) Default cache size (10M) is too small

2017-01-10 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2375: --- Summary: Default cache size (10M) is too small Key: KYLIN-2375 URL: https://issues.apache.org/jira/browse/KYLIN-2375 Project: Kylin Issue Type: Improvement

Re: Error in kylin with standalone HBase cluster

2017-01-09 Thread ShaoFeng Shi
thodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod( > RetryInvocationHandler.java:187) > at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke( > RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy24.getBlockLocations(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations( > DFSClient.java:1169) > ... 20 more -- Best regards, Shaofeng Shi 史少锋

Re: where kylin global dict store?

2017-01-05 Thread ShaoFeng Shi
quot;SEX", > 3, > 1 > ], > [ > "BIRTHDAY", > 9193, > 2 > ], > [ > "NATIVE_PLACE", > 2268, > 2 > ], > [ > "LOCATION", > 15, > 1 > ] > ] > } > ], > "create_time_utc": 1483605652669, > "size_kb": 0, > "input_records_count": 0, > "input_records_size": 0 > } > - > I find all thing begin with /cube or /cube_desc or /model_desc or /table > or /table_ext related to my cube and I can't find anything with Global > Dictionary with my cube > Help me ! > > > > 446463...@qq.com > -- Best regards, Shaofeng Shi 史少锋

Re: Jenkins & SonarQube for Apache Kylin are ready

2017-01-05 Thread ShaoFeng Shi
Added the info to Kylin website: https://kylin.apache.org/development/ 2016-12-29 12:46 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > Great! Thanks Billy! > > 2016-12-29 9:27 GMT+08:00 Billy Liu <billy...@apache.org>: > >> Hi Dev team, >> >> Thank

[jira] [Created] (KYLIN-2359) Update job build step name

2017-01-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2359: --- Summary: Update job build step name Key: KYLIN-2359 URL: https://issues.apache.org/jira/browse/KYLIN-2359 Project: Kylin Issue Type: Improvement

[jira] [Created] (KYLIN-2358) CuboidReducer has too many "if (aggrMask[i])" checks

2017-01-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2358: --- Summary: CuboidReducer has too many "if (aggrMask[i])" checks Key: KYLIN-2358 URL: https://issues.apache.org/jira/browse/KYLIN-2358 Project: Kylin

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
t Kylin cluster mode must be on same nodes that hadoop > cluster. Is it correct? > I was thinked on put Kylin cluster mode on Docker Containers (Isolated from > Hadoop cluster) > > Thanks > > 2017-01-04 15:19 GMT+01:00 ShaoFeng Shi <shaofeng...@apache.org>: >

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
y scenario is for development. 2017-01-04 17:14 GMT+08:00 Alberto Ramón <a.ramonporto...@gmail.com>: > is it the same remote mode than cluster mode ? > <http://kylin.apache.org/docs15/install/kylin_cluster.html> > > > > 2017-01-04 9:35 GMT+01:00 ShaoFeng Shi <shaofeng...@ap

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
well? -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2351) Support using non-default HDFS path as kylin.env.hdfs-working-dir

2017-01-03 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2351: --- Summary: Support using non-default HDFS path as kylin.env.hdfs-working-dir Key: KYLIN-2351 URL: https://issues.apache.org/jira/browse/KYLIN-2351 Project: Kylin

Re: [Announce] New Apache Kylin committer Kaisen Kang

2017-01-03 Thread ShaoFeng Shi
king forward to his continued involvement. > > > > Congratulations and Welcome, Kaisen! > > > -- Best regards, Shaofeng Shi 史少锋

Re: Anomaly in Aggregation

2017-01-02 Thread ShaoFeng Shi
is > set to hll(10) (Error Rate < 9.75%). Please validate my finding. Thank you. > > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Anomaly-in-Aggregation-tp6808p6809.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: version with hive

2017-01-02 Thread ShaoFeng Shi
is message in context: http://apache-kylin.74782.x6. > nabble.com/version-with-hive-tp6793.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2344) Package spark into Kylin binary package

2016-12-30 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2344: --- Summary: Package spark into Kylin binary package Key: KYLIN-2344 URL: https://issues.apache.org/jira/browse/KYLIN-2344 Project: Kylin Issue Type: Task

[jira] [Created] (KYLIN-2343) When syn hive table, got error but actually the table is synced

2016-12-30 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2343: --- Summary: When syn hive table, got error but actually the table is synced Key: KYLIN-2343 URL: https://issues.apache.org/jira/browse/KYLIN-2343 Project: Kylin

[jira] [Created] (KYLIN-2342) When NoClassDefFoundError occurred in building cube, no error in kylin.log

2016-12-30 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2342: --- Summary: When NoClassDefFoundError occurred in building cube, no error in kylin.log Key: KYLIN-2342 URL: https://issues.apache.org/jira/browse/KYLIN-2342 Project

Re: Kybot bug?

2016-12-30 Thread ShaoFeng Shi
om the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Mistake in Kylin doc "Advanced Settings" page

2016-12-30 Thread ShaoFeng Shi
) or "mapreduce.job.queuename". The wrong name will cause it has no effect to YARN: jobs still go to default queue. The website has been corrected now; please refresh : https://kylin.apache.org/docs16/install/advance_settings.html If you ever faced the issue before, please be kindly informed; -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2339) Get error when refresh segment of a streaming cube

2016-12-29 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2339: --- Summary: Get error when refresh segment of a streaming cube Key: KYLIN-2339 URL: https://issues.apache.org/jira/browse/KYLIN-2339 Project: Kylin Issue Type

[jira] [Created] (KYLIN-2334) Investigate using HDFS as Kylin metadata store

2016-12-28 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2334: --- Summary: Investigate using HDFS as Kylin metadata store Key: KYLIN-2334 URL: https://issues.apache.org/jira/browse/KYLIN-2334 Project: Kylin Issue Type: Task

[jira] [Created] (KYLIN-2333) Kylin doesn't need 0-D cuboid, can remove that step

2016-12-28 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2333: --- Summary: Kylin doesn't need 0-D cuboid, can remove that step Key: KYLIN-2333 URL: https://issues.apache.org/jira/browse/KYLIN-2333 Project: Kylin Issue Type

Re: Jenkins & SonarQube for Apache Kylin are ready

2016-12-28 Thread ShaoFeng Shi
ttps://builds.apache.org/analysis/overview?id=org.apache.kylin%3Akylin > Here are still some potential issues found by SonarCube, any fix or > suggestions are welcomed. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2331) By layer Spark cubing

2016-12-28 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2331: --- Summary: By layer Spark cubing Key: KYLIN-2331 URL: https://issues.apache.org/jira/browse/KYLIN-2331 Project: Kylin Issue Type: New Feature

Re: Kylin Performance

2016-12-28 Thread ShaoFeng Shi
cardinality of DIM are 1M, the intermediate > table is only 5% of total: Picture (I don't know you can see pictures in > this mailList) > [image: Imágenes integradas 1] > > > 2016-12-27 2:32 GMT+01:00 ShaoFeng Shi <shaofeng...@apache.org>: > >> Alberto, I didn't test ORC format; b

Re: Kylin Performance

2016-12-26 Thread ShaoFeng Shi
> > Great and Agree! But I still have an question like Alberto, why in an > > AGG > > > > one > > > > dim can use only one regulation(mandatory, join, hierachy)? > > > > > > > > -- > > > > View this message in context: http://apache-kylin.74782.x6. > > > > nabble.com/Kylin-Performance-tp6713p6728.html > > > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: can not get hive table

2016-12-23 Thread ShaoFeng Shi
essorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at org.apache.hadoop.hive.metastore.RetryingHMSHandler. > > invoke(RetryingHMSHandler.java:107) > > at com.sun.proxy.$Proxy48.get_table(Unknown Source) > > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient. > > getTable(HiveMetaStoreClient.java:1208) > > at org.apache.kylin.source.hive.CLIHiveClient.getHiveTableMeta( > > CLIHiveClient.java:79) > > at org.apache.kylin.source.hive.HiveSourceTableLoader. > > extractHiveTables(HiveSourceTableLoader.java:93) > > > -- Best regards, Shaofeng Shi 史少锋

Re: Failed with "Build Cube" in 18 step

2016-12-21 Thread ShaoFeng Shi
0318_m_03_0 > sent hash and received reply > 2016-12-20 14:55:35,108 INFO [fetcher#2] > org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#2 - MergeManager > returned status WAIT ... > 2016-12-20 14:55:35,108 INFO [fetcher#2] > org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: dnode4:13562 > freed by fetcher#2 in 2ms > 2016-12-20 14:55:35,109 INFO [fetcher#2] > org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: Assigning > dnode4:13562 with 1 to fetcher#2 > 2016-12-20 14:55:35,109 INFO [fetcher#2] > org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: assigned 1 > of > 1 to dnode4:13562 to fetcher#2 > > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Failed-with-Build-Cube-in-18-step-tp6700p6729.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Kylin Performance

2016-12-21 Thread ShaoFeng Shi
ogle.com/drive/folders/0B-6nZ2q-HPTNem1KTTRHbDhpOG8?usp= > sharing> > (tell me if there is any problem) > > 2016-12-21 10:27 GMT+01:00 ShaoFeng Shi <shaofeng...@apache.org>: > > > Hi Alberto, where I can preview this doc? Thanks! > > > > 2016-12-21 6:46 GMT+

Re: [DISCUSS] Call the next release v2.0?

2016-12-19 Thread ShaoFeng Shi
ully > > > backward compatible. > > > - KYLIN-2195: All Kylin properties renamed, to follow a convention. > > > - KYLIN-2255: The old HBase storage (called v1 internally) is dropped. > > > Cubes created by v1.3 and before are no longer supported. > > > > > > With all these changes on master, the next release deserves a plus on > the > > > major version. > > > > > > What do you think? > > > > > > > > > Cheers > > > Yang > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: [Discuss] Add "source_type" to cube metadata

2016-12-15 Thread ShaoFeng Shi
Is it possible adding the "source_type" in cube's REST API result, but the value still comes from the fact table? Then in the backend Kylin doesn't need keep this info in two places, then no inconsistent be happen. 2016-12-16 14:25 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>:

Re: [Discuss] Add "source_type" to cube metadata

2016-12-15 Thread ShaoFeng Shi
metadata. > I think we still need "source_type" in cube metadata. > > > Any input is welcomed. > -- > Best regards, > > > kangkaisen -- Best regards, Shaofeng Shi 史少锋

Re: When building streaming cube,i find two issues

2016-12-15 Thread ShaoFeng Shi
e.java:57) at org.apache.kylin.job. > execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) result code:2 -- Best regards, Shaofeng Shi 史少锋

Re: What's the meaning of kylin.job.cuboid.size.memhungry.ratio and kylin.job.cuboid.size.ratio?

2016-12-15 Thread ShaoFeng Shi
behind this, Could anyone tell me? Thanks > > -- > Ding Dinghua > -- Best regards, Shaofeng Shi 史少锋

Re: KYLIN JDBC卡主不动

2016-12-15 Thread ShaoFeng Shi
h (Exception e){} try { if (conn != null) conn.close(); } catch (Exception e){} my pom.xml is: org.apache.kylin kylin-jdbc 1.6.0 gaolv123...@163.com 发件人: ShaoFeng Shi 发送时间: 2016-12-15 12:08 收件人: dev 主题: Re: KYLIN JDBC卡主不动 is there any message in kylin.log? 在 2016年12月15

Re: KYLIN JDBC卡主不动

2016-12-14 Thread ShaoFeng Shi
ctionImpl@4eef38f8 > > 卡在这里不动,CPU变得很高,这是怎么回事啊。 > > > > ------ > gaolv123...@163.com > -- Best regards, Shaofeng Shi 史少锋

[Discuss] Disable/hide "RAW" measure in Kylin web GUI

2016-12-14 Thread ShaoFeng Shi
-- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2282) Step name "Build N-Dimension Cuboid Data : N-Dimension" is inaccurate

2016-12-14 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2282: --- Summary: Step name "Build N-Dimension Cuboid Data : N-Dimension" is inaccurate Key: KYLIN-2282 URL: https://issues.apache.org/jira/browse/KYLIN-2282 Proj

Re: Kylin1.6.0流式Cube查询时间错误

2016-12-12 Thread ShaoFeng Shi
th of them are > operation times on Web GUI?(Also about new streaming cube in Kylin 1.6.0) > > > > 在2016年12月9日 22:46, ShaoFeng Shi<shaofeng...@apache.org>写道: > > Hi guys, sorry for the late response, just back from a travel. > > I think this is expected. The prope

[jira] [Created] (KYLIN-2271) Purge cube may delete building segments

2016-12-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2271: --- Summary: Purge cube may delete building segments Key: KYLIN-2271 URL: https://issues.apache.org/jira/browse/KYLIN-2271 Project: Kylin Issue Type: Bug

[jira] [Created] (KYLIN-2267) When FK/PK column type mismatched, get error on query: "NullPointerException: Column ### does not exist in row key desc"

2016-12-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2267: --- Summary: When FK/PK column type mismatched, get error on query: "NullPointerException: Column ### does not exist in row key desc" Key: KYLIN-2267 URL: https://issues.

Re: Re: 答复: 答复: 回复:答复: Kylin1.6.0流式Cube查询时间错误

2016-12-09 Thread ShaoFeng Shi
; > [cid:1deff3e1$2$158d4477d63$Coremail$skyyws$163.com] > > > > > -- > > > > > [cid:6448603e$3$158d4477d63$Coremail$skyyws$163.com] > > > > > > > > > > 2 > > > > > 您说的流式Cube用脚本启动之后,应该手动加上8小时,是否是指:Kylin服务端在处理kafka的数据时, > 时间会比电脑上的时间少8小时? > > > > > ( > > > > > 我在对流式Cube进行SQL查询时,发现Kylin服务端返回的每一列的时间都比电脑上的晚了8小时) > > > > > > > > > > [cid:d96b8d1$4$158d4477d63$Coremail$skyyws$163.com] > > > > > 这个数据是通过IDEA远程调试得到的来自Kylin服务器的数据,是Kylin通过查询HBase之后返回的。 > > > > > 每一列数据的时间都比电脑上的实际时间晚了8小时。 > > > > > > > > > > > > > > > > > > > > > > > > > 在2016年12月6日 20:30, Mario Copperfield <xwhfcen...@gmail.com x > > > > > whfcen...@gmail.com>> 写道: > > > > > > kylin.properties改的应该是页面上显示的时间,流式Cube用脚本启动了话和页面上的时间是不一致的, > 应该手动加上8小时 > > > > > > > > > > On 6 Dec 2016, 19:40 +0800, 汪胜 > > > > > <sky...@163.com<mailto:sky...@163.com>>, > > > > > wrote: > > > > > > 你好, > > > > > > 我在安装了Kylin1.6.0之后,根据官网的教程,成功地构建了流式cube。但在进行SQL查询的时候, > > > > 发现所有的“MINUTE_START” > > > > > 列的时间比实际时间少了8个小时,而且在WEB页面上,每个流式segment的起始和结束时间也少了8个小时, > > > > > 但是cube的创建时间和最后一次修改时间都是正确的,请问可能是什么原因呢? > > > > > > 注:我在kylin.properties文件中将timezone改为了GMT+8 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Use derived or Joint

2016-12-09 Thread ShaoFeng Shi
yWeek_Txt* > > > Join 2 > > *Year* > > Hierarchy 1 > > Normal 3 > > > SOL 2 is this the best solution ?? > > > > Typical case 2: > > I see the same scenario a lot of times (derived columns with 1:1 Relation) > > Product_ID *(PK)* > > Product_TXT > > TypeProduct_ID > > TypeProduct_TXT > > Country_TXT > > Country_ID > > Optimize queries by product / category / country, are mandatory > > Perhaps, > > Country (lower cardinality) its a good candidate to Join > > I don't want put Product_TXT as Join, because is a long text, and can > be affect Row_Key of HBase, but I need Queries like ... where product_TXT = > ""iRobot Roomba 650 Robotic Vacuum Cleaner > > suggestions ? > -- Best regards, Shaofeng Shi 史少锋

Re: Update default config for sandbox environment

2016-12-09 Thread ShaoFeng Shi
reducing > >> region-cut-gb, > >> hfile-size-gb, max-region-count, reduce-input-mb, > >> max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. > >> > >> What do you think? > >> > >> > >> > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2248) TopN merge further optimization after KYLIN-1917

2016-12-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2248: --- Summary: TopN merge further optimization after KYLIN-1917 Key: KYLIN-2248 URL: https://issues.apache.org/jira/browse/KYLIN-2248 Project: Kylin Issue Type

[jira] [Created] (KYLIN-2247) Automatically flush cache after executing "sample.sh" or "metadata.sh restore"

2016-12-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2247: --- Summary: Automatically flush cache after executing "sample.sh" or "metadata.sh restore" Key: KYLIN-2247 URL: https://issues.apache.org/jira/browse/KYLIN-2247

Re: select * clause still case all regionserver crash

2016-12-02 Thread ShaoFeng Shi
ache-kylin.74782.x6. > nabble.com/select-clause-still-cause-all-regionserver- > crash-tp6474p6481.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2244) "kylin.job.cuboid.size.memhungry.ratio" shouldn't be applied on measures like TopN

2016-12-01 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2244: --- Summary: "kylin.job.cuboid.size.memhungry.ratio" shouldn't be applied on measures like TopN Key: KYLIN-2244 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (KYLIN-2243) TopN memory estimation is inaccurate in some cases

2016-12-01 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2243: --- Summary: TopN memory estimation is inaccurate in some cases Key: KYLIN-2243 URL: https://issues.apache.org/jira/browse/KYLIN-2243 Project: Kylin Issue Type

[Announce] Apache Kylin 1.6.0 released

2016-11-29 Thread ShaoFeng Shi
to report problems, and to get involved, visit the project website at https://kylin.apache.org/ -- Best regards, Shaofeng Shi 史少锋

Re: Apache Kylin 1.6.0 released

2016-11-29 Thread ShaoFeng Shi
Why not :-) 2016-11-30 9:05 GMT+08:00 Luke Han <luke...@gmail.com>: > I think maybe we could resend one...and correct what sebb has mentioned. > > How do you think? > > Thanks. > > > Best Regards! > - > > Luke Han > > On Wed, No

Re: Apache Kylin 1.6.0 released

2016-11-29 Thread ShaoFeng Shi
gt; Luke > > > Best Regards! > - > > Luke Han > > On Tue, Nov 29, 2016 at 6:17 PM, ShaoFeng Shi <shaofeng...@apache.org> > wrote: > > > I updated the template; next time it will exactly follow the guidance. > > Thanks Sebb for the suggestion. > > >

Re: Apache Kylin 1.6.0 released

2016-11-29 Thread ShaoFeng Shi
I updated the template; next time it will exactly follow the guidance. Thanks Sebb for the suggestion. 2016-11-29 17:27 GMT+08:00 sebb <seb...@gmail.com>: > On 28 November 2016 at 07:01, ShaoFeng Shi <shaofeng...@apache.org> wrote: > > The Apache Kylin team is pleased to a

Re: Is kylin not support Hive ORC tables with ACID properties.

2016-11-28 Thread ShaoFeng Shi
.m8...@gmail.com>: > Hi ShaoFeng Shi, > > Thanks for your reply, but here my case is different. I can able to create > and build cube with same data using text formatted tables but when i tried > to build cube with ORC formatted tables with same data facing an issue. > > Regard

Re: problem on using kylin RESTful API "POST /kylin/api/query"

2016-11-28 Thread ShaoFeng Shi
G_COUNT where GROUPID=DS023 group > by GROUPID LIMIT 1\": From line 1, column 101 to line 1, column 109: Column > 'DS023' not found in any table"} > -- Best regards, Shaofeng Shi 史少锋

Re: Is kylin not support Hive ORC tables with ACID properties.

2016-11-28 Thread ShaoFeng Shi
Hi Suresh, Another user also got similar problem, and I replied in the user@ group; Just minute ago I forwarded it to dev@ group; please take a look and let me know whether it is the same:

Apache Kylin 1.6.0 released

2016-11-27 Thread ShaoFeng Shi
/ -- Best regards, Shaofeng Shi 史少锋

[RESULT] [VOTE] Release apache-kylin-1.6.0 (RC2)

2016-11-26 Thread ShaoFeng Shi
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 5 binding +1s: Shaofeng Shi Dong Li Hongbin Ma Dayue Gao Luke Han 0 non-binding +1s: No 0s or -1s. Therefore I am delighted to announce that the proposal to release Apache

[jira] [Created] (KYLIN-2231) Layer cubing tolerates some insane records, but in-mem cubing doesn't

2016-11-25 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2231: --- Summary: Layer cubing tolerates some insane records, but in-mem cubing doesn't Key: KYLIN-2231 URL: https://issues.apache.org/jira/browse/KYLIN-2231 Project: Kylin

Re: cube auto merge bug

2016-11-25 Thread ShaoFeng Shi
list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[VOTE] Release apache-kylin-1.6.0 (RC2)

2016-11-23 Thread ShaoFeng Shi
of at least three +1 PPMC votes are cast. [ ] +1 Release this package as Apache Kylin 1.6.0 [ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (binding) -- Best regards, Shaofeng Shi 史少锋

Re: Kylin Job Server is single-point

2016-11-20 Thread ShaoFeng Shi
Kylin process, you need some tools. Could be as > simple as a cron job, to detect Kylin PID periodically and restart it when > it's dead. > > Yang > > On Sat, Nov 19, 2016 at 6:00 PM, 路加126 <luji...@126.com> wrote: > > > Thanks Shaofeng! > > > > > >

Re: Consulting "Define data model"

2016-11-20 Thread ShaoFeng Shi
e of the day" column” , these two > options is useless? > > Because when the model has been edited, “*Partition Date Column*” can > only be edited. > > > > > > Thanks! > -- Best regards, Shaofeng Shi 史少锋

Re: 答复: [DISCUSS] Naming convention for config files and properties

2016-11-19 Thread ShaoFeng Shi
he result is consistent with common hadoop config names, i.e. > > > dfs.namenode.servicerpc-bind-host > > > > > > Good example: kylin.metadata.url, kylin.storage.url > > > Bad example: kylin.job.mapreduce.mapper.input.rows (should be > > > kyiln.job.mapreduce.mapper-input-rows) > > > Bad example: kylin.cube.dimension.customEncodingFactories (should be > > > kylin.cube.dimension.custom-encoding-factories) > > > > > > Please share your thoughts. Want to have a consensus with the majority > > > first, then comes to implementation. > > > > > > > > > Cheers > > > Yang > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: some confuse about Mandatory Dimensions

2016-11-17 Thread ShaoFeng Shi
and b=yyy” or “select > count(distinct u) from table where A=xxx ”. The result is correct > > In my opinion, all of the query condition must be set (A=xxx,B=,C=zzz) > ,the kylin sql can be wrok, > > The question is How the Kylin query the result and the distinct value is > right ? that is unbelievable > > > > > > -- > > With Warm regards > > Yiming Liu (刘一鸣) > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2205) Use column name as the default measure name

2016-11-17 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2205: --- Summary: Use column name as the default measure name Key: KYLIN-2205 URL: https://issues.apache.org/jira/browse/KYLIN-2205 Project: Kylin Issue Type

[jira] [Created] (KYLIN-2204) Export result set as CSV, the double-byte characters are messed up

2016-11-17 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2204: --- Summary: Export result set as CSV, the double-byte characters are messed up Key: KYLIN-2204 URL: https://issues.apache.org/jira/browse/KYLIN-2204 Project: Kylin

[CANCELLED][VOTE] Release apache-kylin-1.6.0 (RC1)

2016-11-13 Thread ShaoFeng Shi
Hi all, The vote for apache-kylin-1.6.0 (RC1) has been cancelled; The RC2 will be sent out soon; thanks for your attention! -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2178) CubeDescTest is unstable

2016-11-13 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2178: --- Summary: CubeDescTest is unstable Key: KYLIN-2178 URL: https://issues.apache.org/jira/browse/KYLIN-2178 Project: Kylin Issue Type: Bug Components

Re: 答复: [VOTE] Release apache-kylin-1.6.0 (RC1)

2016-11-13 Thread ShaoFeng Shi
org.apache.maven.surefire.booter.ForkedBooter. > >> > invokeProviderInSameClassLoader(ForkedBooter.java:290) > >> > at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess( > >> > ForkedBooter.java:242) > >> > at org.apache.maven.surefire

Re: [VOTE] Release apache-kylin-1.6.0 (RC1)

2016-11-12 Thread ShaoFeng Shi
Dayue, please go ahead; Let's evaluate the impaction after you fix it and then decide whether include it. 2016-11-12 21:01 GMT+08:00 Dayue <dayue_...@163.com>: > Shall we include KYLIN-2173 in this release? I can fix it tonight. > > > > 在 2016年11月10日,下午2:40,ShaoFeng Shi &l

Re: [VOTE] Release apache-kylin-1.6.0 (RC1)

2016-11-12 Thread ShaoFeng Shi
er. > executeTestSet(JUnit4Provider.java:238) > > at org.apache.maven.surefire.junit4.JUnit4Provider.invoke( > JUnit4Provider.java:161) > > at org.apache.maven.surefire.booter.ForkedBooter. > invokeProviderInSameClassLoader(ForkedBooter.java:290) > > at org.apache.maven.sure

Re: Why we need gridtable?

2016-11-10 Thread ShaoFeng Shi
erstand than kylin 1.2, mainly because of > introduce of gridtable, so I want to ask why gridtable is introduced? > > -- > Ding Dinghua > -- Best regards, Shaofeng Shi 史少锋

Re: Cube Merge Error

2016-11-10 Thread ShaoFeng Shi
; at java.lang.Thread.run(Thread.java:745) > > > > > > In /bin/ setenv.sh : > > export KYLIN_JVM_SETTINGS="-Xmx100g -Xms100g -Xmn2g -XX:+UseG1GC > -XX:MaxPermSize=128M -verbose:gc -XX:+PrintGCDetails > > -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ > -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLo > > gFileSize=64M" > > > > but this error still appear,so Where is to modify the parameters can be > solved ?? > > > > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: 答复: hadoop2.2 imcompatible proble

2016-11-10 Thread ShaoFeng Shi
ages > sent to and from us may be monitored to ensure compliance with internal > policies and to protect our business. > Emails are not secure and cannot be guaranteed to be error free as they > can be intercepted, amended, lost or destroyed, or contain viruses. Anyone > who communicates with us by email is taken to accept these risks. > > 收发邮件者请注意: > 本邮件含保密信息,若误收本邮件,请务必通知发送人并直接删去,不得使用、传播或复制本邮件。 > 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 > > > -- Best regards, Shaofeng Shi 史少锋

[VOTE] Release apache-kylin-1.6.0 (RC1)

2016-11-09 Thread ShaoFeng Shi
of at least three +1 PPMC votes are cast. [ ] +1 Release this package as Apache Kylin 1.6.0 [ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (binding) -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2171) Release 1.6.0

2016-11-08 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2171: --- Summary: Release 1.6.0 Key: KYLIN-2171 URL: https://issues.apache.org/jira/browse/KYLIN-2171 Project: Kylin Issue Type: Task Reporter: Shaofeng

Re: Re: Can Kylin load data from Hbase?

2016-11-07 Thread ShaoFeng Shi
his: http://kylin.apache.org/development/plugin_arch.html > > > As Data Source, there is only two types: Hive and Kafka (in progress) > > > > > > (I don't understand your second Q) > > > > > > 2016-11-07 7:40 GMT+01:00 446463...@qq.com <446463...@qq.com>: > > > > > > > Hi: > > > > Can Kylin load data from HBase? > > > > And floating table which is anomaly in Hbase? > > > > > > > > > > > > > > > > 446463...@qq.com > > > > > > > > > > > > > > > -- > > Best regards, > > Amuro Copperfield > > > -- Best regards, Shaofeng Shi 史少锋

Re: Cube Designer, In TOP_N Expression, the choice of SUM|ORDER BY Column and Group By Column are all measures column

2016-11-07 Thread ShaoFeng Shi
in SUM|ORDER BY Column and choose a dimension column in Group BY Column, > > But I only can choose a measure column in all SUM|ORDER BY Column and > > Group By Column. > > > > Is it a bug in shown, how can solve it? Thanks! > > > > ORDER | SUM BY Column > > > > > > > > Group By Column: > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: [Draft][REPORT] Apache Kylin - Nov 2016

2016-11-06 Thread ShaoFeng Shi
urrently 23 PMC members and committers. - No new PMC member committer >> added in the last 3 months - Messages on the dev mailing list after last >> report: 720 - Messages on the user mailing list after last report: 402 - >> 224 JIRA tickets created after last report - 239 JIRA tickets >> closed/resolved after last report ## Releases: - The latest >> release, v1.5.4.1, released on 2016-09-28 - v1.5.4, released on 2016-09-15 > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2165) Use hive table statistics data to get the total count

2016-11-06 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2165: --- Summary: Use hive table statistics data to get the total count Key: KYLIN-2165 URL: https://issues.apache.org/jira/browse/KYLIN-2165 Project: Kylin Issue Type

Re: cube buiding wait in map sort stage

2016-11-04 Thread ShaoFeng Shi
1-Dimension > > wait in this step's map sort stage for 1 hour until map task > timeout,then > > container kill it. > > excuse me, how to deal with this problem.. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2162) Improve the cube validation error message

2016-11-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2162: --- Summary: Improve the cube validation error message Key: KYLIN-2162 URL: https://issues.apache.org/jira/browse/KYLIN-2162 Project: Kylin Issue Type

[jira] [Created] (KYLIN-2161) On saving a DataModel Kylin didn't do enough validation

2016-11-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2161: --- Summary: On saving a DataModel Kylin didn't do enough validation Key: KYLIN-2161 URL: https://issues.apache.org/jira/browse/KYLIN-2161 Project: Kylin Issue

Re: 答复: 答复: how to modify kylin.cube.aggrgroup.max.combination

2016-11-03 Thread ShaoFeng Shi
now the default value, and kylin.properties doesn't > contain it by default > > -邮件原件- > 发件人: ShaoFeng Shi [mailto:shaofeng...@apache.org] > 发送时间: 2016年11月3日 16:48 > 收件人: dev > 主题: Re: 答复: how to modify kylin.cube.aggrgroup.max.combination > > The value need be an

Re: 答复: how to modify kylin.cube.aggrgroup.max.combination

2016-11-03 Thread ShaoFeng Shi
.lang.IllegalStateException: CubeInstance desc not found > 'TXN_CUBE_1010', at /cube/txn_cube_1010.json” >After deleting " kylin.cube.aggrgroup.max.combination= 4294967296", it > go back normal. > > -邮件原件- > 发件人: ShaoFeng Shi [mailto:shaofeng.

Re: 答复: 答复: How to filter dimension from look-up table

2016-11-03 Thread ShaoFeng Shi
Great, enjoy! 2016-11-03 16:21 GMT+08:00 胡志华(万里通科技及数据中心商务智能团队数据分析组) < huzhihua...@pingan.com.cn>: > Thank you shaofeng, > > Done successfully as your tips > > -邮件原件----- > 发件人: ShaoFeng Shi [mailto:shaofeng...@apache.org] > 发送时间: 2016年11月2日 17:24 > 收件人: d

Re: [jira] [Created] (KYLIN-2154) source table rows will be skipped if TOPN's group column contains NULL values

2016-11-03 Thread ShaoFeng Shi
415) > > at org.apache.hadoop.security.UserGroupInformation.doAs( > > UserGroupInformation.java:1628) > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.3.4#6332) > > > -- Best regards, Shaofeng Shi 史少锋

Re: how to modify kylin.cube.aggrgroup.max.combination

2016-11-03 Thread ShaoFeng Shi
信息,若误收本邮件,请务必通知发送人并直接删去,不得使用、传播或复制本邮件。 > 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 > > ******** > -- Best regards, Shaofeng Shi 史少锋

Re: 13 Step Name: Load HFile to HBase Table

2016-11-02 Thread ShaoFeng Shi
sage in context: http://apache-kylin.74782.x6. > nabble.com/13-Step-Name-Load-HFile-to-HBase-Table-tp6144p6164.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: 13 Step Name: Load HFile to HBase Table

2016-11-02 Thread ShaoFeng Shi
bble.com/13-Step-Name-Load-HFile-to-HBase-Table-tp6144p6162.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

<    5   6   7   8   9   10   11   12   13   >