Re: get

2017-03-16 Thread ShaoFeng Shi
-step-18-Build-Cube-tp7378p7452.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: get

2017-03-15 Thread ShaoFeng Shi
uild-Cube-tp7378p7444.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Hbase Region server goes down

2017-03-14 Thread ShaoFeng Shi
.hbase.client.RegionServerCallable. > prepare(RegionServerCallable.java:79) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries( > RpcRetryingCaller.java:124) > ... 18 more > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Hbase-Region-server-goes-down-tp7437.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2504) Clone cube didn't keep the "engine_type" property

2017-03-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2504: --- Summary: Clone cube didn't keep the "engine_type" property Key: KYLIN-2504 URL: https://issues.apache.org/jira/browse/KYLIN-2504 Project: Kylin

[jira] [Created] (KYLIN-2503) Spark cubing step should show YARN app link

2017-03-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2503: --- Summary: Spark cubing step should show YARN app link Key: KYLIN-2503 URL: https://issues.apache.org/jira/browse/KYLIN-2503 Project: Kylin Issue Type

[jira] [Created] (KYLIN-2502) "Create flat table" and "redistribute table" steps don't show YARN application link

2017-03-11 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2502: --- Summary: "Create flat table" and "redistribute table" steps don't show YARN application link Key: KYLIN-2502 URL: https://issues.apache.o

[jira] [Created] (KYLIN-2499) HybridInstance cache wasn't cleaned when underlying Cube be updated

2017-03-10 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2499: --- Summary: HybridInstance cache wasn't cleaned when underlying Cube be updated Key: KYLIN-2499 URL: https://issues.apache.org/jira/browse/KYLIN-2499 Project:

Re: kylin build cube时,状态一直是PENDING

2017-03-10 Thread ShaoFeng Shi
Hive: 0.14.0 > > Zookeeper: 3.4.6 > > Hbase: 0.98.8 > > > > 谢谢 > -- Best regards, Shaofeng Shi 史少锋

Re: 求 编译 kylin 源码 方法

2017-03-10 Thread ShaoFeng Shi
e Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: build cube with spark ERROR

2017-03-09 Thread ShaoFeng Shi
ce( > DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > > at org.datanucleus.NucleusContext.createStoreManagerForPropertie > s(NucleusContext.java:1187) > > at org.datanucleus.NucleusContext.initialise( > NucleusContext.java:356) > > at org.datanucleus.api.jdo.JDOPersistenceManagerFactory. > freezeConfiguration(JDOPersistenceManagerFactory.java:775) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl. > generateDataSources(ConnectionFactoryImpl.java:259) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl. > initialiseDataSources(ConnectionFactoryImpl.java:131) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl.( > ConnectionFactoryImpl.java:85) > > ... 85 more > > > > Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke > the "BONECP" plugin to create a ConnectionPool gave an erro > > r : The specified datastore driver ("com.mysql.jdbc.Driver") was not found > in the CLASSPATH. Please check your CLASSPATH specificatio > > n, and the name of the driver. > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl. > generateDataSources(ConnectionFactoryImpl.java:259) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl. > initialiseDataSources(ConnectionFactoryImpl.java:131) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl.( > ConnectionFactoryImpl.java:85) > > ... 85 more > > Caused by: org.datanucleus.store.rdbms.connectionpool. > DatastoreDriverNotFoundException: The specified datastore driver > ("com.mysql.jd > > bc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH > specification, and the name of the driver. > > at org.datanucleus.store.rdbms.connectionpool. > AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory. > java:58) > > at org.datanucleus.store.rdbms.connectionpool. > BoneCPConnectionPoolFactory.createConnectionPool( > BoneCPConnectionPoolFactory.ja > > va:54) > > at org.datanucleus.store.rdbms.ConnectionFactoryImpl. > generateDataSources(ConnectionFactoryImpl.java:238) > > ... 87 more > -- Best regards, Shaofeng Shi 史少锋

Re: Hive table design (multiple fact tables or rolled up)

2017-03-08 Thread ShaoFeng Shi
;>> tables into a single table vs keeping lots of tables with many fact/lookup >>>> tables. >>>> >>>> In short: >>>> should we do any transformations in Hive before presenting the tables >>>> to kylin for cubing?... >>>> >>>> Thanks >>>> >>> >>> >> >> >> -- >> >> >> Pushpinder S. Heer >> Senior Software Engineer >> m: 360-434-4354 <(360)%20434-4354> h: 509-884-2574 <(509)%20884-2574> >> > > > > -- > > > Pushpinder S. Heer > Senior Software Engineer > m: 360-434-4354 h: 509-884-2574 > -- Best regards, Shaofeng Shi 史少锋

Re: How kylin limit mapreduce task Memory!!!!

2017-03-08 Thread ShaoFeng Shi
秦殇》!健 : > > > When I build cube, it occured a error. as follws: > > Invaild Resouce request, requested Memory <0, or requested Memory> max > > configured, requestedMemory=122880, maxMemory=61440. > > > > > > My env is CDH5.7 + Kylin1.6. > > > > > > Please help me T_T!! > -- Best regards, Shaofeng Shi 史少锋

Re: Which cubing algorithm is better for building a cube

2017-03-07 Thread ShaoFeng Shi
e Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: get

2017-03-07 Thread ShaoFeng Shi
l.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/get-tp7378.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2490) Couldn't get cube size on Azure HDInsight

2017-03-07 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2490: --- Summary: Couldn't get cube size on Azure HDInsight Key: KYLIN-2490 URL: https://issues.apache.org/jira/browse/KYLIN-2490 Project: Kylin Issue Type

Re: error while refreshing cube with curl script

2017-03-06 Thread ShaoFeng Shi
g this script. But Same script is working > properly in Dev environment but facing issue in production. > > > {"url":"http:// > *.*.*.*:7070/kylin/api/cubes/AL_RPT_PROD/rebuild","exception":null} > > > Please help me to solve this issue. > > > Regards, > > Suresh > -- Best regards, Shaofeng Shi 史少锋

Re: 关于kylin的一个问题

2017-03-06 Thread ShaoFeng Shi
t; [image: Catch.jpg] > 想问一下,这里为什么程序会去hdfs上找Hive包?是不是哪里需要配置? > > -- > wang...@aspirecn.com > -- > Best Regard > ZhouQianhao > -- Best regards, Shaofeng Shi 史少锋

Tutorial for Spark cubing

2017-03-06 Thread ShaoFeng Shi
Hi, Newly added a tutorial on using Apache Spark as the cube engine in v2.0 beta; Your comment/suggestion is welcomed: https://kylin.apache.org/docs16/tutorial/cube_spark.html -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2484) Kylin spark engine doesn't support source from Kafka

2017-03-05 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2484: --- Summary: Kylin spark engine doesn't support source from Kafka Key: KYLIN-2484 URL: https://issues.apache.org/jira/browse/KYLIN-2484 Project: Kylin Issue

Re: Kylin with Spark

2017-03-05 Thread ShaoFeng Shi
2017-03-03 11:42 GMT+08:00 Lovelesh Gupta : > Hi Team , > > Any good Blog and examples for using kylin with spark in Java. > Current documentation does not give much idea! > > Thanks > Lovelesh > -- Best regards, Shaofeng Shi 史少锋

Re: 求 编译 kylin 源码 方法

2017-03-05 Thread ShaoFeng Shi
g list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Which version of spark is kylin 2.0 Beta using ?

2017-03-03 Thread ShaoFeng Shi
Xu" wrote: Hi, Shaofeng, Can I change the spark version to spark 2.1 using “kylin.engine.spark-conf” to config ?  在 2017年3月1日,下午5:27,ShaoFeng Shi 写道: Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly introduce this, later will add a doc to instruct on how t

Re: Which version of spark is kylin 2.0 Beta using ?—— How can others get data used in your test?

2017-03-03 Thread ShaoFeng Shi
anks again. Mars. 在 2017年3月1日,下午5:27,ShaoFeng Shi 写道: Kylin embed a spark, which is spark v1.6.3; I added a blog to briefly introduce this, later will add a doc to instruct on how to config: https://kylin.apache.org/blog/2017/02/23/by-layer-spark-cubing/ To customize the spark conf, please

Re: Apache Kylin 2.0.0 beta is ready for download

2017-02-25 Thread ShaoFeng Shi
ng list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2465) Web page still has "Streaming cube build is not supported on UI" statements

2017-02-23 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2465: --- Summary: Web page still has "Streaming cube build is not supported on UI" statements Key: KYLIN-2465 URL: https://issues.apache.org/jira/browse/KYLIN-2465

Re: cube build error in step 4

2017-02-21 Thread ShaoFeng Shi
: Failed to create dictionary on xxx.xxx.ROWKEY > caused by CacheLoader returned null for key 58a0b00d40808. > How to slove this ? > > > 446463...@qq.com > -- Best regards, Shaofeng Shi 史少锋

Re: No build all dimensions

2017-02-21 Thread ShaoFeng Shi
/No-build-all-dimensions-tp7263p7264.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: query meet java heap space,help!

2017-02-17 Thread ShaoFeng Shi
lin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

Re: kylin sql query with Weird error

2017-02-16 Thread ShaoFeng Shi
mn", "value": > "IS_OUT_SESSION", "next_parameter": null }, "returntype": "bigint" }, > "dependent_measure_ref": null } ], "dictionaries": [], "rowkey": { > "rowkey_columns": [ { "column": "YEAR", "encoding": "dict", "isShardBy": > false }, { "column": "QUARTER_CN", "encoding": "dict", "isShardBy": false > }, { "column": "MONTH_CN", "encoding": "dict", "isShardBy": false }, { > "column": "DATE", "encoding": "dict", "isShardBy": false }, { "column": > "PROVINCE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "CITY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_COUNTRY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_REGION_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_PROVINCE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_CITY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "TYPE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "LOGIN_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "REGISTER_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "BROWSER_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "VERSION_NAME", "encoding": "dict", "isShardBy": false } ] }, > "hbase_mapping": { "column_family": [ { "name": "F1", "columns": [ { > "qualifier": "M", "measure_refs": [ "_COUNT_", "PV", "OUT_SESSION" ] } ] }, > { "name": "F2", "columns": [ { "qualifier": "M", "measure_refs": [ "UV" ] } > ] } ] }, "aggregation_groups": [ { "includes": [ "YEAR", "QUARTER_CN", > "MONTH_CN", "DATE", "PROVINCE_NAME", "CITY_NAME", "USER_COUNTRY_NAME", > "USER_REGION_NAME", "USER_PROVINCE_NAME", "USER_CITY_NAME", "TYPE_NAME", > "LOGIN_NAME", "REGISTER_NAME", "BROWSER_NAME", "VERSION_NAME" ], > "select_rule": { "hierarchy_dims": [ [ "YEAR", "QUARTER_CN", "MONTH_CN", > "DATE" ], [ "PROVINCE_NAME", "CITY_NAME" ], [ "USER_COUNTRY_NAME", > "USER_REGION_NAME", "USER_PROVINCE_NAME", "USER_CITY_NAME" ] ], > "mandatory_dims": [], "joint_dims": [] } } ], "signature": > "odaKh4hY+Mb4uRggEJJanQ==", "notify_list": [], "status_need_notify": [ > "ERROR", "DISCARDED", "SUCCEED" ], "partition_date_start": 148590720, > "partition_date_end": 31536, "auto_merge_time_ranges": [ 60480, > 241920 ], "retention_range": 0, "engine_type": 2, "storage_type": 2, > "override_kylin_properties": { "kylin.cube.aggrgroup.max.combination": > "16384" } } > > > > > > > -- > 祝身体健康,万事如意 ^_^ > -- Best regards, Shaofeng Shi 史少锋

Re: Help needed - Issue when build the cube

2017-02-16 Thread ShaoFeng Shi
xecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > result code:2 > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Help-needed-Issue-when-build-the-cube-tp7191.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Trouble with building cube at step 3!!cry!cry!

2017-02-16 Thread ShaoFeng Shi
er( > ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > result code:2 > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Trouble-with-building-cube-at-step-3-cry-cry-tp7172.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: 你好,请教下关于kylin高可用的一些问题,谢谢

2017-02-16 Thread ShaoFeng Shi
高可用的一些问题 > > 1、集群模式下的负载均衡器使用哪种开源产品?ngnix是否可以满足kylin的要求? > > 2、job engine在集群中只能部署一个,job节点如何实现高可用? > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Kylie Cube Building,Merging and Refreshing getting stucked at Create HFile Step

2017-02-16 Thread ShaoFeng Shi
ermission > issues, and thus all other subsequent requests for creating hbase table > handlers started queueing.Got the issue while looking at hbase dump. > > Regards,Ketan > > > Sent from my Samsung Galaxy smartphone. > ---- Original message From: ShaoFeng Shi <

Re: Kylie Cube Building,Merging and Refreshing getting stucked at Create HFile Step

2017-02-15 Thread ShaoFeng Shi
e a cube which is getting stucked for all type of jobs: > Build,Refresh and Merge on the “Create Hfile” step. > > Any idea, how can I get back to normalcy. > We are using Kylin 1.5.4.1 and the cube is being built incrementally on an > hourly basis. > > > Thanks, > kdcool6932 -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2445) UI: select cube engine in "Advanced setting" page

2017-02-13 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2445: --- Summary: UI: select cube engine in "Advanced setting" page Key: KYLIN-2445 URL: https://issues.apache.org/jira/browse/KYLIN-2445 Project: Kylin

[jira] [Created] (KYLIN-2444) Kylin should release different hadoop versions to maven repository

2017-02-13 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2444: --- Summary: Kylin should release different hadoop versions to maven repository Key: KYLIN-2444 URL: https://issues.apache.org/jira/browse/KYLIN-2444 Project: Kylin

Re: org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.NoSuchMethodError: org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashLong(J

2017-02-12 Thread ShaoFeng Shi
rg-apache- > kylin-job-exception-ExecuteException-javaI-tp7126p7138.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: URGENT !!! Help needed - Issue executing query in Kylin

2017-02-11 Thread ShaoFeng Shi
e.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)> > > > > at > > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run( > RpcExecutor.java:185)> > > > > at > > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run( > RpcExecutor.java:165)> > > > > Caused by: java.lang.NoSuchMethodError: > > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment. > getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;> > > > > at > > org.apache.kylin.storage.hbase.cube.v2.coprocessor. > endpoint.CubeVisitService.visitCube(CubeVisitService.java:177)> > > > > at > > org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated. > CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164)> > > > > at > > org.apache.hadoop.hbase.regionserver.HRegion. > execService(HRegion.java:7797)> > > > > at > > org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion( > RSRpcServices.java:1982)> > > > > at > > org.apache.hadoop.hbase.regionserver.RSRpcServices. > execService(RSRpcServices.java:1964)> > > > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2. > callBlockingMethod(ClientProtos.java:33652)> > > > > at > > org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)> > > > > ... 3 more> > > > at > > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1269)> > > > > > at > > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod( > AbstractRpcClient.java:226)> > > > > at > > org.apache.hadoop.hbase.ipc.AbstractRpcClient$ > BlockingRpcChannelImplementation.callBlockingMethod( > AbstractRpcClient.java:331)> > > > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$ > BlockingStub.execService(ClientProtos.java:34118)> > > > > at > > org.apache.hadoop.hbase.protobuf.ProtobufUtil. > execService(ProtobufUtil.java:1627)> > > > > ... 13 more> > > > > > Thanks,> > > Manoj Jain> > > > > > > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/URGENT-Help-needed-Issue-executing-query-in- > Kylin-tp7046p7131.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: about ConnectException when Extract Fact Table Distinct Columns

2017-02-11 Thread ShaoFeng Shi
View this message in context: http://apache-kylin.74782.x6. > nabble.com/about-ConnectException-when-Extract- > Fact-Table-Distinct-Columns-tp5253p7143.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Problem while querying Cube

2017-02-06 Thread ShaoFeng Shi
seem be :) pls ignore and go ahead. Lei, pls use your origin thread to discuss the problem. Get Outlook for iOS On Mon, Feb 6, 2017 at 10:39 PM +0900, "hongbin ma" wrote: are we mixing two separate issues in the same thread here? On Mon, Feb 6, 2017 at 9:26 PM, Sh

Re: Problem while querying Cube

2017-02-06 Thread ShaoFeng Shi
53,817 DEBUG [pool-8-thread-3] dao.ExecutableDao:210 : updating job output, id: baa9531d-fc11-4a60-aa5e-a069d4bee3c2-02 Thx Lei Wang ________ From: ShaoFeng Shi Sent: Sunday, February 5, 2017 22:19 To: dev Subject: Re: Problem while querying Cube Most of the developers are

Re: Problem while querying Cube

2017-02-05 Thread ShaoFeng Shi
eption. What will happen if we remove this check? > > Thanks. > > On Sat, Feb 4, 2017 at 8:12 PM, ShaoFeng Shi > wrote: > > > I don't know what that check is necessary, as there is a todo which says > > will remove that someday: > > https://github.com/apac

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
s.defaultFS'? > > > 发自我的 iPhone > > > 在 2017年2月5日,21:38,ShaoFeng Shi 写道: > > > > No, from the output of "hdfs fs -ls" the file does exist on HDFS; In > hive's > > context its default fs is HDFS, so it works as expected. The behavior of

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
derstanding correct? > > 发自我的 iPhone > > > 在 2017年2月5日,20:54,ShaoFeng Shi 写道: > > > > From the error it tried to seek the file from local file system, instead > of > > HDFS. Please check whethere "fs.DefaultFS" in your environment was set to > &

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
at org.apache.hadoop.fs.FileSystem.listStatus( > FileSystem.java:1515) > at org.apache.hadoop.fs.FileSystem.listStatus( > FileSystem.java:1555) > at org.apache.hadoop.fs.ChecksumFileSystem.listStatus( > ChecksumFileSystem.java:574) > at org.apache.kylin.source.hive.HiveMRInput$ > RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:338) > at org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable.java:113) > at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork( > DefaultChainedExecutable.java:57) > at org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable.java:113) > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$ > JobRunner.run(DefaultScheduler.java:136) > > > > Thx > Lei Wang > > -- Best regards, Shaofeng Shi 史少锋

Re: Re: Re: build cube and hbase coprocessor not avaliable

2017-02-05 Thread ShaoFeng Shi
#x27;ADMIN'}} > > COLUMN FAMILIES DESCRIPTION > > {NAME => 'F1', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'NONE', > REPLICATION_SCOPE => '0', VERSIONS => '1 > > ', COMPRESSION => 'NONE', MI

Re: Problem while querying Cube

2017-02-04 Thread ShaoFeng Shi
08:00 Shailesh Prajapati : > Hi ShaoFeng, > > Here is the gist link, > > https://gist.github.com/shaipraj/780a3dcc80aa2080911b7348c76f5b88 > > Thanks. > > On Sat, Feb 4, 2017 at 2:24 PM, ShaoFeng Shi > wrote: > > > Hi Shailesh, there is no attachement (attacheme

[jira] [Created] (KYLIN-2424) Optimize the integration test's performance

2017-02-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2424: --- Summary: Optimize the integration test's performance Key: KYLIN-2424 URL: https://issues.apache.org/jira/browse/KYLIN-2424 Project: Kylin Issue

Re: Problem while querying Cube

2017-02-04 Thread ShaoFeng Shi
On Sat, Feb 4, 2017 at 12:59 PM, ShaoFeng Shi > wrote: > >> Hi Shailesh, >> >> Could you please provide the error trace? We need to know where the error >> got thrown. Thanks. >> >> 2017-02-03 18:06 GMT+08:00 Shailesh Prajapati : >> >>

Re: Problem while querying Cube

2017-02-03 Thread ShaoFeng Shi
ne more observation, with limit less than 5 even non aggregate queries are > also working. > Please help us resolving this issue. let us know for any other information. > > Thanks > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2422) NumberDictionary support for decimal with extra 0 after "."

2017-02-03 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2422: --- Summary: NumberDictionary support for decimal with extra 0 after "." Key: KYLIN-2422 URL: https://issues.apache.org/jira/browse/KYLIN-2422 Proj

[jira] [Created] (KYLIN-2421) Add spark engine to Integration Test

2017-02-03 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2421: --- Summary: Add spark engine to Integration Test Key: KYLIN-2421 URL: https://issues.apache.org/jira/browse/KYLIN-2421 Project: Kylin Issue Type: Sub-task

Re: Urgent!!! exact sum of values are not coming in kylin when executed query in Insight

2017-02-02 Thread ShaoFeng Shi
by date means it should give like below, > > sum date > 21 1/12/2017 > 1 1/13/2017 > > But kylin when i execute sum query getting below result, > > sum date > 16 1/12/2017 > 1 1/13/2017 > > We can see clearly its considering distinct values for summarizing result. &

Re: Re: build cube and hbase coprocessor not avaliable

2017-02-02 Thread ShaoFeng Shi
and when build cube done . I execute query "select * from xxx" hits > error below: > >> > >> > >> org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No > registered coprocessor service found for name CubeVisitService in region > KYLIN_GA464RXYQX,,1484635706236.9aac092cc3bd8855999c7a8734f09842. > >> > >> > >> how can I resolve this problem? > >> > >> > >> thanks > >> > -- Best regards, Shaofeng Shi 史少锋

Re: Does Kylin support percent function?

2017-02-02 Thread ShaoFeng Shi
t; 3google 987 > > 4google 789 > > SELECT (SUM(CASE WHEN (host="local") then 1 else 0 end)/count(*)*100) > > as hostpercent FROM table1 > > The sql result is 50%. Mysql or hive can do this,but i don't know does > > kylin support this action? > > please help me, thank you! > > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: New document: "How to optimize cube build"

2017-01-25 Thread ShaoFeng Shi
partitions not > thousand (Hive 9452 there is an idea to solve this isn’t in progress)*" > > In Hive 2.0 will be a preview (only for testing) to solve this > > 2017-01-25 9:46 GMT+01:00 ShaoFeng Shi : > >> Hello, >> >> A new document is added for the prac

Re: org.apache.kylin.dict.CachedTreeMap use a couple classes from org.apache.hadoop.fs

2017-01-25 Thread ShaoFeng Shi
rialized CachedTreeMap to spark executor > tasks? Maybe every tasks initialize own CacheTreeMap instance is another > choice; > >Please feel free to change the code if you really need to serialize > CachedTreeMap, and let me know if there’s somewhere I could help. > > > 在

New document: "How to optimize cube build"

2017-01-25 Thread ShaoFeng Shi
Hello, A new document is added for the practices of cube build. Any suggestion or comment is welcomed. We can update the doc later with feedbacks; Here is the link: https://kylin.apache.org/docs16/howto/howto_optimize_build.html -- Best regards, Shaofeng Shi 史少锋

Re: 答复: more than 1 append dict for globalDict

2017-01-24 Thread ShaoFeng Shi
ike : "./bin/metastore.sh remove /dict/APPLYDATA_DSDCADMIN.FLY_ ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict" 2017-01-24 20:14 GMT+08:00 ShaoFeng Shi : > @Yerui, do you have suggestion on this? > > 2017-01-24 18:19 GMT+08:00 市场中心-ZHANGDA32698 : > >> Hi Sh

Re: 答复: more than 1 append dict for globalDict

2017-01-24 Thread ShaoFeng Shi
ut 2’ > > How can I resolve this and continue my cube building? Any suggestion is > appreciated, thanks! > > > > > > *发件人:* ShaoFeng Shi [mailto:shaofeng...@apache.org] > *发送时间:* 2017年1月24日 17:01 > *收件人:* user > *主题:* Re: more than 1 append dict for globalDict >

Re: kylin cube migrating

2017-01-22 Thread ShaoFeng Shi
w this message in context: http://apache-kylin.74782.x6. > nabble.com/kylin-cube-migrating-tp6000p7014.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Proposal for updating master branch to use HBase 1.x

2017-01-22 Thread ShaoFeng Shi
pacted and should choose correct branch > > > according to corresponding Hadoop distros. > > > > > > 1. HBase 0.98 users should use master-hbase0.98 branch > > > > > > 2. Others please use master branch > > > > > > Developers may

Re: sample.sh running error

2017-01-21 Thread ShaoFeng Shi
manager) > hive 1.1.0-cdh5.9.0(cloudera manager) > kylin apache-kylin-1.6.0-cdh5.7-bin > > > Thanks. > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/sample-sh-running-error-tp2952p6993.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: cube已经为ready状态,但无法执行查询语句

2017-01-21 Thread ShaoFeng Shi
:2278575...@qq.com > 联系地址: 深圳市福田区滨河大道澳知浩大楼5A室 -- Best regards, Shaofeng Shi 史少锋

Re: Guava version error

2017-01-21 Thread ShaoFeng Shi
guava versions. Should i > downgrade hadoop libs? > -- > Vasiliy I Ozerov > Sent with Airmail -- Best regards, Shaofeng Shi 史少锋

Re: build kylin code exception

2017-01-21 Thread ShaoFeng Shi
k1.8.0_45. > > > > -- 原始邮件 -- > *发件人:* "ShaoFeng Shi";; > *发送时间:* 2017年1月20日(星期五) 晚上11:20 > *收件人:* "dev"; > *主题:* Re: build kylin code exception > > Hi, could you please provide the command that you are executing? >

Re: build kylin code exception

2017-01-20 Thread ShaoFeng Shi
n.doExecute(DefaultMaven.java:347) > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154) > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582) > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214) > at org.apache.maven.cli.MavenCli.main(MavenCli.java:158) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at org.codehaus.plexus.classworlds.launcher.Launcher. > launchEnhanced(Launcher.java:289) > at org.codehaus.plexus.classworlds.launcher.Launcher. > launch(Launcher.java:229) > at org.codehaus.plexus.classworlds.launcher.Launcher. > mainWithExitCode(Launcher.java:415) > at org.codehaus.plexus.classworlds.launcher.Launcher. > main(Launcher.java:356) > [DEBUG] URLResourceLoader: Exception when looking for 'LICENSE.txt > java.net.MalformedURLException: no protocol: LICENSE.txt > -- Best regards, Shaofeng Shi 史少锋

Re: Streaming Build Timestamp parse error

2017-01-20 Thread ShaoFeng Shi
I think you would prefer to write SQL like "where partitionCol > ''2017-01-12 04:00:00", instead of "where partitionCol > 1484236798000", right? 2017-01-20 16:00 GMT+08:00 ShaoFeng Shi : > Hi Wu xin, > > It is okay to use the unix time as the &quo

Re: Streaming Build Timestamp parse error

2017-01-20 Thread ShaoFeng Shi
e, not epoch time in miliseconds; 2017-01-20 14:22 GMT+08:00 Mario Copperfield : > I think it might make a confuse, because we usually define timestamp as an > unix timestamp such as “1484236798000" > > On 20 Jan 2017, 13:25 +0800, ShaoFeng Shi , wrote: > > "When i use timestamp

org.apache.kylin.dict.CachedTreeMap use a couple classes from org.apache.hadoop.fs

2017-01-19 Thread ShaoFeng Shi
.SLR_SEGMENT_CD=org.apache.kylin.dict.TrieDictionaryForest@dcfc7d11, DEFAULT.TEST_KYLIN_FACT.CAL_DT=DateStrDictionary [pattern=-MM-dd, baseId=0]}) -- Best regards, Shaofeng Shi 史少锋

Re: Streaming Build Timestamp parse error

2017-01-19 Thread ShaoFeng Shi
y configure. > > I change a little codes so that it can work but i found that the master > branch has changed the class “UpdateCubeInfoAfterBuildStep” so much that I > did not pull request. > > This is what my change: > https://github.com/xwhfcenter/kylin/commit/3e13bf244b82387107b

Re: Streaming Build Timestamp parse error

2017-01-19 Thread ShaoFeng Shi
Kylin's DefaultTimeParser will parse the long typed timestamp value like "1484236798000", so in this case you don't need specify "tsParser". 2017-01-19 22:18 GMT+08:00 : > I'll check tomorrow > > Sent from Alto > On Thursday, January 19, 2017 a

Re: Streaming Build Timestamp parse error

2017-01-19 Thread ShaoFeng Shi
{Nd}++) > > my kylin version is 1.6.0 > > > > -- > > View this message in context: http://apache-kylin.74782.x6. > > nabble.com/Streaming-Build-Timestamp-parse-error-tp6952.html > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2411) Kill MR job on pause

2017-01-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2411: --- Summary: Kill MR job on pause Key: KYLIN-2411 URL: https://issues.apache.org/jira/browse/KYLIN-2411 Project: Kylin Issue Type: Improvement

Re: Streaming Cube Bulkload

2017-01-18 Thread ShaoFeng Shi
hbase.RegionTooBusyException: org.apache.hadoop.hbase. > RegionTooBusyException: > > failed to get a lock in 6 ms. > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2409) Performance tunning for in-mem cubing

2017-01-18 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2409: --- Summary: Performance tunning for in-mem cubing Key: KYLIN-2409 URL: https://issues.apache.org/jira/browse/KYLIN-2409 Project: Kylin Issue Type: Improvement

[jira] [Created] (KYLIN-2404) Add "hive.merge.mapfiles" and "hive.merge.mapredfiles" to kylin_hive_conf.xml

2017-01-17 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2404: --- Summary: Add "hive.merge.mapfiles" and "hive.merge.mapredfiles" to kylin_hive_conf.xml Key: KYLIN-2404 URL: https://issues.apache.org/jira/browse/KYLIN-2404

Re: kylin1.6& HuaWei FusionInsight: load hive table on page. retrun error "java.lang.ClassNotFoundException:org.apache.hive.jdbc.HiveDriver"

2017-01-15 Thread ShaoFeng Shi
m > CLI beeline ) > > > 9. kylin.sh start (find: > /opt/huawei/Bigdata/FusionInsight_V100R002C60U10/ > FusionInsight-Hive-1.3.0/hive-1.3.0/bin/..//lib > :No such file or directory) > > > 10. Click "Model" on page. Click "load hive table" . Then retun error > message. > > > > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2393) Add "hive.auto.convert.join" and "hive.stats.autogather" to kylin_hive_conf.xml

2017-01-15 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2393: --- Summary: Add "hive.auto.convert.join" and "hive.stats.autogather" to kylin_hive_conf.xml Key: KYLIN-2393 URL: https://issues.apache.org/jira/browse/KYLIN-2393

Re: Cube build error @First Step : DFS Command incompatible with Ranger

2017-01-12 Thread ShaoFeng Shi
anger > is > > active > > > > Does it mean i can not use Kylin OLAP on Hortonworks with ranger > activated > > ? > > > > -- > > View this message in context: http://apache-kylin.74782.x6. > > nabble.com/Cube-build-error-First-Step-DFS-Command- > > incompatible-with-Ranger-tp6924.html > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

Re: Cube build error @First Step : DFS Command incompatible with Ranger

2017-01-12 Thread ShaoFeng Shi
on Hortonworks with ranger activated > ? > > -- > View this message in context: http://apache-kylin.74782.x6.n > abble.com/Cube-build-error-First-Step-DFS-Command-incompatib > le-with-Ranger-tp6924.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Spatial Functions on top of Kylin

2017-01-11 Thread ShaoFeng Shi
4782.x6. > nabble.com/Spatial-Functions-on-top-of-Kylin-tp6902.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Error in kylin with standalone HBase cluster

2017-01-11 Thread ShaoFeng Shi
Good; enjoy! 2017-01-11 18:01 GMT+08:00 柯南 : > Thank you! It works,and the file "yarn-site.xml" and "mapred-site.xml" > also should be replaced. > > > > > -- 原始邮件 -- > 发件人: "ShaoFeng Shi";; > 发送时间: 2017年1

Re: Consulting "EXTENDED_COLUMN"

2017-01-11 Thread ShaoFeng Shi
11-30 11:35 GMT+01:00 仇同心 : > > > > > > > Hi ,all > > > > I don’t understand the usage scenarios of EXTENDED_COLUMN,although I > saw > > > > this article “https://issues.apache.org/jira/browse/KYLIN-1313”. > > > > What,s the means about parameters of “

Re: Error in kylin with standalone HBase cluster

2017-01-10 Thread ShaoFeng Shi
ylin.properties,when to use for? > If I need to dependent hbase and hive/mapreduce cluster's on diffent hdfs, > counld kylin support that? Or may be in the future? > > > -- 原始邮件 -- > 发件人: "ShaoFeng Shi";; > 发送时间: 2017年1月10

[jira] [Created] (KYLIN-2375) Default cache size (10M) is too small

2017-01-10 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2375: --- Summary: Default cache size (10M) is too small Key: KYLIN-2375 URL: https://issues.apache.org/jira/browse/KYLIN-2375 Project: Kylin Issue Type: Improvement

Re: Error in kylin with standalone HBase cluster

2017-01-09 Thread ShaoFeng Shi
iveMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod( > RetryInvocationHandler.java:187) > at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke( > RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy24.getBlockLocations(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations( > DFSClient.java:1169) > ... 20 more -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2371) Allow overwrite default spark conf at cube level

2017-01-08 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2371: --- Summary: Allow overwrite default spark conf at cube level Key: KYLIN-2371 URL: https://issues.apache.org/jira/browse/KYLIN-2371 Project: Kylin Issue Type: Sub

Re: where kylin global dict store?

2017-01-05 Thread ShaoFeng Shi
1 > ], > [ > "BIRTHDAY", > 9193, > 2 > ], > [ > "NATIVE_PLACE", > 2268, > 2 > ], > [ > "LOCATION", > 15, > 1 > ] > ] > } > ], > "create_time_utc": 1483605652669, > "size_kb": 0, > "input_records_count": 0, > "input_records_size": 0 > } > - > I find all thing begin with /cube or /cube_desc or /model_desc or /table > or /table_ext related to my cube and I can't find anything with Global > Dictionary with my cube > Help me ! > > > > 446463...@qq.com > -- Best regards, Shaofeng Shi 史少锋

Re: Jenkins & SonarQube for Apache Kylin are ready

2017-01-05 Thread ShaoFeng Shi
Added the info to Kylin website: https://kylin.apache.org/development/ 2016-12-29 12:46 GMT+08:00 ShaoFeng Shi : > Great! Thanks Billy! > > 2016-12-29 9:27 GMT+08:00 Billy Liu : > >> Hi Dev team, >> >> Thanks the Apache INFR team, now we have set up the Jenkins Bui

[jira] [Created] (KYLIN-2359) Update job build step name

2017-01-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2359: --- Summary: Update job build step name Key: KYLIN-2359 URL: https://issues.apache.org/jira/browse/KYLIN-2359 Project: Kylin Issue Type: Improvement

[jira] [Created] (KYLIN-2358) CuboidReducer has too many "if (aggrMask[i])" checks

2017-01-04 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2358: --- Summary: CuboidReducer has too many "if (aggrMask[i])" checks Key: KYLIN-2358 URL: https://issues.apache.org/jira/browse/KYLIN-2358 Project: Kylin

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
ame nodes that hadoop > cluster. Is it correct? > I was thinked on put Kylin cluster mode on Docker Containers (Isolated from > Hadoop cluster) > > Thanks > > 2017-01-04 15:19 GMT+01:00 ShaoFeng Shi : > > > Here is the explainations in > > examples/test_case_dat

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
scenario is for development. 2017-01-04 17:14 GMT+08:00 Alberto Ramón : > is it the same remote mode than cluster mode ? > <http://kylin.apache.org/docs15/install/kylin_cluster.html> > > > > 2017-01-04 9:35 GMT+01:00 ShaoFeng Shi : > > > Hi Lei, > > > >

Re: When kylin.job.run.as.remote.cmd=true, all hadoop、hive、hbase commands can be in remote cluster?

2017-01-04 Thread ShaoFeng Shi
also work well? -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2351) Support using non-default HDFS path as kylin.env.hdfs-working-dir

2017-01-03 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2351: --- Summary: Support using non-default HDFS path as kylin.env.hdfs-working-dir Key: KYLIN-2351 URL: https://issues.apache.org/jira/browse/KYLIN-2351 Project: Kylin

Re: [Announce] New Apache Kylin committer Kaisen Kang

2017-01-03 Thread ShaoFeng Shi
gt; > > Congratulations and Welcome, Kaisen! > > > -- Best regards, Shaofeng Shi 史少锋

Re: Anomaly in Aggregation

2017-01-02 Thread ShaoFeng Shi
hll(10) (Error Rate < 9.75%). Please validate my finding. Thank you. > > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Anomaly-in-Aggregation-tp6808p6809.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: version with hive

2017-01-02 Thread ShaoFeng Shi
n context: http://apache-kylin.74782.x6. > nabble.com/version-with-hive-tp6793.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

[jira] [Created] (KYLIN-2344) Package spark into Kylin binary package

2016-12-30 Thread Shaofeng SHI (JIRA)
Shaofeng SHI created KYLIN-2344: --- Summary: Package spark into Kylin binary package Key: KYLIN-2344 URL: https://issues.apache.org/jira/browse/KYLIN-2344 Project: Kylin Issue Type: Task

<    5   6   7   8   9   10   11   12   13   14   >