Re: Is TOP_N measure support multiple dimensions(columns) in one group?

2016-05-15 Thread ShaoFeng Shi
(dim_city), (dim_industry) and (dim_city, dim_industrt), is kylin > 1.5.1 support those tow use cases? > > Best Regards! > -- Best regards, Shaofeng Shi

Re: how can i increase the count of reduce at Kylin_HFile_Generator_cubename_step?

2016-05-13 Thread ShaoFeng Shi
crease my reduces of mr at > Kylin_HFile_Generator_cubename_step? > > -- Best regards, Shaofeng Shi

Re: Can't build and refresh job

2016-05-13 Thread ShaoFeng Shi
t; javax.servlet.http.HttpServlet.service(HttpServlet.java:649) > > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > > > > I can use others kylin server to build or refresh this cube. > > And, when I restart this kylin server, I can refresh and build > this cube. So, I think that maybe a bug. > > Can you help me to fix it. > > > > Thanks, > > Yapu > > > -- Best regards, Shaofeng Shi

Re: Can't build and refresh job

2016-05-13 Thread ShaoFeng Shi
error job for > the cube. > If there have any running or error job about this cube. I think I can't > use others Kylin server to build and refresh it. > In this case, it may be a Kylin bug. > > -Original Message- > From: ShaoFeng Shi [mailto:shaofeng...@apache.org] > Sent: Friday

Re: java.lang.IllegalArgumentException:KeyValue size too large at Building Dimension Dictionary

2016-05-03 Thread ShaoFeng Shi
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable > .java:114) > > at > > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS > cheduler.java:124) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 > 45) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 > 15) > > at java.lang.Thread.run(Thread.java:745) > > > > result code:2 > > > > Thanks, > > > > Laney. > > > > -- Best regards, Shaofeng Shi

Re: 答复: problem happened at step "build base cuboid data"

2016-05-06 Thread ShaoFeng Shi
int > pt_log_dstring > partner_id string > > # Partition Information > # col_name data_type comment > > pt_log_dstring > partner_id string > Time taken: 0.124 seconds, Fetched: 47 row(s) > hive>

Re: CubeHFileJob fail with mr job configuration

2016-05-04 Thread ShaoFeng Shi
N_JOB_NAME)); > > -- > View this message in context: > http://apache-kylin.74782.x6.nabble.com/CubeHFileJob-fail-with-mr-job-configuration-tp4331p4387.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi

Re: Exception at build streaming cube

2016-05-05 Thread ShaoFeng Shi
t; > > > On Tue, May 3, 2016 at 3:59 PM, Nick Dimiduk <ndimi...@apache.org> > wrote: > > > > > On Thu, Apr 14, 2016 at 3:24 AM, ShaoFeng Shi <shaofeng...@apache.org> > > > wrote: > > > > > >> For how to setup streaming cubing,

Re: Table 'KYLIN_CAL_DT' not found

2016-05-05 Thread ShaoFeng Shi
> saml.context.serverName=hostname > saml.context.serverPort=443 > saml.context.contextPath=/kylin > > > ganglia.group= > ganglia.port=8664 > > ## Config for mail service > > # If true, will send email notification; > mail.enabled=false > mail.host= > mail.username= > mail.password= > mail.sender= > > ###config info for web### > > #help info ,format{name|displayName|link} ,optional > kylin.web.help.length=4 > kylin.web.help.0=start|Getting Started| > kylin.web.help.1=odbc|ODBC Driver| > kylin.web.help.2=tableau|Tableau Guide| > kylin.web.help.3=onboard|Cube Design Tutorial| > > #guide user how to build streaming cube > kylin.web.streaming.guide=http://kylin.apache.org/ > > #hadoop url link ,optional > kylin.web.hadoop= > #job diagnostic url link ,optional > kylin.web.diagnostic= > #contact mail on web page ,optional > kylin.web.contact_mail= > > ###config info for front### > > #env DEV|QA|PROD > deploy.env=DEV > > ###deprecated configs### > kylin.sandbox=true > kylin.web.hive.limit=2 > > > > wenye...@163.com > -- Best regards, Shaofeng Shi

Re: Re: Table 'KYLIN_CAL_DT' not found

2016-05-06 Thread ShaoFeng Shi
If the cube wasn't built, the query won't work; what's the error of the cube build? 2016-05-06 12:12 GMT+08:00 wenye...@163.com <wenye...@163.com>: > Build cube falied also! > But in the version of kylin 1.3 successfully ,before build cuse! > > > > wenye...@163.com >

Re: 答复: Can't create EnumerableAggregate

2016-05-05 Thread ShaoFeng Shi
e the > integrity of this email message, nor that this email message is free of > viruses, interceptions, or interference. > > > > > > 本邮件(包括任何附件)内容机密并受法律保护。如果您意外地收到这封邮件,请回复通知发件人并从当前系统中删除本邮件。任何未经授权者,严禁使用并部分或者全部的转发本条信息。任何未经授权的使用或传播都是被严格禁止的。远景能源与其分公司不对因不正确和不完整的转发此邮件包含的信息以及因任何因邮件延迟或对你的系统造成的损害而负责。远景能源不能保证此邮件的真实完整性,也不能确定此邮件是否含有病毒或者监听程序。 > This email message (including any attachments) is confidential and may be > legally privileged. If you have received it by mistake, please notify the > sender by return email and delete this message from your system. Any > unauthorized use or dissemination of this message in whole or in part is > strictly prohibited. Envision Energy Limited and all its subsidiaries shall > not be liable for the improper or incomplete transmission of the > information contained in this email nor for any delay in its receipt or > damage to your system. Envision Energy Limited does not guarantee the > integrity of this email message, nor that this email message is free of > viruses, interceptions, or interference. > -- Best regards, Shaofeng Shi

Re: kylin Exception log

2016-04-15 Thread ShaoFeng Shi
che.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > at > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > at > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > at > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > at > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) > -- Best regards, Shaofeng Shi

Re: kylin Exception log

2016-04-18 Thread ShaoFeng Shi
and $KYLIN_HOME/bin/find-hive-dependency.sh, and then check the output of them to see whether there is such a jar. 2016-04-18 9:56 GMT+08:00 陈佛林 <chenfo...@gmail.com>: > kylin version:1.5.0 > hadoop version: 2.5.0-cdh5.3.2 > hbase version : 0.98.6-cdh5.3.2 > > > 2016-04-16 9:14 GMT+08:0

Re: unable to login after adding user

2016-04-14 Thread ShaoFeng Shi
E_MODELER, ROLE_ANALYST" /> > > > > But unable to login. > > > > > > > > > > > > 郑江雨 云平台 > > Phone: 15155195496 > > > -- Best regards, Shaofeng Shi

Re: kylin 1.5.1 start failed on hdpp 2.2.4

2016-05-09 Thread ShaoFeng Shi
jar > /usr/hdp/2.2.4.2-2/hbase/lib/ranger-hbase-plugin-0.4.0.2.2.4.2-2.jar > /usr/hdp/2.2.4.2-2/hbase/lib/ranger-plugins-audit-0.4.0.2.2.4.2-2.jar > /usr/hdp/2.2.4.2-2/hbase/lib/ranger-plugins-common-0.4.0.2.2.4.2-2.jar > /usr/hdp/2.2.4.2-2/hbase/lib/ranger-plugins-cred-0.4.0.2.2.4.2-2.jar > /usr/hdp/2.2.4.2-2/hbase/lib/ranger-plugins-impl-0.4.0.2.2.4.2-2.jar > /usr/hdp/2.2.4.2-2/hbase/lib/servlet-api-2.5-6.1.14.jar > /usr/hdp/2.2.4.2-2/hbase/lib/servlet-api-2.5.jar > /usr/hdp/2.2.4.2-2/hbase/lib/slf4j-api-1.6.4.jar > /usr/hdp/2.2.4.2-2/hbase/lib/snappy-java-1.0.4.1.jar > /usr/hdp/2.2.4.2-2/hbase/lib/xercesImpl-2.9.1.jar > /usr/hdp/2.2.4.2-2/hbase/lib/xml-apis-1.3.04.jar > /usr/hdp/2.2.4.2-2/hbase/lib/xmlenc-0.52.jar > /usr/hdp/2.2.4.2-2/hbase/lib/xz-1.0.jar > /usr/hdp/2.2.4.2-2/hbase/lib/zookeeper.jar -what's the > problem? -- Best regards, Shaofeng Shi

[Cancelled] [VOTE] Release apache-kylin-1.5.3 (release candidate 2)

2016-07-24 Thread ShaoFeng Shi
Dong Li found the package has nodejs modules included. They were ignored by Git but wasn't excluded by the release tool. Cancell this vote and will run release with a clean repository. 2016-07-25 7:57 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > Hi All, > > I have

[RESULT] [VOTE] Release apache-kylin-1.5.3 (release candidate 3)

2016-07-28 Thread ShaoFeng Shi
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 6 binding +1s: Shaofeng Shi Dong Li Yang Li Hongbin Ma Luke Han Xiaoyu Wang 0 non-binding +1s: No 0s or -1s. Therefore I am delighted to announce that the proposal to release

[Cancelled] [VOTE] Release apache-kylin-1.5.3 (release candidate 1)

2016-07-21 Thread ShaoFeng Shi
Several new issues (KYLIN-1909, KYLIN-1911, KYLIN-1912) were reported in these days, so cancel this vote and will prepare RC2 soon. 2016-07-20 23:05 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > Hi Dayue, is this a severe bug, or it is just an "improvement"? When you &g

Re: Turn off some hive configuration commands when they are not allowed

2016-07-31 Thread ShaoFeng Shi
llowed to modify at runtime". It will break the build. > > Maybe there are some other hard code hadoop commands still. It will be more > friendly if they could turn off on demand. > > -- > With Warm regards > > Yiming Liu (刘一鸣) > -- Best regards, Shaofeng Shi

Re: kylin查询栈溢出异常

2016-08-02 Thread ShaoFeng Shi
Compiler.java:320) > at > org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:383) > at > org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:315) > at > org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:233) > at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:192) > at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:47) > at > org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:340) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.getBindable(EnumerableInterpretable.java:140) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:105) > ... 90 more > -- Best regards, Shaofeng Shi

Re: kylin查询栈溢出异常

2016-08-02 Thread ShaoFeng Shi
is 1.5.3 > > > > > -- 原始邮件 ---------- > 发件人: "ShaoFeng Shi";<shaofeng...@apache.org>; > 发送时间: 2016年8月3日(星期三) 上午8:55 > 收件人: "dev"<dev@kylin.apache.org>; > > 主题: Re: kylin查询栈溢出异常 > > > > Hi, which version of Kylin

Re: kylin 1.5.3 Bug

2016-08-02 Thread ShaoFeng Shi
> 把1.5.2的替换1.5.3之后正常启动 > 环境 ubuntu 12.04 > jdk 1.7 > hadoop 2.7.1 > hbase 1.2.1 > hive 1.2.1 > -- Best regards, Shaofeng Shi

Re: Kylin Cube Performance

2016-08-02 Thread ShaoFeng Shi
, > "isShardBy": false > }, > { > "column": "DEVICE_TYPE", > "encoding": "dict", > "isShardBy": false > }, > { > "column": "

Re: Derived measures in Kylin

2016-08-11 Thread ShaoFeng Shi
t: http://apache-kylin.74782.x6. > nabble.com/Derived-measures-in-Kylin-tp5513p.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi

Re: Derived measures in Kylin

2016-08-09 Thread ShaoFeng Shi
is message in context: http://apache-kylin.74782.x6. > nabble.com/Derived-measures-in-Kylin-tp5513p5530.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi

Re: Kylin Cube Performance

2016-08-04 Thread ShaoFeng Shi
d_id, > search_type, hotel_id, report_date > User: ADMIN > Success: true > Duration: 56.152 > Project: koddi_dev > Realization Names: [hpa_reporting2_cube_clone] > Cuboid Ids: [992] > Total scan count: 12306477 > Result row count: 5 > Accept Partial: true > Is Parti

Re: Keep the column order when parsing the streaming message

2016-07-20 Thread ShaoFeng Shi
better if Kylin could keep the same >> column order with the origin message. >> >> [image: 内嵌图片 1] >> -- >> With Warm regards >> >> Yiming Liu (刘一鸣) >> > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) > -- Best regards, Shaofeng Shi

Re: JDBC query result Date column get wrong value

2016-07-20 Thread ShaoFeng Shi
DATE'2012-01-01' group by part_dt order by part_dt"; > >> > >> PreparedStatement stmt = conn.prepareStatement(sql); > >> try { > >> try (ResultSet rs = stmt.executeQuery()) { > >> while (rs.next()) { > >> > > System.out.println(rs.getString(1)+"\t"+rs.getString(2)+"\t"+rs.getString(3)); > >> } > >> } > >> } finally { > >> stmt.close(); > >> } > >> > >> I got the result: > >> 2011-12-31 466.9037 12 > >> > >> It seems something wrong when JDBC parse the result. > >> > >> -- > >> With Warm regards > >> > >> Yiming Liu (刘一鸣) > >> > > > > > > > > -- > > With Warm regards > > > > Yiming Liu (刘一鸣) > > > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) > -- Best regards, Shaofeng Shi

Re: Keep the column order when parsing the streaming message

2016-07-20 Thread ShaoFeng Shi
; JSON record, and click ">>" to generate Table columns. > > 2016-07-21 8:35 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > > > Hi Yiming, > > > > the embeded screenshot couldn't show up, which step there? > > > > 2016-07-20 18

[VOTE] Release apache-kylin-1.5.3 (release candidate 1)

2016-07-19 Thread ShaoFeng Shi
-- Best regards, Shaofeng Shi

Re: Kylin job failed

2016-07-12 Thread ShaoFeng Shi
hi yiming, kylin relies on calcite to parse the SQL so we may not able to give warning before calcite throws error; but we can add that to some doc or faq;  Regards, Shaofeng Shi shaofeng...@gmail.com >From Outlook Mobile On Tue, Jul 12, 2016 at 5:31 PM +0800, "Yiming Liu&quo

Re: Timeout visiting cube

2016-07-04 Thread ShaoFeng Shi
-- 原始邮件 -- > 发件人: "移动苏州研发中心-陈雷雷";<775620...@qq.com>; > 发送时间: 2016年6月21日(星期二) 晚上11:14 > 收件人: "dev"<dev@kylin.apache.org>; > > 主题: 回复: Timeout visiting cube > > > > > > > > > > -- 原始邮件 ---

Re: A question about the start date of cube in Kylin

2016-07-05 Thread ShaoFeng Shi
20160101? > Thanks. -- Best regards, Shaofeng Shi

Re: [Discuss] Prepare releasing v1.5.3

2016-07-05 Thread ShaoFeng Shi
We should release on regular basis.​ > > Some known bugs could easily become show stoppers for new comers. > > > > On Mon, Jul 4, 2016 at 5:24 PM, ShaoFeng Shi <shaofeng...@apache.org> > > wrote: > > > > > Hi Kylin developers, > > > > >

Re: A question about the start date of cube in Kylin

2016-07-06 Thread ShaoFeng Shi
rieve the data since 20160101. After finish, your cube will have the data of this year. BTW, we don't suggest to manually edit metadata, unless you're very clear and careful about what you're doing. Backup is always needed. 2016-07-06 22:57 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: >

Re: cube选择问题

2016-07-06 Thread ShaoFeng Shi
aStatement.java:135) > ... 80 more > Caused by: org.apache.calcite.rel.InvalidRelException: distinct > aggregation not supported > at > org.apache.calcite.adapter.enumerable.EnumerableAggregate.(EnumerableAggregate.java:71) > at > org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:368) > ... 95 more > 2016-07-05 21:19:19,993 INFO [http-bio-7070-exec-5] > service.QueryService:248 : > ==[QUERY]=== > SQL: select count(DISTINCT order_title) from hwb_alipay_trade > User: ADMIN > Success: false > Duration: 0.0 > Project: test > Realization Names: [hwb_alipay_cube] > Cuboid Ids: [] > Total scan count: 0 > Result row count: 0 > Accept Partial: true > Is Partial Result: false > Hit Exception Cache: false > Storage cache used: false > Message: Error while executing SQL "select count(DISTINCT order_title) > from hwb_alipay_trade LIMIT 5": Can't create EnumerableAggregate! > > > 2016-07-06 > > > langren_qing > > -- > View this message in context: > http://apache-kylin.74782.x6.nabble.com/cube-tp5225.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi

Re: 答复: cube查询问题

2016-07-11 Thread ShaoFeng Shi
hi,we have kick off the 1.5.3 release process in last week, will invite the community to do testing soon. The plan is to finish it in this month. For this "union" bug, since it wasn't marked as "resolved" when kick off the release, it may have to defer to next version.  Re

Re: Kylin job failed

2016-07-11 Thread ShaoFeng Shi
name for better readability; could you please open a JIRA for tracking? To bypass it,  Regards, Shaofeng Shi shaofeng...@gmail.com >From Outlook Mobile On Mon, Jul 11, 2016 at 2:30 PM +0800, "Karthigeyan K" <karthigeyan.t...@gmail.com> wrote: Hi , I was able t

Re: Derived measures in Kylin

2016-08-07 Thread ShaoFeng Shi
usceptible to alteration. Neither SOCIETE GENERALE nor any > of its subsidiaries or affiliates shall be liable for the message > if altered, changed or falsified. > > = > -- Best regards, Shaofeng Shi

Re: Can not load hive table in Kylin

2016-07-22 Thread ShaoFeng Shi
can find hive-exec-1.2.1.jar in the result > of find-hive-dependency.sh and the process, is there any other > configuration I missed? > > > 2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > >> It seems the request wasn't received by the server at all;

Re: kylin查询,报超时异常:Timeout visiting cube!

2016-06-30 Thread ShaoFeng Shi
... 4 more > > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1235) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:222 > t > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpc > Client.java:323) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.j > ava:32855) > at > org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1618) > ... 13 more > > 目前cube size : 1.5GB source records :8亿 > 版本: hadoop2.6 hive -hive-1.2.1 hbase 1.1.5kylin 1.5.2 > > 还有问题就是都有哪些点可以优化查询??? > > 谢谢! > > > > > -- Best regards, Shaofeng Shi

Re: Questions / Clarifications for 1.5.x

2016-06-30 Thread ShaoFeng Shi
query server serve only one > cube or all servers serve all cubes. If its one cube only in one server, > how does kylin handle a server going down. > > > 6) Incremental build > Is there a document on how incremental build works ? We want to understand > limitations / assumptions for this. > > Regards, > Abhilash > -- Best regards, Shaofeng Shi

Re: 答复: kylin查询,报超时异常:Timeout visiting cube!

2016-06-30 Thread ShaoFeng Shi
二、如果group by的是nomal维度则查询很快 > >三、如果增加hbase的regionderver是否可解决此问题 ? > > > 谢谢~ > > > > -邮件原件- > 发件人: ShaoFeng Shi [mailto:shaofeng...@apache.org] > 发送时间: 2016年7月1日 9:32 > 收件人: dev > 抄送: u...@kylin.apache.org >

Re: Kylin使用kafka作为数据源的一些讨论

2016-07-03 Thread ShaoFeng Shi
二、如果Kafka集群不稳定到某个partition根本没有数据,Kylin的build过程将陷入死循环 > >对于实时业务来说,这种延迟的都呈现出不确定性 > -- > Best regards, > Amuro Copperfield > -- Best regards, Shaofeng Shi

[Discuss] Prepare releasing v1.5.3

2016-07-04 Thread ShaoFeng Shi
, then the community can start testing on that; This link shows the bug fix and enhancements in v1.5.3: https://issues.apache.org/jira/issues/?filter=12334169=project%20%3D%20KYLIN%20AND%20fixVersion%20%3D%20v1.5.3 Please share your thought and comment. Thanks! -- Best regards, Shaofeng Shi

Re: I got a problem in #2 Step Name: Extract Fact Table Distinct Columns

2016-07-04 Thread ShaoFeng Shi
s apache-kylin-1.5.2.1 and my Hadoop env is CDH5.2.0. > > > Appreciate response from Kylin experts. > > > > > > > > > -- Best regards, Shaofeng Shi

Re: Re: build cube and hbase coprocessor not avaliable

2017-02-02 Thread ShaoFeng Shi
t; today I add kylin.coprocessor.local.jar arguments in kylin.property > file and when build cube done . I execute query "select * from xxx" hits > error below: > >> > >> > >> org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No > registered coprocessor service found for name CubeVisitService in region > KYLIN_GA464RXYQX,,1484635706236.9aac092cc3bd8855999c7a8734f09842. > >> > >> > >> how can I resolve this problem? > >> > >> > >> thanks > >> > -- Best regards, Shaofeng Shi 史少锋

Re: Problem while querying Cube

2017-02-06 Thread ShaoFeng Shi
6, 2017 at 9:26 PM, ShaoFeng Shi wrote: > NoSuchObjectException(message:default.kylin_intermediate_ > kylin_sales_cube_desc_3a41df7d_93e1_4445_9 > usually this error indicates that hive-site.xml wasn't on classpath; a > quick workaround is copying that file to $KYLIN_HOME/conf; the ult

Re: Does Kylin support percent function?

2017-02-02 Thread ShaoFeng Shi
321 > > 2local123 > > 3google 987 > > 4google 789 > > SELECT (SUM(CASE WHEN (host="local") then 1 else 0 end)/count(*)*100) > > as hostpercent FROM table1 > > The sql result is 50%. Mysql or hive can do this,but i don't know does > > kylin support this action? > > please help me, thank you! > > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Urgent!!! exact sum of values are not coming in kylin when executed query in Insight

2017-02-02 Thread ShaoFeng Shi
g distinct values for summarizing result. > > > Please help to resolve this asap > > Regards, > Suresh > -- Best regards, Shaofeng Shi 史少锋

Re: Problem while querying Cube

2017-02-05 Thread ShaoFeng Shi
> am keep getting this exception. What will happen if we remove this check? > > Thanks. > > On Sat, Feb 4, 2017 at 8:12 PM, ShaoFeng Shi <shaofeng...@apache.org> > wrote: > > > I don't know what that check is necessary, as there is a todo which says > > will re

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
.hadoop.fs.FileSystem.listStatus( > FileSystem.java:1515) > at org.apache.hadoop.fs.FileSystem.listStatus( > FileSystem.java:1555) > at org.apache.hadoop.fs.ChecksumFileSystem.listStatus( > ChecksumFileSystem.java:574) > at org.apache.kylin.source.hive.HiveMRInput$ > RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:338) > at org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable.java:113) > at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork( > DefaultChainedExecutable.java:57) > at org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable.java:113) > at org.apache.kylin.job.impl.threadpool.DefaultScheduler$ > JobRunner.run(DefaultScheduler.java:136) > > > > Thx > Lei Wang > > -- Best regards, Shaofeng Shi 史少锋

Re: Re: Re: build cube and hbase coprocessor not avaliable

2017-02-05 Thread ShaoFeng Shi
NE', MIN_VERSIONS => '0', TTL => 'FOREVER', > KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '6553 > > 6', IN_MEMORY => 'false', BLOCKCACHE => 'true'} > > 1 row(s) in 0.1060 seconds > > 4,kylin-coprocessor-1.6.0.jar, kylin-jdbc-1.6.0.jar, kylin-job-1.6.0.jar > &

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
d. > Is my understanding correct? > > 发自我的 iPhone > > > 在 2017年2月5日,20:54,ShaoFeng Shi <shaofeng...@apache.org> 写道: > > > > From the error it tried to seek the file from local file system, instead > of > > HDFS. Please check whethere "fs.DefaultFS&

Re: Can not build sample cube

2017-02-05 Thread ShaoFeng Shi
ay 'fs.defaultFS'? > > > 发自我的 iPhone > > > 在 2017年2月5日,21:38,ShaoFeng Shi <shaofeng...@apache.org> 写道: > > > > No, from the output of "hdfs fs -ls" the file does exist on HDFS; In > hive's > > context its default fs is HDFS, so it works

Re: Problem while querying Cube

2017-02-04 Thread ShaoFeng Shi
Shailesh Prajapati <shail...@infoworks.io>: > Hi ShaoFeng, > > Here is the gist link, > > https://gist.github.com/shaipraj/780a3dcc80aa2080911b7348c76f5b88 > > Thanks. > > On Sat, Feb 4, 2017 at 2:24 PM, ShaoFeng Shi <shaofeng...@apache.org> > wrote:

Re: kylin cube migrating

2017-01-22 Thread ShaoFeng Shi
的话, 如何同步呢? > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/kylin-cube-migrating-tp6000p7014.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Proposal for updating master branch to use HBase 1.x

2017-01-22 Thread ShaoFeng Shi
> > > 2. Others please use master branch > > > > > > Developers may need to upgrade your dev environments. > > > > > > Best regards, > > > > > > Kaige Liu(刘凯歌) > > > Mail: kaige@kyligence.io<mailto:kaige@kyligence.io> > > > Shanghai Kyligence Information Technology Co., Ltd > > > 上海市浦东新区亮秀路112号Y1座405室 > > > > > > "Do small things with great love." > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: build kylin code exception

2017-01-21 Thread ShaoFeng Shi
Neon.1a Release (4.6.1), JRE jdk1.8.0_45. > > > > -- 原始邮件 ------ > *发件人:* "ShaoFeng Shi";<shaofeng...@apache.org>; > *发送时间:* 2017年1月20日(星期五) 晚上11:20 > *收件人:* "dev"<dev@kylin.apache.org>; > *主题:* Re: build kyl

Re: 答复: more than 1 append dict for globalDict

2017-01-24 Thread ShaoFeng Shi
in/metastore.sh remove /dict/APPLYDATA_DSDCADMIN.FLY_ ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict" 2017-01-24 20:14 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > @Yerui, do you have suggestion on this? > > 2017-01-24 18:19 GMT+08:00 市场中心-ZHANGDA32698 <zd

Re: 答复: more than 1 append dict for globalDict

2017-01-24 Thread ShaoFeng Shi
d have 0 or 1 append dict but 2’ > > How can I resolve this and continue my cube building? Any suggestion is > appreciated, thanks! > > > > > > *发件人:* ShaoFeng Shi [mailto:shaofeng...@apache.org] > *发送时间:* 2017年1月24日 17:01 > *收件人:* user > *主题:* Re: more than 1 appen

New document: "How to optimize cube build"

2017-01-25 Thread ShaoFeng Shi
Hello, A new document is added for the practices of cube build. Any suggestion or comment is welcomed. We can update the doc later with feedbacks; Here is the link: https://kylin.apache.org/docs16/howto/howto_optimize_build.html -- Best regards, Shaofeng Shi 史少锋

Re: org.apache.kylin.dict.CachedTreeMap use a couple classes from org.apache.hadoop.fs

2017-01-25 Thread ShaoFeng Shi
e really need to serialized CachedTreeMap to spark executor > tasks? Maybe every tasks initialize own CacheTreeMap instance is another > choice; > >Please feel free to change the code if you really need to serialize > CachedTreeMap, and let me know if there’s somewhere I could h

Re: New document: "How to optimize cube build"

2017-01-25 Thread ShaoFeng Shi
is meant for few hundred of partitions not > thousand (Hive 9452 there is an idea to solve this isn’t in progress)*" > > In Hive 2.0 will be a preview (only for testing) to solve this > > 2017-01-25 9:46 GMT+01:00 ShaoFeng Shi <shaofeng...@apache.org>: > >> Hello, >

Re: No build all dimensions

2017-02-21 Thread ShaoFeng Shi
2.x6. > nabble.com/No-build-all-dimensions-tp7263p7264.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Apache Kylin 2.0.0 beta is ready for download

2017-02-25 Thread ShaoFeng Shi
nt from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.NoSuchMethodError: org.apache.hadoop.yarn.proto.YarnProtos$LocalResourceProto.hashLong(J

2017-02-12 Thread ShaoFeng Shi
ache- > kylin-job-exception-ExecuteException-javaI-tp7126p7138.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: 你好,请教下关于kylin高可用的一些问题,谢谢

2017-02-16 Thread ShaoFeng Shi
>写道: > 请教下关于kylin高可用的一些问题 > > 1、集群模式下的负载均衡器使用哪种开源产品?ngnix是否可以满足kylin的要求? > > 2、job engine在集群中只能部署一个,job节点如何实现高可用? > > > > > -- Best regards, Shaofeng Shi 史少锋

Re: Trouble with building cube at step 3!!cry!cry!

2017-02-16 Thread ShaoFeng Shi
er( > ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > result code:2 > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Trouble-with-building-cube-at-step-3-cry-cry-tp7172.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: Kylie Cube Building,Merging and Refreshing getting stucked at Create HFile Step

2017-02-16 Thread ShaoFeng Shi
ot able to drop a table due to hbase permission > issues, and thus all other subsequent requests for creating hbase table > handlers started queueing.Got the issue while looking at hbase dump. > > Regards,Ketan > > > Sent from my Samsung Galaxy smartphone. > ---- Orig

Re: Help needed - Issue when build the cube

2017-02-16 Thread ShaoFeng Shi
hreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > result code:2 > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Help-needed-Issue-when-build-the-cube-tp7191.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: kylin sql query with Weird error

2017-02-16 Thread ShaoFeng Shi
r": null }, "returntype": "bigint" }, > "dependent_measure_ref": null } ], "dictionaries": [], "rowkey": { > "rowkey_columns": [ { "column": "YEAR", "encoding": "dict", "isShardBy": > false }, { "column": "QUARTER_CN", "encoding": "dict", "isShardBy": false > }, { "column": "MONTH_CN", "encoding": "dict", "isShardBy": false }, { > "column": "DATE", "encoding": "dict", "isShardBy": false }, { "column": > "PROVINCE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "CITY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_COUNTRY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_REGION_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_PROVINCE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "USER_CITY_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "TYPE_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "LOGIN_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "REGISTER_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "BROWSER_NAME", "encoding": "dict", "isShardBy": false }, { "column": > "VERSION_NAME", "encoding": "dict", "isShardBy": false } ] }, > "hbase_mapping": { "column_family": [ { "name": "F1", "columns": [ { > "qualifier": "M", "measure_refs": [ "_COUNT_", "PV", "OUT_SESSION" ] } ] }, > { "name": "F2", "columns": [ { "qualifier": "M", "measure_refs": [ "UV" ] } > ] } ] }, "aggregation_groups": [ { "includes": [ "YEAR", "QUARTER_CN", > "MONTH_CN", "DATE", "PROVINCE_NAME", "CITY_NAME", "USER_COUNTRY_NAME", > "USER_REGION_NAME", "USER_PROVINCE_NAME", "USER_CITY_NAME", "TYPE_NAME", > "LOGIN_NAME", "REGISTER_NAME", "BROWSER_NAME", "VERSION_NAME" ], > "select_rule": { "hierarchy_dims": [ [ "YEAR", "QUARTER_CN", "MONTH_CN", > "DATE" ], [ "PROVINCE_NAME", "CITY_NAME" ], [ "USER_COUNTRY_NAME", > "USER_REGION_NAME", "USER_PROVINCE_NAME", "USER_CITY_NAME" ] ], > "mandatory_dims": [], "joint_dims": [] } } ], "signature": > "odaKh4hY+Mb4uRggEJJanQ==", "notify_list": [], "status_need_notify": [ > "ERROR", "DISCARDED", "SUCCEED" ], "partition_date_start": 148590720, > "partition_date_end": 31536, "auto_merge_time_ranges": [ 60480, > 241920 ], "retention_range": 0, "engine_type": 2, "storage_type": 2, > "override_kylin_properties": { "kylin.cube.aggrgroup.max.combination": > "16384" } } > > > > > > > -- > 祝身体健康,万事如意 ^_^ > -- Best regards, Shaofeng Shi 史少锋

Re: Kylie Cube Building,Merging and Refreshing getting stucked at Create HFile Step

2017-02-15 Thread ShaoFeng Shi
: > Hi Guys, > We have a cube which is getting stucked for all type of jobs: > Build,Refresh and Merge on the “Create Hfile” step. > > Any idea, how can I get back to normalcy. > We are using Kylin 1.5.4.1 and the cube is being built incrementally on an > hourly basis. > > > Thanks, > kdcool6932 -- Best regards, Shaofeng Shi 史少锋

Re: query meet java heap space,help!

2017-02-17 Thread ShaoFeng Shi
7237p7238.html > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

Re: Streaming Cube Bulkload

2017-01-18 Thread ShaoFeng Shi
.RpcRetryingCaller@39f0ffa8, > > org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase. > RegionTooBusyException: > > failed to get a lock in 6 ms. > > > -- Best regards, Shaofeng Shi 史少锋

Re: Streaming Build Timestamp parse error

2017-01-19 Thread ShaoFeng Shi
}++)\Q > > \E(\p{Nd}++)\Q:\E(\p{Nd}++)\Q:\E(\p{Nd}++) > > my kylin version is 1.6.0 > > > > -- > > View this message in context: http://apache-kylin.74782.x6. > > nabble.com/Streaming-Build-Timestamp-parse-error-tp6952.html > > Sent from the Apache Kylin mailing list archive at Nabble.com. > > > -- Best regards, Shaofeng Shi 史少锋

org.apache.kylin.dict.CachedTreeMap use a couple classes from org.apache.hadoop.fs

2017-01-19 Thread ShaoFeng Shi
SEGMENT_CD=org.apache.kylin.dict.TrieDictionaryForest@dcfc7d11, DEFAULT.TEST_KYLIN_FACT.CAL_DT=DateStrDictionary [pattern=-MM-dd, baseId=0]}) -- Best regards, Shaofeng Shi 史少锋

Re: Streaming Build Timestamp parse error

2017-01-20 Thread ShaoFeng Shi
I think you would prefer to write SQL like "where partitionCol > ''2017-01-12 04:00:00", instead of "where partitionCol > 1484236798000", right? 2017-01-20 16:00 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>: > Hi Wu xin, > > It is okay to use the u

Re: Streaming Build Timestamp parse error

2017-01-20 Thread ShaoFeng Shi
alue, not epoch time in miliseconds; 2017-01-20 14:22 GMT+08:00 Mario Copperfield <xwhfcen...@gmail.com>: > I think it might make a confuse, because we usually define timestamp as an > unix timestamp such as “1484236798000" > > On 20 Jan 2017, 13:25 +0800, ShaoFeng Shi <shaofeng...

Re: Dup key found when building cube

2016-08-20 Thread ShaoFeng Shi
Usually the join is on PK, which need be unique; if not fulfilled kylin will report such error, you need clean the data to remove this ambiguity. Regards, Shaofeng Shi shaofeng...@gmail.com >From Outlook Mobile On Fri, Aug 19, 2016 at 7:22 PM +0800, "北极晨光" <176897.

Re: About lookup table fileds in filter

2016-08-20 Thread ShaoFeng Shi
I know there is such an issue; please open a JIRA; and try this workaround: find the HQL that kylin uses to create the flat table, get the table alias, and the use this alias in the model's "filter condition" Regards, Shaofeng Shi shaofeng...@gmail.com >From Outlook Mobile O

Re: Kylin Cube Performance

2016-08-03 Thread ShaoFeng Shi
AggregateRel(group=[{0, 1, 2, 3}], EXPR$0=[SUM($4)]) > OLAPProjectRel(SITE_ID=[$9], CHILD_ID=[$3], REPORT_DATE=[$0], > HOTEL_ID=[$2], CLICKS=[$10]) > OLAPTableScan(table=[[DEFAULT, HPA_REPORTING2]], fields=[[0, 1, 2, 3, 4, 5, > 6, 7, 8, 9, 10, 11]]) > > On Tue, Aug 2, 2016 at 7:46 P

Re: [jira] [Created] (KYLIN-1994) Build Cube failing

2016-09-02 Thread ShaoFeng Shi
k] > > <https://www.facebook.com/XactlyCorp> [image: YouTube] > > <http://www.youtube.com/xactlycorporation> > > > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) > > -- > > > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> > > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] > <https://twitter.com/Xactly> [image: Facebook] > <https://www.facebook.com/XactlyCorp> [image: YouTube] > <http://www.youtube.com/xactlycorporation> > -- Best regards, Shaofeng Shi

Re: version compability

2016-08-30 Thread ShaoFeng Shi
me query in hive and get the results. > > So if you want to update your jira ticket that is the solution. > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/version-compability-tp5636p5693.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi

Re: Hive server vs Impala server

2016-09-07 Thread ShaoFeng Shi
; with 1.1 and t did not work. The webapp does not use routing and it is > > documented. No worries on the webapp part. > > > > Is there any docs on uninstalling Kylin -remove all HBase tables, > > Meta-Data and wipe of all intermediate hive tables etc..? > > &g

Re: Kylin: On the Fly Calcs

2016-09-07 Thread ShaoFeng Shi
at Mr > Price Group Limited (http://www.mrpricegroup.com/LegalNotice.aspx). If > you cannot access the legal notice, kindly contact us on +27 800 21 2535 > and a copy will be provided to you. > > Mr Price Group is an Authorised Financial Services and Credit Provider: > FSP31450 and NCRCP46. > > PPlease consider the environment before printing this email. > -- Best regards, Shaofeng Shi

Re: Hive server vs Impala server

2016-08-29 Thread ShaoFeng Shi
ail.com> wrote: > > > This is cool. I will have this tried this week. Thanks! > > On Aug 28, 2016 7:36 PM, "ShaoFeng Shi" <shaofeng...@apache.org> wrote: > > > >> The current implementation only route to hive, via Hive JDBC (add > >> hive.

Re: how to kill a query request in kylin

2016-08-31 Thread ShaoFeng Shi
t; > Regards, > > > > On 31/08/16 04:04, 李寅威 wrote: > >> Hi all: >> >>I have a query consuming so long time until it return an error says >> time out, does kylin has rest api or other methods that can kill a query >> request? >> > > -- Best regards, Shaofeng Shi

Re: Hive server vs Impala server

2016-08-30 Thread ShaoFeng Shi
keep telling > customers too. It is a great optimization that is really awesome. > I will test it with Impala and post back. > -- Best regards, Shaofeng Shi

Re: [jira] [Created] (KYLIN-2015) replace h2 with alternatives like sqllite or mysql

2016-09-14 Thread ShaoFeng Shi
ed to be very slow on H2. > > > > > > > > > > > > -- > > This message was sent by Atlassian JIRA > > (v6.3.4#6332) > > > -- Best regards, Shaofeng Shi

Re: Hive server vs Impala server

2016-09-08 Thread ShaoFeng Shi
ry>","exception":"Error while > executing SQL \"select productid, sum(quantity) from hive_aggregator_kylin > where quantity < 100 group by productid\": null"} > >at org.apache.kylin.jdbc.KylinClient.asIOException( > *KylinClient.java:414*) > &g

Re: Kylin: On the Fly Calcs

2016-09-12 Thread ShaoFeng Shi
indly contact us on +27 800 21 2535 > and a copy will be provided to you. > > Mr Price Group is an Authorised Financial Services and Credit Provider: > FSP31450 and NCRCP46. > > PPlease consider the environment before printing this email. > -- Best regards, Shaofeng Shi

Re: Fwd: About Kafka in Kylin

2016-09-13 Thread ShaoFeng Shi
sorry one correction: " if some messages are arrived late than the margin, it will not be lost" should be: " if some messages are arrived late than the margin, it will be lost" Mario, you can try to set a bigger margin value to reduce the possibility. 2016-09-13 22:05 GM

[VOTE] Release apache-kylin-1.5.4 (release candidate 1)

2016-09-11 Thread ShaoFeng Shi
and passes if a majority of at least three +1 PMC votes are cast. [ ] +1 Release this package as Apache Kylin 1.5.4 [ ] 0 I don't feel strongly about it, but I'm okay with the release [ ] -1 Do not release this package because... Here is my vote: +1 (binding) -- Best regards, Shaofeng Shi

[VOTE] Release apache-kylin-1.5.4.1 (release candidate 1)

2016-09-23 Thread ShaoFeng Shi
regards, Shaofeng Shi 史少锋

Re: Update from Kylin 1.2 to 1.5.3 ERROR

2016-09-24 Thread ShaoFeng Shi
gt; at > org.apache.kylin.rest.service.CubeService.listAllCubes( > CubeService.java:215) > at > org.apache.kylin.rest.service.CubeService.listAllCubes( > CubeService.java:103) > at > org.apache.kylin.rest.service.CubeService.getCubes(CubeService.java:135) > at > org.apache.kylin.rest.service.CubeService$$FastClassByCGLIB$ > $17a07c0e.invoke() > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Update-from-Kylin-1-2-to-1-5-3-ERROR-tp5858.html > Sent from the Apache Kylin mailing list archive at Nabble.com. > -- Best regards, Shaofeng Shi 史少锋

Re: support count or count distinct measure in multiple column topN

2016-09-24 Thread ShaoFeng Shi
gt; > > 珠海市魅族科技有限公司 > > MEIZU Technology Co., Ltd. > > 广东省珠海市科技创新海岸魅族科技楼 > > MEIZU Tech Bldg., Technology & Innovation Coast > > Zhuhai, 519085, Guangdong, China > > meizu.com > -- Best regards, Shaofeng Shi 史少锋

[VOTE] Release apache-kylin-1.5.4.1 (release candidate 2)

2016-09-24 Thread ShaoFeng Shi
regards, Shaofeng Shi 史少锋

Re: cube详细页的SQL标签页报错

2016-10-09 Thread ShaoFeng Shi
mpl.java:57) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > > > > > > > -- Best regards, Shaofeng Shi 史少锋

[RESULT] [VOTE] Release apache-kylin-1.5.4 (release candidate 1)

2016-09-14 Thread ShaoFeng Shi
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 5 binding +1s: Shaofeng Shi Dong Li Xiaoyu Wang Hongbin Ma Yang Li 0 non-binding +1; No 0 or -1. Therefore I am delighted to announce that the proposal to release Apache-Kylin

<    1   2   3   4   5   6   7   8   9   10   >