Re:RE: kylin 2.3.1 - System Cube Tables

2018-04-04 Thread Ma Gang
Hi Manoj, You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default. At 2018-04-02 22:26:56, "Kumar, Manoj H" wrote: Can it be possible to create the table in project Database as we d

Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables

2018-04-04 Thread Ma Gang
Re:RE: kylin 2.3.1 - System Cube Tables Thanks for the input.. Let me try it out & I will update you. Regards, Manoj From: Ma Gang [mailto:mg4w...@163.com] Sent: Wednesday, April 04, 2018 2:30 PM To:user@kylin.apache.org Subject: Re:RE: kylin 2.3.1 - System Cube Tables Hi Manoj, You c

Re:Re: 紧急求救,kylin Query机 查询运行20多分钟后死机

2018-04-23 Thread Ma Gang
You may check the region server log, is the related region server OOM or overload? 在 2018-04-24 13:47:08,"沈鲁威" 写道: > >异常补充 >ylin.log:Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: >org.apache.hadoop.hbase.DoNotRetryIOException: Coprocessor passed deadline! >Maybe server is overload

Re:kylin help

2018-04-24 Thread Ma Gang
Hi, Currently you can set the configuration: "kylin.storage.partition.max-scan-bytes", to limit the scan bytes for each region to prevent possible OOM in region server, by default it is set to 3GB, and the configuration can be set in cube level. In our private Kylin env, we add two new configur

Re:Re: Reg:Doubt About Cube Planner

2018-05-09 Thread Ma Gang
Hi Kumar, The dashboard data is from system cube, not from jmx, you can run sql in 'KYLIN_SYSTEM' project to check whether the data has been collected correctly or not: "select KDAY_DATE, count(1) from HIVE_METRICS_QUERY_PROD where KDAY_DATE > '2018-05-01' group by KDAY_DATE order by KDAY_DATE d

Re:Logging kylin server log to console

2018-09-26 Thread Ma Gang
You may check the file: logs/kylin.out At 2018-09-26 15:16:23, "Francis Liang" wrote: Hi: I am using kylin-2.5.0-bin-cdh57 and want to output kylin sever logs to console. I modified the conf/kylin-server-log4j.properties file as follows: log4j.appender.file=org.apache.log4j.RollingF

Re: 【kylin2.5.0】 NullPointerException

2018-10-13 Thread Ma Gang
That may caused by the sql that use preparedStatement didnot hit any cube, @guofeng, could you provide the sql? | | Ma Gang | | 邮箱:mg4w...@163.com | Signature is customized by Netease Mail Master On 10/13/2018 22:41, ShaoFeng Shi wrote: It looks more like a bug related with prepared statement

Re:Re: 【kylin2.5.0】 NullPointerException

2018-10-13 Thread Ma Gang
tCache":false, ... } At 2018-10-13 23:16:13, "Ma Gang" wrote: That may caused by the sql that use preparedStatement didnot hit any cube, @guofeng, could you provide the sql? | | Ma Gang | | 邮箱:mg4w...@163.com | Signature is customized by Netease Mail Master On 10/13

Re:Re:Re: 【kylin2.5.0】 NullPointerException

2018-10-13 Thread Ma Gang
when there is no realization exists in the OLAPContext. 在 2018-10-14 00:06:24,"Ma Gang" 写道: @guofeng, You may first disable the preparestatement cache to see it works or not, it can be disabled by setting the "enableStatementCache" field to false in the sql

Re:Re:Re:Re: 【kylin2.5.0】 NullPointerException

2018-10-13 Thread Ma Gang
Disable the preparedStatement cache should workaround this issue per my test, it can be disabled by setting the "enableStatementCache" field to false in the sql request, like: { "sql":"...", "project":"...", "enableStatementCac

Re:Re: 【kylin2.5.0】 NullPointerException

2018-10-13 Thread Ma Gang
No, that doesn't work. 在 2018-10-14 10:53:44,"jinhongfei" 写道: @Ma Gang Hello kylin.properties中把这个配置项 #kylin.query.cache-enabled=true 改成 kylin.query.cache-enabled=false 是不是就禁用enableStatementCache=false了啊? -- 靳洪飞 百乘金科/金蛋分期/技术部 北京

Re:Re: [Announce] Welcome new Apache Kylin committer :Allen Ma

2018-10-16 Thread Ma Gang
ards Billy Liu Luke Han 于2018年10月16日周二下午11:28写道: > > I am very pleased to announce that the Project Management Committee (PMC) of > Apache Kylin has asked Allen Ma (Gang Ma) to become Apache Kylin committer, > and he has already accepted. > > Allen has already made many

Re:Coprocessor passed deadline error!

2018-10-22 Thread Ma Gang
The error means the HBase coprocessor execution timeout, you need to identify why it is timeout: 1. Is the cube design not properly, so coprocessor need to scan too many rows to do runtime aggregation or can not skipped rows efficiently if the rowkey is not properly designed? 2. Is the related r

Re:Re: Kylin Count(col) Result Is Different With Hive

2018-10-23 Thread Ma Gang
Is the column 'fzy_bill_code' from fact table? If so, the issue should have been fixed in https://issues.apache.org/jira/browse/KYLIN-3360, you need to enable dynamic column configuration. At 2018-10-23 14:11:46, "you Zhuang" wrote: http://apache-kylin.74782.x6.nabble.com/about-count-a-partic

Re:答复: Re: Kylin Count(col) Result Is Different With Hive

2018-10-23 Thread Ma Gang
You may set the configuration 'kylin.query.enable-dynamic-column' to true in kylin.properties, and test it. At 2018-10-23 15:15:00, " 宋海涛" wrote: Hi Ma Gang, Thank you for your reply. The column ‘fzy_bill_code’ is from fact table. But what do you mean a

Re:Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Ma Gang
You may post your query related log here, there should be some query log that indicated whether the filter is push down or not, at least in the returned response stats, there's some log show how many rows are filtered in the coprocessor side. At 2018-10-23 16:58:36, "Sachin Aggarwal" wrote: >

Re:Problematic thread for Query, BadQueryDetector

2018-10-23 Thread Ma Gang
Hi Shrikant, The log indicated that your query needs data from a snapshot lookup table(your query columns should contain some derived columns defined in the cube), but the snapshot is very large, so the query is very slow. You may check the snapshot size in the lookup table's snapshot tab in K

Re:Re: Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Ma Gang
fic module(s) as required for tracing latencies -- would you please suggest any classes which can provide me these details? Regards, Shrikant Bang. On Tue, Oct 23, 2018 at 3:09 PM Ma Gang wrote: You may post your query related log here, there should be some query log that indicated whether the

Re: Why does BatchCubingJobBuilder2.java calculate building level in a different way ?

2018-11-01 Thread Ma Gang
as I remember, this is related to cube planner feature, because the cube planner algorithm will run after the statistics step, and it will change the cuboid tree structure defined by static rules, also the layer’s number will be changed accordingly, if use the original layer number some layer cu

Re:Re: Evaluate Kylin on Parquet

2018-12-18 Thread Ma Gang
Awesome! Looking forward to the improvement. For dictionary, keep the dictionary in query engine, most time is not good since it brings lots of pressure to Kylin server, but sometimes it has benefit, for example, some segments can be pruned very early when filter value is not in the dictionary,

Kylin real-time streaming is ready on realtime-streaming branch

2018-12-22 Thread Ma Gang
und in the attachment of jira: https://issues.apache.org/jira/browse/KYLIN-3654. This is just the first version, any comments and pull request are welcome! Thanks, Ma,Gang

Re:[DISCUSS] Kylin 3.0 alpha and beta release before GA

2019-03-25 Thread Ma Gang
Thanks ShaoFeng, The plan looks good. At 2019-03-25 09:24:00, "ShaoFeng Shi" wrote: Hello, About two months ago, we raised the "[Discuss] Moving toward Apache Kylin 3.0" in the developer group, all agree to use 3.0 as the next major release version when the Real-Time feature released. Now

Re:Plan to host the first "Kylin Data Summit" event

2019-05-30 Thread Ma Gang
+1, looking forward to it. At 2019-05-30 14:29:24, "ShaoFeng Shi" wrote: >Hello Kylin developers and users, > > > >We (Kyligence Inc) planned to host the first "Kylin Data Summit" event at >Shanghai, China. This event is going to provide a place to share, discuss >the technology and trends in B

Re: Kylin streaming questions

2019-06-13 Thread Ma Gang
9.0 cannot work, but not sure 0.9.x can work or not | | Ma Gang | | 邮箱:mg4w...@163.com | 签名由 网易邮箱大师 定制 On 06/13/2019 18:01, Andras Nagy wrote: Greetings, I have a few questions related to the new streaming (real-time OLAP) implementation. 1) Is there a way to have data reprocessed from

Re:Re: Kylin streaming questions

2019-06-13 Thread Ma Gang
rhaps this is something that's only available on the API?) Many thanks, Andras On Thu, Jun 13, 2019 at 1:14 PM Ma Gang wrote: Hi Andras, 1)yes, you can specify a configuration in the new cube, to consume data from start offset 2)It should work, but I haven't tested it yet 3)as

Re: Kylin streaming questions

2019-06-27 Thread Ma Gang
cube, it should be a bug | | Ma Gang | | 邮箱:mg4w...@163.com | 签名由 网易邮箱大师 定制 On 06/27/2019 11:31, Xiaoxiang Yu wrote: Hi, As far as I know, "best practice"(in my mind) of lambda mode should looks like this. Here, I use "batch segment" to refer to segment which source fr

Re:Re: Issue when recreating EMR cluster with HBase data on S3

2019-06-28 Thread Ma Gang
Hi Andras, Yes, Kylin real-time assignments currently is stored in zookeeper, you need to backup the streaming metadata information, and restored to the new zookeeper. If you don't care the previous assignments and the real-time data, you can just disable the cube, and enable it to start strea

Re: Issue when recreating EMR cluster with HBase data on S3

2019-06-29 Thread Ma Gang
Hi Shaofeng, Yes, that's true, the server address changed, the previous assignment info makes no sense anymore | | Ma Gang | | 邮箱:mg4w...@163.com | 签名由 网易邮箱大师 定制 On 06/28/2019 18:19, ShaoFeng Shi wrote: Hi Gang, On the cloud, after the cluster re-creation, the RT nodes' addr