[jira] [Created] (KYLIN-2454) Data generation tool will fail if column name is hive reserved keyword

[Dong Li (JIRA)]

Dong Li created KYLIN-2454:
--

 Summary: Data generation tool will fail if column name is hive 
reserved keyword
 Key: KYLIN-2454
 URL: https://issues.apache.org/jira/browse/KYLIN-2454
 Project: Kylin
  Issue Type: Bug
Reporter: Dong Li
Assignee: Dong Li
Priority: Minor


Error message:
FAILED: ParseException line 3:4 Failed to recognize predicate 'DATE'. Failed 
rule: 'identifier' in column specification



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (KYLIN-2453) The "Monitor" page consumes high CPU

[liyang (JIRA)]

liyang created KYLIN-2453:
-

 Summary: The "Monitor" page consumes high CPU
 Key: KYLIN-2453
 URL: https://issues.apache.org/jira/browse/KYLIN-2453
 Project: Kylin
  Issue Type: Bug
Reporter: liyang
Assignee: Zhixiong Chen


Maybe some auto-pulling runs in the background?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: query meet java heap space，help！

[ShaoFeng Shi]

 you can also try to allocate more memory to kylin:
https://kylin.apache.org/docs16/install/advance_settings.html

2017-02-17 17:53 GMT+08:00 Billy Liu :

> You'd better define the distinct count as a measure, with Error Rate <
> 9.75%, and have a try.
>
> 2017-02-17 16:05 GMT+08:00 yangchao :
>
> > PS：
> > if I remove distinct count condition， it can work！
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/query-meet-java-heap-space-help-tp7237p7238.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>



-- 
Best regards,

Shaofeng Shi 史少锋

Re: .jHist file got deleted on /user/kylin/.staging/ directory

[Billy Liu]

You could discard the failed build job and rebuild that segment, instead of
purging the whole cube.

2017-02-14 19:33 GMT+08:00 ketan dikshit :

> Hi Guys,
> The required .jHist file that the cube build,refresh,merge jobs wants to
> search is not available, and we get this error;
>
> java.io.FileNotFoundException: File does not exist:
> /user/kylin/.staging/job_1486033012567_33450/job_
> 1486033012567_33450_2.jhist
> …
> …
> Caused by: 
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException):
> File does not exist: /user/kylin/.staging/job_1486033012567_33450/job_
> 1486033012567_33450_2.jhist
>
>
> How can we possible ignore this in cube building(refreshing and merge as
> well) process, to get us back on the normal behaviour again;
> Dropping the entire cube and recreating it though possible but is really
> expensive operation for us.
>
>
> Thanks,
> kdcool6932
>
>
>
>

Re: query meet java heap space，help！

[Billy Liu]

You'd better define the distinct count as a measure, with Error Rate <
9.75%, and have a try.

2017-02-17 16:05 GMT+08:00 yangchao :

> PS：
> if I remove distinct count condition， it can work！
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/query-meet-java-heap-space-help-tp7237p7238.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: kylin sql query with Weird error

[Billy Liu]

The Query works fine on the latest master, could not be reproduced. It
returns 255 records:

Toys & Hobbies
59.5137
1
Toys & Hobbies
87.3074
1
Toys & Hobbies
177.3848
3
Toys & Hobbies
59.9071
2
Toys & Hobbies
128.2840
3

2017-02-17 12:40 GMT+08:00 xl l :

> hi,Billy Liu
>   thanks.  I did the same sql with sample cube, and the query result ｉｓ
> ａｌｓｏ ｅｒｒｏｒ .
>
> sql :
> select
> META_CATEG_NAME,
> sum(price) as total_selled,
> count(distinct seller_id) as sellers
> from kylin_sales
> inner join KYLIN_CATEGORY_GROUPINGS
> on KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID=KYLIN_SALES.LEAF_CATEG_ID
> and KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
> where part_dt>='2012-01-01'
> and part_dt<='2013-01-01'
> and KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME in ('Toys & Hobbies','Cameras
> & Photo')
> group by part_dt,KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME
>
>
> ｄｅｔａｉｌ ｄｅｓｃｒｉｐｔｉｏｎ ｓｅｅ ：
> http://note.youdao.com/noteshare?id=df34c64dcf3cf801a9c085be0c3f5f21=
> 7BB3043221BA44E4BAF5760339280480
>
>
>
>
> 2017-02-16 23:27 GMT+08:00 Billy Liu :
>
> > Could you reproduce this issue on the sample cube? That would help the
> dev
> > team to identify the root cause quickly.
> >
> > 2017-02-16 20:30 GMT+08:00 xl l :
> >
> > > HI, I am sure hbase is ok.
> > > 而且只有这个sql抛异常，且能稳定复现。 sql稍微改一下，就正常。
> > >
> > > 从异常日志看， 首先抛出异常的是
> > > Caused by: java.lang.NullPointerException
> > > at com.google.common.base.Preconditions.checkNotNull(
> > > Preconditions.java:191)
> > > at
> > > org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1$1.next(
> > > HBaseReadonlyStore.java:131)
> > >
> > > 对应于 代码：
> > >
> > > Pair hbaseColumn = hbaseColumns.get(i);
> > > Cell cell = findCell(oneRow, hbaseColumn.getFirst(),
> > > hbaseColumn.getSecond());
> > > Preconditions.checkNotNull(cell);
> > >
> > > cell 啥时候 会为空？
> > >
> > >
> > >
> > >
> > >
> > > 2017-02-16 17:47 GMT+08:00 ShaoFeng Shi :
> > >
> > > > "java.net.SocketTimeoutException", did you check HBase's healthy
> > status?
> > > > Are all regions of the table "KYLIN_F2H6NPOLR7" online?
> > > >
> > > > 2017-02-16 16:10 GMT+08:00 xl l :
> > > >
> > > > > see  :
> > > > >
> > > > > http://note.youdao.com/noteshare?id=df34c64dcf3cf801a9c085be0c3f5f
> > > > 21=
> > > > > 7BB3043221BA44E4BAF5760339280480
> > > > >
> > > > >
> > > > > kylin 1.6 问题记录：
> > > > > 版本：apache-kylin-1.6.0-hbase1.x-bin
> > > > >
> > > > >
> > > > > select
> > > > > cast(SUM(pv) as double) as pv,
> > > > > cast( count(distinct user_id) as double) as user_id
> > > > > from olap.olap_log_accs_page_di
> > > > > inner join DIM.DIM_LOG_USER_LOCATION on
> > > > > DIM.DIM_LOG_USER_LOCATION.user_city_code=olap.olap_log_
> > > > > accs_page_di.location
> > > > >
> > > > > inner join DIM.DIM_PUBLIC_DATE_INFO on
> > > > > DIM.DIM_PUBLIC_DATE_INFO."DATE"=olap.olap_log_accs_page_di."DATE"
> > > > > where
> > > > > DIM.DIM_PUBLIC_DATE_INFO."DATE" >=20170117
> > > > > and DIM.DIM_PUBLIC_DATE_INFO."DATE" <=20170215
> > > > > and DIM.DIM_LOG_USER_LOCATION.user_region_name in ('华东')
> > > > > group by
> > > > > DIM.DIM_LOG_USER_LOCATION.user_country_name,DIM.DIM_LOG_
> > > > > USER_LOCATION.user_province_name,DIM.DIM_LOG_USER_
> > > > > LOCATION.user_region_name
> > > > >
> > > > > order by DIM.DIM_LOG_USER_LOCATION.user_province_name ASC
> > > > >
> > > > > 上面这个ｓｑｌ执行ＯＫ，符合预期。
> > > > > 但是 如果 仅仅把 in ('华东') 改成 in ('华东','华南') 则 sql执行就会报错。
> > > > >
> > > > >
> > > > > 错误信息如下所示：
> > > > >
> > > > >
> > > > > 查看 kylin.log日志， 详细的异常信息 ：
> > > > > http://note.youdao.com/noteshare?id=a1c257599774c4bccb0c6763923359
> > > > d5=
> > > > > 11C6AA36AC894EDD9006DDAE17B16747
> > > > > 2017-02-16 15:32:51,080 WARN [kylin-coproc--pool3-t5578]
> > > > > ipc.CoprocessorRpcChannel:58 : Call failed on IOException
> > > > > java.net.SocketTimeoutException: callTimeout=6,
> > > callDuration=114625:
> > > > > row ' ' on table 'KYLIN_F2H6NPOLR7' at
> > > > > region=KYLIN_F2H6NPOLR7,,1487168742102.
> > 433e266be82448c5380610e9e77046
> > > > 58.,
> > > > > hostname=jx-db-hbase03.22lll.com,16020,1480406440673, seqNum=2
> > > > > at
> > > > > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(
> > > > > RpcRetryingCaller.java:159)
> > > > > at
> > > > > org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.
> > > callExecService(
> > > > > RegionCoprocessorRpcChannel.java:95)
> > > > > at
> > > > > org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(
> > > > > CoprocessorRpcChannel.java:56)
> > > > > at
> > > > > org.apache.kylin.storage.hbase.cube.v2.coprocessor.
> > endpoint.generated.
> > > > > CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.
> > > > java:4178)
> > > > >
> > > > > Caused by: java.lang.NullPointerException
> > > > > at com.google.common.base.Preconditions.checkNotNull(
> > > > > Preconditions.java:191)
> > > > > at
> > > > > org.apache.kylin.storage.hbase.cube.v2.
> 

query meet java heap space，help！

[yangchao]

==[QUERY]===
Query Id: b3c72902-f5f8-47f0-b08c-d7318817236c
SQL: select
DWD_LOOKUP_TIME_A_ONCE.TIME_DD,
DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.GOODS_ID ,
count(*) as ordersNum,
sum(DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.GOODS_NUM) as orderGoodsNum,
count(distinct(DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.GROUP_ORDER_ID)) as
successGroupNum,
sum(DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.ORDER_AMOUNT) as gmv
from DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D
join DWD_LOOKUP_TIME_A_ONCE on
DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.CONFIRM_TIME_HH
=DWD_LOOKUP_TIME_A_ONCE.TIME_HH
where DWD_LOOKUP_TIME_A_ONCE.TIME_DD='20161212'
group by DWD_LOOKUP_TIME_A_ONCE.TIME_DD
,DWD_FACT_SUCCESS_GROUP_ORDERS_I_1D.GOODS_ID
limit 1 offset 0
User: ADMIN
Success: false
Duration: 0.0
Project: pro_ares
Realization Names: [pro_ares_orders_model_cube]
Cuboid Ids: [12]
Total scan count: 0
Result row count: 0
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Message: Java heap space
==[QUERY]===

2017-02-17 16:02:12,114 ERROR [http-bio-7070-exec-13]
controller.BasicController:44 :
org.apache.kylin.rest.exception.InternalErrorException: Java heap space
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:389)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:69)
at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/query-meet-java-heap-space-help-tp7237.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: query meet java heap space，help！

[yangchao]

PS：
if I remove distinct count condition， it can work！ 

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/query-meet-java-heap-space-help-tp7237p7238.html
Sent from the Apache Kylin mailing list archive at Nabble.com.