Re: [jira] [Created] (KYLIN-1304) Incorrect count estimate upon HyperLogLogPlusCounter.merge operation.

[vipul jhawar]

Hi Luke

Please check this issue once and let us know if merging HLL for uniques has
been stable for you guys as we recently saw a huge discrepancy and wanted
to check with you guys if we should totally avoid this option for now or we
can bank upon it.

Thanks

On Tue, Jan 12, 2016 at 9:04 AM, Prashant Prakash (JIRA) 
wrote:

> Prashant Prakash created KYLIN-1304:
> ---
>
>  Summary: Incorrect count estimate upon
> HyperLogLogPlusCounter.merge operation.
>  Key: KYLIN-1304
>  URL: https://issues.apache.org/jira/browse/KYLIN-1304
>  Project: Kylin
>   Issue Type: Bug
> Affects Versions: v1.1.1
> Reporter: Prashant Prakash
>
>
> We are using hllc12 for evaluating uniques for a measure in a table
> partitioned over date.
> The uniques estimate for individual dates 2016-01-07, 2016-01-08,
> 2016-01-09 are 93,728,324, 90,982,364, 45,485,278 respectively.
> But the uniques across days, which is calculated through
> HyperLogLogPlusCounter.merge operation gives a incorrect value 67,980,576.
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

[jira] [Created] (KYLIN-1312) Enhance DeployCoprocessorCLI to support Cube level filter

[Dong Li (JIRA)]

Dong Li created KYLIN-1312:
--

 Summary: Enhance DeployCoprocessorCLI to support Cube level filter
 Key: KYLIN-1312
 URL: https://issues.apache.org/jira/browse/KYLIN-1312
 Project: Kylin
  Issue Type: Improvement
  Components: Tools, Build and Test
Affects Versions: v2.0
Reporter: Dong Li
Assignee: Dong Li
Priority: Minor


Currently DeployCoprocessorCLI can only filter on Tables, we need to enhance it 
to support Cube as filter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: can we support adding mapping cube columns to hive table columns

[yu feng]

My suggestion is adding this mapping by creating view, So, you can change
the column name in hive table, and recreate the view, that will do not have
any effect to cube. and need not sync hive table. this view only used at
the first step, so can recreate it in any time except doing this step.

2016-01-12 16:13 GMT+08:00 dong wang :

> sometimes, we have to change column names of the source hive table AFTER
> the cube is built successfully, actually, in most cases, we may just change
> the column name without changing the column type, but now, we have to
> refresh the cube again when we change the column name of the source hive
> table and sync the hive table to kylin meta. in common sense, we want that
> the cube data may be avoid being calculated again since just changing the
> column name.
>

Re: Re: encouter Deserialization error when load hive table

[wangxianbin1...@gmail.com]

HDP2.3.2, kylin 2.0-rc



wangxianbin1...@gmail.com
 
From: Li Yang
Date: 2016-01-12 17:40
To: dev
Subject: Re: Re: encouter Deserialization error when load hive table
Then let's confirm your Kylin version and HDP version first. What are they?
 
On Thu, Jan 7, 2016 at 4:03 PM, xianbin wang 
wrote:
 
> hi all!
>
> I use HDP sandbox, there is only one node, table is loaded into hive by hdp
> sandbox hive script, I don't think there will have a hive consistent
> problem
>
> best regards!
>
> --
> View this message in context:
> http://apache-kylin.74782.x6.nabble.com/encouter-Deserialization-error-when-load-hive-table-tp2888p3118.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

can we support adding mapping cube columns to hive table columns

[dong wang]

sometimes, we have to change column names of the source hive table AFTER
the cube is built successfully, actually, in most cases, we may just change
the column name without changing the column type, but now, we have to
refresh the cube again when we change the column name of the source hive
table and sync the hive table to kylin meta. in common sense, we want that
the cube data may be avoid being calculated again since just changing the
column name.

Re: A problem,maybe a bug, when querying in kylin using not in

[Li Yang]

Let's continue discussion in the JIRA, KYLIN-1294.

On Thu, Jan 7, 2016 at 11:40 AM, 曾经 <1032195...@qq.com> wrote:

> In the case you give lstg_format_name does not contain null, I tested in
> my cube, and it's also OK.
> While, when I execute the following sql, where "a" is not a member of
> "KYLIN_TV_DIM_AREA"."TV_AREA_NAME", the "TV_AREA_NAME" of "默认地区" will still
> appear in the result set. I think this is also a related problem in the bug.
> The sql is :
> SELECT "KYLIN_TV_DIM_AREA"."TV_AREA_NAME" AS "TV_AREA_NAME",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."INCOME") AS "sum_INCOME_ok",
>SUM(1) AS "sum_Number_of_Records_ok",
>SUM("KYLIN_VIEW_TVAD_SUMMARY"."VV") AS "sum_VV_ok"
>   FROM "BD_WAREHOUSE"."KYLIN_VIEW_TVAD_SUMMARY" "KYLIN_VIEW_TVAD_SUMMARY"
>   LEFT JOIN "BD_WAREHOUSE"."KYLIN_TV_DIM_AREA" "KYLIN_TV_DIM_AREA"
> ON ("KYLIN_VIEW_TVAD_SUMMARY"."AREA" =
> "KYLIN_TV_DIM_AREA"."TV_AREA_ID")
>  WHERE (("KYLIN_TV_DIM_AREA"."TV_AREA_NAME" not IN ('美剧', 'a', '默认地区')))
>  GROUP BY "KYLIN_TV_DIM_AREA"."TV_AREA_NAME";
>
>
>
>
>
> -- 原始邮件 --
> 发件人: "Li Yang";;
> 发送时间: 2016年1月7日(星期四) 中午11:28
> 收件人: "dev";
>
> 主题: Re: A problem,maybe a bug, when querying in kylin using not in
>
>
>
> Confirm this is a bug. Not sure of the root cause, but I can reproduce on
> 2.x and 1.x branch though they yield different exceptions.
>
> Open a JIRA for further investigation.
> https://issues.apache.org/jira/browse/KYLIN-1294
>
> Meanwhile worth note that this test query did pass with no problem, giving
> ('FP-GTC', 'ABIN') are valid values and lstg_format_name does not contain
> null.
>
> select lstg_format_name, sum(price) as GMV
>  from test_kylin_fact
>  where lstg_format_name not in ('FP-GTC', 'ABIN')
>  group by lstg_format_name
>
>
>
> On Thu, Jan 7, 2016 at 10:54 AM, Julian Hyde  wrote:
>
> > Yes, it looks as if you have a data correctness issue. If I were you I’d
> > look at the plan and see if it makes sense.
> >
> > Your second case, the NullPointerException issue, looks a lot like
> > https://issues.apache.org/jira/browse/CALCITE-980 <
> > https://issues.apache.org/jira/browse/CALCITE-980>.
> >
> > Julian
> >
> > > On Jan 6, 2016, at 6:28 PM, 曾经 <1032195...@qq.com> wrote:
> > >
> > > Thank you very much for answering.
> > > In my result ,not only null in H_CODE is skipped, but also the lines
> > where H_CODE is 'B' and 'O' are all skipped. I think this.
> > > I think this is not only a prolem of null.
> >
> >
>

[jira] [Created] (KYLIN-1308) query storage v2 enable parallel cube visiting

[hongbin ma (JIRA)]

hongbin ma created KYLIN-1308:
-

 Summary: query storage v2 enable parallel cube visiting
 Key: KYLIN-1308
 URL: https://issues.apache.org/jira/browse/KYLIN-1308
 Project: Kylin
  Issue Type: Improvement
Reporter: hongbin ma
Assignee: hongbin ma


Currently if a cube has multiple segments, the endpoint coprocessor invocations 
for each segments are executed sequentially. We'll try to parallize it to see 
how it contributes to performance



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Cube build error - file kylin_job_meta does not exist

[Li Yang]

Seems your hadoop has a problem when copying distributed cache.  However
not sure of the best way to troubleshoot the problem.  Maybe try a simpler
MR job that involves distributed cache?

On Fri, Jan 8, 2016 at 7:05 AM, Jiunn Jye Ng  wrote:

> Hi,
> I am trying out Kylin 1.2.
> While building the cube, the job has failed at Step 2 with error file
> "kylin_job_meta546890272332823064/meta" does not exist.
> Can someone provide some hint what could have gone wrong.
>
> Below are more detail of the error captured
>
> AM Container for appattempt_1452159080124_0008_01 exited with exitCode:
> -1000
> For more detailed output, check application tracking page:
> http://:8088/cluster/app/application_1452159080124_0008Then, click on
> links to logs of each attempt.
> Diagnostics: File
>
> file:/root/Downloads/apache-kylin-1.2-bin/tomcat/temp/kylin_job_meta546890272332823064/meta
> does not exist
> java.io.FileNotFoundException: File
>
> file:/root/Downloads/apache-kylin-1.2-bin/tomcat/temp/kylin_job_meta546890272332823064/meta
> does not exist
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
> at
>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
> at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:253)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:361)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:358)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Failing this attempt
>
>
> Thank you very much.
>
> Rgds,
> Ng Jiunn Jye (黄俊杰）
>

[jira] [Created] (KYLIN-1310) Auto Merge Time Ranges are filled with default options when edit a cube without auto merge

[Dong Li (JIRA)]

Dong Li created KYLIN-1310:
--

 Summary: Auto Merge Time Ranges are filled with default options 
when edit a cube without auto merge
 Key: KYLIN-1310
 URL: https://issues.apache.org/jira/browse/KYLIN-1310
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Affects Versions: v2.0
Reporter: Dong Li
Assignee: Zhong,Jason


1. Create a cube without auto merge
2. Edit the cube, switch to "Refresh Setting" tabpage
3. Found Auto merge time ranges are filled with default options.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-1309) cordova-plugin-inappbrowser v1.1.1 android PDF rendering

[Wade Cantrell (JIRA)]

Wade Cantrell created KYLIN-1309:


 Summary: cordova-plugin-inappbrowser v1.1.1 android PDF rendering
 Key: KYLIN-1309
 URL: https://issues.apache.org/jira/browse/KYLIN-1309
 Project: Kylin
  Issue Type: Bug
  Components: Web 
Reporter: Wade Cantrell
Assignee: Zhong,Jason






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)