[jira] [Created] (KYLIN-3645) Kylin does not clean table metadata when drop project(Kafka Source)

2018-10-23 Thread rongchuan.jin (JIRA)
rongchuan.jin created KYLIN-3645: Summary: Kylin does not clean table metadata when drop project(Kafka Source) Key: KYLIN-3645 URL: https://issues.apache.org/jira/browse/KYLIN-3645 Project: Kylin

Re:Re: Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Ma Gang
Hi Shrikant, Which Kylin version you used? By default, most of the filters will be pushed down to coprocessor, you may grep "Total filtered row:" in your log to see how many rows are filtered from coprocessor side. If you want to change code to add some log there, you can add log at method:

Re: Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Shrikant Bang
Hi Team, I tried debugging code to understand query execution (on Kylin v2.5 hbase 1.x) . As I understood query is formed and executed using Calcite APIs. Could you please help me to get answer of below? 1. With my understanding default implementation of filter operation (WHERE clause)

Re: join column derived vs normal

2018-10-23 Thread 陶 加涛
Hi Sachin Aggarwal When a lookup table is stored as snapshot, all columns in that table will be taken as derived dimension by default during cube design. Compare to normal dimension, derived dimension does not participate in cube calculation directly. Instead, it is represented by its FK

??????join column derived vs normal

2018-10-23 Thread Chao Long
I hope this may help you. http://kylin.apache.org/docs/howto/howto_optimize_cubes.html -- -- ??: "Sachin Aggarwal"; : 2018??10??23??(??) 7:25 ??: "user";"dev"; : join column derived vs normal Hi Team, while creating

Re: Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Shrikant Bang
Thank you Ma for response. I didn't find any logs about applying filters in co-processors. Also I don't see logs for logical/physical query plans in kylin.log. Am I missing anything? I would like to put logger statements in code and build specific module(s) as required for tracing latencies --

join column derived vs normal

2018-10-23 Thread Sachin Aggarwal
Hi Team, while creating cube what are recommendations across join column between fact and dimension. should we mark it as derived or normal while selecting it from dimension column. what factors play a role in making this decision. -- Thanks & Regards Sachin Aggarwal

Re: Problematic thread for Query, BadQueryDetector

2018-10-23 Thread Shrikant Bang
Thank you Ma for reply. I have one of dimension of size 1.2GB with records ~ 13 millions and its a UHC. I am trying with higher size of 'kylin.dictionary.forest-trie-max-mb' to 2GB and marking 'kylin.dictionary.reusable' true as dimensions are static. Are there any other setting which could help

Re:Problematic thread for Query, BadQueryDetector

2018-10-23 Thread Ma Gang
Hi Shrikant, The log indicated that your query needs data from a snapshot lookup table(your query columns should contain some derived columns defined in the cube), but the snapshot is very large, so the query is very slow. You may check the snapshot size in the lookup table's snapshot tab in

[jira] [Created] (KYLIN-3644) NumberFormatExcetion on null values when building cube with Spark

2018-10-23 Thread Hubert STEFANI (JIRA)
Hubert STEFANI created KYLIN-3644: - Summary: NumberFormatExcetion on null values when building cube with Spark Key: KYLIN-3644 URL: https://issues.apache.org/jira/browse/KYLIN-3644 Project: Kylin

Problematic thread for Query, BadQueryDetector

2018-10-23 Thread Shrikant Bang
Hi Team, I am observing some queries are running for longer time and getting failed. Here is exception in kylin logs for "BadQueryDetector". Could someone please help me to understand about Bad Queries and meaning of below stack trace : 2018-10-23 09:36:40,647 DEBUG

Re:Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Ma Gang
You may post your query related log here, there should be some query log that indicated whether the filter is push down or not, at least in the returned response stats, there's some log show how many rows are filtered in the coprocessor side. At 2018-10-23 16:58:36, "Sachin Aggarwal" wrote:

Re: Slow Query Performance With 'WHERE' Clause

2018-10-23 Thread Sachin Aggarwal
Hi Team, Could you please confirm if filtering of rows (WHERE clause) is done in CoProcessor side? Is there any APIs/logging to get physical plan of query? It will help us in optimising the cube. On Mon, Oct 22, 2018 at 8:58 PM Shrikant Bang wrote: > Thanks ShaoFeng for response.

Re: Derived column from windowSpec not working in where

2018-10-23 Thread DDDQ
Hi Yi, I have reproduced this issue using the sample dataset of kylin, and I've opened a JIRA to track this issue. https://issues.apache.org/jira/browse/KYLIN-3643 Best Regards, May -- Sent from: http://apache-kylin.74782.x6.nabble.com/

[jira] [Created] (KYLIN-3643) Derived column from windowSpec not working in where

2018-10-23 Thread May Zhou (JIRA)
May Zhou created KYLIN-3643: --- Summary: Derived column from windowSpec not working in where Key: KYLIN-3643 URL: https://issues.apache.org/jira/browse/KYLIN-3643 Project: Kylin Issue Type: Bug