Hi ShaoFeng, Exactly. It will solve the problem for all query engines whatsoever.
Best Regards, Hari On Sat, Oct 6, 2018 at 6:51 PM ShaoFeng Shi <[email protected]> wrote: > Hi Hari, > > I see now; it is very similar to this integration with spark: > https://kylin.apache.org/docs/tutorial/spark.html > > Spark can register a SQL as a temp table, and then use that temp table for > subsequent analysis. But other engines may not have this function, so a > common view in Kylin may help to fill that gap, am I correct? > > > > hosur narahari <[email protected]> 于2018年10月2日周二 下午12:48写道： > >> Hi ShaoFeng, >> >> It could be either like lambda architecture or just like merging cube >> data with latest data for which cube has not yet been generated(in case of >> periodic cube generation). >> >> Now coming to aggregate pushdown. I'll give a very simple scenario. >> Consider below query. >> >> *Select sum(price) from kylin_sales* >> >> when we execute it in any query engine like presto, spark, drill etc., it >> doesn't execute above query but execute *Select price from kylin_sales.* >> After >> getting all price values, it does *map-reduce* to calculate sum. It's >> good for normal data sources but not for cube, since this is already >> pre-calculated. Even if in future aggregate pushdown is provided, it'll be >> very limited because we can't pushdown everything. For example, what do we >> do when we've joins and then aggregate. And AFAIK, unlike indexes(used for >> predicate pushdown), JDBC driver doesn't provide any information on pre >> aggregated data. >> >> How does views, solve this problem? >> >> Let's say we create view on above query like *Create view sum_view as >> Select sum(price) from kylin_sales;* >> >> In query engine we execute, *Select * from sum_view.* So kylin is >> queried for sum_view, which is basically *Select sum(price) from >> kylin_sales.* In this way, we can solve this problem using views. >> >> Please let me know, if I'm not being clear. >> >> Best Regards, >> Hari >> >> On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <[email protected]> >> wrote: >> >>> Hi Hari, >>> >>> Sorry for the late response. Does it like a lambda architecture? How can >>> add a view to supporting aggregation pushdown? I'm not clear on this, >>> please elaborate. Thank you! >>> >>> hosur narahari <[email protected]> 于2018年9月30日周日 下午1:50写道： >>> >>>> Hi ShaoFeng, >>>> >>>> Is it possible to provide that support. Because, in many cases cubes >>>> are used with latest transaction data to get up to date analysis and some >>>> kind of query engine will be used for merging cube with non-cube data. And >>>> most of the query engines don't have aggregate pushdown, which makes it >>>> unable to query from cube. If we provide view, we can solve this problem >>>> for all query engines irrespective of whether they provide aggregate >>>> pushdown or not, making kylin more adaptable. >>>> >>>> Also it's just a conceptual view, not adding any overhead. >>>> >>>> Best Regards, >>>> Hari >>>> >>>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <[email protected]> >>>> wrote: >>>> >>>>> Hi Hari, >>>>> >>>>> Creating view on top of kylin tables is not supported in Kylin. Kylin >>>>> is read-only. >>>>> >>>>> hosur narahari <[email protected]> 于2018年9月28日周五 下午1:09写道： >>>>> >>>>>> Hi, >>>>>> >>>>>> Is it possible to create views on the lines of rdbms or hive on top >>>>>> of kylin tables. >>>>>> >>>>>> Best Regards, >>>>>> Hari >>>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Shaofeng Shi 史少锋 >>>>> >>>>> >>> >>> -- >>> Best regards, >>> >>> Shaofeng Shi 史少锋 >>> >>> > > -- > Best regards, > > Shaofeng Shi 史少锋 > >
