Hi Hari, I see now; it is very similar to this integration with spark: https://kylin.apache.org/docs/tutorial/spark.html
Spark can register a SQL as a temp table, and then use that temp table for subsequent analysis. But other engines may not have this function, so a common view in Kylin may help to fill that gap, am I correct? hosur narahari <[email protected]> 于2018年10月2日周二 下午12:48写道： > Hi ShaoFeng, > > It could be either like lambda architecture or just like merging cube data > with latest data for which cube has not yet been generated(in case of > periodic cube generation). > > Now coming to aggregate pushdown. I'll give a very simple scenario. > Consider below query. > > *Select sum(price) from kylin_sales* > > when we execute it in any query engine like presto, spark, drill etc., it > doesn't execute above query but execute *Select price from kylin_sales.* After > getting all price values, it does *map-reduce* to calculate sum. It's > good for normal data sources but not for cube, since this is already > pre-calculated. Even if in future aggregate pushdown is provided, it'll be > very limited because we can't pushdown everything. For example, what do we > do when we've joins and then aggregate. And AFAIK, unlike indexes(used for > predicate pushdown), JDBC driver doesn't provide any information on pre > aggregated data. > > How does views, solve this problem? > > Let's say we create view on above query like *Create view sum_view as > Select sum(price) from kylin_sales;* > > In query engine we execute, *Select * from sum_view.* So kylin is queried > for sum_view, which is basically *Select sum(price) from kylin_sales.* In > this way, we can solve this problem using views. > > Please let me know, if I'm not being clear. > > Best Regards, > Hari > > On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <[email protected]> > wrote: > >> Hi Hari, >> >> Sorry for the late response. Does it like a lambda architecture? How can >> add a view to supporting aggregation pushdown? I'm not clear on this, >> please elaborate. Thank you! >> >> hosur narahari <[email protected]> 于2018年9月30日周日 下午1:50写道： >> >>> Hi ShaoFeng, >>> >>> Is it possible to provide that support. Because, in many cases cubes are >>> used with latest transaction data to get up to date analysis and some kind >>> of query engine will be used for merging cube with non-cube data. And most >>> of the query engines don't have aggregate pushdown, which makes it unable >>> to query from cube. If we provide view, we can solve this problem for all >>> query engines irrespective of whether they provide aggregate pushdown or >>> not, making kylin more adaptable. >>> >>> Also it's just a conceptual view, not adding any overhead. >>> >>> Best Regards, >>> Hari >>> >>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <[email protected]> >>> wrote: >>> >>>> Hi Hari, >>>> >>>> Creating view on top of kylin tables is not supported in Kylin. Kylin >>>> is read-only. >>>> >>>> hosur narahari <[email protected]> 于2018年9月28日周五 下午1:09写道： >>>> >>>>> Hi, >>>>> >>>>> Is it possible to create views on the lines of rdbms or hive on top of >>>>> kylin tables. >>>>> >>>>> Best Regards, >>>>> Hari >>>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> Shaofeng Shi 史少锋 >>>> >>>> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> >> -- Best regards, Shaofeng Shi 史少锋
