Hi Hari, Would you like to open a JIRA to Kylin with this? Thanks!
hosur narahari <[email protected]> 于2018年10月6日周六 下午9:25写道： > Hi ShaoFeng, > > Exactly. It will solve the problem for all query engines whatsoever. > > Best Regards, > Hari > > On Sat, Oct 6, 2018 at 6:51 PM ShaoFeng Shi <[email protected]> > wrote: > >> Hi Hari, >> >> I see now; it is very similar to this integration with spark: >> https://kylin.apache.org/docs/tutorial/spark.html >> >> Spark can register a SQL as a temp table, and then use that temp table >> for subsequent analysis. But other engines may not have this function, so a >> common view in Kylin may help to fill that gap, am I correct? >> >> >> >> hosur narahari <[email protected]> 于2018年10月2日周二 下午12:48写道： >> >>> Hi ShaoFeng, >>> >>> It could be either like lambda architecture or just like merging cube >>> data with latest data for which cube has not yet been generated(in case of >>> periodic cube generation). >>> >>> Now coming to aggregate pushdown. I'll give a very simple scenario. >>> Consider below query. >>> >>> *Select sum(price) from kylin_sales* >>> >>> when we execute it in any query engine like presto, spark, drill etc., >>> it doesn't execute above query but execute *Select price from >>> kylin_sales.* After getting all price values, it does *map-reduce* to >>> calculate sum. It's good for normal data sources but not for cube, since >>> this is already pre-calculated. Even if in future aggregate pushdown is >>> provided, it'll be very limited because we can't pushdown everything. For >>> example, what do we do when we've joins and then aggregate. And AFAIK, >>> unlike indexes(used for predicate pushdown), JDBC driver doesn't provide >>> any information on pre aggregated data. >>> >>> How does views, solve this problem? >>> >>> Let's say we create view on above query like *Create view sum_view as >>> Select sum(price) from kylin_sales;* >>> >>> In query engine we execute, *Select * from sum_view.* So kylin is >>> queried for sum_view, which is basically *Select sum(price) from >>> kylin_sales.* In this way, we can solve this problem using views. >>> >>> Please let me know, if I'm not being clear. >>> >>> Best Regards, >>> Hari >>> >>> On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <[email protected]> >>> wrote: >>> >>>> Hi Hari, >>>> >>>> Sorry for the late response. Does it like a lambda architecture? How >>>> can add a view to supporting aggregation pushdown? I'm not clear on this, >>>> please elaborate. Thank you! >>>> >>>> hosur narahari <[email protected]> 于2018年9月30日周日 下午1:50写道： >>>> >>>>> Hi ShaoFeng, >>>>> >>>>> Is it possible to provide that support. Because, in many cases cubes >>>>> are used with latest transaction data to get up to date analysis and some >>>>> kind of query engine will be used for merging cube with non-cube data. And >>>>> most of the query engines don't have aggregate pushdown, which makes it >>>>> unable to query from cube. If we provide view, we can solve this problem >>>>> for all query engines irrespective of whether they provide aggregate >>>>> pushdown or not, making kylin more adaptable. >>>>> >>>>> Also it's just a conceptual view, not adding any overhead. >>>>> >>>>> Best Regards, >>>>> Hari >>>>> >>>>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Hari, >>>>>> >>>>>> Creating view on top of kylin tables is not supported in Kylin. Kylin >>>>>> is read-only. >>>>>> >>>>>> hosur narahari <[email protected]> 于2018年9月28日周五 下午1:09写道： >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Is it possible to create views on the lines of rdbms or hive on top >>>>>>> of kylin tables. >>>>>>> >>>>>>> Best Regards, >>>>>>> Hari >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> >>>>>> Shaofeng Shi 史少锋 >>>>>> >>>>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> Shaofeng Shi 史少锋 >>>> >>>> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> >> -- Best regards, Shaofeng Shi 史少锋
