Here's JIRA: https://issues.apache.org/jira/browse/KYLIN-599
Best Regards! --------------------- Luke Han On Thu, Sep 17, 2015 at 1:09 AM, Henry Saputra <[email protected]> wrote: > That is good to know. Li Yang, Luke, could one of you share the design > document for this realtime OLAP query in the JIRA? > > Thanks, > > - Henry > > On Tue, Sep 15, 2015 at 11:12 PM, Li Yang <[email protected]> wrote: > >> There will be incremental updates on the existing cubes, but during > >> that updates I suppose no queries will be ran against them? > > > > Yes, it's mini batch, usually at minutes interval. And of course cube CAN > > serve query while the mini incremental is under built. How can we let the > > cube offline every few minutes, that's impossible. :-) > > > > On Tue, Sep 15, 2015 at 6:41 PM, Sarnath <[email protected]> wrote: > > > >> Inverted index? That sounds interesting. We use inverted index to serve > the > >> cubes in our internal implementation. > >> > >> I come from Big Data Center of excellence from an Indian IT major. > >> > >> We have been experimenting with the idea of serving cubes through > >> ElasticSearch REST API. This is not related to Kylin. This is our own > >> internal development. > >> > >> The motivation for this is --- Once the cube is built, it needs to be > >> served. > >> > >> The query looks somewhat like this: > >> > >> "Given ProductID=*, Year=2015, Fetch All Quantities Sold" > >> > >> "Given ProductID=XX, Fetch how much it has sold every Month" > >> > >> Find all entries that match K1=V1, K2=V2 > >> > >> This relieves us from lot of things - storage, REST API etc. and makes > the > >> cubes easily searchable. > >> > >> However, we don't do SQL/MDX on top of it. Tableau 9.1Beta is > >> experimenting with Web-Data-Connector which we believe can be used for > >> Visualization... Apart from that, we experimented with a few > >> auto-generated Kibana dashboards which were just okay. But Kibana was > not > >> designed for Cubes and so it has its own limitations. > >> > >> Appreciate any feedback! > >> > >> Thanks, > >> > >> Best, > >> > >> Sarnath > >> I also think that it's a mini batch cubing. It's time to bring back > the > >> inverted index into roadmap. The inverted index will be the true > real-time > >> solution and can provide the low-level query capability on the raw data. > >> > >> > >> Thanks! > >> JiangXu > >> > >> > >> ------------------ 原始邮件 ------------------ > >> 发件人: "Henry Saputra";<[email protected]>; > >> 发送时间: 2015年9月15日(星期二) 中午12:39 > >> 收件人: "[email protected]"<[email protected]>; > >> > >> 主题: Re: Kylin Real time > >> > >> > >> > >> Ok, but that still seems like mini batch to me. > >> > >> There will be incremental updates on the existing cubes, but during > >> that updates I suppose no queries will be ran against them? > >> > >> - Henry > >> > >> On Mon, Sep 14, 2015 at 12:33 AM, Li Yang <[email protected]> wrote: > >> > Streaming OLAP provides Near-Realtime analysis where data delay can > be as > >> > short as a few minutes. > >> > > >> > Traditional daily build allows user to analyze yesterday's data. If > >> > increase the frequency to hourly, then user can analyze last hour's > data. > >> > Further down the line, how about incremental build every 5 minutes > from a > >> > streaming source? Then user can analyze data 5 minutes ago. That's > >> > Streaming OLAP! > >> > > >> > On Mon, Sep 14, 2015 at 12:43 AM, Henry Saputra < > [email protected] > >> > > >> > wrote: > >> > > >> >> Hi Luke, > >> >> > >> >> Could you clarify again what is the streaming OLAP means here? > >> >> > >> >> By definition OLAP work with historical data. > >> >> > >> >> Maybe I missed it but was there any discussions or proposed design > for > >> it? > >> >> > >> >> Thanks, > >> >> > >> >> - Henry > >> >> > >> >> On Monday, August 3, 2015, Luke Han <[email protected]> wrote: > >> >> > >> >> > Hi Siddharth, > >> >> > Kylin's next majority release (0.8.x) will support Streaming > OLAP > >> >> which > >> >> > will coming in Q4 since it still under development now, as Hongbin > >> >> > mentioned above. > >> >> > Could you please drop me a mail about your case? I would like > to > >> >> > better understand your scenario to well manage coming features? > >> >> > > >> >> > Thanks. > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > Best Regards! > >> >> > --------------------- > >> >> > > >> >> > Luke Han > >> >> > > >> >> > On Wed, Jul 29, 2015 at 2:08 PM, hongbin ma <[email protected] > >> >> > <javascript:;>> wrote: > >> >> > > >> >> > > For current 0.7 releases, you cannot. > >> >> > > > >> >> > > Real time data processing and querying will be added in 0.8 > release. > >> It > >> >> > is > >> >> > > still under development and testing. We have achieved good > progress > >> on > >> >> > it, > >> >> > > please wait for announcements. > >> >> > > > >> >> > > On Wed, Jul 29, 2015 at 2:02 PM, Siddharth Ubale < > >> >> > > [email protected] <javascript:;>> wrote: > >> >> > > > >> >> > > > Hi , > >> >> > > > > >> >> > > > I would like to ask whether Kylin can be used as a real time > >> querying > >> >> > > > system? > >> >> > > > The process of building a cube , makes it look like a batch > >> process > >> >> > after > >> >> > > > which the queries are with low latency.. however can > >> >> > > > We get a real time idea of what the OLAP system's state is at > the > >> >> query > >> >> > > > instance? > >> >> > > > > >> >> > > > Thanks, > >> >> > > > Siddharth > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > -- > >> >> > > Regards, > >> >> > > > >> >> > > *Bin Mahone | 马洪宾* > >> >> > > Apache Kylin: http://kylin.io > >> >> > > Github: https://github.com/binmahone > >> >> > > > >> >> > > >> >> > >> >
