The inverted index development is paused a while, agree to Xu, it's time to resume it back for extreme low latency cases.
Best Regards! --------------------- Luke Han On Wed, Sep 16, 2015 at 2:12 PM, Li Yang <[email protected]> wrote: > > There will be incremental updates on the existing cubes, but during > > that updates I suppose no queries will be ran against them? > > Yes, it's mini batch, usually at minutes interval. And of course cube CAN > serve query while the mini incremental is under built. How can we let the > cube offline every few minutes, that's impossible. :-) > > On Tue, Sep 15, 2015 at 6:41 PM, Sarnath <[email protected]> wrote: > > > Inverted index? That sounds interesting. We use inverted index to serve > the > > cubes in our internal implementation. > > > > I come from Big Data Center of excellence from an Indian IT major. > > > > We have been experimenting with the idea of serving cubes through > > ElasticSearch REST API. This is not related to Kylin. This is our own > > internal development. > > > > The motivation for this is --- Once the cube is built, it needs to be > > served. > > > > The query looks somewhat like this: > > > > "Given ProductID=*, Year=2015, Fetch All Quantities Sold" > > > > "Given ProductID=XX, Fetch how much it has sold every Month" > > > > Find all entries that match K1=V1, K2=V2 > > > > This relieves us from lot of things - storage, REST API etc. and makes > the > > cubes easily searchable. > > > > However, we don't do SQL/MDX on top of it. Tableau 9.1Beta is > > experimenting with Web-Data-Connector which we believe can be used for > > Visualization... Apart from that, we experimented with a few > > auto-generated Kibana dashboards which were just okay. But Kibana was not > > designed for Cubes and so it has its own limitations. > > > > Appreciate any feedback! > > > > Thanks, > > > > Best, > > > > Sarnath > > I also think that it's a mini batch cubing. It's time to bring back the > > inverted index into roadmap. The inverted index will be the true > real-time > > solution and can provide the low-level query capability on the raw data. > > > > > > Thanks! > > JiangXu > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: "Henry Saputra";<[email protected]>; > > 发送时间: 2015年9月15日(星期二) 中午12:39 > > 收件人: "[email protected]"<[email protected]>; > > > > 主题: Re: Kylin Real time > > > > > > > > Ok, but that still seems like mini batch to me. > > > > There will be incremental updates on the existing cubes, but during > > that updates I suppose no queries will be ran against them? > > > > - Henry > > > > On Mon, Sep 14, 2015 at 12:33 AM, Li Yang <[email protected]> wrote: > > > Streaming OLAP provides Near-Realtime analysis where data delay can be > as > > > short as a few minutes. > > > > > > Traditional daily build allows user to analyze yesterday's data. If > > > increase the frequency to hourly, then user can analyze last hour's > data. > > > Further down the line, how about incremental build every 5 minutes > from a > > > streaming source? Then user can analyze data 5 minutes ago. That's > > > Streaming OLAP! > > > > > > On Mon, Sep 14, 2015 at 12:43 AM, Henry Saputra < > [email protected] > > > > > > wrote: > > > > > >> Hi Luke, > > >> > > >> Could you clarify again what is the streaming OLAP means here? > > >> > > >> By definition OLAP work with historical data. > > >> > > >> Maybe I missed it but was there any discussions or proposed design for > > it? > > >> > > >> Thanks, > > >> > > >> - Henry > > >> > > >> On Monday, August 3, 2015, Luke Han <[email protected]> wrote: > > >> > > >> > Hi Siddharth, > > >> > Kylin's next majority release (0.8.x) will support Streaming > OLAP > > >> which > > >> > will coming in Q4 since it still under development now, as Hongbin > > >> > mentioned above. > > >> > Could you please drop me a mail about your case? I would like > to > > >> > better understand your scenario to well manage coming features? > > >> > > > >> > Thanks. > > >> > > > >> > > > >> > > > >> > > > >> > Best Regards! > > >> > --------------------- > > >> > > > >> > Luke Han > > >> > > > >> > On Wed, Jul 29, 2015 at 2:08 PM, hongbin ma <[email protected] > > >> > <javascript:;>> wrote: > > >> > > > >> > > For current 0.7 releases, you cannot. > > >> > > > > >> > > Real time data processing and querying will be added in 0.8 > release. > > It > > >> > is > > >> > > still under development and testing. We have achieved good > progress > > on > > >> > it, > > >> > > please wait for announcements. > > >> > > > > >> > > On Wed, Jul 29, 2015 at 2:02 PM, Siddharth Ubale < > > >> > > [email protected] <javascript:;>> wrote: > > >> > > > > >> > > > Hi , > > >> > > > > > >> > > > I would like to ask whether Kylin can be used as a real time > > querying > > >> > > > system? > > >> > > > The process of building a cube , makes it look like a batch > > process > > >> > after > > >> > > > which the queries are with low latency.. however can > > >> > > > We get a real time idea of what the OLAP system's state is at > the > > >> query > > >> > > > instance? > > >> > > > > > >> > > > Thanks, > > >> > > > Siddharth > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > -- > > >> > > Regards, > > >> > > > > >> > > *Bin Mahone | 马洪宾* > > >> > > Apache Kylin: http://kylin.io > > >> > > Github: https://github.com/binmahone > > >> > > > > >> > > > >> > > >
