Re: Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-03 Thread Billy Liu
Cool. It extends Kylin scenario into Real Time Query. With Warm regards Billy Liu ShaoFeng Shi 于2018年11月2日周五 下午7:17写道: > > Hi Gang, I appreciate your hard work! > > Ma Gang 于2018年11月1日周四 下午3:29写道: > > > Hi ShaoFeng, > > For streaming ingest/query performance, there is a doc: > >

Re: Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-02 Thread ShaoFeng Shi
Hi Gang, I appreciate your hard work! Ma Gang 于2018年11月1日周四 下午3:29写道: > Hi ShaoFeng, > For streaming ingest/query performance, there is a doc: > https://drive.google.com/file/d/1GSBMpRuVQRmr8Ev2BWvssfMd-Rck9vsH/view?ths=true > , it is also in the design doc's 'performance' section attached in

Re:Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-01 Thread Ma Gang
Hi ShaoFeng, For streaming ingest/query performance, there is a doc: https://drive.google.com/file/d/1GSBMpRuVQRmr8Ev2BWvssfMd-Rck9vsH/view?ths=true , it is also in the design doc's 'performance' section attached in the jira: https://issues.apache.org/jira/browse/KYLIN-3654 For stability, it is

Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-01 Thread Xiaoxiang Yu
Thank you for your reply. Maybe I can help to improve your Kylin Streaming Solution in the future. Best wishes, Xiaoxiang Yu On [DATE], "[NAME]" <[ADDRESS]> wrote: Thanks Xiaoxiang, Very good questions! Please see my comments started with [Gang]: 1.

Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-11-01 Thread ShaoFeng Shi
Hi Gang, Thank you for the information, that is helpful for understanding the overall design and implementation. Do you have some statistical information, like performance, throughput, stability, etc.? Besides, what's the plan of contributing it to the community? Thanks! Ma Gang 于2018年11月1日周四

Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-10-31 Thread Xiaoxiang Yu
Hi gang, I am so glad to know that eBay has a solution for realtime olap on kylin. I have some small question: 1. Is it possible to use Yarn as cluster manager for index task. Coordinator process will set up them at specificed period. Yarn will manage : a) retry these task if some

Re:Re: Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-10-31 Thread Ma Gang
Hi ShaoFeng, Sorry I reply the wrong email, copy my reply here for further discussion :)Very good questions, please see my comments start with [Gang]:1) How to bridge the real-time cube with a cube built from Hive? You know, in Kylin the source type is marked at the table level, which means a

Re: [DISCUSS] New Kylin Streaming Solution From eBay

2018-10-30 Thread ShaoFeng Shi
Hi Gang, The design doc is still missing; can you upload it to somewhere and then provide a link? Ma Gang 于2018年10月30日周二 下午8:35写道: > Resend the design doc, not sure why the attachment is removed in the > previous mail. > > At 2018-10-30 15:24:01, "Ma Gang" wrote: > > Hi all, > > eBay Kylin

Re:[DISCUSS] New Kylin Streaming Solution From eBay

2018-10-30 Thread Ma Gang
Resend the design doc, not sure why the attachment is removed in the previous mail. At 2018-10-30 15:24:01, "Ma Gang" wrote: Hi all, eBay Kylin team has developed a new Kylin streaming solution, the basic idea is to build a streaming cluster to ingest data from streaming source(Kafka), and