Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-30 Thread Yu Li
of the feature, I've already mentioned my > comments > > > in > > > > > the > > > > > > original discussion thread. > > > > > > > > > > > > Cheers, > > > > > > Gordon > > > > > > >

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-29 Thread Zili Chen
> > On Thu, Aug 15, 2019 at 5:53 PM Yun Tang wrote: > > > > > > > > > > > Big +1 for this feature. > > > > > > > > > > > > Our customers including me, have ever met dilemma where we have > to > > &g

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-20 Thread Yu Li
real-time > monitoring. > > > The > > > > > larger of timer and window state, the poor performance of RocksDB. > > > > However, > > > > > switching to use FsStateBackend would always make me feel fear > about > > > the > &g

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-16 Thread Till Rohrmann
t; > > > larger of timer and window state, the poor performance of RocksDB. > > > However, > > > > switching to use FsStateBackend would always make me feel fear about > > the > > > > OOM errors. > > > > > > > > Look for

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Congxian Qiu
> > to achieve better performance together. > > > > > > Best > > > Yun Tang > > > > > > From: Stephan Ewen > > > Sent: Thursday, August 15, 2019 23:07 > > > To: dev > > > Subject: Re: [DIS

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Yu Li
owerful enrichment to state-backend, and help > Flink > > to achieve better performance together. > > > > Best > > Yun Tang > > ________________ > > From: Stephan Ewen > > Sent: Thursday, August 15, 2019 23:07 > > To: dev >

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Tzu-Li (Gordon) Tai
together. > > Best > Yun Tang > > From: Stephan Ewen > Sent: Thursday, August 15, 2019 23:07 > To: dev > Subject: Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend > > +1 for this feature. I think this will be appreciated by users, as a way to > use the HeapStat

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Yun Tang
: Spill-able Heap Keyed State Backend +1 for this feature. I think this will be appreciated by users, as a way to use the HeapStateBackend with a safety-net against OOM errors. And having had major production exposure is great. >From the implementation plan, it looks like this exists pur

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-15 Thread Stephan Ewen
+1 for this feature. I think this will be appreciated by users, as a way to use the HeapStateBackend with a safety-net against OOM errors. And having had major production exposure is great. >From the implementation plan, it looks like this exists purely in a new module and does not require any

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-13 Thread Yu Li
Sorry for forgetting to give the link of the FLIP, here it is: https://cwiki.apache.org/confluence/display/FLINK/FLIP-50%3A+Spill-able+Heap+Keyed+State+Backend Thanks! Best Regards, Yu On Tue, 13 Aug 2019 at 18:06, Yu Li wrote: > Hi All, > > We ever held a discussion about this feature

[DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-13 Thread Yu Li
Hi All, We ever held a discussion about this feature before [1] but now opening another thread because after a second thought introducing a new backend instead of modifying the existing heap backend is a better option to prevent causing any regression or surprise to existing in-production usage.