Hi Aljoscha, I am using event Time based tumbling window wherein the allowedLateness is kept to Long.MAX_VALUE and I have custom trigger which is similar to 1.0.3 where Flink was not discarding late elements (we have discussed this scenario before).
The watermark is working correctly because I have validated the records earlier. I was doubtful that the RocksDB statebackend is not set , but in the logs I can clearly see that RocksDB is initialized successfully, so that should not be an issue. Even I have not changed any major code from the last performance test I had done. The snapshot I had attached is of Off-heap memory, I have only assigned 12GB heap memory per TM Regards, Vinay Patil On Wed, Jun 28, 2017 at 8:43 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, > > Just a quick question, because I’m not sure whether this came up in the > discussion so far: what kind of windows are you using? Processing > time/event time? Sliding Windows/Tumbling Windows? Allowed lateness? How is > the watermark behaving? > > Also, the latest memory usage graph you sent, is that heap memory or > off-heap memory or both? > > Best, > Aljoscha > > > On 27. Jun 2017, at 11:45, vinay patil <vinay18.pa...@gmail.com> wrote: > > > > Hi Stephan, > > > > I am observing similar issue with Flink 1.2.1 > > > > The memory is continuously increasing and data is not getting flushed to > > disk. > > > > I have attached the snapshot for reference. > > > > Also the data processed till now is only 17GB and above 120GB memory is > > getting used. > > > > Is there any change wrt RocksDB configurations > > > > <http://apache-flink-user-mailing-list-archive.2336050.n4. > nabble.com/file/n14013/TM_Memory_Usage.png> > > > > Regards, > > Vinay Patil > > > > > > > > -- > > View this message in context: http://apache-flink-user-maili > ng-list-archive.2336050.n4.nabble.com/Re-Checkpointing- > with-RocksDB-as-statebackend-tp11752p14013.html > > Sent from the Apache Flink User Mailing List archive. mailing list > archive at Nabble.com. > >
