Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
Right now, I don’t think there is a way of doing that. I don’t think there is something fundament against having a method that drops a state complete, data and registered meta data. But so far that never existed and it seems nobody ever needed it (or asked for it at least). The closest thing

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
I had a quick look at it, and we could do that, even for RocksDB: the method does a meta data lookup similar to what state registration does, remove the meta data and drop the column family. But until then, there is currently no complete dropping a keyed state. > Am 22.02.2018 um 12:19 schrieb

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Gyula Fóra
Do you have any suggestion how to completely delete an operator and keyed state? For operator state this seems to be easy enough, but what about completely dropping a keyed state? Gyula Stefan Richter ezt írta (időpont: 2018. febr. 22., Cs, 11:46): > > Hi, > > I

Re: Serious memory leak in DefaultOperatorStateBackend

2018-02-22 Thread Stefan Richter
Hi, I don’t think that this is a bug, but rather a necessity that comes with the (imo questionable) design of allowing lazy state registration. In this design, just because a state is *currently* not registered does not mean that you can simply drop it. Imagine that your code did *not yet*