Sure Till, I would love to also make the patch but need to prioritize some other things these days. At least I will dig and see how complex this is regarding the different backends. I also have some follow-up questions, in case anybody has thought about these things already (or is simply interested): - Do you think it would make sense to automatically garbage collect empty states in general? - Shouldn't this happen already during snapshot compaction (in rocksdb) and would that violate any user assumptions in your view?
> On 24 Jan 2017, at 11:44, Till Rohrmann <[email protected]> wrote: > > Hi Paris, > > if there is no such issue open, then please open one so that we can track > the issue. If you have time to work on that even better :-) > > Cheers, > Till > > On Tue, Jan 24, 2017 at 10:25 AM, Paris Carbone <[email protected]> wrote: > >> Any thoughts/plans? >> So should I open a Jira and add this? >> >> Paris >> >> On Jan 21, 2017, at 5:17 PM, Paris Carbone <[email protected]<mailto:parisc@ >> kth.se>> wrote: >> >> Thank you for the answer Ufuk! >> >> To elaborate a bit more, I am not using keyed state, it would be indeed >> tricky in that case to discard everything. >> >> I need that for operator state, in my loop fault tolerance PR [1]. The >> idea is to tag a ListState (upstream log) per snapshot id. >> When a concurent snapshot is commited I want to simply remove everything >> related to that ListState (not just clear it). This would also eliminate a >> memory leak in case many empty logs accumulate in time (and thus state >> entries). >> Hope that makes it a bit more clear. Thanks again :) >> >> Paris >> >> [1] https://github.com/apache/flink/pull/1668 >> >> >> On 21 Jan 2017, at 17:10, Ufuk Celebi <[email protected]<mailto:uce@ >> apache.org>> wrote: >> >> Hey Paris! >> >> As far as I know it's not possible at the moment and not planned. Does >> not sound to hard to add though. @Stefan: correct? >> >> You can currently only clear the state via #clear in the scope of the >> key for keyed state or the whole operator when used with operator >> state. In case of keyed state it's indeed hard to clear all state for >> operator state it's slightly better. I'm curious what your use case >> is? >> >> - Ufuk >> >> >> On Fri, Jan 20, 2017 at 5:59 PM, Paris Carbone <[email protected]<mailto: >> [email protected]>> wrote: >> Hi folks, >> >> I have a little question regarding the managed store operator backend, in >> case someone can help. >> >> Is there some convenient way (planned or under development) to completely >> unregister a state entry (e.g. a ListState) with a given id from the >> backend? >> It is fairly easy to register new states dynamically (i.e. with >> getOperatorState(...)), why not being able to discard it as well? >> >> I would find this feature extremely convenient to a fault tolerance >> related PR I am working on but I can think of many use cases that might >> need it. >> >> >> Paris >> >> >>
