Hi, folks,

I'm in the middle of implementing the MapState and SetState in our Samza
runner. We noticed that the state returns the Java Iterable for reading
entries, keys, etc. For state backed by file-based kv store like rocksDb,
we need to be able to let users explicitly close iterator/iterable to
release the resources.Otherwise we have to load the iterable into memory so
we can safely close the underlying rocksDb iterator, similar to Flink's
implementation. But this won't work for states that don't fit into memory.
I chatted with Kenn and he also agrees we need this capability to avoid
bulk read/write. This seems to be a general use case and I'm wondering if
we can add the support to it? I am happy to contribute to this if needed.
Any feedback is highly appreciated.

Thanks,
Xinyu

Reply via email to