Hi! I do slightly disagree with Timo. Custom memory management is always useful, also in the Streaming API. It makes execution more robust.
If you use RocksDB as a state backend, you get memory management from RocksDB - effectively all your program key/value state is off-heap. Flink's own state backends have not yet implemented custom memory management (it is quite a bit more complex in a true streaming environment than in batch), but it will come as a feature (though not officially tracked as a jira). Stephan On Thu, Dec 15, 2016 at 10:43 AM, Tao Meng <oatg...@gmail.com> wrote: > Thanks a lot. > > On 12ζ 15 2016, at 5:39 δΈε, Timo Walther <twal...@apache.org> wrote: > >> Hi Tao, >> >> no, streaming jobs do not use managed memory yet. Managed memory is >> useful for sorting, joining and grouping bounded data. Unbounded stream do >> not need that. >> >> It could be used in the future e.g. to store state or for new operators, >> but is this is not on the roadmap so far. >> >> Regards, >> Timo >> >> >> Am 15/12/16 um 10:30 schrieb Tao Meng: >> >> Hi all, >> >> I have some questions about memory management in the Streaming mode. >> >> Do the Streaming jobs use the memory management module ? >> If they don't, for what considerations do not ? Because Data exchange >> is too frequent ? >> Is there a plan to let streaming job use it ? >> >> Thanks a lot. >> >> >>
