Hi,
Chiwan is correct. The reason why we're (not yet) using managed memory in
the streaming API (DataStream) is that it was easier to get things up and
running by just using JVM heap. We're hoping to change this at some point
in the future, though.
Cheers,
Aljoscha
On Wed, 22 Jun 2016 at 14:05 Ch
Hi,
I’m not sure about the reason to use JVM heap instead of managed memory, but It
seems that the reason is using JVM heap makes development easier. Maybe Stephan
can give exact answer to you. I think managed memory still has benefit in terms
of GC time and memory utilization.
The Flink comm
Thank you for your answer to my question, Chiwan :)
Can I ask another question?
> On Jun 22, 2016, at 7:22 PM, Chiwan Park wrote:
>
> Hi Tae-Geon,
>
> AFAIK, spilling *data* to disk happens only when managed memory is used.
> Currently, streaming API (DataStream) doesn’t use managed memor
Hi Tae-Geon,
AFAIK, spilling *data* to disk happens only when managed memory is used.
Currently, streaming API (DataStream) doesn’t use managed memory yet.
`MutableHashTable` is one of representative usage of managed memory with disk
spilling. Note that some special structures such as `Compacti
I have another question.
Is the spilling only executed on batch mode?
What happen on streaming mode?
> On Jun 22, 2016, at 1:48 PM, Tae-Geon Um wrote:
>
> Hi, all
>
> As far as I know, Flink spills data (states?) to disk if the data exceeds
> memory threshold or there exists memory pressur
Hi, all
As far as I know, Flink spills data (states?) to disk if the data exceeds
memory threshold or there exists memory pressure.
i’d like to know the detail of how Flink spills data to disk.
Could you please let me know which codes do I have to investigate?
Thanks,
Taegeon
