Hi, the heap mem usage should be available via Flink's metrics system. Not sure if that also captures spilled data. Chesnay (in CC) should know that.
If the spilled data is not available as a metric, you can try to write a small script that monitors the directories to which Flink spills (Config parameter: taskmanager.tmp.dirs [1]). The script would repeatedly list all files and keep for each file the max size (files are deleted once the are not used anymore). This is not super precise but might be good enough. Hope this helps, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.1/setup/config.html#jobmanager-amp-taskmanager 2016-12-09 14:12 GMT+01:00 otherwise777 <wou...@onzichtbaar.net>: > Currently i'm doing some analysis for some algorithms that i use in Flink, > I'm interested in the Space and time it takes to execute them. For the Time > i used getNetRuntime() in the executionenvironment, but I have no idea how > to analyse the amount of space an algorithm uses. > Space can mean different things here, like Heap space, disk space, overal > memory or allocated memory. I would like to analyze some of these. > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/How-to-analyze- > space-usage-of-Flink-algorithms-tp10555.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >
