Github user ramkrish86 commented on the issue:
    Thank you. I agree this is a super important piece of FLINK where the 
memory is involved and that is one reason why I wanted to know its managment 
and usage and wanted to work on this. I have already split this into 2 tasks. 
One that deals with sorters and another is with the Iterators.
    In an iterative task I found that memory is being allocated with the 
iterators and then with the sorters.
    This PR is only for sorters.
    the other one is for the iterators
    @ggevay  is sheperding it and helping me out with the idea of making 
ResettableDrivers which I was not fully sure if it can be done.
    I had added some docs to the JIRA itself about splitting up the tasks and 
what is being addressed here and how it can be addressed.
    `SorterMemoryAllocators` are nothing but just holders of the memory 
segments that needs to be passed to the sorters. They hold the memory so that 
the read buffers, write buffers and large memory buffers can pull in memory and 
put back the memory to them at the end of each iteration. I initially 
understood that changing this should have minimal impact on other areas which 
are not impacted without which it is going to be difficult to review. 
    As said above it was just to show the intention of the changes and I am 
very much aware that these change needs time to get a thorough review and any 
design needs to be focussed on future enhancements as I had mentioned in the 
doc. A PR will always help to understand better because the changes are in code.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

Reply via email to