> > 1. Should the managed memory be part of direct memory? > The answer is no. Managed memory is currently allocated by accessing to private field of Unsafe. It is not accounted for in JVM's direct memory limit and corresponding metrics. To that end, it is equivalent to native memory.
> 2. Should the shuffle memory also be part of the managed memory? I don't think so. Shuffle (Network) memory is allocated with direct buffers, and accounted for in JVM's direct memory limit and corresponding metrics. Moreover, the FLIP-49 memory model expose network memory and managed memory as two independent components of the overall memory footprint. Thank you~ Xintong Song On Fri, Feb 21, 2020 at 11:45 AM Kurt Young <ykt...@gmail.com> wrote: > Some questions related to "managed memory": > > 1. Should the managed memory be part of direct memory? > 2. Should the shuffle memory also be part of the managed memory? > > Best, > Kurt > > > On Fri, Feb 21, 2020 at 10:41 AM Xintong Song <tonysong...@gmail.com> > wrote: > > > Thanks for driving this FLIP, Yadong. > > > > +1 (non-binding) for the FLIP in general. I think this really helps our > > users to understand and use the new FLIP-49 memory configuration. > > > > I have a few minor comments. > > - There's a frame "Other" in the frame "Non-Heap", besides "JVM Overhead" > > and "JVM Metaspace". IIUC, the purpose of this is to explain the > > mismatching between the metric "non-heap maximum" and the sum of the > > configurations "JVM metaspace" & "JVM Overhead". However, from the > > perspective of FLIP-49, JVM Overhead accounts for all the JVM non-heap > > memory usages except for metaspace. The metrics does not match the > > configuration because we did not set the a JVM parameter for "max > non-heap > > memory" (actually I'm not sure whether it can be specified in java 8). > The > > current UI might confuse people making them think there are other > non-heap > > memory usages not accounted by the configurations. Therefore, I would > > suggest to remove the "Other" frame, but add another frame inside "JVM > > Overhead", besides "Configuration", with "JVM limit" as the title and > > "non-heap max metric minus metaspace configuration" as the value . > > > > - In the final release, we have changed "shuffle memory" to "network > > memory" because the latter is easier to understand for users. I think we > > should be updated it in this FLIP as well. > > > > - There's a typo "Directed" (should be "Direct") at the direct memory > > metric. > > > > Thank you~ > > > > Xintong Song > > > > > > > > On Thu, Feb 20, 2020 at 5:52 PM Yadong Xie <vthink...@gmail.com> wrote: > > > > > Hi all > > > > > > I want to start the vote for FLIP-102, which proposes to add more > metrics > > > to the task manager in web UI. > > > > > > To help everyone better understand the proposal, we spent some efforts > on > > > making an online POC > > > > > > previous web: > > > > > > > > > http://101.132.122.69:8081/#/task-manager/6df6c5f37b2bff125dbc3a7388128559/metrics > > > POC web: > > > > > > > > > http://101.132.122.69:8081/web/#/task-manager/6df6c5f37b2bff125dbc3a7388128559/metrics > > > > > > > > > The vote will last for at least 72 hours, following the consensus > voting > > > process. > > > > > > FLIP wiki: > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-102%3A+Add+More+Metrics+to+TaskManager > > > > > > Discussion thread: > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html > > > > > > Thanks, > > > > > > Yadong > > > > > >
