The snapshotting stops much before the actual JVM exits - when a map/reduce task is deemed done, the metric collection stops. The JVM exits a little later.
Thanks, +Vinod On Jul 12, 2013, at 3:57 PM, hadoop qi wrote: > If a program terminates normally, i would assume its memory usage close to > zero or at least very small, because the program release most memory it was > allocated. That would mean all jobs in Hadoop had very small memory usage, > because only the last point was measured. Actually, this is not case. We can > see some programs with several G memory usage. Is that means at the end of > the program it still hold several G memory? I am still confused. > > Regards, > Qi > > > On Fri, Jul 12, 2013 at 1:51 PM, Shahab Yunus <[email protected]> wrote: > As Vinod Kumar Vavilapalli they are indeed snapshots in point and time. So > they are neither the peak usage from the whole duration of the job, nor > cumulative aggregate that increases over time. > > Regards, > Shahab > > > On Fri, Jul 12, 2013 at 4:47 PM, hadoop qi <[email protected]> wrote: > Thanks for the response. So they represent the total physical memory (virtual > memory) has been allocated to the job (e.g., from heap and stack) during its > entire life time? I am still confused how to get the cumulative number from > /proc/meminfo. I think from /proc/meminfo we can only get the memory usage of > a process in a particular time point (looked like a snapshot of the status > of the process). If these numbers are added, the sum would be much more than > memory allocated to the program. > > > On Fri, Jul 12, 2013 at 6:47 AM, Shahab Yunus <[email protected]> wrote: > I think they are cumulative but per task. > > Physical memory bytes > (PHYSICAL_MEMORY_BYTES) > The physical memory being used by a task in bytes, as reported by > /proc/meminfo. > Virtual memory bytes > (VIRTUAL_MEMORY_BYTES) > The virtual memory being used by a task in bytes, as reported by > /proc/meminfo. > > This is from the Definitive Guide book. Page 260. > > Regards, > Shhab > > > On Thu, Jul 11, 2013 at 12:47 PM, hadoop qi <[email protected]> wrote: > Hello, > > I am wondering how memory counters 'PHYSICAL_MEMORY_BYTES' and > 'VIRTUAL_MEMORY_BYTES' are calculated? They are peaks of memory usage or > cumulative usage? > > Thanks for help, > > > >
