Re: "ps -o %mem" and free memory in Linux
Victor Sudakov writes: > I summed up with awk the values of %mem, which are supposed to be "ratio > of the process's resident set size to the physical memory", correct? > > In my understanding, the value of %mem should indicate how much physical > memory is spent on the "individual" part of the process, otherwise the > parameter is either useless or misdocumented. No, in resident memory you can find memory private to a process as well as memory shared between processes. And in memory private to a process, you can find resident memory and non-resident memory. According to Linux's Documentation/filesystems/proc.txt, we can find this information in /proc/PID/status: VmData size of private data segments e.g. VmData: 123004 kB This looks like the amount of private memory, but I'm not sure. -- Fabrice BAUZAC-STEHLY PGP 015AE9B25DCB0511D200A75DE5674DEA514C891D
Re: "ps -o %mem" and free memory in Linux
to...@tuxteam.de wrote: > > > Those are kind of virtual things, as far as I understand. If not %mem, then > > what `ps` parameter can show me how many php-fpm workers I can safely start > > before RAM is exhausted? > > This is a seemingly easy question with a surprisingly difficult answer. > > I don't have a good reference ready for you, but I poked a bit the > Intertubes and this [1] at least might give you a rough impression > of what kinds of things are involved. > > Cheers > > [1] > https://stackoverflow.com/questions/131303/how-can-i-measure-the-actual-memory-usage-of-an-application-or-process > Thank you, a very helpful, and about /proc/*/spams too. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: "ps -o %mem" and free memory in Linux
Klaus Singvogel wrote: > Victor Sudakov wrote: > > > Perhaps because the php-fpm workers were forked from the same parent > > > and so a lot of theie 'physical' RAM is actually the same RAM as each > > > other, because it's not been modified? > > > > I see your point, but ps(1) talks about real physical RAM: > > > > %mem%MEM ratio of the process's resident set size to the > > physical memory on the machine, expressed as a percentage. (alias pmem). > > > > If those php-fpm workers share a lot of virtual (?) memory between one > > another, shouldn't `ps` show it as such? > > You sum up this: > > < php-fpm individual 1>< php-fpm shared > > < php-fpm individual 2>< php-fpm shared > > ... > < php-fpm individual n>< php-fpm shared > > > You summed up with awk: indivual[1..n] + n * shared I summed up with awk the values of %mem, which are supposed to be "ratio of the process's resident set size to the physical memory", correct? In my understanding, the value of %mem should indicate how much physical memory is spent on the "individual" part of the process, otherwise the parameter is either useless or misdocumented. > > But the real memory sum is: indivual[1..n] + 1 * shared > > Do you see the difference? I see your point but still don't understand how that comes from the definition of "%mem" in the man page. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ signature.asc Description: PGP signature
Re: "ps -o %mem" and free memory in Linux
Victor Sudakov wrote: > > Perhaps because the php-fpm workers were forked from the same parent > > and so a lot of theie 'physical' RAM is actually the same RAM as each > > other, because it's not been modified? > > I see your point, but ps(1) talks about real physical RAM: > > %mem%MEM ratio of the process's resident set size to the > physical memory on the machine, expressed as a percentage. (alias pmem). > > If those php-fpm workers share a lot of virtual (?) memory between one > another, shouldn't `ps` show it as such? You sum up this: < php-fpm individual 1>< php-fpm shared > < php-fpm individual 2>< php-fpm shared > ... < php-fpm individual n>< php-fpm shared > You summed up with awk: indivual[1..n] + n * shared But the real memory sum is: indivual[1..n] + 1 * shared Do you see the difference? Regards, Klaus. -- Klaus Singvogel GnuPG-Key-ID: 1024R/5068792D 1994-06-27
Re: "ps -o %mem" and free memory in Linux
On Tue, Sep 29, 2020 at 10:19:17PM +0700, Victor Sudakov wrote: [...] > Those are kind of virtual things, as far as I understand. If not %mem, then > what `ps` parameter can show me how many php-fpm workers I can safely start > before RAM is exhausted? This is a seemingly easy question with a surprisingly difficult answer. I don't have a good reference ready for you, but I poked a bit the Intertubes and this [1] at least might give you a rough impression of what kinds of things are involved. Cheers [1] https://stackoverflow.com/questions/131303/how-can-i-measure-the-actual-memory-usage-of-an-application-or-process - t signature.asc Description: Digital signature
Re: "ps -o %mem" and free memory in Linux
to...@tuxteam.de wrote:
> On Tue, Sep 29, 2020 at 10:24:35AM +0700, Victor Sudakov wrote:
> > Dear Colleagues,
> >
> > Could you please clarify for me how the following is possible. `ps` shows
> > that the php-fpm workers have occupied 62% of physical memory, while
> > `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> >
> > $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
>
> Ah, but you're adding the resident set sizes of many processes here,
> right?
Indeed, as written in the man page:
%mem%MEM ratio of the process's resident set size to the physical
memory on the machine, expressed as a percentage. (alias pmem).
And it is even not mine, but Zabbix' idea, but that's another story.
>
> Remember that they do share quite a bit of that set: libraries, binaries,
> and so on: you are counting that shared stuff more than once.
Those are kind of virtual things, as far as I understand. If not %mem, then
what `ps` parameter can show me how many php-fpm workers I can safely start
before RAM is exhausted?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: "ps -o %mem" and free memory in Linux
Tixy wrote:
> >
> > Could you please clarify for me how the following is possible. `ps` shows
> > that the php-fpm workers have occupied 62% of physical memory, while
> > `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
> >
> > $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
> > 62.1
> > $ free -h
> > totalusedfree shared buff/cache
> > available
> > Mem: 7.5Gi 1.3Gi 4.4Gi 113Mi 1.8Gi
> > 5.8Gi
> > Swap:0B 0B 0B
> > $
>
> Perhaps because the php-fpm workers were forked from the same parent
> and so a lot of theie 'physical' RAM is actually the same RAM as each
> other, because it's not been modified?
I see your point, but ps(1) talks about real physical RAM:
%mem%MEM ratio of the process's resident set size to the physical
memory on the machine, expressed as a percentage. (alias pmem).
If those php-fpm workers share a lot of virtual (?) memory between one
another, shouldn't `ps` show it as such?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
Re: "ps -o %mem" and free memory in Linux
On Tue, Sep 29, 2020 at 10:24:35AM +0700, Victor Sudakov wrote:
> Dear Colleagues,
>
> Could you please clarify for me how the following is possible. `ps` shows
> that the php-fpm workers have occupied 62% of physical memory, while
> `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
>
> $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
Ah, but you're adding the resident set sizes of many processes here,
right?
Remember that they do share quite a bit of that set: libraries, binaries,
and so on: you are counting that shared stuff more than once.
Cheers
- t
signature.asc
Description: Digital signature
Re: "ps -o %mem" and free memory in Linux
On Tue, 2020-09-29 at 10:24 +0700, Victor Sudakov wrote:
> Dear Colleagues,
>
> Could you please clarify for me how the following is possible. `ps` shows
> that the php-fpm workers have occupied 62% of physical memory, while
> `free` shows that only 1.3Gi (which is 17% of total RAM) is used:
>
> $ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
> 62.1
> $ free -h
> totalusedfree shared buff/cache available
> Mem: 7.5Gi 1.3Gi 4.4Gi 113Mi 1.8Gi
> 5.8Gi
> Swap:0B 0B 0B
> $
Perhaps because the php-fpm workers were forked from the same parent
and so a lot of theie 'physical' RAM is actually the same RAM as each
other, because it's not been modified?
--
Tixy
"ps -o %mem" and free memory in Linux
Dear Colleagues,
Could you please clarify for me how the following is possible. `ps` shows
that the php-fpm workers have occupied 62% of physical memory, while
`free` shows that only 1.3Gi (which is 17% of total RAM) is used:
$ ps axww -o cmd,%mem |awk '/php-fpm/{sum+=$NF}END{print sum}'
62.1
$ free -h
totalusedfree shared buff/cache available
Mem: 7.5Gi 1.3Gi 4.4Gi 113Mi 1.8Gi
5.8Gi
Swap:0B 0B 0B
$
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
signature.asc
Description: PGP signature
