> Did those jobs share nodes -- sometimes two or more jobs using the same
> nodes? I am sure SGI has such users too, though such job mixes make
> the runtimes of specific jobs less obvious, so customers are more
> tolerant of variations and some inefficiencies, as they get hidden in
> the mix.
Hm
Kosaki-san wrote:
> Yes.
> Fujitsu HPC middleware watching sum of memory consumption of the job
> and, if over-consumption happened, kill process and remove job schedule.
Did those jobs share nodes -- sometimes two or more jobs using the same
nodes? I am sure SGI has such users too, though such j
Rik wrote:
> In that case the user is better off having that job killed and
> restarted elsewhere, than having all of the jobs on that node
> crawl to a halt due to swapping.
>
> Paul, is this guess correct? :)
Not for the loads I focus on. Each job gets exclusive use of its own
dedicated set of
Hi Rik
> > Sounds like a job for memory limits (ulimit?), not for OOM
> > notification, right?
>
> I suspect one problem could be that an HPC job scheduling program
> does not know exactly how much memory each job can take, so it can
> sometimes end up making a mistake and overcommitting the memo
On Tue, 19 Feb 2008 23:28:28 +0100
Pavel Machek <[EMAIL PROTECTED]> wrote:
> Sounds like a job for memory limits (ulimit?), not for OOM
> notification, right?
I suspect one problem could be that an HPC job scheduling program
does not know exactly how much memory each job can take, so it can
somet
Pavel, responding to pj:
> > There is not much my customers HPC jobs can do with notification before
> > swap. Their jobs either have the main memory they need to perform the
> > requested calculations with the desired performance, or their job is
> > useless and should be killed. Unlike the appl
On Tue 2008-02-19 09:00:08, Paul Jackson wrote:
> Kosaki-san wrote:
> > Thank you for wonderful interestings comment.
>
> You're most welcome. The pleasure is all mine.
>
> > you think kill the process just after swap, right?
> > but unfortunately, almost user hope receive notification before sw
pj, talking to himself:
> Of course
> for embedded use, I'd have to adapt it to a non-cpuset based mechanism
> (not difficult), as embedded definitely doesn't do cpusets.
I'm forgetting an important detail here. Kosaki-san has clearly stated
that this hook, at vmscan's writepage, is too late for
Rik wrote:
> Basically in all situations, the kernel needs to warn at the same point
> in time: when the system is about to run out of RAM for anonymous pages.
>
> ...
>
> In the HPC case, it leads to swapping (and a management program can kill or
> restart something else).
Thanks for stopping by
On Tue, 19 Feb 2008 09:00:08 -0600
Paul Jackson <[EMAIL PROTECTED]> wrote:
> Depending on what we're trying to do:
> 1) warn applications of swap coming soon (your case),
> 2) show how close we are to swapping,
> 3) show how much swap has happened already,
> 4) kill instantly if try to swap (m
Kosaki-san wrote:
> Thank you for wonderful interestings comment.
You're most welcome. The pleasure is all mine.
> you think kill the process just after swap, right?
> but unfortunately, almost user hope receive notification before swap ;-)
> because avoid swap.
There is not much my customers H
Hi Paul,
Thank you for wonderful interestings comment.
your comment is really nice.
I was HPC guy with large NUMA box at past.
I promise i don't ignroe hpc user.
but unfortunately I didn't have experience of use CPUSET
because at that point, it was under development yet.
I hope discuss you that
I just noticed this patchset, kosaki-san. It looks quite interesting;
my apologies for not commenting earlier.
I see mention somewhere that mem_notify is of particular interest to
embedded systems.
I have what seems, intuitively, a similar problem at the opposite
end of the world, on big-honkin
> > the Linux Today article is very nice description. (great works by Jake Edge)
> > http://www.linuxworld.com/news/2008/020508-kernel.html
>
> Just for future reference...the above-mentioned article is from LWN,
> syndicated onto LinuxWorld. It has, so far as I know, never been near
> Linux Today
> the Linux Today article is very nice description. (great works by Jake Edge)
> http://www.linuxworld.com/news/2008/020508-kernel.html
Just for future reference...the above-mentioned article is from LWN,
syndicated onto LinuxWorld. It has, so far as I know, never been near
Linux Today.
Glad you
Hi Rik
> More importantly, all gtk+ programs, as well as most databases and other
> system daemons have a poll() loop as their main loop.
not only gtk+, may be all modern GUI program :)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL P
On Sun, 10 Feb 2008 01:33:49 +0900
"KOSAKI Motohiro" <[EMAIL PROTECTED]> wrote:
> > Where is the netlink interface? Polling an FD is so last century :)
>
> to be honest, I don't know anyone use netlink and why hope receive
> low memory notify by netlink.
>
> poll() is old way, but it works good
Hi
> Interesting patch series (I am being yuppie and reading this thread
> from my iPhone on a treadmill at the gym - so further comments later).
> I think that this is broadly along the lines that I was thinking, but
> this should be an RFC only patch series for now.
sorry, I fixed at next post.
Yo,
Interesting patch series (I am being yuppie and reading this thread
from my iPhone on a treadmill at the gym - so further comments later).
I think that this is broadly along the lines that I was thinking, but
this should be an RFC only patch series for now.
Some initial questions:
Wh
Hi
The /dev/mem_notify is low memory notification device.
it can avoid swappness and oom by cooperationg with the user process.
the Linux Today article is very nice description. (great works by Jake Edge)
http://www.linuxworld.com/news/2008/020508-kernel.html
When memory gets tight, it is quite
20 matches
Mail list logo
