Andi Kleen wrote:
OK, so what is the alternative? Well, if we had a va_start and
va_end (or a va_start and length) we could move the shared object
once using a call of the form
migrate_pages(pid, va_start, va_end, count, old_node_list,
new_node_list);
with old_node_list = 0 1 2 ... 31
On Tue, Feb 22, 2005 at 12:45:21PM -0600, Ray Bryant wrote:
> Andi Kleen wrote:
>
> >
> >How about you add the va_start, va_end but only accept them
> >when pid is 0 (= current process). Otherwise enforce with EINVAL
> >that they are both 0. This way you could map the
> >shared object into the
Andi Kleen wrote:
How about you add the va_start, va_end but only accept them
when pid is 0 (= current process). Otherwise enforce with EINVAL
that they are both 0. This way you could map the
shared object into the batch manager, migrate it there, then
mark it somehow to not be migrated further,
On Mon, Feb 21, 2005 at 11:12:14AM -0600, Ray Bryant wrote:
> Andi Kleen wrote:
>
>
> >
> >I wouldn't bother fixing up VMA policies.
> >
> >
>
> How would these policies get changed so that they represent the
> reality of the new node location(s) then? Doesn't this have to
> happen as part of
> OK, so what is the alternative? Well, if we had a va_start and
> va_end (or a va_start and length) we could move the shared object
> once using a call of the form
>
>migrate_pages(pid, va_start, va_end, count, old_node_list,
> new_node_list);
>
> with old_node_list = 0 1 2 ... 31
>
OK, so what is the alternative? Well, if we had a va_start and
va_end (or a va_start and length) we could move the shared object
once using a call of the form
migrate_pages(pid, va_start, va_end, count, old_node_list,
new_node_list);
with old_node_list = 0 1 2 ... 31
On Mon, Feb 21, 2005 at 11:12:14AM -0600, Ray Bryant wrote:
Andi Kleen wrote:
I wouldn't bother fixing up VMA policies.
How would these policies get changed so that they represent the
reality of the new node location(s) then? Doesn't this have to
happen as part of
Andi Kleen wrote:
How about you add the va_start, va_end but only accept them
when pid is 0 (= current process). Otherwise enforce with EINVAL
that they are both 0. This way you could map the
shared object into the batch manager, migrate it there, then
mark it somehow to not be migrated further,
On Tue, Feb 22, 2005 at 12:45:21PM -0600, Ray Bryant wrote:
Andi Kleen wrote:
How about you add the va_start, va_end but only accept them
when pid is 0 (= current process). Otherwise enforce with EINVAL
that they are both 0. This way you could map the
shared object into the batch
Andi Kleen wrote:
OK, so what is the alternative? Well, if we had a va_start and
va_end (or a va_start and length) we could move the shared object
once using a call of the form
migrate_pages(pid, va_start, va_end, count, old_node_list,
new_node_list);
with old_node_list = 0 1 2 ... 31
Andi,
Oops. It's late. The pargraph below in my previous note confused
cpus and nodes. It should have read as follows:
Let's suppose that nodes 0-1 of a 64 node [was: CPU] system have graphics
pipes. To keep it simple, we will assume that there are 2 cpus
per node like an Altix [128 CPUS in
Andi,
I went back and did some digging on one the issues that has dropped
off the list here: the case where the set of old nodes and new
nodes overlap in some way. No one could provide me with a specific
example, but the thread was that "This did happen in certain scenarios".
Part of these
Andi Kleen wrote:
I wouldn't bother fixing up VMA policies.
How would these policies get changed so that they represent the
reality of the new node location(s) then? Doesn't this have to
happen as part of migrate_pages()?
--
Best Regards,
Ray
---
On Mon, Feb 21, 2005 at 02:42:16AM -0600, Ray Bryant wrote:
> All,
>
> Just an update on the idea of migrating a process without suspending
> it.
>
> The hard part of the problem here is to make sure that the page_migrate()
> system call sees all of the pages to migrate. If the process that is
Ray wrote:
> As I understood it, we were converging on the following:
> (1) ...
> (2) ...
> (3) ...
> This is different than your reply above, which seems to imply that:
> (A) ...
> (B) ...
Andi reacted to various details of (A) and (B).
Any chance, Andi, of you directly stating
On Mon, Feb 21, 2005 at 01:29:41AM -0600, Ray Bryant wrote:
> This is different than your reply above, which seems to imply that:
>
> (A) Step 1 is to migrate mapped files using mbind(). I don't understand
> how to do this in general, because:
> (a) I don't know how to make a
All,
Just an update on the idea of migrating a process without suspending
it.
The hard part of the problem here is to make sure that the page_migrate()
system call sees all of the pages to migrate. If the process that is
being migrated can still allocate pages, then the page_migrate() call
may
All,
Just an update on the idea of migrating a process without suspending
it.
The hard part of the problem here is to make sure that the page_migrate()
system call sees all of the pages to migrate. If the process that is
being migrated can still allocate pages, then the page_migrate() call
may
On Mon, Feb 21, 2005 at 01:29:41AM -0600, Ray Bryant wrote:
This is different than your reply above, which seems to imply that:
(A) Step 1 is to migrate mapped files using mbind(). I don't understand
how to do this in general, because:
(a) I don't know how to make a non-racy
Ray wrote:
As I understood it, we were converging on the following:
(1) ...
(2) ...
(3) ...
This is different than your reply above, which seems to imply that:
(A) ...
(B) ...
Andi reacted to various details of (A) and (B).
Any chance, Andi, of you directly stating whether you
On Mon, Feb 21, 2005 at 02:42:16AM -0600, Ray Bryant wrote:
All,
Just an update on the idea of migrating a process without suspending
it.
The hard part of the problem here is to make sure that the page_migrate()
system call sees all of the pages to migrate. If the process that is
being
Andi Kleen wrote:
I wouldn't bother fixing up VMA policies.
How would these policies get changed so that they represent the
reality of the new node location(s) then? Doesn't this have to
happen as part of migrate_pages()?
--
Best Regards,
Ray
---
Andi,
I went back and did some digging on one the issues that has dropped
off the list here: the case where the set of old nodes and new
nodes overlap in some way. No one could provide me with a specific
example, but the thread was that This did happen in certain scenarios.
Part of these
Andi,
Oops. It's late. The pargraph below in my previous note confused
cpus and nodes. It should have read as follows:
Let's suppose that nodes 0-1 of a 64 node [was: CPU] system have graphics
pipes. To keep it simple, we will assume that there are 2 cpus
per node like an Altix [128 CPUS in
Paul Jackson wrote:
You have to walk to full node mapping for each array, but
even with hundreds of nodes that should not be that costly
I presume if you knew that the job only had pages on certain nodes,
perhaps due to aggressive use of cpusets, that you would only have to
walk those nodes,
Andi Kleen wrote:
Do you have any better way to suggest, Andi, for a batch manager to
relocate a job? The typical scenario, as Ray explained it to me, is
- Give the shared libraries and any other files a suitable policy
(by mapping them and applying mbind)
- Then execute migrate_pages() for
Andi Kleen wrote:
But we are least at the level of agreeing that the new system
call looks something like the following:
migrate_pages(pid, count, old_list, new_list);
right?
For the external case probably yes. For internal (process does this
on its own address space) it should be hooked into
> - Give the shared libraries and any other files a suitable policy
> (by mapping them and applying mbind)
Ah - I think you've said this before, and I'm being a bit retarded.
You're saying that one could horse around with the physical placement of
existing files mapped into another tasks space
> Do you have any better way to suggest, Andi, for a batch manager to
> relocate a job? The typical scenario, as Ray explained it to me, is
- Give the shared libraries and any other files a suitable policy
(by mapping them and applying mbind)
- Then execute migrate_pages() for the anonymous
Andi wrote:
> I still think it's fundamentally unclean and racy. External processes
> shouldn't mess with virtual addresses of other processes.
It's not really messing with (changing) the virtual addresses of
another process. It's messing with the physical placement. It's
using the virtual
> >Perhaps node masks would be better and teaching the kernel to handle
> >relative distances inside the masks transparently while migrating?
> >Not sure how complicated this would be to implement though.
> >
> >Supporting interleaving on the new nodes may be also useful, that would
> >need a
Perhaps node masks would be better and teaching the kernel to handle
relative distances inside the masks transparently while migrating?
Not sure how complicated this would be to implement though.
Supporting interleaving on the new nodes may be also useful, that would
need a policy argument
Andi wrote:
I still think it's fundamentally unclean and racy. External processes
shouldn't mess with virtual addresses of other processes.
It's not really messing with (changing) the virtual addresses of
another process. It's messing with the physical placement. It's
using the virtual
Do you have any better way to suggest, Andi, for a batch manager to
relocate a job? The typical scenario, as Ray explained it to me, is
- Give the shared libraries and any other files a suitable policy
(by mapping them and applying mbind)
- Then execute migrate_pages() for the anonymous
- Give the shared libraries and any other files a suitable policy
(by mapping them and applying mbind)
Ah - I think you've said this before, and I'm being a bit retarded.
You're saying that one could horse around with the physical placement of
existing files mapped into another tasks space by
Andi Kleen wrote:
But we are least at the level of agreeing that the new system
call looks something like the following:
migrate_pages(pid, count, old_list, new_list);
right?
For the external case probably yes. For internal (process does this
on its own address space) it should be hooked into
Andi Kleen wrote:
Do you have any better way to suggest, Andi, for a batch manager to
relocate a job? The typical scenario, as Ray explained it to me, is
- Give the shared libraries and any other files a suitable policy
(by mapping them and applying mbind)
- Then execute migrate_pages() for
Paul Jackson wrote:
You have to walk to full node mapping for each array, but
even with hundreds of nodes that should not be that costly
I presume if you knew that the job only had pages on certain nodes,
perhaps due to aggressive use of cpusets, that you would only have to
walk those nodes,
Andi Kleen wrote:
[Enjoy your vacation]
[I am thanks -- or I was -- I go home tomorrow]
I assume they would allow marking arbitary segments with specific
policies, so it should be possible.
An alternative way to handle shared libraries BTW would be to add the ELF
headers Steve did in his patch.
Andi Kleen wrote:
You and Robin mentioned some problems with "double migration"
with that, but it's still not completely clear to me what
problem you're solving here. Perhaps that needs to be reexamined.
There is one other case where Robin and I have talked about double
migration. That is the
Andi, et al:
I see that several messages have been sent in the interim.
I apologize for being "out of sync", but today is my last
day to go skiing and it is gorgeous outside. I'll try
to catch up and digest everthing later.
--
---
Ray Bryant
Here's an interface proposal that may be a middle ground and
should satisfy both small and large system requirements:
The system call interface would be:
page_migrate(pid, va_start, va_end, count, old_node_list, new_node_list);
(e. g. same as before, but please keep reading):
The following
Andi wrote:
> Problem is what happens
> when some memory is in some other node due to memory pressure fallbacks.
> Your scheme would not migrate this memory at all.
The arrays of old and new nodes handle this fine.
Include that 'other node' in the array of old nodes,
and the corresponding new
Andi wrote:
> e.g. job runs threads on nodes 0,1,2,3 and you want it to move
> to nodes 4,5,6,7 with all memory staying staying in the same
> distance from the new CPUs as it were from the old CPUs, right?
>
> It explains why you want old_node, you would do
> (assuming node mask arguments)
Andi wrote:
> I don't like old_node* very much because it's imho unreliable
> (because you can usually never fully know on which nodes the old
> process was and there can be good reasons to just migrate everything)
That's one way that the arrays of old and new nodes pays off.
You can list any old
Andi - what does this line mean:
+ node mask length.
I guess its the names of the parameters in a proposed
migration system call. Length of what, mask of what,
what's the node mean, huh?
--
I won't rest till it's the best ...
Programmer, Linux
[Enjoy your vacation]
On Fri, Feb 18, 2005 at 02:38:42AM -0600, Ray Bryant wrote:
>
> Let's start off with at least one thing we can agree on. If xattrs
> are already part of XFS, then it seems reasonable to use an extended
> attribute to mark certain files as non-migratable. (Some further
>
Andi Kleen wrote:
[Sorry for the late answer.]
No problem, remember, I'm supposed to be on vacation, anyway. :-)
Let's start off with at least one thing we can agree on. If xattrs
are already part of XFS, then it seems reasonable to use an extended
attribute to mark certain files as
Andi Kleen wrote:
[Sorry for the late answer.]
No problem, remember, I'm supposed to be on vacation, anyway. :-)
Let's start off with at least one thing we can agree on. If xattrs
are already part of XFS, then it seems reasonable to use an extended
attribute to mark certain files as
[Enjoy your vacation]
On Fri, Feb 18, 2005 at 02:38:42AM -0600, Ray Bryant wrote:
Let's start off with at least one thing we can agree on. If xattrs
are already part of XFS, then it seems reasonable to use an extended
attribute to mark certain files as non-migratable. (Some further
Andi - what does this line mean:
+ node mask length.
I guess its the names of the parameters in a proposed
migration system call. Length of what, mask of what,
what's the node mean, huh?
--
I won't rest till it's the best ...
Programmer, Linux
Andi wrote:
I don't like old_node* very much because it's imho unreliable
(because you can usually never fully know on which nodes the old
process was and there can be good reasons to just migrate everything)
That's one way that the arrays of old and new nodes pays off.
You can list any old
Andi wrote:
e.g. job runs threads on nodes 0,1,2,3 and you want it to move
to nodes 4,5,6,7 with all memory staying staying in the same
distance from the new CPUs as it were from the old CPUs, right?
It explains why you want old_node, you would do
(assuming node mask arguments)
Yup -
Andi wrote:
Problem is what happens
when some memory is in some other node due to memory pressure fallbacks.
Your scheme would not migrate this memory at all.
The arrays of old and new nodes handle this fine.
Include that 'other node' in the array of old nodes,
and the corresponding new node,
Here's an interface proposal that may be a middle ground and
should satisfy both small and large system requirements:
The system call interface would be:
page_migrate(pid, va_start, va_end, count, old_node_list, new_node_list);
(e. g. same as before, but please keep reading):
The following
Andi Kleen wrote:
You and Robin mentioned some problems with double migration
with that, but it's still not completely clear to me what
problem you're solving here. Perhaps that needs to be reexamined.
There is one other case where Robin and I have talked about double
migration. That is the case
Andi, et al:
I see that several messages have been sent in the interim.
I apologize for being out of sync, but today is my last
day to go skiing and it is gorgeous outside. I'll try
to catch up and digest everthing later.
--
---
Ray Bryant
512-453-9679
Andi Kleen wrote:
[Enjoy your vacation]
[I am thanks -- or I was -- I go home tomorrow]
I assume they would allow marking arbitary segments with specific
policies, so it should be possible.
An alternative way to handle shared libraries BTW would be to add the ELF
headers Steve did in his patch.
[Sorry for the late answer.]
On Tue, Feb 15, 2005 at 09:44:41PM -0600, Ray Bryant wrote:
> >
> >
> >Sorry, but the only real difference between your API and mbind is that
> >yours has a pid argument.
> >
>
> That may be true, but the internals of the implementations have got
> to be pretty
[Sorry for the late answer.]
On Tue, Feb 15, 2005 at 09:44:41PM -0600, Ray Bryant wrote:
Sorry, but the only real difference between your API and mbind is that
yours has a pid argument.
That may be true, but the internals of the implementations have got
to be pretty different as near
Andi Kleen wrote:
Making memory migration a subset of page migration is not a general
solution. It only works for programs that are using memory policy
to control placement. As I've tried to point out multiple times
before, most programs that I am aware of use placement based on
first-touch.
Thanks Andi for your effort to present your case more completely.
I agree that there is some 'talking by each other' going on.
Dave Hansen has publically (and Ray privately) sought to
move this discussion to linux-mm (or more specifically,
off lkml for now).
Any chance, Andi, that you could
> Making memory migration a subset of page migration is not a general
> solution. It only works for programs that are using memory policy
> to control placement. As I've tried to point out multiple times
> before, most programs that I am aware of use placement based on
> first-touch. When we
Andi Kleen wrote:
[Sorry, didn't answer to everything in your mail the first time.
See previous mail for beginning]
On Mon, Feb 14, 2005 at 06:29:45PM -0600, Ray Bryant wrote:
migrating, and figure out from that what portions of which pid's
address spaces need to migrated so that we satisfy the
[Sorry, didn't answer to everything in your mail the first time.
See previous mail for beginning]
On Mon, Feb 14, 2005 at 06:29:45PM -0600, Ray Bryant wrote:
> migrating, and figure out from that what portions of which pid's
> address spaces need to migrated so that we satisfy the constraints
>
[Sorry, didn't answer to everything in your mail the first time.
See previous mail for beginning]
On Mon, Feb 14, 2005 at 06:29:45PM -0600, Ray Bryant wrote:
migrating, and figure out from that what portions of which pid's
address spaces need to migrated so that we satisfy the constraints
Andi Kleen wrote:
[Sorry, didn't answer to everything in your mail the first time.
See previous mail for beginning]
On Mon, Feb 14, 2005 at 06:29:45PM -0600, Ray Bryant wrote:
migrating, and figure out from that what portions of which pid's
address spaces need to migrated so that we satisfy the
Making memory migration a subset of page migration is not a general
solution. It only works for programs that are using memory policy
to control placement. As I've tried to point out multiple times
before, most programs that I am aware of use placement based on
first-touch. When we
Thanks Andi for your effort to present your case more completely.
I agree that there is some 'talking by each other' going on.
Dave Hansen has publically (and Ray privately) sought to
move this discussion to linux-mm (or more specifically,
off lkml for now).
Any chance, Andi, that you could
Andi Kleen wrote:
Making memory migration a subset of page migration is not a general
solution. It only works for programs that are using memory policy
to control placement. As I've tried to point out multiple times
before, most programs that I am aware of use placement based on
first-touch.
70 matches
Mail list logo
