Re: Concerning a post that you made about expandable anonymous shared mappings
Stas Sergeev wrote: Hi. William Tambe wrote: I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. Could you please provide a detailed list of the problems you have with shm_open? If they are valid, then I can bet the patch will be applied, no matter what. :) In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. In this case you need to use shm_unlink() right after shm_open(). Then this shm will be accessable only to your process and its children, via an fd, and not to anyone else. And you still can do anything with it (ftruncate/mmap/mremap whatever). Ok, now I find myself without any other arguments :-) shm_unlink() right after shm_open() is a solution. And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. In what way, exactly? I wrote the above not knowing that I could use shm_unlink() right after shm_open(). But still, I have lost a considerable amount of time trying to figure that out. It appeared all natural to me that I could just remap ANONYMOUS and get what I wanted. And the worst thing here is that the man pages do not let you know about that. Sincerely, William Tambe - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Stas Sergeev wrote: Hi. William Tambe wrote: I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. Could you please provide a detailed list of the problems you have with shm_open? If they are valid, then I can bet the patch will be applied, no matter what. :) In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. In this case you need to use shm_unlink() right after shm_open(). Then this shm will be accessable only to your process and its children, via an fd, and not to anyone else. And you still can do anything with it (ftruncate/mmap/mremap whatever). Ok, now I find myself without any other arguments :-) shm_unlink() right after shm_open() is a solution. And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. In what way, exactly? I wrote the above not knowing that I could use shm_unlink() right after shm_open(). But still, I have lost a considerable amount of time trying to figure that out. It appeared all natural to me that I could just remap ANONYMOUS and get what I wanted. And the worst thing here is that the man pages do not let you know about that. Sincerely, William Tambe - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi. William Tambe wrote: I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. Could you please provide a detailed list of the problems you have with shm_open? If they are valid, then I can bet the patch will be applied, no matter what. :) In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. In this case you need to use shm_unlink() right after shm_open(). Then this shm will be accessable only to your process and its children, via an fd, and not to anyone else. And you still can do anything with it (ftruncate/mmap/mremap whatever). And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. In what way, exactly? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hugh Dickins wrote: On Mon, 9 Jul 2007, William Tambe wrote: Hugh Dickins wrote: I've come right around to your original view, Stas, and William's: if that mmap creates such an object, then the expanding mremap really ought to be useful, and allow the underlying object to be expanded. The shared anonymous object is already anomalous: expanding it on fault makes it more consistent with its own nature, not less. ... Here's a patch against 2.6.22-rc7: would you, Stas, put your Signed-off-by into this, and accept authorship - although I'm sending this back to you, it's very much your idea, and only trivially modified from your three-year-old patch by me. If you're agreeable, I can then forward it or its shmem_zero_fault equivalent to Andrew when we see which way 2.6.23 is going. ... Will this patch be added to stable versions of the linux kernel? Please let me know. I confess that the lukewarm response from Stas cooled my enthusiasm, and left me feeling that perhaps I'm an idiot to be adding such a feature so many years too late; and my old caution about the way a child could use up memory not freed on child's exit, unknown to parent, returned to haunt me. That could be documented for new usages, but I just don't know what usages are already out there, and fear I'd be introducing an exploit. It most certainly will not be added to a stable version of the linux kernel, if by that you mean 2.6.22.N or 2.6.21.N etc. Though it can be viewed as a bugfix, the patch as it stands seems in danger of introducing its own bug, and it's just too much of a feature to be suitable for a -stable release. But more probably you meant, will it be in 2.6.23 or 2.6.24? Sorry to be such a vacillatiing wimp, but I don't know. How well are you managing with the shm_open approach? Hugh I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. shm_open is great, but it doesn't quite fit my needs. And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. Can that feature be added at least to release candidates series so that everybody can try it and see how well it does? Sincerely, William Tambe - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
On Mon, 9 Jul 2007, William Tambe wrote: > Hugh Dickins wrote: > > > > I've come right around to your original view, Stas, and William's: > > if that mmap creates such an object, then the expanding mremap really > > ought to be useful, and allow the underlying object to be expanded. > > The shared anonymous object is already anomalous: expanding it on > > fault makes it more consistent with its own nature, not less. > > ... > > Here's a patch against 2.6.22-rc7: would you, Stas, put your > > Signed-off-by into this, and accept authorship - although I'm > > sending this back to you, it's very much your idea, and only > > trivially modified from your three-year-old patch by me. If > > you're agreeable, I can then forward it or its shmem_zero_fault > > equivalent to Andrew when we see which way 2.6.23 is going. >... > Will this patch be added to stable versions of the linux kernel? > Please let me know. I confess that the lukewarm response from Stas cooled my enthusiasm, and left me feeling that perhaps I'm an idiot to be adding such a feature so many years too late; and my old caution about the way a child could use up memory not freed on child's exit, unknown to parent, returned to haunt me. That could be documented for new usages, but I just don't know what usages are already out there, and fear I'd be introducing an exploit. It most certainly will not be added to a stable version of the linux kernel, if by that you mean 2.6.22.N or 2.6.21.N etc. Though it can be viewed as a bugfix, the patch as it stands seems in danger of introducing its own bug, and it's just too much of a feature to be suitable for a -stable release. But more probably you meant, will it be in 2.6.23 or 2.6.24? Sorry to be such a vacillatiing wimp, but I don't know. How well are you managing with the shm_open approach? Hugh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
On Mon, 9 Jul 2007, William Tambe wrote: Hugh Dickins wrote: I've come right around to your original view, Stas, and William's: if that mmap creates such an object, then the expanding mremap really ought to be useful, and allow the underlying object to be expanded. The shared anonymous object is already anomalous: expanding it on fault makes it more consistent with its own nature, not less. ... Here's a patch against 2.6.22-rc7: would you, Stas, put your Signed-off-by into this, and accept authorship - although I'm sending this back to you, it's very much your idea, and only trivially modified from your three-year-old patch by me. If you're agreeable, I can then forward it or its shmem_zero_fault equivalent to Andrew when we see which way 2.6.23 is going. ... Will this patch be added to stable versions of the linux kernel? Please let me know. I confess that the lukewarm response from Stas cooled my enthusiasm, and left me feeling that perhaps I'm an idiot to be adding such a feature so many years too late; and my old caution about the way a child could use up memory not freed on child's exit, unknown to parent, returned to haunt me. That could be documented for new usages, but I just don't know what usages are already out there, and fear I'd be introducing an exploit. It most certainly will not be added to a stable version of the linux kernel, if by that you mean 2.6.22.N or 2.6.21.N etc. Though it can be viewed as a bugfix, the patch as it stands seems in danger of introducing its own bug, and it's just too much of a feature to be suitable for a -stable release. But more probably you meant, will it be in 2.6.23 or 2.6.24? Sorry to be such a vacillatiing wimp, but I don't know. How well are you managing with the shm_open approach? Hugh - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hugh Dickins wrote: On Mon, 9 Jul 2007, William Tambe wrote: Hugh Dickins wrote: I've come right around to your original view, Stas, and William's: if that mmap creates such an object, then the expanding mremap really ought to be useful, and allow the underlying object to be expanded. The shared anonymous object is already anomalous: expanding it on fault makes it more consistent with its own nature, not less. ... Here's a patch against 2.6.22-rc7: would you, Stas, put your Signed-off-by into this, and accept authorship - although I'm sending this back to you, it's very much your idea, and only trivially modified from your three-year-old patch by me. If you're agreeable, I can then forward it or its shmem_zero_fault equivalent to Andrew when we see which way 2.6.23 is going. ... Will this patch be added to stable versions of the linux kernel? Please let me know. I confess that the lukewarm response from Stas cooled my enthusiasm, and left me feeling that perhaps I'm an idiot to be adding such a feature so many years too late; and my old caution about the way a child could use up memory not freed on child's exit, unknown to parent, returned to haunt me. That could be documented for new usages, but I just don't know what usages are already out there, and fear I'd be introducing an exploit. It most certainly will not be added to a stable version of the linux kernel, if by that you mean 2.6.22.N or 2.6.21.N etc. Though it can be viewed as a bugfix, the patch as it stands seems in danger of introducing its own bug, and it's just too much of a feature to be suitable for a -stable release. But more probably you meant, will it be in 2.6.23 or 2.6.24? Sorry to be such a vacillatiing wimp, but I don't know. How well are you managing with the shm_open approach? Hugh I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. shm_open is great, but it doesn't quite fit my needs. And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. Can that feature be added at least to release candidates series so that everybody can try it and see how well it does? Sincerely, William Tambe - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi. William Tambe wrote: I understand your concern. But since I am working on a dynamic memory management code that I wish to use with other projects that I have, I didn't find appropriate to use shm_open. Could you please provide a detailed list of the problems you have with shm_open? If they are valid, then I can bet the patch will be applied, no matter what. :) In fact there is a name associated with the shared memory requested with shm_open, so that it can be mmap(ed) in another process. And I do not wish to have it accessible by any other process, unless I choose to do so. In this case you need to use shm_unlink() right after shm_open(). Then this shm will be accessable only to your process and its children, via an fd, and not to anyone else. And you still can do anything with it (ftruncate/mmap/mremap whatever). And I think remap(ing) ANONYMOUS memory kind of make a lot of things easier. In what way, exactly? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hugh Dickins wrote:
On Mon, 2 Jul 2007, Stas Sergeev wrote:
Hugh Dickins wrote:
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
IIRC your argument, that made sense to me,
was that with such an approach, you can only
expand the backing-store, but never shrink.
I agree this is a problem from some point of
view. I have no idea whether it is important
or not, but it definitely _looks_ not very good.
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I've come right around to your original view, Stas, and William's:
if that mmap creates such an object, then the expanding mremap really
ought to be useful, and allow the underlying object to be expanded.
The shared anonymous object is already anomalous: expanding it on
fault makes it more consistent with its own nature, not less.
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
Well, this generates a bus error, while, for
example, doing the same trick with having a
/dev/mem as a backing-store, simply maps the
"enlarged" space with the anonymous memory,
and so does not generate a SIGBUS (not producing
a desired effect either, of course).
Why do we have it both ways? Shouldn't they
(i.e. /dev/zero and /dev/mem mapping) behave
the same after expanding with mremap?
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
I haven't given it any thought since then:
I was thinking about it a bit, and I imagined
we need something like
int mopen(void *addr, size_t len, unsigned int flags);
which will give you an fd for the memory area,
which you can then ftruncate() and mmap() (and
mremap).
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem. As last time around, you were suggesting .mremap
callouts; but I much prefer your original shmem_zero_nopage,
which is a solution of the scale appropriate to the problem.
Such a thing could solve that mremap() problem
that me and William Tambe have.
If you're thinking of going that way, for this problem it
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap, and would
like to put in a patch for that. This isn't a particularly good
time for such a patch - it's unclear right now whether 2.6.23 will
have shmem_nopage like 2.6.22 or shmem_fault like 2.6.22-rc-mm;
but we can easily adjust to whichever.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
[I'll fill in the patch comment later]
Signed-off-by: Hugh Dickins <[EMAIL PROTECTED]>
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma->vm_file->f_dentry->d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma->vm_start) +
+ ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
+ spin_lock(>lock);
+ i_size = i_size_read(inode);
+ if (i_size < vm_size && vm_size <= SHMEM_MAX_BYTES &&
+ !shmem_acct_size(info->flags, vm_size - i_size)) {
+
Re: Concerning a post that you made about expandable anonymous shared mappings
Hugh Dickins wrote:
On Mon, 2 Jul 2007, Stas Sergeev wrote:
Hugh Dickins wrote:
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
IIRC your argument, that made sense to me,
was that with such an approach, you can only
expand the backing-store, but never shrink.
I agree this is a problem from some point of
view. I have no idea whether it is important
or not, but it definitely _looks_ not very good.
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I've come right around to your original view, Stas, and William's:
if that mmap creates such an object, then the expanding mremap really
ought to be useful, and allow the underlying object to be expanded.
The shared anonymous object is already anomalous: expanding it on
fault makes it more consistent with its own nature, not less.
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
Well, this generates a bus error, while, for
example, doing the same trick with having a
/dev/mem as a backing-store, simply maps the
enlarged space with the anonymous memory,
and so does not generate a SIGBUS (not producing
a desired effect either, of course).
Why do we have it both ways? Shouldn't they
(i.e. /dev/zero and /dev/mem mapping) behave
the same after expanding with mremap?
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
I haven't given it any thought since then:
I was thinking about it a bit, and I imagined
we need something like
int mopen(void *addr, size_t len, unsigned int flags);
which will give you an fd for the memory area,
which you can then ftruncate() and mmap() (and
mremap).
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem. As last time around, you were suggesting .mremap
callouts; but I much prefer your original shmem_zero_nopage,
which is a solution of the scale appropriate to the problem.
Such a thing could solve that mremap() problem
that me and William Tambe have.
If you're thinking of going that way, for this problem it
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap, and would
like to put in a patch for that. This isn't a particularly good
time for such a patch - it's unclear right now whether 2.6.23 will
have shmem_nopage like 2.6.22 or shmem_fault like 2.6.22-rc-mm;
but we can easily adjust to whichever.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
[I'll fill in the patch comment later]
Signed-off-by: Hugh Dickins [EMAIL PROTECTED]
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma-vm_file-f_dentry-d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma-vm_start) +
+ ((loff_t)vma-vm_pgoff PAGE_SHIFT);
+ spin_lock(info-lock);
+ i_size = i_size_read(inode);
+ if (i_size vm_size vm_size = SHMEM_MAX_BYTES
+ !shmem_acct_size(info-flags, vm_size - i_size)) {
+
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi.
Hugh Dickins wrote:
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I simply think that the anonymous shared mapping
is a bit of an ad-hoc interface. The result is that
whenever you extend it in some way, you get a problem
elsewhere. So I just stopped using it.
The shared anonymous object is already anomalous:
Exactly.
expanding it on
fault makes it more consistent with its own nature, not less.
That's certainly true, you were agree with this even in
the past. It is more consistent to have it that way. The
only question is whether the side-effects you outlined,
do outweight the benefit, or not. But since it was you
who outlined the problems, I guess you have the right to
say that they, while being valid, do not outweight the benefit. :)
I am not going to oppose my own patch, of course.
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
OK. And I guess comparing /dev/mem mapping with
/dev/shm/xxx mapping is not valid too, since the
/dev/mem perhaps doesn't permit ftruncate().
So do you say, silently mapping the private anonymous
pages for /dev/mem is a correct behaveour, even though
the /proc/pid/maps shows that the mapping is shared?
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem.
Well, I simply thought that something like this is
needed anyway, not only for that problem. But then
vmsplice() appeared, so of course such an mopen() is
no longer much relevant. I am just puzzled as to why
the pipe was used for vmsplice() and can't find an
answer, but I am probably missing something obvious...
Such a thing could solve that mremap() problem
that me and William Tambe have.
If you're thinking of going that way, for this problem it
No, of course not, at least, not any more.
Back in 2004 it was a good idea, but now there is no
longer a vacant place for this wonderfull syscall. :)
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
The only scenario I can image where you would prefer
the anonymous mmaps over the Posix SHM, is when you want
to manage many small memory areas separately, and do not
want to track them back to their fd's. But even then this
probably doesn't save more than a few dozens of code lines. :)
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap,
IIRC this view was never under any doubts. The discussion
was only whether or not to count a few problems you pointed.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
OK, no problems. Except that I've already forgotten the
details, and pretty much lost the need for that patch.
So let's assume the authorship is yours. :)
Signed-off-by: Hugh Dickins <[EMAIL PROTECTED]>
Signed-off-by: Stas Sergeev <[EMAIL PROTECTED]>
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma->vm_file->f_dentry->d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma->vm_start) +
+ ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
+ spin_lock(>lock);
+ i_size = i_size_read(inode);
+ if (i_size < vm_size && vm_size <= SHMEM_MAX_BYTES &&
+
Re: Concerning a post that you made about expandable anonymous shared mappings
On Mon, 2 Jul 2007, Stas Sergeev wrote:
> Hugh Dickins wrote:
> > You've answered your own question: we did not make the change Stas
> > suggested, IIRC because I remained a little uneasy with that change
> > in behaviour, and nobody else spoke up for it.
> IIRC your argument, that made sense to me,
> was that with such an approach, you can only
> expand the backing-store, but never shrink.
> I agree this is a problem from some point of
> view. I have no idea whether it is important
> or not, but it definitely _looks_ not very good.
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I've come right around to your original view, Stas, and William's:
if that mmap creates such an object, then the expanding mremap really
ought to be useful, and allow the underlying object to be expanded.
The shared anonymous object is already anomalous: expanding it on
fault makes it more consistent with its own nature, not less.
>
> > > //why does this failed. I am well in the interval [4096, 8192]
> > > *(unsigned int *)(ptr + 4096 + 8)= 10;
> >> }
> Well, this generates a bus error, while, for
> example, doing the same trick with having a
> /dev/mem as a backing-store, simply maps the
> "enlarged" space with the anonymous memory,
> and so does not generate a SIGBUS (not producing
> a desired effect either, of course).
> Why do we have it both ways? Shouldn't they
> (i.e. /dev/zero and /dev/mem mapping) behave
> the same after expanding with mremap?
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
>
> > I haven't given it any thought since then:
> I was thinking about it a bit, and I imagined
> we need something like
>int mopen(void *addr, size_t len, unsigned int flags);
> which will give you an fd for the memory area,
> which you can then ftruncate() and mmap() (and
> mremap).
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem. As last time around, you were suggesting .mremap
callouts; but I much prefer your original shmem_zero_nopage,
which is a solution of the scale appropriate to the problem.
> Such a thing could solve that mremap() problem
> that me and William Tambe have.
If you're thinking of going that way, for this problem it
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap, and would
like to put in a patch for that. This isn't a particularly good
time for such a patch - it's unclear right now whether 2.6.23 will
have shmem_nopage like 2.6.22 or shmem_fault like 2.6.22-rc-mm;
but we can easily adjust to whichever.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
[I'll fill in the patch comment later]
Signed-off-by: Hugh Dickins <[EMAIL PROTECTED]>
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma->vm_file->f_dentry->d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma->vm_start) +
+ ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
+ spin_lock(>lock);
+ i_size = i_size_read(inode);
+ if (i_size < vm_size && vm_size <= SHMEM_MAX_BYTES &&
+
Re: Concerning a post that you made about expandable anonymous shared mappings
On Mon, 2 Jul 2007, Stas Sergeev wrote:
Hugh Dickins wrote:
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
IIRC your argument, that made sense to me,
was that with such an approach, you can only
expand the backing-store, but never shrink.
I agree this is a problem from some point of
view. I have no idea whether it is important
or not, but it definitely _looks_ not very good.
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I've come right around to your original view, Stas, and William's:
if that mmap creates such an object, then the expanding mremap really
ought to be useful, and allow the underlying object to be expanded.
The shared anonymous object is already anomalous: expanding it on
fault makes it more consistent with its own nature, not less.
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
Well, this generates a bus error, while, for
example, doing the same trick with having a
/dev/mem as a backing-store, simply maps the
enlarged space with the anonymous memory,
and so does not generate a SIGBUS (not producing
a desired effect either, of course).
Why do we have it both ways? Shouldn't they
(i.e. /dev/zero and /dev/mem mapping) behave
the same after expanding with mremap?
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
I haven't given it any thought since then:
I was thinking about it a bit, and I imagined
we need something like
int mopen(void *addr, size_t len, unsigned int flags);
which will give you an fd for the memory area,
which you can then ftruncate() and mmap() (and
mremap).
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem. As last time around, you were suggesting .mremap
callouts; but I much prefer your original shmem_zero_nopage,
which is a solution of the scale appropriate to the problem.
Such a thing could solve that mremap() problem
that me and William Tambe have.
If you're thinking of going that way, for this problem it
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap, and would
like to put in a patch for that. This isn't a particularly good
time for such a patch - it's unclear right now whether 2.6.23 will
have shmem_nopage like 2.6.22 or shmem_fault like 2.6.22-rc-mm;
but we can easily adjust to whichever.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
[I'll fill in the patch comment later]
Signed-off-by: Hugh Dickins [EMAIL PROTECTED]
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma-vm_file-f_dentry-d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma-vm_start) +
+ ((loff_t)vma-vm_pgoff PAGE_SHIFT);
+ spin_lock(info-lock);
+ i_size = i_size_read(inode);
+ if (i_size vm_size vm_size = SHMEM_MAX_BYTES
+ !shmem_acct_size(info-flags, vm_size - i_size)) {
+
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi.
Hugh Dickins wrote:
You were gracious enough to accept my arguments back then, but after
mulling this over overnight, I've come to think I was just too timid
back then, and gave too much weight to the issue of there being no
shrink, and to the issue that child might expand the object without
parent knowing (that surplus remaining allocated until both exited).
I simply think that the anonymous shared mapping
is a bit of an ad-hoc interface. The result is that
whenever you extend it in some way, you get a problem
elsewhere. So I just stopped using it.
The shared anonymous object is already anomalous:
Exactly.
expanding it on
fault makes it more consistent with its own nature, not less.
That's certainly true, you were agree with this even in
the past. It is more consistent to have it that way. The
only question is whether the side-effects you outlined,
do outweight the benefit, or not. But since it was you
who outlined the problems, I guess you have the right to
say that they, while being valid, do not outweight the benefit. :)
I am not going to oppose my own patch, of course.
They've very different: mapping /dev/mem maps an existing object;
whereas mapping /dev/zero is a convention by which a new object
is created - and we can't afford the memory to make it of infinite
extent, so have limited it to the mapped size.
OK. And I guess comparing /dev/mem mapping with
/dev/shm/xxx mapping is not valid too, since the
/dev/mem perhaps doesn't permit ftruncate().
So do you say, silently mapping the private anonymous
pages for /dev/mem is a correct behaveour, even though
the /proc/pid/maps shows that the mapping is shared?
Ah, if we added an mopen(), there's no end to the discussions
we could have about what it can do. It may be a great idea:
but it's really not needed to solve this particular little
problem.
Well, I simply thought that something like this is
needed anyway, not only for that problem. But then
vmsplice() appeared, so of course such an mopen() is
no longer much relevant. I am just puzzled as to why
the pipe was used for vmsplice() and can't find an
answer, but I am probably missing something obvious...
Such a thing could solve that mremap() problem
that me and William Tambe have.
If you're thinking of going that way, for this problem it
No, of course not, at least, not any more.
Back in 2004 it was a good idea, but now there is no
longer a vacant place for this wonderfull syscall. :)
would be better just to stick with POSIX shm_open, as Christoph
suggested before, and you suggest to William in other mail.
The only scenario I can image where you would prefer
the anonymous mmaps over the Posix SHM, is when you want
to manage many small memory areas separately, and do not
want to track them back to their fd's. But even then this
probably doesn't save more than a few dozens of code lines. :)
But I now accept your view, that the shared anonymous object
is not behaving consistently in response to mremap,
IIRC this view was never under any doubts. The discussion
was only whether or not to count a few problems you pointed.
Here's a patch against 2.6.22-rc7: would you, Stas, put your
Signed-off-by into this, and accept authorship - although I'm
sending this back to you, it's very much your idea, and only
trivially modified from your three-year-old patch by me. If
you're agreeable, I can then forward it or its shmem_zero_fault
equivalent to Andrew when we see which way 2.6.23 is going.
OK, no problems. Except that I've already forgotten the
details, and pretty much lost the need for that patch.
So let's assume the authorship is yours. :)
Signed-off-by: Hugh Dickins [EMAIL PROTECTED]
Signed-off-by: Stas Sergeev [EMAIL PROTECTED]
mm/shmem.c | 41 -
1 file changed, 40 insertions(+), 1 deletion(-)
--- 2.6.22-rc7/mm/shmem.c 2007-06-17 10:51:02.0 +0100
+++ linux/mm/shmem.c2007-07-03 15:35:32.0 +0100
@@ -1320,6 +1320,36 @@ static struct page *shmem_nopage(struct
return page;
}
+struct page *shmem_zero_nopage(struct vm_area_struct *vma,
+ unsigned long address, int *type)
+{
+ struct page *page;
+
+ page = shmem_nopage(vma, address, type);
+ if (unlikely(page == NOPAGE_SIGBUS)) {
+ struct inode *inode = vma-vm_file-f_dentry-d_inode;
+ struct shmem_inode_info *info = SHMEM_I(inode);
+ loff_t vm_size, i_size;
+
+ /*
+* If mremap has been used to extend the vma,
+* now extend the underlying object to include this page.
+*/
+ vm_size = (PAGE_ALIGN(address) - vma-vm_start) +
+ ((loff_t)vma-vm_pgoff PAGE_SHIFT);
+ spin_lock(info-lock);
+ i_size = i_size_read(inode);
+ if (i_size vm_size vm_size = SHMEM_MAX_BYTES
+ !shmem_acct_size(info-flags, vm_size -
Re: Concerning a post that you made about expandable anonymous shared mappings
Hello. William Tambe wrote: And it just doesn't make sens to have mmap() map ANONYMOUS shared memory and mremap() not to expand it and make the expanded area available. I agree with this, but the argument against that approach was that then you can only enlarge the backing-store, but never shrink. I personally think it is a valid argument, even though the problem is probably not very important. Also, you can't expand the SysV SHM with mremap just as well - it will give you a SIGBUS too IIRC. So for that discussion of 2004, I lost the battle and was convinced that the proposed approach is not very good... Would you happen to know how I can work around that issue for now, and make writing in an expended area not to generate a Bus error? Have you tried the Posix SHM instead? It works very well for me. Back in 2004 the glibc had bugs, so I couldn't easily use posix shm and was thinking about the different approaches. But now it should suffice. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi. Hugh Dickins wrote: You've answered your own question: we did not make the change Stas suggested, IIRC because I remained a little uneasy with that change in behaviour, and nobody else spoke up for it. IIRC your argument, that made sense to me, was that with such an approach, you can only expand the backing-store, but never shrink. I agree this is a problem from some point of view. I have no idea whether it is important or not, but it definitely _looks_ not very good. //why does this failed. I am well in the interval [4096, 8192] *(unsigned int *)(ptr + 4096 + 8)= 10; } Well, this generates a bus error, while, for example, doing the same trick with having a /dev/mem as a backing-store, simply maps the "enlarged" space with the anonymous memory, and so does not generate a SIGBUS (not producing a desired effect either, of course). Why do we have it both ways? Shouldn't they (i.e. /dev/zero and /dev/mem mapping) behave the same after expanding with mremap? I haven't given it any thought since then: I was thinking about it a bit, and I imagined we need something like int mopen(void *addr, size_t len, unsigned int flags); which will give you an fd for the memory area, which you can then ftruncate() and mmap() (and mremap). It looks like vmsplice() is a very close one, but unfortunately it involves pipe, which will not give you an ability to ftruncate() I suppose. I even asked Jens Axboe about the possibility to have mopen(). He said it might be a good optimization (having only one syscall whereas now 2 are needed (pipe/vmsplice)), but not worth an efforts. Now, if maybe someone have a time and patience, he can explain me what was an advantage of using pipe with vmsplice() at all. Why it was not possible to have something like the mopen() above, that will give you an fd right away, without a pipe, so that ftruncate/mmap/lseek etc can be used on it? Well, I guess this was discussed many times, but I failed to google anything relevant... Such a thing could solve that mremap() problem that me and William Tambe have. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Yes, I have a good case, but my case may not sound interesting until you
see it working.
Ok, I am developing a dynamic memory allocation routine which takes
direct advantage of the ability of a machine to use Virtual Memory to
make everything look contiguous and fast.
And it just doesn't make sens to have mmap() map ANONYMOUS shared memory
and mremap() not to expand it and make the expanded area available.
I am not a 100% sure, but I don't think correcting its behavior would
cause a problem.
Would you happen to know how I can work around that issue for now, and
make writing in an expended area not to generate a Bus error?
Sincerely,
William Tambe
Hugh Dickins wrote:
On Fri, 29 Jun 2007, William Tambe wrote:
I read a post that you made about not being able to expand anonymous shared
mapping with mremap(). And I am actually having that issue now.
I guess you're referring to the thread at
http://lkml.org/lkml/2004/6/16/155
and you're asking either Stas or me.
You made the post in 2004 and we are now in 2007. I would like to know if that
feature was added because the code below always fail with bus error on my
machine. I use glibc 2.5
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
I haven't given it any thought since then:
do you have a good case for us to reconsider it?
Hugh
Thank you for helping.
#define _GNU_SOURCE
#include
#include
#include
main() {
void *ptr;
if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
printf("failed to mmap\n");
return;
}
if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
printf("failed to mremap\n");
return;
}
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
On Fri, 29 Jun 2007, William Tambe wrote:
> I read a post that you made about not being able to expand anonymous shared
> mapping with mremap(). And I am actually having that issue now.
I guess you're referring to the thread at
http://lkml.org/lkml/2004/6/16/155
and you're asking either Stas or me.
>
> You made the post in 2004 and we are now in 2007. I would like to know if that
> feature was added because the code below always fail with bus error on my
> machine. I use glibc 2.5
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
I haven't given it any thought since then:
do you have a good case for us to reconsider it?
Hugh
>
> Thank you for helping.
>
> #define _GNU_SOURCE
> #include
> #include
>
> #include
>
> main() {
> void *ptr;
> if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
> MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
> printf("failed to mmap\n");
> return;
> }
>
> if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
> printf("failed to mremap\n");
> return;
> }
>
> //why does this failed. I am well in the interval [4096, 8192]
> *(unsigned int *)(ptr + 4096 + 8)= 10;
> }
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
On Fri, 29 Jun 2007, William Tambe wrote:
I read a post that you made about not being able to expand anonymous shared
mapping with mremap(). And I am actually having that issue now.
I guess you're referring to the thread at
http://lkml.org/lkml/2004/6/16/155
and you're asking either Stas or me.
You made the post in 2004 and we are now in 2007. I would like to know if that
feature was added because the code below always fail with bus error on my
machine. I use glibc 2.5
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
I haven't given it any thought since then:
do you have a good case for us to reconsider it?
Hugh
Thank you for helping.
#define _GNU_SOURCE
#include sys/mman.h
#include unistd.h
#include stdio.h
main() {
void *ptr;
if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
printf(failed to mmap\n);
return;
}
if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
printf(failed to mremap\n);
return;
}
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Yes, I have a good case, but my case may not sound interesting until you
see it working.
Ok, I am developing a dynamic memory allocation routine which takes
direct advantage of the ability of a machine to use Virtual Memory to
make everything look contiguous and fast.
And it just doesn't make sens to have mmap() map ANONYMOUS shared memory
and mremap() not to expand it and make the expanded area available.
I am not a 100% sure, but I don't think correcting its behavior would
cause a problem.
Would you happen to know how I can work around that issue for now, and
make writing in an expended area not to generate a Bus error?
Sincerely,
William Tambe
Hugh Dickins wrote:
On Fri, 29 Jun 2007, William Tambe wrote:
I read a post that you made about not being able to expand anonymous shared
mapping with mremap(). And I am actually having that issue now.
I guess you're referring to the thread at
http://lkml.org/lkml/2004/6/16/155
and you're asking either Stas or me.
You made the post in 2004 and we are now in 2007. I would like to know if that
feature was added because the code below always fail with bus error on my
machine. I use glibc 2.5
You've answered your own question: we did not make the change Stas
suggested, IIRC because I remained a little uneasy with that change
in behaviour, and nobody else spoke up for it.
I haven't given it any thought since then:
do you have a good case for us to reconsider it?
Hugh
Thank you for helping.
#define _GNU_SOURCE
#include sys/mman.h
#include unistd.h
#include stdio.h
main() {
void *ptr;
if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
printf(failed to mmap\n);
return;
}
if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
printf(failed to mremap\n);
return;
}
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hi. Hugh Dickins wrote: You've answered your own question: we did not make the change Stas suggested, IIRC because I remained a little uneasy with that change in behaviour, and nobody else spoke up for it. IIRC your argument, that made sense to me, was that with such an approach, you can only expand the backing-store, but never shrink. I agree this is a problem from some point of view. I have no idea whether it is important or not, but it definitely _looks_ not very good. //why does this failed. I am well in the interval [4096, 8192] *(unsigned int *)(ptr + 4096 + 8)= 10; } Well, this generates a bus error, while, for example, doing the same trick with having a /dev/mem as a backing-store, simply maps the enlarged space with the anonymous memory, and so does not generate a SIGBUS (not producing a desired effect either, of course). Why do we have it both ways? Shouldn't they (i.e. /dev/zero and /dev/mem mapping) behave the same after expanding with mremap? I haven't given it any thought since then: I was thinking about it a bit, and I imagined we need something like int mopen(void *addr, size_t len, unsigned int flags); which will give you an fd for the memory area, which you can then ftruncate() and mmap() (and mremap). It looks like vmsplice() is a very close one, but unfortunately it involves pipe, which will not give you an ability to ftruncate() I suppose. I even asked Jens Axboe about the possibility to have mopen(). He said it might be a good optimization (having only one syscall whereas now 2 are needed (pipe/vmsplice)), but not worth an efforts. Now, if maybe someone have a time and patience, he can explain me what was an advantage of using pipe with vmsplice() at all. Why it was not possible to have something like the mopen() above, that will give you an fd right away, without a pipe, so that ftruncate/mmap/lseek etc can be used on it? Well, I guess this was discussed many times, but I failed to google anything relevant... Such a thing could solve that mremap() problem that me and William Tambe have. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Concerning a post that you made about expandable anonymous shared mappings
Hello. William Tambe wrote: And it just doesn't make sens to have mmap() map ANONYMOUS shared memory and mremap() not to expand it and make the expanded area available. I agree with this, but the argument against that approach was that then you can only enlarge the backing-store, but never shrink. I personally think it is a valid argument, even though the problem is probably not very important. Also, you can't expand the SysV SHM with mremap just as well - it will give you a SIGBUS too IIRC. So for that discussion of 2004, I lost the battle and was convinced that the proposed approach is not very good... Would you happen to know how I can work around that issue for now, and make writing in an expended area not to generate a Bus error? Have you tried the Posix SHM instead? It works very well for me. Back in 2004 the glibc had bugs, so I couldn't easily use posix shm and was thinking about the different approaches. But now it should suffice. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Concerning a post that you made about expandable anonymous shared mappings
I read a post that you made about not being able to expand anonymous
shared mapping with mremap(). And I am actually having that issue now.
You made the post in 2004 and we are now in 2007. I would like to know
if that feature was added because the code below always fail with bus
error on my machine. I use glibc 2.5
Thank you for helping.
#define _GNU_SOURCE
#include
#include
#include
main() {
void *ptr;
if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
printf("failed to mmap\n");
return;
}
if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
printf("failed to mremap\n");
return;
}
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Concerning a post that you made about expandable anonymous shared mappings
I read a post that you made about not being able to expand anonymous
shared mapping with mremap(). And I am actually having that issue now.
You made the post in 2004 and we are now in 2007. I would like to know
if that feature was added because the code below always fail with bus
error on my machine. I use glibc 2.5
Thank you for helping.
#define _GNU_SOURCE
#include sys/mman.h
#include unistd.h
#include stdio.h
main() {
void *ptr;
if ((ptr=mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_ANONYMOUS|MAP_SHARED|MAP_GROWSDOWN, 0, 0)) == -1) {
printf(failed to mmap\n);
return;
}
if ((ptr=mremap(ptr, 4096, 8192, MREMAP_MAYMOVE)) == -1) {
printf(failed to mremap\n);
return;
}
//why does this failed. I am well in the interval [4096, 8192]
*(unsigned int *)(ptr + 4096 + 8)= 10;
}
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
