Re: shm_mkstemp(3) without the file name

[bytevolcano]

Philip Guenther wrote:


Well, I am amazed. I guess I just have to do some more investigation into
workarounds for this, as RAM-based tmpfs file systems will get full very
quickly with shared memory segments, and large segments result in high disk
activity when munmap() is called. And SysV shared memory is too limiting for
my purposes.


Could you clarify what underlying problem you're trying to solve?
Maybe there's just a misunderstanding about the POSIX APIs, or about
how OpenBSD implements them, or what alternative APIs exist.


Philip Guenther



Basically:

1. shm_mkstemp() will result in the creation of a file with a long 
random name in /tmp, which is quite small on the target system (512M 
mfs). The system has 4GB of RAM.


 - The buffers tend to be somewhat large (several dozen MB 
occasionally). I cannot create a shared memory segment larger than the 
remaining free space on the /tmp FS. This creates an issue where 
multiple applications have a buffer each.


 - On a system where /tmp is a disk partition, this file does not 
actually occupy any space until munmap() is called; this results in a 
long pause as it writes the data to the disk.
   This occurs even if shm_unlink() is called before the unmap/close. 
In fact, shm_unlink() is called immediately after shm_mkstemp(), because 
I only am interested in the descriptor from now on. This disk write is 
unnecessary for shared memory; I don't want the file to persist at all. 
When both the application and server are done with the buffer, all 
should be gone.


 - The only way to resolve the issue with disk I/O is to call 
ftruncate(fd, 0) before munmap(), and even this only works where /tmp is 
a huge MFS partition or a huge disk partition.


 - There is a brief point in time (between shm_mkstemp() and 
shm_unlink()) where a rogue application can grab the buffer file, call 
ftruncate(), and map its contents, without either my server or 
applications knowing. Whilst file permissions are good at stopping other 
users accessing the segment, this goes against the idea that the app


To summarise, the major issue I am having is that the "shared memory" 
buffer looks like a file on disk, and acts like a file on disk, rather 
than a section of memory that I am sharing in between processes. As a 
result, I need to have a huge /tmp file system just to accommodate the 
buffer, regardless of how much RAM I have.



The problem I have with the SysV shm* API is that it gets hard to clean 
up shared memory segments. Whilst that I can live with to some degree, 
the biggest issue with it is the 32MB limit per segment 
(kern.shminfo.shmmax).

Re: shm_mkstemp(3) without the file name

[Philip Guenther]

On Fri, Jul 15, 2016 at 1:56 AM,   wrote:
> Ted Unangst wrote:
>>
>> bytevolc...@safe-mail.net wrote:
>>>
>>> I see where you are coming from, but what I am getting at is, where in
>>> the POSIX standard does it say that it needs to be anywhere in the file
>>> system at all? If it is shared memory, then surely this doesn't require
>>> backing up.
>>
>> Oh. It doesn't have to be a file. It can be a file. (And you can't asusme
>> it's not a file.)
>
> Well, I am amazed. I guess I just have to do some more investigation into
> workarounds for this, as RAM-based tmpfs file systems will get full very
> quickly with shared memory segments, and large segments result in high disk
> activity when munmap() is called. And SysV shared memory is too limiting for
> my purposes.

Could you clarify what underlying problem you're trying to solve?
Maybe there's just a misunderstanding about the POSIX APIs, or about
how OpenBSD implements them, or what alternative APIs exist.


Philip Guenther

Re: shm_mkstemp(3) without the file name

[bytevolcano]

Ted Unangst wrote:

bytevolc...@safe-mail.net wrote:

I see where you are coming from, but what I am getting at is, where in
the POSIX standard does it say that it needs to be anywhere in the file
system at all? If it is shared memory, then surely this doesn't require
backing up.


Oh. It doesn't have to be a file. It can be a file. (And you can't asusme it's
not a file.)



Well, I am amazed. I guess I just have to do some more investigation 
into workarounds for this, as RAM-based tmpfs file systems will get full 
very quickly with shared memory segments, and large segments result in 
high disk activity when munmap() is called. And SysV shared memory is 
too limiting for my purposes.

Thanks for the info, Ted.

Re: shm_mkstemp(3) without the file name

[Ted Unangst]

bytevolc...@safe-mail.net wrote:
> I see where you are coming from, but what I am getting at is, where in 
> the POSIX standard does it say that it needs to be anywhere in the file 
> system at all? If it is shared memory, then surely this doesn't require 
> backing up.

Oh. It doesn't have to be a file. It can be a file. (And you can't asusme it's
not a file.)

Re: shm_mkstemp(3) without the file name

[bytevolcano]

Theo de Raadt wrote:

Is using ftruncate(2) to lengthen the segment the right way to do it, or
is this yet another stupid limitation of POSIX shared memory?


If you are getting the picture that the standards commitee cobbled
together a bunch of junk and expected a good outcome, you are well
on your way to being on the same page as us...



I agree. It all makes me wonder how long we have before IPv6 support is 
mandated for UNIX domain sockets.

Re: shm_mkstemp(3) without the file name

[bytevolcano]

Ted Unangst wrote:

bytevolc...@safe-mail.net wrote:

When I use ftruncate(2) to actually allocate the segment, the file is as
long as the segment that is allocated.

Even if the file is unlinked before ftruncate(2) is called, enough free
space in the /tmp *file system* is required for the shared memory segment.

Is using ftruncate(2) to lengthen the segment the right way to do it, or
is this yet another stupid limitation of POSIX shared memory?


ftruncate is really the only way to do it.

There's nothing special about shm_open. It's just open with a different name.
If you want the files on a different filesystem, you can use open.



I see where you are coming from, but what I am getting at is, where in 
the POSIX standard does it say that it needs to be anywhere in the file 
system at all? If it is shared memory, then surely this doesn't require 
backing up.

Re: shm_mkstemp(3) without the file name

[Ted Unangst]

bytevolc...@safe-mail.net wrote:
> When I use ftruncate(2) to actually allocate the segment, the file is as 
> long as the segment that is allocated.
> 
> Even if the file is unlinked before ftruncate(2) is called, enough free 
> space in the /tmp *file system* is required for the shared memory segment.
> 
> Is using ftruncate(2) to lengthen the segment the right way to do it, or 
> is this yet another stupid limitation of POSIX shared memory?

ftruncate is really the only way to do it.

There's nothing special about shm_open. It's just open with a different name.
If you want the files on a different filesystem, you can use open.

Re: shm_mkstemp(3) without the file name

[Theo de Raadt]

> Is using ftruncate(2) to lengthen the segment the right way to do it, or 
> is this yet another stupid limitation of POSIX shared memory?

If you are getting the picture that the standards commitee cobbled
together a bunch of junk and expected a good outcome, you are well
on your way to being on the same page as us...

Re: shm_mkstemp(3) without the file name

[bytevolcano]

Ted Unangst wrote:

bytevolc...@safe-mail.net wrote:

Yes, it seems to create files with long names (that have nothing to do
with the template I provide) in the /tmp root.

If it doesn't respect the path or template, what is the point of having
this argument there in the first place, and what is the point of even
having a file on the file system?


The file is the backing for the memory. The path is so that you can refer to
the same object. There's no requirement that the path be the name of the file
in the filesystem, since you don't use normal file functions to talk to it.


When I use ftruncate(2) to actually allocate the segment, the file is as 
long as the segment that is allocated.


Even if the file is unlinked before ftruncate(2) is called, enough free 
space in the /tmp *file system* is required for the shared memory segment.


Is using ftruncate(2) to lengthen the segment the right way to do it, or 
is this yet another stupid limitation of POSIX shared memory?
I did give SysV shared memory segments a go, but work required for 
cleanup is even more insane when the program crashes!





I do not see any promises here. The only promise here is that the shared
memory object will be created atomically. I found it creates long-name
files with mode 0600. If the implementation promises these permissions
for shm_mkstemp(3), then fantastic; it should really be mentioned in the
man page though.


"This implementation forces the mode to be 0600"


Thanks for pointing that out; I saw that, but I got confused as it was 
in the paragraph describing shm_open(3).





There's still an issue of the stale files on the file system, should
there be a crash, interruption, signal, or something like that. Even
with close(2), the file remains until shm_unlink(3) is called.


That's not unique to shm functions. Everything creating files can do that.



Indeed, but I don't see a use case for having shared memory backed 
verbatim by a file, to the point where the amount of shared memory that 
can be allocated depends on how much free space left in the /tmp file 
system.


Say I want to allocate 2GB shared memory object, I cannot do so with 
ftruncate(2) because I only have 128MB left in /tmp.

Re: shm_mkstemp(3) without the file name

[Ted Unangst]

bytevolc...@safe-mail.net wrote:
> Yes, it seems to create files with long names (that have nothing to do 
> with the template I provide) in the /tmp root.
> 
> If it doesn't respect the path or template, what is the point of having 
> this argument there in the first place, and what is the point of even 
> having a file on the file system?

The file is the backing for the memory. The path is so that you can refer to
the same object. There's no requirement that the path be the name of the file
in the filesystem, since you don't use normal file functions to talk to it.

> I do not see any promises here. The only promise here is that the shared 
> memory object will be created atomically. I found it creates long-name 
> files with mode 0600. If the implementation promises these permissions 
> for shm_mkstemp(3), then fantastic; it should really be mentioned in the 
> man page though.

"This implementation forces the mode to be 0600"

> There's still an issue of the stale files on the file system, should 
> there be a crash, interruption, signal, or something like that. Even 
> with close(2), the file remains until shm_unlink(3) is called.

That's not unique to shm functions. Everything creating files can do that.

Re: shm_mkstemp(3) without the file name

[bytevolcano]

Jeremie Courreges-Anglas wrote:

bytevolc...@safe-mail.net writes:


Hello,

I am writing a local server which requires the use of shared memory
objects. Essentially, other applications communicate to this server by
connecting to a UNIX domain socket within the file system.

Occasionally such an application may require a shared memory buffer to
share large quantities of information with the server. In doing this,
the server uses shm_mkstemp(3) to create the shared memory objects, and
then sends the file descriptor over the connection.

char snm[25] = "/tmp/megaserv/XX";


I doubt that our shm_open(3) implementation will respect the path you
provide here.


Yes, it seems to create files with long names (that have nothing to do 
with the template I provide) in the /tmp root.


If it doesn't respect the path or template, what is the point of having 
this argument there in the first place, and what is the point of even 
having a file on the file system?





int fd;

fd = shm_mkstemp(snm);

/* Error handling omitted for clarity. */
/* Potential race condition here */

shm_unlink(snm);

...

/* Send fd over connected socket. */

Whilst the setup I have works well, I see a potential race condition,
albeit a very small one, in the position indicated above; an external
process, malicious or otherwise, can connect to the object in between
the shm_mkstmp() and shm_unlink() calls.


Not if the permissions on the file prevent this.

  "This implementation forces the mode to be 0600 or 0400, and prohibits
  sharing between different UIDs."


Furthermore, there is a small possibility for a stale file to be present
in the file system, should there be a crash.


Why not trust the promise made by shm_mkstemp(3)?


" If a temporary shared memory object is desired, the shm_mkstemp() 
function should be preferred as it avoids several possible security 
holes that tend to appear in programs trying to create their own unique 
temporary names.  The template argument is a string with at least six 
trailing Xs as described in mkstemp(3)."


I do not see any promises here. The only promise here is that the shared 
memory object will be created atomically. I found it creates long-name 
files with mode 0600. If the implementation promises these permissions 
for shm_mkstemp(3), then fantastic; it should really be mentioned in the 
man page though.


But I suppose if the malicious process is running as the user, then it 
can wreak all kinds of other havoc on the system; no need to protect the 
shared memory buffers then.


There's still an issue of the stale files on the file system, should 
there be a crash, interruption, signal, or something like that. Even 
with close(2), the file remains until shm_unlink(3) is called.





Is there a way of creating these objects without having to actually
create a file in the file system, something like pipe() or socketpair()?

For example:

int shm_create(int flags);

That would basically eliminate the race condition, the possibility of
a stale object, and make shm_unlink() unnecessary in this case.

Any advice/suggestions?

Re: shm_mkstemp(3) without the file name

[Jeremie Courreges-Anglas]

bytevolc...@safe-mail.net writes:

> Hello,
>
> I am writing a local server which requires the use of shared memory
> objects. Essentially, other applications communicate to this server by
> connecting to a UNIX domain socket within the file system.
>
> Occasionally such an application may require a shared memory buffer to
> share large quantities of information with the server. In doing this,
> the server uses shm_mkstemp(3) to create the shared memory objects, and
> then sends the file descriptor over the connection.
>
>   char snm[25] = "/tmp/megaserv/XX";

I doubt that our shm_open(3) implementation will respect the path you
provide here.

>   int fd;
>
>   fd = shm_mkstemp(snm);
>
>   /* Error handling omitted for clarity. */
>   /* Potential race condition here */
>
>   shm_unlink(snm);
>
>   ...
>
>   /* Send fd over connected socket. */
>
> Whilst the setup I have works well, I see a potential race condition,
> albeit a very small one, in the position indicated above; an external
> process, malicious or otherwise, can connect to the object in between
> the shm_mkstmp() and shm_unlink() calls.

Not if the permissions on the file prevent this.

 "This implementation forces the mode to be 0600 or 0400, and prohibits
 sharing between different UIDs."

> Furthermore, there is a small possibility for a stale file to be present
> in the file system, should there be a crash.

Why not trust the promise made by shm_mkstemp(3)?

> Is there a way of creating these objects without having to actually
> create a file in the file system, something like pipe() or socketpair()?
>
> For example:
>
>   int shm_create(int flags);
>
> That would basically eliminate the race condition, the possibility of
> a stale object, and make shm_unlink() unnecessary in this case.
>
> Any advice/suggestions?
>

--
jca | PGP: 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE

[demime 1.01d removed an attachment of type application/pgp-signature which had 
a name of signature.asc]

shm_mkstemp(3) without the file name

[bytevolcano]

Hello,

I am writing a local server which requires the use of shared memory 
objects. Essentially, other applications communicate to this server by 
connecting to a UNIX domain socket within the file system.


Occasionally such an application may require a shared memory buffer to 
share large quantities of information with the server. In doing this, 
the server uses shm_mkstemp(3) to create the shared memory objects, and 
then sends the file descriptor over the connection.


char snm[25] = "/tmp/megaserv/XX";
int fd;

fd = shm_mkstemp(snm);

/* Error handling omitted for clarity. */
/* Potential race condition here */

shm_unlink(snm);

...

/* Send fd over connected socket. */

Whilst the setup I have works well, I see a potential race condition, 
albeit a very small one, in the position indicated above; an external 
process, malicious or otherwise, can connect to the object in between 
the shm_mkstmp() and shm_unlink() calls.


Furthermore, there is a small possibility for a stale file to be present 
in the file system, should there be a crash.


Is there a way of creating these objects without having to actually 
create a file in the file system, something like pipe() or socketpair()?


For example:

int shm_create(int flags);

That would basically eliminate the race condition, the possibility of a 
stale object, and make shm_unlink() unnecessary in this case.


Any advice/suggestions?