Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On Fri, Nov 19, 2021 at 08:59:29PM +, Stuart Henderson wrote: > On 2021/11/19 11:12, Steve Williams wrote: > > Thanks very much for that! I'll have a look as it's more contemporary than > > the port that Stuart pointed to. > > yep, I mostly lost interest a while ago and haven't been tracking it > closely (this was after getting motion to work - that's more my sort > of program than zoneminder - maybe zm is better now but it felt > rather disjointed, I seem to remember it feeling rather slow on > OpenBSD as well - suspect the tsc timecounter changes a few releases > ago will have helped that, at least on single-socket machines) It runs well enough for a small environment. It's still a giant turd. Looking at this new version, I'm quickly losing interest as well. Sorry Ian! -- Tracey Emery
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On 2021/11/19 11:12, Steve Williams wrote: > Thanks very much for that! I'll have a look as it's more contemporary than > the port that Stuart pointed to. yep, I mostly lost interest a while ago and haven't been tracking it closely (this was after getting motion to work - that's more my sort of program than zoneminder - maybe zm is better now but it felt rather disjointed, I seem to remember it feeling rather slow on OpenBSD as well - suspect the tsc timecounter changes a few releases ago will have helped that, at least on single-socket machines)
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On 19/11/2021 7:53 a.m., Tracey Emery wrote: On Fri, Nov 19, 2021 at 07:14:04AM -0800, Steve Williams wrote: Stuart, You are absolutely amazing! I have no idea how you find time to respond to all these requests and still manage to accomplish what you do. Thanks very much for the pointer to a WIP port. I'll have a look at it. Cheers, Steve Williams On 19/11/2021 12:21 a.m., Stuart Henderson wrote: zoneminder has fallback code, they just unconditionally include zm_sendfile.h even where sendfile was not detected, and that has a hard #error. See https://github.com/jasperla/openbsd-wip/tree/master/multimedia/zoneminder, there is a patch for this. -- Sent from a phone, apologies for poor formatting. On 19 November 2021 05:31:32 George Koehler wrote: On Thu, 18 Nov 2021 18:04:26 -0800 Steve Williams wrote: Hi, I'm trying to compile a Linux/FreeBSD application (zoneminder) under OpenBSD 7.0. I'm slowly working through it but have gotten stuck at the point where it has a dependency on sendfile(2) which OpenBSD doesn't have. Doing some research, I found one reference that sosplice(2) might be able to be used in place of sendfile. Elsewhere, I have found reference to a "simple-sendfile" package, but that doesn't seem to exist. Linux sendfile(2) and FreeBSD sendfile(2) are different functions. simple_sendfile() is a function in lang/ruby that wraps FreeBSD's sendfile to act like Linux's. simple_sendfile() is not for OpenBSD. Linux: https://man7.org/linux/man-pages/man2/sendfile.2.html FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=sendfile simple_sendfile: https://github.com/ruby/ruby/blob/0db68f023372b634603c74fca94588b457be084c/io.c#L11320 OpenBSD's SO_SPLICE can only "splice together two TCP or UDP sockets", says setsockopt(2). You can't SO_SPLICE if your code copies a regular file to a socket. Is this an issue people have already resolved with other ports? Or is it a deal breaker? I guess the devil is in the details, but I thought I'd reach out here first in case there's already a good solution to this. Check what your code does if sendfile fails. If it falls back to a read/write loop, then you might patch it to skip sendfile. If it doesn't have a read/write loop, then you might need to add one. --George Thanks, Steve Williams If you want a zoneminder that is currently working in OpenBSD, go here: https://github.com/basepr1me/openbsd-ports-mystuff/tree/master/multimedia/zoneminder I haven't touched it for awhile and the README still needs updated, but that is running at my house right now. It uses httpd outside of chroot. Everything works as intended for me. The README file that is there is full of notes that might be able to get you through a setup. Ian was working on an updated README, but he hasn't finished. I haven't looked at newer versions. Hi Tracey, Thanks very much for that! I'll have a look as it's more contemporary than the port that Stuart pointed to. I'll have a look :) Cheers, Steve W.
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On Fri, Nov 19, 2021 at 08:53:19AM -0700, Tracey Emery wrote: > > If you want a zoneminder that is currently working in OpenBSD, go here: > https://github.com/basepr1me/openbsd-ports-mystuff/tree/master/multimedia/zoneminder > > I haven't touched it for awhile and the README still needs updated, but > that is running at my house right now. It uses httpd outside of chroot. > Everything works as intended for me. > > The README file that is there is full of notes that might be able to get > you through a setup. Ian was working on an updated README, but he hasn't > finished. > > I haven't looked at newer versions. > > -- > > Tracey Emery It looks like there are several breaking changes from 1.34.26 to 1.36.11. It'll be awhile before I can thoroughly review. -- Tracey Emery
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On Fri, Nov 19, 2021 at 07:14:04AM -0800, Steve Williams wrote: > Stuart, > > You are absolutely amazing! I have no idea how you find time to respond to > all these requests and still manage to accomplish what you do. > > Thanks very much for the pointer to a WIP port. I'll have a look at it. > > Cheers, > Steve Williams > > On 19/11/2021 12:21 a.m., Stuart Henderson wrote: > > zoneminder has fallback code, they just unconditionally include > > zm_sendfile.h even where sendfile was not detected, and that has a hard > > #error. See > > https://github.com/jasperla/openbsd-wip/tree/master/multimedia/zoneminder, > > there is a patch for this. > > > > -- > > Sent from a phone, apologies for poor formatting. > > > > > > On 19 November 2021 05:31:32 George Koehler wrote: > > > > > On Thu, 18 Nov 2021 18:04:26 -0800 > > > Steve Williams wrote: > > > > > > > Hi, > > > > > > > > I'm trying to compile a Linux/FreeBSD application (zoneminder) under > > > > OpenBSD 7.0. > > > > > > > > I'm slowly working through it but have gotten stuck at the point where > > > > it has a dependency on sendfile(2) which OpenBSD doesn't have. > > > > > > > > Doing some research, I found one reference that sosplice(2) might be > > > > able to be used in place of sendfile. Elsewhere, I have found reference > > > > to a "simple-sendfile" package, but that doesn't seem to exist. > > > > > > Linux sendfile(2) and FreeBSD sendfile(2) are different functions. > > > simple_sendfile() is a function in lang/ruby that wraps FreeBSD's > > > sendfile to act like Linux's. simple_sendfile() is not for OpenBSD. > > > > > > Linux: https://man7.org/linux/man-pages/man2/sendfile.2.html > > > FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=sendfile > > > simple_sendfile: > > > https://github.com/ruby/ruby/blob/0db68f023372b634603c74fca94588b457be084c/io.c#L11320 > > > > > > OpenBSD's SO_SPLICE can only "splice together two TCP or UDP sockets", > > > says setsockopt(2). You can't SO_SPLICE if your code copies a regular > > > file to a socket. > > > > > > > Is this an issue people have already resolved with other ports? Or is > > > > it a deal breaker? > > > > > > > > I guess the devil is in the details, but I thought I'd reach out here > > > > first in case there's already a good solution to this. > > > > > > Check what your code does if sendfile fails. If it falls back to a > > > read/write loop, then you might patch it to skip sendfile. If it > > > doesn't have a read/write loop, then you might need to add one. > > > > > > --George > > > > > > > > > > > Thanks, > > > > Steve Williams > > > > > > If you want a zoneminder that is currently working in OpenBSD, go here: https://github.com/basepr1me/openbsd-ports-mystuff/tree/master/multimedia/zoneminder I haven't touched it for awhile and the README still needs updated, but that is running at my house right now. It uses httpd outside of chroot. Everything works as intended for me. The README file that is there is full of notes that might be able to get you through a setup. Ian was working on an updated README, but he hasn't finished. I haven't looked at newer versions. -- Tracey Emery
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
Stuart, You are absolutely amazing! I have no idea how you find time to respond to all these requests and still manage to accomplish what you do. Thanks very much for the pointer to a WIP port. I'll have a look at it. Cheers, Steve Williams On 19/11/2021 12:21 a.m., Stuart Henderson wrote: zoneminder has fallback code, they just unconditionally include zm_sendfile.h even where sendfile was not detected, and that has a hard #error. See https://github.com/jasperla/openbsd-wip/tree/master/multimedia/zoneminder, there is a patch for this. -- Sent from a phone, apologies for poor formatting. On 19 November 2021 05:31:32 George Koehler wrote: On Thu, 18 Nov 2021 18:04:26 -0800 Steve Williams wrote: Hi, I'm trying to compile a Linux/FreeBSD application (zoneminder) under OpenBSD 7.0. I'm slowly working through it but have gotten stuck at the point where it has a dependency on sendfile(2) which OpenBSD doesn't have. Doing some research, I found one reference that sosplice(2) might be able to be used in place of sendfile. Elsewhere, I have found reference to a "simple-sendfile" package, but that doesn't seem to exist. Linux sendfile(2) and FreeBSD sendfile(2) are different functions. simple_sendfile() is a function in lang/ruby that wraps FreeBSD's sendfile to act like Linux's. simple_sendfile() is not for OpenBSD. Linux: https://man7.org/linux/man-pages/man2/sendfile.2.html FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=sendfile simple_sendfile: https://github.com/ruby/ruby/blob/0db68f023372b634603c74fca94588b457be084c/io.c#L11320 OpenBSD's SO_SPLICE can only "splice together two TCP or UDP sockets", says setsockopt(2). You can't SO_SPLICE if your code copies a regular file to a socket. Is this an issue people have already resolved with other ports? Or is it a deal breaker? I guess the devil is in the details, but I thought I'd reach out here first in case there's already a good solution to this. Check what your code does if sendfile fails. If it falls back to a read/write loop, then you might patch it to skip sendfile. If it doesn't have a read/write loop, then you might need to add one. --George Thanks, Steve Williams
Re: OpenBSD wrapper for Linux/FreeBSD sendfile? (for ZoneMinder)
On Thu, Nov 18, 2021 at 06:04:26PM -0800, Steve Williams wrote: > Hi, > > I'm trying to compile a Linux/FreeBSD application (zoneminder) under OpenBSD > 7.0. > > I'm slowly working through it but have gotten stuck at the point where it > has a dependency on sendfile(2) which OpenBSD doesn't have. It's generally good to start with a "Is anybody else working on a ZoneMinder port?"
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
zoneminder has fallback code, they just unconditionally include zm_sendfile.h even where sendfile was not detected, and that has a hard #error. See https://github.com/jasperla/openbsd-wip/tree/master/multimedia/zoneminder, there is a patch for this. -- Sent from a phone, apologies for poor formatting. On 19 November 2021 05:31:32 George Koehler wrote: On Thu, 18 Nov 2021 18:04:26 -0800 Steve Williams wrote: Hi, I'm trying to compile a Linux/FreeBSD application (zoneminder) under OpenBSD 7.0. I'm slowly working through it but have gotten stuck at the point where it has a dependency on sendfile(2) which OpenBSD doesn't have. Doing some research, I found one reference that sosplice(2) might be able to be used in place of sendfile. Elsewhere, I have found reference to a "simple-sendfile" package, but that doesn't seem to exist. Linux sendfile(2) and FreeBSD sendfile(2) are different functions. simple_sendfile() is a function in lang/ruby that wraps FreeBSD's sendfile to act like Linux's. simple_sendfile() is not for OpenBSD. Linux: https://man7.org/linux/man-pages/man2/sendfile.2.html FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=sendfile simple_sendfile: https://github.com/ruby/ruby/blob/0db68f023372b634603c74fca94588b457be084c/io.c#L11320 OpenBSD's SO_SPLICE can only "splice together two TCP or UDP sockets", says setsockopt(2). You can't SO_SPLICE if your code copies a regular file to a socket. Is this an issue people have already resolved with other ports? Or is it a deal breaker? I guess the devil is in the details, but I thought I'd reach out here first in case there's already a good solution to this. Check what your code does if sendfile fails. If it falls back to a read/write loop, then you might patch it to skip sendfile. If it doesn't have a read/write loop, then you might need to add one. --George Thanks, Steve Williams
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
On Thu, 18 Nov 2021 18:04:26 -0800 Steve Williams wrote: > Hi, > > I'm trying to compile a Linux/FreeBSD application (zoneminder) under > OpenBSD 7.0. > > I'm slowly working through it but have gotten stuck at the point where > it has a dependency on sendfile(2) which OpenBSD doesn't have. > > Doing some research, I found one reference that sosplice(2) might be > able to be used in place of sendfile. Elsewhere, I have found reference > to a "simple-sendfile" package, but that doesn't seem to exist. Linux sendfile(2) and FreeBSD sendfile(2) are different functions. simple_sendfile() is a function in lang/ruby that wraps FreeBSD's sendfile to act like Linux's. simple_sendfile() is not for OpenBSD. Linux: https://man7.org/linux/man-pages/man2/sendfile.2.html FreeBSD: https://www.freebsd.org/cgi/man.cgi?query=sendfile simple_sendfile: https://github.com/ruby/ruby/blob/0db68f023372b634603c74fca94588b457be084c/io.c#L11320 OpenBSD's SO_SPLICE can only "splice together two TCP or UDP sockets", says setsockopt(2). You can't SO_SPLICE if your code copies a regular file to a socket. > Is this an issue people have already resolved with other ports? Or is > it a deal breaker? > > I guess the devil is in the details, but I thought I'd reach out here > first in case there's already a good solution to this. Check what your code does if sendfile fails. If it falls back to a read/write loop, then you might patch it to skip sendfile. If it doesn't have a read/write loop, then you might need to add one. --George > > Thanks, > Steve Williams >
Re: OpenBSD wrapper for Linux/FreeBSD sendfile?
Hi, Steve Williams wrote on Thu, Nov 18, 2021 at 06:04:26PM -0800: > I'm trying to compile a Linux/FreeBSD application (zoneminder) under > OpenBSD 7.0. > > I'm slowly working through it but have gotten stuck at the point where > it has a dependency on sendfile(2) which OpenBSD doesn't have. Judging from the Linux manual page https://man.bsd.lv/Linux-5.13/sendfile you should be able to trivially implement a replacement function for sendfile(2) using lseek(2), read(2), and write(2). Of course, that will be slower than doing the copying inside the kernel as sendfile(2) does. But it should get you going in order to make the program run at all. Try that unless somebody else has a better idea. > Doing some research, I found one reference that sosplice(2) might be > able to be used in place of sendfile. A system call soplice(2) does not exist. The sosplice(9) manual page describes kernel internals, so to implement sendfile(2) based on sosplice(9), you would have to do some real kernel hacking. Given the questions you are asking, you are clearly not up to that, and even if you were - hacking up the kernel just to port some software is usually not the best idea. Yours, Ingo
OpenBSD wrapper for Linux/FreeBSD sendfile?
Hi, I'm trying to compile a Linux/FreeBSD application (zoneminder) under OpenBSD 7.0. I'm slowly working through it but have gotten stuck at the point where it has a dependency on sendfile(2) which OpenBSD doesn't have. Doing some research, I found one reference that sosplice(2) might be able to be used in place of sendfile. Elsewhere, I have found reference to a "simple-sendfile" package, but that doesn't seem to exist. Is this an issue people have already resolved with other ports? Or is it a deal breaker? I guess the devil is in the details, but I thought I'd reach out here first in case there's already a good solution to this. Thanks, Steve Williams
