Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-11-19 Thread Thomas Munro
On Sat, Nov 20, 2021 at 9:34 AM Tom Lane wrote: > Thomas Munro writes: > > This has been fixed. So now there are working basic futexes on Linux, > > macOS, {Free,Open,Net,Dragonfly}BSD (though capabilities beyond basic > > wait/wake vary, as do APIs). So the question is whether it would be > >

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-11-19 Thread Tom Lane
Thomas Munro writes: > This has been fixed. So now there are working basic futexes on Linux, > macOS, {Free,Open,Net,Dragonfly}BSD (though capabilities beyond basic > wait/wake vary, as do APIs). So the question is whether it would be > worth trying to do our own futex-based semaphores, as sketc

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-11-19 Thread Thomas Munro
On Fri, Oct 29, 2021 at 4:54 PM Thomas Munro wrote: > On Sun, Oct 24, 2021 at 10:50 PM Thomas Munro wrote: > > Sadly, although the attached proof-of-concept patch allows a > > PREFERRED_SEMAPHORES=FUTEX build to pass tests on macOS (which also > > lacks native unnamed semas), FreeBSD and Linux (w

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-28 Thread Thomas Munro
On Sun, Oct 24, 2021 at 10:50 PM Thomas Munro wrote: > Sadly, although the attached proof-of-concept patch allows a > PREFERRED_SEMAPHORES=FUTEX build to pass tests on macOS (which also > lacks native unnamed semas), FreeBSD and Linux (which don't need this > but are interesting to test), and it a

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-25 Thread Mikhail
On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: > Also, you haven't explained why the existing (and much safer) recycling > logic in IpcSemaphoreCreate doesn't solve your problem. I think I'll drop the diffs, you're right that current proven logic need not to be changed for such rare cor

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-24 Thread Thomas Munro
On Mon, Oct 18, 2021 at 10:07 AM Thomas Munro wrote: > Note: The best kind would be *unnamed* POSIX semas, where we get to > control their placement in existing memory; that's what we do on Linux > and FreeBSD. They weren't supported on OpenBSD last time we checked: > it rejects requests for shar

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-23 Thread Mikhail
On Sun, Oct 17, 2021 at 10:52:38AM -0400, Tom Lane wrote: > I am, however, concerned that this'll just trade off one hazard for > another. Instead of a risk of failing with ENOSPC (which the DBA > can fix), we'll have a risk of kneecapping some other process at > random (which the DBA can do nothi

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-23 Thread Mikhail
On Fri, Oct 22, 2021 at 09:00:31PM -0400, Tom Lane wrote: > I tried this on an OpenBSD 6.0 image I had handy. The good news is > that it works, and I can successfully start the postmaster with a lot > of semaphores (I tried with max_connections=1) without any special > system configuration. T

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Tom Lane
Mikhail writes: > +# OpenBSD 5.5 (2014) gained named POSIX semaphores. They work out of the box > +# without changing any sysctl settings, unlike System V semaphores. > +USE_NAMED_POSIX_SEMAPHORES=1 I tried this on an OpenBSD 6.0 image I had handy. The good news is that it works, and I can succ

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Tom Lane
Mikhail writes: > On Fri, Oct 22, 2021 at 03:43:00PM -0400, Tom Lane wrote: >> Official support or no, we have OpenBSD 5.9 in our buildfarm, so >> ignoring the case isn't going to fly. > 5.9 has support for unnamed POSIX semas. Do you think new machine with > OpenBSD <5.5 (when unnamed POSIX sema

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Mikhail
On Fri, Oct 22, 2021 at 03:43:00PM -0400, Tom Lane wrote: > Mikhail writes: > > In your patch I've removed testing for 5.x versions, because official > > releases are supported only for one year, no need to worry about them. > > Official support or no, we have OpenBSD 5.9 in our buildfarm, so > i

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Thomas Munro
On Sat, Oct 23, 2021 at 8:43 AM Tom Lane wrote: > Mikhail writes: > > In your patch I've removed testing for 5.x versions, because official > > releases are supported only for one year, no need to worry about them. > > Official support or no, we have OpenBSD 5.9 in our buildfarm, so > ignoring th

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Tom Lane
Mikhail writes: > In your patch I've removed testing for 5.x versions, because official > releases are supported only for one year, no need to worry about them. Official support or no, we have OpenBSD 5.9 in our buildfarm, so ignoring the case isn't going to fly. regards,

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-22 Thread Mikhail
On Mon, Oct 18, 2021 at 10:07:40AM +1300, Thomas Munro wrote: > On Mon, Oct 18, 2021 at 4:49 AM Mikhail wrote: > > The logic works - the initial call to semget() in > > InternalIpcSemaphoreCreate returns -1 and errno is set to ENOSPC - I > > tested the patch on OpenBSD 7.0, it successfully recycle

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread Thomas Munro
On Mon, Oct 18, 2021 at 4:49 AM Mikhail wrote: > The logic works - the initial call to semget() in > InternalIpcSemaphoreCreate returns -1 and errno is set to ENOSPC - I > tested the patch on OpenBSD 7.0, it successfully recycles sem's after > previous "pkill -6 postgres". Verified it with 'ipcs -

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread Mikhail
On Sun, Oct 17, 2021 at 10:52:38AM -0400, Tom Lane wrote: > Mikhail writes: > > On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: > >> AFAICS, this patch could be disastrous. What if the semaphore in > >> question belongs to some other postmaster? > > > Does running more than one postmas

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread Tom Lane
Mikhail writes: > On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: >> AFAICS, this patch could be disastrous. What if the semaphore in >> question belongs to some other postmaster? > Does running more than one postmaster on the same PGDATA is supported at > all? Currently seed for the s

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread Mikhail
On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: > mp39...@gmail.com writes: > > We might be in situation when we have "just enough" semaphores in the > > system limit to start but previously crashed unexpectedly, in that case > > we won't be able to start again - semget() will return ENOS

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread Tom Lane
mp39...@gmail.com writes: > We might be in situation when we have "just enough" semaphores in the > system limit to start but previously crashed unexpectedly, in that case > we won't be able to start again - semget() will return ENOSPC, despite > the semaphores are ours, and we can recycle them, so

[PATCH] Make ENOSPC not fatal in semaphore creation

2021-10-17 Thread mp39590
From: Mikhail We might be in situation when we have "just enough" semaphores in the system limit to start but previously crashed unexpectedly, in that case we won't be able to start again - semget() will return ENOSPC, despite the semaphores are ours, and we can recycle them, so check this situat