Re: [HACKERS] kqueue

On Wed, Jan 29, 2020 at 11:54 AM Thomas Munro wrote: > If there are no further objections, I'm planning to commit this sooner > rather than later, so that it gets plenty of air time on developer and > build farm machines. If problems are discovered on a particular > platform, there's a pretty

Re: [HACKERS] kqueue

On Sat, Jan 25, 2020 at 11:29:11AM +1300, Thomas Munro wrote: > On Thu, Jan 23, 2020 at 9:38 AM Rui DeSousa wrote: > > On Jan 22, 2020, at 2:19 PM, Tom Lane wrote: > >> It's certainly possible that to see any benefit you need stress > >> levels above what I can manage on the small box I've got

Re: [HACKERS] kqueue

On Sat, Jan 25, 2020 at 11:29 AM Thomas Munro wrote: > On Thu, Jan 23, 2020 at 9:38 AM Rui DeSousa wrote: > > Here is two charts comparing a patched and unpatched system. > > These systems are very large and have just shy of thousand > > connections each with averages of 20 to 30 active queries

Re: [HACKERS] kqueue

On Thu, Jan 23, 2020 at 9:38 AM Rui DeSousa wrote: > On Jan 22, 2020, at 2:19 PM, Tom Lane wrote: >> It's certainly possible that to see any benefit you need stress >> levels above what I can manage on the small box I've got these >> OSes on. Still, it'd be nice if a performance patch could

Re: [HACKERS] kqueue

I wrote: > This just says it doesn't lock up, of course. I've not attempted > any performance-oriented tests. I've now done some light performance testing -- just stuff like pgbench -S -M prepared -c 20 -j 20 -T 60 bench I cannot see any improvement on either FreeBSD 12 or NetBSD 8.1, either as

Re: [HACKERS] kqueue

Matteo Beccati writes: > On 22/01/2020 17:06, Tom Lane wrote: >> Matteo Beccati writes: >>> I had a NetBSD 8.0 VM lying around and I gave the patch a spin on latest >>> master. >>> With the kqueue patch, a pgbench -c basically hangs the whole postgres >>> instance. Not sure if it's a kernel

Re: [HACKERS] kqueue

On 22/01/2020 17:06, Tom Lane wrote: > Matteo Beccati writes: >> On 21/01/2020 02:06, Thomas Munro wrote: >>> [1] >>> https://www.postgresql.org/message-id/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com > >> I had a NetBSD 8.0 VM lying around and I gave the patch a

Re: [HACKERS] kqueue

Matteo Beccati writes: > On 21/01/2020 02:06, Thomas Munro wrote: >> [1] >> https://www.postgresql.org/message-id/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com > I had a NetBSD 8.0 VM lying around and I gave the patch a spin on latest > master. > With the kqueue

Re: [HACKERS] kqueue

Hi, On 21/01/2020 02:06, Thomas Munro wrote: > [1] > https://www.postgresql.org/message-id/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com I had a NetBSD 8.0 VM lying around and I gave the patch a spin on latest master. With the kqueue patch, a pgbench -c basically

Re: [HACKERS] kqueue

On Tue, Jan 21, 2020 at 8:03 AM Tom Lane wrote: > I observe very similar behavior on FreeBSD/amd64 12.0-RELEASE-p12, > so it's not just macOS. Thanks for testing. Fixed by handling the new exit_on_postmaster_death flag from commit cfdf4dc4. On Tue, Jan 21, 2020 at 5:55 AM Tom Lane wrote: >

Re: [HACKERS] kqueue

I wrote: > Peter Eisentraut writes: >> I took this patch for a quick spin on macOS. The result was that the >> test suite hangs in the test src/test/recovery/t/017_shm.pl. I didn't >> see any mentions of this anywhere in the thread, but that test is newer >> than the beginning of this

Re: [HACKERS] kqueue

On Tue, Jan 21, 2020 at 2:34 AM Peter Eisentraut wrote: > I took this patch for a quick spin on macOS. The result was that the > test suite hangs in the test src/test/recovery/t/017_shm.pl. I didn't > see any mentions of this anywhere in the thread, but that test is newer > than the beginning

Re: [HACKERS] kqueue

Thomas Munro writes: > [ 0001-Add-kqueue-2-support-for-WaitEventSet-v13.patch ] I haven't read this patch in any detail, but a couple quick notes: * It needs to be rebased over the removal of pg_config.h.win32 --- it should be touching Solution.pm instead, I believe. * I'm disturbed by the

Re: [HACKERS] kqueue

Peter Eisentraut writes: > I took this patch for a quick spin on macOS. The result was that the > test suite hangs in the test src/test/recovery/t/017_shm.pl. I didn't > see any mentions of this anywhere in the thread, but that test is newer > than the beginning of this thread. Can anyone

Re: [HACKERS] kqueue

On 2019-12-20 01:26, Thomas Munro wrote: It's still my intention to get this committed eventually, but I got a bit frazzled by conflicting reports on several operating systems. For FreeBSD, performance was improved in many cases, but there were also some regressions that seemed to be related to

Re: [HACKERS] kqueue

Thanks Thomas, Just a quick update. I just deployed this patch into a lower environment yesterday running FreeBSD 12.1 and PostgreSQL 11.6. I see a significant reduction is CPU/system load from load highs of 500+ down to the low 20’s. System CPU time has been reduced to practically nothing.

Re: [HACKERS] kqueue

On Fri, Dec 20, 2019 at 1:26 PM Thomas Munro wrote: > On Fri, Dec 20, 2019 at 12:41 PM Rui DeSousa wrote: > > PostgreSQL 11 BTW, PostgreSQL 12 has an improvement that may be relevant for your case: it suppresses a bunch of high frequency reads on the "postmaster death" pipe in some scenarios,

Re: [HACKERS] kqueue

On Fri, Dec 20, 2019 at 12:41 PM Rui DeSousa wrote: > I’m instrested in the kqueue patch and would like to know its current state > and possible timeline for inclusion in the base code. I have several large > FreeBSD systems running PostgreSQL 11 that I believe currently displays this >

Re: [HACKERS] kqueue

> On Apr 10, 2018, at 9:05 PM, Thomas Munro > wrote: > > On Wed, Dec 6, 2017 at 12:53 AM, Thomas Munro > wrote: >> On Thu, Jun 22, 2017 at 7:19 PM, Thomas Munro >> wrote: >>> I don't plan to resubmit this patch myself, but I was doing some >>> spring cleaning and rebasing today and I

Re: [HACKERS] kqueue

On Tue, Oct 2, 2018 at 6:28 AM Andres Freund wrote: > On 2018-10-01 19:25:45 +0200, Matteo Beccati wrote: > > On 01/10/2018 01:09, Thomas Munro wrote: > > > I don't know why the existence of the kqueue should make recvfrom() > > > slower on the pgbench side. That's probably something to look

Re: [HACKERS] kqueue

On 2018-10-01 19:25:45 +0200, Matteo Beccati wrote: > On 01/10/2018 01:09, Thomas Munro wrote: > > I don't know why the existence of the kqueue should make recvfrom() > > slower on the pgbench side. That's probably something to look into > > off-line with some FreeBSD guru help. Degraded

Re: [HACKERS] kqueue

On Sun, Sep 30, 2018 at 9:49 PM Matteo Beccati wrote: > On 30/09/2018 04:36, Thomas Munro wrote: > > On Sat, Sep 29, 2018 at 7:51 PM Matteo Beccati wrote: > >> Out of curiosity, I've installed FreBSD on an identically specced VM, > >> and the select benchmark was ~75k tps for kqueue vs ~90k tps

Re: [HACKERS] kqueue

Hi Thomas, On 30/09/2018 04:36, Thomas Munro wrote: > On Sat, Sep 29, 2018 at 7:51 PM Matteo Beccati wrote: >> Out of curiosity, I've installed FreBSD on an identically specced VM, >> and the select benchmark was ~75k tps for kqueue vs ~90k tps on >> unpatched master, so maybe there's something

Re: [HACKERS] kqueue

On Sat, Sep 29, 2018 at 7:51 PM Matteo Beccati wrote: > On 28/09/2018 14:19, Thomas Munro wrote: > > On Fri, Sep 28, 2018 at 11:09 AM Andres Freund wrote: > >> On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: > >>> Matteo Beccati reported a 5-10% performance drop on a > >>> low-end Celeron

Re: [HACKERS] kqueue

On 28/09/2018 14:19, Thomas Munro wrote: > On Fri, Sep 28, 2018 at 11:09 AM Andres Freund wrote: >> On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: >>> Matteo Beccati reported a 5-10% performance drop on a >>> low-end Celeron NetBSD box which we have no explanation for, and we >>> have no

Re: [HACKERS] kqueue

On Fri, Sep 28, 2018 at 11:09 AM Andres Freund wrote: > On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: > > Matteo Beccati reported a 5-10% performance drop on a > > low-end Celeron NetBSD box which we have no explanation for, and we > > have no reports from server-class machines on that OS --

Re: [HACKERS] kqueue

Hi Thomas, On 28/09/2018 00:55, Thomas Munro wrote: > I would like to commit this patch for PostgreSQL 12, based on this > report. We know it helps performance on macOS developer machines and > big FreeBSD servers, and it is the right kernel interface for the job > on principle. Matteo Beccati

Re: [HACKERS] kqueue

Hi, On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: > On Tue, May 22, 2018 at 12:07 PM Thomas Munro > wrote: > > On Mon, May 21, 2018 at 7:27 PM, Mateusz Guzik wrote: > > > I have benchmarked the change on a FreeBSD box and found an big > > > performance win once the number of clients goes

Re: [HACKERS] kqueue

On Tue, May 22, 2018 at 12:07 PM Thomas Munro wrote: > On Mon, May 21, 2018 at 7:27 PM, Mateusz Guzik wrote: > > I have benchmarked the change on a FreeBSD box and found an big > > performance win once the number of clients goes beyond the number of > > hardware threads on the target machine.

Re: [HACKERS] kqueue

On Mon, May 21, 2018 at 7:27 PM, Mateusz Guzik wrote: > I have benchmarked the change on a FreeBSD box and found an big > performance win once the number of clients goes beyond the number of > hardware threads on the target machine. For smaller number of clients > the win was

Re: [HACKERS] kqueue

On Mon, May 21, 2018 at 9:03 AM, Thomas Munro wrote: > On Wed, Apr 11, 2018 at 1:05 PM, Thomas Munro > wrote: > > I heard through the grapevine of some people currently investigating > > performance problems on busy FreeBSD systems,

Re: [HACKERS] kqueue

On Wed, Apr 11, 2018 at 1:05 PM, Thomas Munro wrote: > I heard through the grapevine of some people currently investigating > performance problems on busy FreeBSD systems, possibly related to the > postmaster pipe. I suspect this patch might be a part of the

Re: [HACKERS] kqueue

On Wed, Dec 6, 2017 at 12:53 AM, Thomas Munro wrote: > On Thu, Jun 22, 2017 at 7:19 PM, Thomas Munro > wrote: >> I don't plan to resubmit this patch myself, but I was doing some >> spring cleaning and rebasing today and I figured it

Re: [HACKERS] kqueue

On Thu, Jun 22, 2017 at 7:19 PM, Thomas Munro wrote: > I don't plan to resubmit this patch myself, but I was doing some > spring cleaning and rebasing today and I figured it might be worth > quietly leaving a working patch here just in case anyone from the > various