Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
Hi, On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > This brings up a question about the prefetching. We never had to have > this discussion for sequential scan streaming read because it didn't > (and still doesn't) do prefetching. But, if we push the streaming read > code down into the

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
reaming read) to review? Is it > v1-0001-revert-Generalize-relation-analyze-in-table-AM-in.patch? I think so. It's at least what I've been looking at. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
Hi, On 2024-04-10 16:05:21 -0400, Tom Lane wrote: > Andres Freund writes: > > I think it could exascerbate the issue. Parag reported ~7k connections on a > > 128 core machine. The buffer replacement logic in < 16 tries to lock the old > > and new lock partitions at once

Re: broken JIT support on Fedora 40

2024-04-10 Thread Andres Freund
ich is probably not needed. At the same time I can't get this error > from the verify pass with LLVM-12 or LLVM-15 (I have those at hand by > accident). To make it even more confusing I've found a few similar > examples in other projects, where this was really triggered by an issue > in LLVM [1]. I'm afraid that it actually has changed over time, I'm fairly sure that it was required at some point. Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
o it made sense for 041b96802e to create the stream in acquire_sample_rows() and have block_sampling_read_stream_next() be in analyze.c. But eventually that should be in access/heap/. Compared to 16, the state post the revert does tie analyze.c a bit closer to the internals of the AM than before, but I'm not sure the increase matters. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
greatly exacerbated by the issue fixed in a4adc31f690, once the waitqueue is long, the spinlock will be held for an extended amount of time. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
own, without coordination. That obviously isn't a reason to not fix the delay logic in lwlock.c. Looks like the wrong logic was introduced by me in commit 008608b9d51061b1f598c197477b3dc7be9c4a64 Author: Andres Freund Date: 2016-04-10 20:12:32 -0700 Avoid the use of a separate spinloc

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
ock usage pattern that is > causing this spot to become so heavily contended.) My suspicion is that it's a4adc31f690 which we only recently backpatched to < 16. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
h to manager our ratelimiting counters. What version of PG was this? I think it's much more likely that you're hitting a bug that caused a lot more contention inside lwlocks. That was fixed for 16+ in a4adc31f690 on 2022-11-20, but only backpatched to 12-15 on 2024-01-18. Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-10 Thread Andres Freund
he commit message it's hard to evaluate how this actually intended to be used. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-10 Thread Andres Freund
m that RNG unfairness is a real part of the problem here. Greetings, Andres Freund

Re: Speed up clean meson builds by ~25%

2024-04-09 Thread Andres Freund
Hi, On 2024-04-09 20:12:48 -0400, Tom Lane wrote: > Andres Freund writes: > > FWIW, just redefining mm_strdup() that way doesn't help much here either, > > even with an affected compiler. The gain increases substantially after > > simplifying unreserved_keyword etc to just u

Re: Speed up clean meson builds by ~25%

2024-04-09 Thread Andres Freund
clang-19 -O2 "plain"0m24.354s mm_strdup redefined0m23.741s +use default action 0m14.218s Greetings, Andres Freund

Re: Speed up clean meson builds by ~25%

2024-04-09 Thread Andres Freund
Hi, On 2024-04-09 19:00:41 -0400, Tom Lane wrote: > Andres Freund writes: > > I think we need to do something about the compile time of this file, even > > with > > gcc. Our main grammar already is an issue and stacking all the ecpg stuff on > > top makes it co

Re: Speed up clean meson builds by ~25%

2024-04-09 Thread Andres Freund
Hi, On 2024-04-09 15:33:10 -0700, Andres Freund wrote: > Which leads to a control flow graph with *many* incoming edges to the basic > block containing the function call to mm_strdup(), triggering a normally > harmless O(N^2) or such. With clang-16 -O2 there is a basic block with 3904

Re: Speed up clean meson builds by ~25%

2024-04-09 Thread Andres Freund
ng-16 -O2 from 345.188 to 158.084s clang-19 -O2 from 26.018s to 15.200s I suspect what is happening is that clang tries to optimize the number of calls to mm_strdup(), by separating the argument setup from the function call. Which leads to a control flow graph with *many* incoming edges to the basic block containing the function call to mm_strdup(), triggering a normally harmless O(N^2) or such. Greetings, Andres Freund

Re: Issue with the PRNG used by Postgres

2024-04-09 Thread Andres Freund
, I did some > statistical analysis on the RNG generator that PG uses to create the > backoff for the spin locks. ISTM that the fix here is to not use a spinlock for whatever the contention is on, rather than improve the RNG. Greetings, Andres Freund

Re: simplehash.h: "SH_SCOPE static" causes warnings

2024-04-09 Thread Andres Freund
tiation. Might make it slightly easier to notice obsoleted uses of simplehash. Greetings, Andres Freund

Re: Annoying build warnings from latest Apple toolchain

2024-04-09 Thread Andres Freund
Hi, Looks like I forgot to update the thread to note that I finally committed the remaining warning fixes. I had already fixed a bunch of others in upstream meson. commit a3da95deee38ee067b0bead639c830eacbe894d5 Author: Andres Freund Date: 2024-03-13 01:40:53 -0700 meson: macos: Avoid

Re: macOS Ventura won't generate core dumps

2024-04-09 Thread Andres Freund
n without a problem. Perhaps it's worth picking that up again? https://github.com/anarazel/postgres/tree/meson-rpath https://github.com/anarazel/postgres/commit/46f1963fee7525c3cc3837ef8423cbf6cb08d10a Greetings, Andres Freund

Re: Synchronizing slots from primary to standby

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 16:01:41 +0530, Amit Kapila wrote: > Pushed. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder=2024-04-08%2012%3A04%3A27 This unfortunately is a commit after commit 6f3d8d5e7cc Author: Amit Kapila Date: 2024-04-08 13:21:55 +0530 Fix the intermittent

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
On 2024-04-08 08:37:44 -0700, Andres Freund wrote: > On 2024-04-08 11:17:51 +0400, Pavel Borisov wrote: > > On Mon, 8 Apr 2024 at 03:25, Alexander Korotkov > > > I was under the impression there are not so many out-of-core table > > > AMs, which have non-dummy analysis

Re: Table AM Interface Enhancements

2024-04-08 Thread Andres Freund
the new API doesn't look well polished either, so it's not a question of a smoother transition or something like that. I don't think redesigning extension APIs at this stage of the release cycle makes sense. Greetings, Andres Freund

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Andres Freund
Hi, On 2024-04-08 09:26:09 -0400, Robert Haas wrote: > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > And maybe we need to think of a way to further mitigate this crush of > last minute commits. e.g. In the last week, you can't have more > feature commits, or more lines of insertions in

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 16:18:21 +1200, David Rowley wrote: > On Mon, 8 Apr 2024 at 16:08, Andres Freund wrote: > > I think visible would be ok, the serialization checks are IMO about > > visibility too. But if you'd prefer I'd also be ok with something like > > page_collect_tup

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 15:43:12 +1200, David Rowley wrote: > On Mon, 8 Apr 2024 at 15:13, Andres Freund wrote: > > Off-list Melanie suggested heap_page_collect_visible_tuples(). Given the > > separate callsites (making long names annoying) and the fact that it's > > really >

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 14:43:21 +1200, David Rowley wrote: > On Sun, 7 Apr 2024 at 19:30, Andres Freund wrote: > > Good call. Added and pushed. > > I understand you're already aware of the reference in the comment to > heapgetpage(), which no longer exists as of 44086b097. Yea, h

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Andres Freund
On 2024-04-07 21:41:23 -0400, Tom Lane wrote: > Andres Freund writes: > > I can't see a way it could hurt in the back branches, so I'm inclined to > > backpatch the pfree? > > +1 Done

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Andres Freund
t now. If coverity had started to complain after da41d71070d, I'd understand, but that was ~5 years ago. I can't see a way it could hurt in the back branches, so I'm inclined to backpatch the pfree? Greetings, Andres Freund

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Andres Freund
useful to be able to evict arbitrary blocks" that seems to describe things at least as well as "invalidate"? Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 an

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Andres Freund
nteresting for Makefile.global - they're msvc flags - because you can't use that with msvc. I'm also doubtful that it's worth supporting warning_level=3/everything, you end up with a completely flood of warnings that way. Greetings, Andres Freund

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
total_allocated == > context->mem_allocated"), File: "bump.c", Line: 808, PID: 7809 > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=turaco=2024-04-07%2022%3A42%3A54 What makes you think that's unrelated? To me that looks like the same issue? Greetings, Andres Freund

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Andres Freund
etty much free to call another callback there. Greetings, Andres Freund

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Andres Freund
gt; > Author: Melanie Plageman, Nazir Bilal Yavuz > Discussion: > https://postgr.es/m/flat/CAN55FZ0UhXqk9v3y-zW_fp4-WCp43V8y0A72xPmLkOM%2B6M%2BmJg%40mail.gmail.com The justification here seems somewhat odd. Sure, the previous commit stopped using BlockSampler_HasMore in acquire_sample_rows - but only because it was moved to block_sampling_streaming_read_next()? Greetings, Andres Freund

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
res reimplementing all of analyze.c. What am I missing here? Greetings, Andres Freund

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
ext->mem_allocated); > (gdb) p total_allocated > $1 = 8120 > (gdb) p context->mem_allocated > $2 = 8192 Greetings, Andres Freund

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread Andres Freund
ut you're storing a BlocktableEntry in a char[]. You can't just do that. Look at how we do that for e.g. PGAlignedblock. With the attached minimal fix, the tests pass again. Greetings, Andres Freund diff --git i/src/backend/access/common/tidstore.c w/src/backend/access/common/tidstore.c index e1

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 18:51:40 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 18:28, Andres Freund wrote: > > > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > >>> On 2024-04-06 Sa 20:49

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > > On 2024-04-06 Sa 20:49, Andres Freund wrote: > > >> That's probably unnecessary optimization, but it seems a tad silly to read > >> an > >> ent

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 12:07:22 +0700, John Naylor wrote: > Just in time ;-) The commit message should also have "reviewed by > Zhang Mingli" and "tested by Quan Zongliang", who shared results in a > thread for a withrawn CF entry with a similar idea but covering fewer > cases: Good call. Added and

Re: Improve heapgetpage() performance, overhead from serializable

2024-04-06 Thread Andres Freund
Hi, On 2024-01-22 13:01:31 +0700, John Naylor wrote: > On Mon, Jul 17, 2023 at 9:58 PM Andres Freund wrote: > > And 397->320ms > > for something as core as this, is imo worth considering on its own. > > Hi Andres, this interesting work seems to have fallen off the

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Andres Freund
Hi, On 2024-04-07 00:45:31 +0200, Jelte Fennema-Nio wrote: > On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote: > > The small regression for small results is still kinda visible, I haven't yet > > tested the patch downthread. > > Thanks a lot for the faster test script,

Re: Cluster::restart dumping logs when stop fails

2024-04-06 Thread Andres Freund
Hi, On 2024-04-07 00:19:35 +0200, Daniel Gustafsson wrote: > > On 6 Apr 2024, at 23:44, Andres Freund wrote: > > > It might be useful to print a few lines, but the whole log files can be > > several megabytes worth of output. > > The non-context aware fix would be

Cluster::restart dumping logs when stop fails

2024-04-06 Thread Andres Freund
he "normal" contents of regress_log* continue. Greetings, Andres Freund

Re: Synchronizing slots from primary to standby

2024-04-06 Thread Andres Freund
n a bunch of failures of this test locally. Greetings, Andres Freund

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Andres Freund
l results is still kinda visible, I haven't yet tested the patch downthread. Greetings, Andres Freund #!/bin/bash set -e dbname=postgres port=5440 host=/tmp host=localhost test_cases=( "100 100 100" # only 100 bytes "1024 10240 10"# 1Kb and 10Kb &

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Andres Freund
g about this from our side as well, I suspect. The times aren't great with gcc either, even if not as bad as with clang. Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-04-04 Thread Andres Freund
Hi, On 2024-04-04 19:00:00 +0300, Alexander Lakhin wrote: > 26.03.2024 10:59, Andres Freund wrote: > > Late, will try to look more in the next few days. > > > > AFAICS, last 027_streaming_regress.pl failures on calliphoridae, > culicidae, tamandua occurred be

Re: Streaming read-ready sequential scan code

2024-04-04 Thread Andres Freund
efault at some point - but we probably should overhaul the infrastructure first... Greetings, Andres Freund

Re: Security lessons from liblzma - libsystemd

2024-04-03 Thread Andres Freund
Hi, On 2024-04-03 17:58:55 -0400, Tom Lane wrote: > Magnus Hagander writes: > > On Wed, Apr 3, 2024 at 7:57 PM Andres Freund wrote: > >> Openssh has now integrated [1] a patch to remove the dependency on > >> libsystemd > >> for triggering service manager

Re: Streaming read-ready sequential scan code

2024-04-03 Thread Andres Freund
kes that effect worse in.a BAS_VACUUM strategy, > by taking away space that is effectively deferring WAL flushes, so I'd at > least like to double the size for if we use "/ 2" above. I think for VACUUM we should probably go a bit further. There's no comparable L1/L2 issue, because the per-buffer processing + WAL insertion is a lot more expensive, compared to a seqscan. I'd go or at lest 4x-8x. Greetings, Andres Freund

Re: Parent/child context relation in pg_get_backend_memory_contexts()

2024-04-03 Thread Andres Freund
etings, Andres Freund

Re: Security lessons from liblzma - libsystemd

2024-04-03 Thread Andres Freund
that can rely on the server actually having started up. Greetings, Andres Freund [1] https://bugzilla.mindrot.org/show_bug.cgi?id=2641 [2] https://www.freedesktop.org/software/systemd/man/devel/systemd.html#Readiness%20Protocol [3] https://www.freedesktop.org/software/systemd/man/devel

Re: Security lessons from liblzma

2024-03-31 Thread Andres Freund
an empty string but can be overridden? Greetings, Andres Freund

Re: Security lessons from liblzma

2024-03-30 Thread Andres Freund
parately downloadable archive. We already stopped including other generated files recently. > We shouldn't make the mistake of assuming that bad things can't happen to > us. +1 Greetings, Andres Freund

Re: Security lessons from liblzma

2024-03-29 Thread Andres Freund
hard to hide something sneaky. I comparison to that hiding something in configure.ac seems less likely to succeed IMO, that imo tends to be more scrutinized. And hiding just in configure directly wouldn't get you far, it'd just get removed when the committer or some other committer at a later time, regenerates configure. Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-03-26 Thread Andres Freund
look more in the next few days. Greetings, Andres Freund

Re: make dist using git archive

2024-03-26 Thread Andres Freund
Hi, On 2024-03-26 08:36:58 +0100, Peter Eisentraut wrote: > On 26.03.24 01:23, Andres Freund wrote: > > On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > > > Done and committed. > > > > This triggered a new warning for me: > > > > ../../../../../h

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-26 00:00:38 -0400, Tom Lane wrote: > Andres Freund writes: > > I think there must be some actual regression involved. The frequency of > > failures on HEAD vs failures on 16 - both of which run the tests > > concurrently > > via meson - is just vastly

Re: Recent 027_streaming_regress.pl hangs

2024-03-25 Thread Andres Freund
Hi, On 2024-03-20 17:41:45 -0700, Andres Freund wrote: > On 2024-03-14 16:56:39 -0400, Tom Lane wrote: > > Also, this is probably not > > helping anything: > > > >

Re: Sync scan & regression tests

2024-03-25 Thread Andres Freund
Hi, On 2024-03-24 11:28:12 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 19/09/2023 01:57, Andres Freund wrote: > >> On 2023-09-18 13:49:24 +0300, Heikki Linnakangas wrote: > >>> d) Copy fewer rows to the table in the test. If we copy only 6 rows

Re: RFC: Logging plan of the running query

2024-03-25 Thread Andres Freund
to do this? No. I think it's acceptable though. However it might be worth inventing an executor tree walker in a preliminary step. We have already quite a few switches over all plan nodes, which we could largely replace with a helper. Greetings, Andres Freund

Re: make dist using git archive

2024-03-25 Thread Andres Freund
Hi, On 2024-03-25 06:44:33 +0100, Peter Eisentraut wrote: > Done and committed. This triggered a new warning for me: ../../../../../home/andres/src/postgresql/meson.build:3422: WARNING: Project targets '>=0.54' but uses feature introduced in '0.55.0': Passing executable/found program object

Re: pgsql: Clean up role created in new subscription test.

2024-03-25 Thread Andres Freund
d what your patch is doing, but perhaps the issue is that it's seeing global objects concurrently created by another test? Greetings, Andres Freund

Re: Trying to build x86 version on windows using meson

2024-03-22 Thread Andres Freund
--cross-file) might give you a start: [properties] needs_exe_wrapper = false [binaries] c = 'cl' cpp = 'cl' ar = 'lib' windres = 'rc' [host_machine] system = 'windows' cpu_family = 'x86_64' cpu = 'x86_64' endian = 'little' Greetings, Andres Freund

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
Hi, On 2024-03-22 15:02:45 -0400, Robert Haas wrote: > On Fri, Mar 22, 2024 at 2:38 PM Andres Freund wrote: > > I wonder if we ought to make configure warn if it sees -Werror in CFLAGS - > > this is far from the first time somebody stumbling with -Werror. Including a > > few

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
to use Makefile.custom. Greetings, Andres Freund

Re: Cannot find a working 64-bit integer type on Illumos

2024-03-22 Thread Andres Freund
s an unrelated warning triggering the configure test to fail. We'd need to see config.log to see what that is. Greetings, Andres Freund

Re: An improved README experience for PostgreSQL

2024-03-21 Thread Andres Freund
On 2024-02-28 14:59:16 -0600, Nathan Bossart wrote: > On Wed, Feb 28, 2024 at 02:21:49PM -0600, Nathan Bossart wrote: > > On Wed, Feb 28, 2024 at 02:07:34PM -0600, Andrew Atkinson wrote: > >> I agree that starting with the direct conversion is reasonable. Markdown > >> "modernizes" the file using

Re: Trying to build x86 version on windows using meson

2024-03-21 Thread Andres Freund
s Use --disable-stdcall-fixup to disable these fixups /usr/bin/i686-w64-mingw32-ld: warning: resolving _DllUnregisterServer by linking to _DllUnregisterServer@0 Greetings, Andres Freund

Re: Recent 027_streaming_regress.pl hangs

2024-03-21 Thread Andres Freund
Hi, On 2024-03-20 17:41:45 -0700, Andres Freund wrote: > 2024-03-20 22:14:01.904 UTC [56343][client backend][6/1925:0] LOG: > connection authorized: user=bf database=postgres > application_name=027_stream_regress.pl > 2024-03-20 22:14:01.930 UTC [56343][client backend][6

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:41:47 -0700, Andres Freund wrote: > There's a lot of other animals on the same machine, however it's rarely fuly > loaded (with either CPU or IO). > > I don't think the test just being slow is the issue here, e.g. in the last > failing iteration > > [

Re: Recent 027_streaming_regress.pl hangs

2024-03-20 Thread Andres Freund
le coverage. I probably could reduce the number of animals using it though. Greetings, Andres Freund

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-21 11:02:27 +1300, David Rowley wrote: > On Thu, 21 Mar 2024 at 11:00, Andres Freund wrote: > > > > On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > > > First off this is on an ARM64 machine > > > > Uh, that's a fairly crucial bit - you

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
Hi, On 2024-03-20 17:49:14 -0400, Dave Cramer wrote: > On Wed, 20 Mar 2024 at 17:11, Andres Freund wrote: > > On 2024-03-20 16:14:23 -0400, Dave Cramer wrote: > > > I am getting the following error > > > > > > meson.build:1479:17: ERROR: Can not

Re: Trying to build x86 version on windows using meson

2024-03-20 Thread Andres Freund
s86` This is not enough information to debug anything. At the very least we need the exact steps performed to set up the build and meson-logs/meson-log.txt > The docs say it is possible to build postgres for x86. Are there specific > instructions ? It should work. Greetings, Andres Freund

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-17 Thread Andres Freund
Hi, On 2024-03-16 21:25:18 +0100, Tomas Vondra wrote: > On 3/16/24 20:12, Andres Freund wrote: > > That would address some of the worst behaviour, but it doesn't really seem > > to > > address the underlying problem of the two iterators being modified > > independently

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-16 Thread Andres Freund
Hi, On 2024-03-15 18:42:29 -0400, Melanie Plageman wrote: > On Fri, Mar 15, 2024 at 5:14 PM Andres Freund wrote: > > On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote: > > I spent a good amount of time looking into this with Melanie. After a bunch > > of > >

Re: Vectored I/O in bulk_write.c

2024-03-16 Thread Andres Freund
of combinding the two operations? I'm not dead-set against this, just not really convinced that it's a good idea to combine the operations. Greetings, Andres Freund

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-15 Thread Andres Freund
stantial - we're moving the two iterators independently from each other, in multiple processes, without useful locking. I'm inclined to think this is a bug we ought to fix in the backbranches. Greetings, Andres Freund

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-06 Thread Andres Freund
Hi, On March 6, 2024 9:06:50 AM GMT+01:00, John Naylor wrote: >On Wed, Mar 6, 2024 at 3:02 PM Masahiko Sawada wrote: >> >> On Wed, Mar 6, 2024 at 4:41 PM Andres Freund wrote: > >> > A few ARM buildfarm animals are complaining: >> > >> > https://bu

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-03-05 Thread Andres Freund
07%3A33%3A18 Greetings, Andres Freund

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-03-04 Thread Andres Freund
Hi, On 2024-03-04 08:47:11 -0300, Ranier Vilela wrote: > Does filling a memory area, one by one, with branches, need strong evidence > to prove that it is better than filling a memory area, all at once, without > branches? That's a bogus comparison: a) the memset version modifies the whole

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-03-04 Thread Andres Freund
in [1] (aset.c is > > just there for comparisons between slab and generation) > > > > The attached includes some additional tuning to generation.c: > > I've now pushed this. Thanks for working on all these, much appreciated! Greetings, Andres Freund

Re: Remove AIX Support (was: Re: Relation bulk write facility)

2024-02-29 Thread Andres Freund
Hi, On 2024-02-29 10:24:24 +0100, Michael Banck wrote: > On Thu, Feb 29, 2024 at 12:57:31AM -0800, Andres Freund wrote: > > On 2024-02-29 09:13:04 +0100, Michael Banck wrote: > > > The commit message says there is not a lot of user demand and that might > > > be r

Re: Remove AIX Support (was: Re: Relation bulk write facility)

2024-02-29 Thread Andres Freund
this. Then these users should have paid somebody to actually do maintenance work on the AIX support,o it doesn't regularly stand in the way of implementing various things. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-28 Thread Andres Freund
Hi, On 2024-02-27 12:59:14 -0800, Andres Freund wrote: > On 2024-02-27 15:45:45 -0500, Tom Lane wrote: > > Heikki Linnakangas writes: > > With AIX out of the picture, lapwing will be the only remaining > > animal testing MAXALIGN less than 8. That seems like a single

Re: Relation bulk write facility

2024-02-28 Thread Andres Freund
t; the > -- catalog, since packagers can override NAMEDATALEN to an odd number. > +-- (XXX: I'm not sure if any of the supported platforms have > MAXIMUM_ALIGNOF==8 and > +-- ALIGNOF_DOUBLE==4. Perhaps we should just require that > +-- ALIGNOF_DOUBLE==MAXIMUM_ALIGNOF) > -- > WITH check_columns AS ( > SELECT relname, attname, I agree, this should be an error, and we should then remove the test. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-27 Thread Andres Freund
animals? I had supposed that my faithful old PPC animal mamba > was helping to check this, but I see that under NetBSD it's > joined the ALIGNOF_DOUBLE==8 crowd. I can set up a i386 animal, albeit on an amd64 kernel. But I don't think the latter matters. Greetings, Andres Freund

Re: Relation bulk write facility

2024-02-24 Thread Andres Freund
tion. Let's just drop AIX. This isn't the only alignment issue we've found and the solution for those isn't so much a fix as forcing everyone to carefully only look into one direction and not notice the cliffs to either side. Greetings, Andres Freund [1] https://www.ibm.com/docs/en/SSGH2K_13.1.2/com.ibm.compilers.aix.doc/proguide.pdf

Re: locked reads for atomics

2024-02-24 Thread Andres Freund
tomic operation. And it's a bit simpler to analyze on which "side" of the read/write the barrier is needed. Greetings, Andres Freund

Re: locked reads for atomics

2024-02-24 Thread Andres Freund
the underlying variable isn't actually shared. In those cases using the other variants is a bug. The only use of pg_atomic_unlocked_write_u32() is temp-table buffers which share the data structure with the shared buffers case. Greetings, Andres Freund

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-02-22 Thread Andres Freund
; @@ -1061,6 +1072,16 @@ MemoryContextAlloc(MemoryContext context, Size size) > > context->isReset = false; > For a moment this made me wonder if we could move the isReset handling into the allocator slow paths as well - it's annoying to write that bit (and thus dirty the cacheline) over and ove. But it'd be somewhat awkward due to pre-allocated blocks. So that'd be a larger change better done separately. Greetings, Andres Freund

Re: Why is pq_begintypsend so slow?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 10:02:52 +0900, Sutou Kouhei wrote: > In <20240218200906.zvihkrs46yl2j...@awork3.anarazel.de> > "Re: Why is pq_begintypsend so slow?" on Sun, 18 Feb 2024 12:09:06 -0800, > Andres Freund wrote: > > >> [1] > >&

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 13:54:01 -0500, Joe Conway wrote: > On 2/19/24 13:13, Andres Freund wrote: > > On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > > > Couldn't we scale the rounding, e.g. allow small allocations as we do now, > > > but above some number always rou

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Andres Freund
Hi, On 2024-02-19 09:19:16 -0500, Joe Conway wrote: > On 2/18/24 15:35, Andres Freund wrote: > > On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > > > How many people set shared_buffers to something that's not a whole > > > number of GB these days? > > > >

Re: PGC_SIGHUP shared_buffers?

2024-02-18 Thread Andres Freund
Hi, On 2024-02-18 17:06:09 +0530, Robert Haas wrote: > On Sat, Feb 17, 2024 at 12:38 AM Andres Freund wrote: > > IMO the ability to *shrink* shared_buffers dynamically and cheaply is more > > important than growing it in a way, except that they are related of > > co

Re: Why is pq_begintypsend so slow?

2024-02-18 Thread Andres Freund
Hi, On 2024-02-18 17:38:09 +0900, Sutou Kouhei wrote: > In <20240218015955.rmw5mcmobt5hb...@awork3.anarazel.de> > "Re: Why is pq_begintypsend so slow?" on Sat, 17 Feb 2024 17:59:55 -0800, > Andres Freund wrote: > > > v3-0008-wip-make-in-out-send-

Re: Why is pq_begintypsend so slow?

2024-02-17 Thread Andres Freund
Hi, I was reminded of the patchset I had posted in this thread by https://postgr.es/m/679d5455cbbb0af667ccb753da51a475bae1eaed.camel%40cybertec.at On 2020-07-31 13:35:43 -0400, Robert Haas wrote: > On Fri, Jul 31, 2020 at 1:00 PM Andres Freund wrote: > > > Perhaps we could add a c

<    1   2   3   4   5   6   7   8   9   10   >