Re: move some bitmapset.c macros to bitmapset.h

2022-12-05 Thread John Naylor
On Mon, Dec 5, 2022 at 9:33 PM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2022-Dec-05, John Naylor wrote: > >> -#define WORDNUM(x) ((x) / BITS_PER_BITMAPWORD) > >> -#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD) > > > In this locatio

move some bitmapset.c macros to bitmapset.h

2022-12-05 Thread John Naylor
versions there makes sense. But that way is not a clear win, so I didn't go that far. [1] https://www.postgresql.org/message-id/CAFBsxsHgP5LP9q%3DRq_3Q2S6Oyut67z%2BVpemux%2BKseSPXhJF7sg%40mail.gmail.com -- John Naylor EDB: http://www.enterprisedb.com src/backend/nodes/bitmapset.c | 24

Re: slab allocator performance issues

2022-12-05 Thread John Naylor
On Mon, Dec 5, 2022 at 3:02 PM David Rowley wrote: > > On Fri, 11 Nov 2022 at 22:20, John Naylor wrote: > > #define SLAB_FREELIST_COUNT ((1<<3) + 1) > > index = (freecount & (SLAB_FREELIST_COUNT - 2)) + (freecount != 0); > > Doesn't this create a sort of round

Re: Optimizing Node Files Support

2022-12-02 Thread John Naylor
. > 5. Avoid useless attribution, ignoring the result of pg_strtok when it is unnecessary. Looks worse. -- John Naylor EDB: http://www.enterprisedb.com

Re: initdb: Refactor PG_CMD_PUTS loops

2022-12-01 Thread John Naylor
member. In other > cases, one might want to use PG_CMD_PRINTF() instead, to parametrize a > command, but that would require breaking up the loop or using > workarounds like replace_token(). This patch unwinds all that; it's > much simpler that way. +1, I can't think of a reason to ke

Re: [PATCH] Add native windows on arm64 support

2022-12-01 Thread John Naylor
MV8_CRC32C', false) +cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1) +have_optimized_crc = true + else That seems like a heavy-handed way to force it. Could we just use the same gating in the test program that the patch puts in the code of interest? Namely: +#ifndef _MSC_VER #include +#endif -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-01 Thread John Naylor
On Thu, Dec 1, 2022 at 3:03 PM Masahiko Sawada wrote: > > On Thu, Dec 1, 2022 at 4:00 PM John Naylor wrote: > > > > The bigger question in my mind is: Why is there an atomic variable in backend-local memory? > > Because I use the same radix_tree and radix_tree_control st

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-30 Thread John Naylor
On Wed, Nov 30, 2022 at 2:28 PM Masahiko Sawada wrote: > > On Fri, Nov 25, 2022 at 5:00 PM John Naylor > wrote: > > These hacks were my work, but I think we can improve that by having two versions of NODE_HAS_FREE_SLOT -- one for fixed- and one for variable-sized nodes. F

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-30 Thread John Naylor
cated by aset.c is 4-bytes aligned so these atomic > variables are not always 8-bytes aligned. Is there any way to enforce > 8-bytes aligned memory allocations in 32-bit machines? The bigger question in my mind is: Why is there an atomic variable in backend-local memory? --

Re: Improve performance of pg_strtointNN functions

2022-11-30 Thread John Naylor
imal literal patch needs to be "immediately" faster? If doing this first leads to less code churn, that's another consideration, but you haven't made that argument. -- John Naylor EDB: http://www.enterprisedb.com

Re: Prefetch the next tuple's memory during seqscans

2022-11-30 Thread John Naylor
ms Testing a5 latency average = 1737.173 ms Testing a6 latency average = 1828.239 ms Testing a7 latency average = 1920.909 ms Testing a8 latency average = 2036.922 ms [1] https://www.postgresql.org/message-id/CAFBsxsHqmH_S%3D4apc5agKsJsF6xZ9f6NaH0Z83jUYv3EgySHfw%40mail.gmail.com --

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread John Naylor
There are a few things up in the air, so I'm coming back to this list to summarize and add a recent update: On Mon, Nov 14, 2022 at 7:59 PM John Naylor wrote: > > - See how much performance we actually gain from tagging the node kind. Needs a benchmark that has enough branch mispredicts

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-28 Thread John Naylor
UUM Oops. I meant to finish with "Since VACUUM doesn't perform deletion we didn't have an opportunity to detect this during that operation." -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-28 Thread John Naylor
sts didn't hit this. I do still intend to replace this code with something faster, but before I do so the tests should probably exercise the deletion paths more. Since VACUUM -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
e open questions about the design around shared memory. To keep unnecessary churn to a minimum, perhaps we should limit size class expansion to just one (or 5 total size classes) for the near future? -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
that by having two versions of NODE_HAS_FREE_SLOT -- one for fixed- and one for variable-sized nodes. For that to work, in "init-node" we'd need a branch to set fanout to zero for node256. That should be fine -- it already has to branch for memset'ing node128's indexes to 0xFF. -- John Naylor EDB: http://www.enterprisedb.com

Re: Non-decimal integer literals

2022-11-23 Thread John Naylor
Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/path/to/srcdir/src/include -I/path/to/builddir/src/include -D_GNU_SOURCE

Re: Non-decimal integer literals

2022-11-22 Thread John Naylor
On Tue, Nov 22, 2022 at 8:36 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > On 15.11.22 11:31, Peter Eisentraut wrote: > > On 14.11.22 08:25, John Naylor wrote: > >> Regarding the patch, it looks good overall. My only suggestion would > >>

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread John Naylor
On Wed, Nov 23, 2022 at 5:00 AM David Rowley wrote: > > On Thu, 3 Nov 2022 at 22:09, John Naylor wrote: > > I tried a similar test, but with text fields of random length, and there is improvement here: > > Thank you for testing that. Can you share which CPU this was on? That

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-21 Thread John Naylor
On Mon, Nov 21, 2022 at 3:43 PM Masahiko Sawada wrote: > > On Mon, Nov 21, 2022 at 4:20 PM John Naylor > wrote: > > Assuming the smallest node is fixed size (i.e. fanout/capacity member not part of the common set, so only part of variable-sized nodes), 3 has a nice property: no

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-20 Thread John Naylor
node4: 5 + 4+(7) + 4*8 = 48 bytes node3: 5 + 3 + 3*8 = 32 -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-20 Thread John Naylor
On Fri, Nov 18, 2022 at 8:20 PM Masahiko Sawada wrote: > > On Thu, Nov 17, 2022 at 12:24 AM Masahiko Sawada wrote: > > > > On Wed, Nov 16, 2022 at 4:39 PM John Naylor > > wrote: > > > That means my idea for the pointer struct might have some problems, at leas

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-18 Thread John Naylor
<24)). Ugh, sign extension, brain fade on my part. Thanks, I'm glad there was a straightforward explanation. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-17 Thread John Naylor
I'm thinking of it: rt_alloc_node should be always-inlined and the memset done separately (i.e. not *AllocZero). That way the compiler should be able generate more efficient zeroing code for smaller nodes. I'll test the numbers on this sometime in the future. -- John Naylor EDB: http://www.enterpri

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
On Wed, Nov 16, 2022 at 12:33 PM Masahiko Sawada wrote: > > On Wed, Nov 16, 2022 at 1:46 PM John Naylor > wrote: > > > > > > On Tue, Nov 15, 2022 at 11:59 AM Masahiko Sawada wrote: > > > Thanks! Please let me know if there is something I can help with. &

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
On Wed, Nov 16, 2022 at 11:46 AM John Naylor wrote: > > > On Tue, Nov 15, 2022 at 11:59 AM Masahiko Sawada wrote: > > Thanks! Please let me know if there is something I can help with. > > I didn't get very far because the tests fail on 0004 in rt_verify_node: > > TRAP

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
src/backend/lib/radixtree.c", Line: 2186, PID: 18242 -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-14 Thread John Naylor
e duplicated. I'll set the patch to "waiting on author", but in this case the author is me. -- John Naylor EDB: http://www.enterprisedb.com

Re: Non-decimal integer literals

2022-11-13 Thread John Naylor
s there an ETA yet? Also, it's not this patch's job to do it, but there are at least a half dozen places that open-code turning a hex char into a number. It might be a good easy "todo item" to unify that. -- John Naylor EDB: http://www.enterprisedb.com

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-13 Thread John Naylor
nboxing of the redirect headers. To my mind the main point of detecting this case is to save memory, so if that's not possible/convenient, special-casing doesn't seem worth it. - Assert((char *) chunk > (char *) block); + Assert((char *) chunk >= (char *) block); Is this related or independent? -- John Naylor EDB: http://www.enterprisedb.com

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-11-11 Thread John Naylor
On Fri, Nov 11, 2022 at 2:12 PM Michael Paquier wrote: > > On Fri, Nov 11, 2022 at 11:53:08AM +0530, Bharath Rupireddy wrote: > > On Fri, Nov 11, 2022 at 11:14 AM John Naylor < john.nay...@enterprisedb.com> wrote: > >> Was there supposed to be an attachment here? >

Re: slab allocator performance issues

2022-11-11 Thread John Naylor
nse with both freelist_shift and the loop that computes it? -- John Naylor EDB: http://www.enterprisedb.com

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-11-10 Thread John Naylor
ents don't need to be that long, so I have tweaked these to be > minimal, and updated a few more areas. Note that this has been split > into two commits: one to introduce the new routine in file_utils.c and > a second for the switch in walmethods.c. Was there supposed to be an attachment here?

Re: Direct I/O

2022-11-09 Thread John Naylor
but if allocating pages on the heap is hot enough, maybe we should consider raising that limit. Although then, aligned-to-4kB requests would result in 16kB chunks requested unless a different allocator was used. -- John Naylor EDB: http://www.enterprisedb.com

Re: Add palloc_aligned() to allow arbitrary power of 2 memory alignment

2022-11-07 Thread John Naylor
undary * * Note: we rely on zeroing to initialize all the dlist headers correctly */ sz = sizeof(CatCache) + PG_CACHE_LINE_SIZE; cp = (CatCache *) CACHELINEALIGN(palloc0(sz)); -- John Naylor EDB: http://www.enterprisedb.com

Re: Improve logging when using Huge Pages

2022-11-06 Thread John Naylor
e > if it tries and fails? I think the best thing to do is change huge_pages='on' to log a WARNING and fallback to regular pages if the mapping fails. That way, both dev and prod can keep the same settings, since 'on' will have both visibility and robustness. I don't see a good reason to refuse t

Re: remap the .text segment into huge pages at run time

2022-11-06 Thread John Naylor
d in https://releases.llvm.org/14.0.0/tools/lld/docs/index.html it says I can run this and it should show “Linker: LLD”, but that doesn't appear for me: $ readelf --string-dump .comment inst-perf/bin/postgres String dump of section '.comment': [ 0] GCC: (GNU) 12.2.1 20220819 (Red Hat 12.2.1-2) -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-05 Thread John Naylor
d shared radix tree functionality. It does help, good to see this. -- John Naylor EDB: http://www.enterprisedb.com

Re: remap the .text segment into huge pages at run time

2022-11-04 Thread John Naylor
er only > maps the necessary part, not all the zero padding. > > This means that if we were to issue a MADV_COLLAPSE, we can before it do an > mremap() to increase the length of the mapping. I see, interesting. What location are you passing for madvise() and mremap()? The beginning of the segment (for me has .init/.plt) or an aligned boundary within .text? -- John Naylor EDB: http://www.enterprisedb.com

Re: Incorrect include file order in guc-file.l

2022-11-03 Thread John Naylor
On Fri, Nov 4, 2022 at 5:42 AM Michael Paquier wrote: > > The CI is able to complete without it. Would you mind if it is > removed? If you don't want us to poke more at the bear, that's a nit > so leaving things as they are is also fine by me. I've removed it. -- John Nayl

Re: Incorrect include file order in guc-file.l

2022-11-03 Thread John Naylor
On Thu, Nov 3, 2022 at 6:40 PM Michael Paquier wrote: > > On Thu, Nov 03, 2022 at 12:40:19PM +0700, John Naylor wrote: > > On Wed, Nov 2, 2022 at 1:01 PM Julien Rouhaud wrote: > >> Agreed, it's apparently an oversight in dac048f71eb. +1 for the patch. > > &

Re: Prefetch the next tuple's memory during seqscans

2022-11-03 Thread John Naylor
s Testing a6 latency average = 1149.418 ms Testing a7 latency average = 1213.134 ms Testing a8 latency average = 1282.760 ms -- John Naylor EDB: http://www.enterprisedb.com

Re: Incorrect include file order in guc-file.l

2022-11-02 Thread John Naylor
gt; there. > > Agreed, it's apparently an oversight in dac048f71eb. +1 for the patch. I've pushed this, thanks! -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-02 Thread John Naylor
at's the simplest thing we can do to make that possible? Other use cases can change to more sophisticated schemes if need be. -- John Naylor EDB: http://www.enterprisedb.com

Re: Incorrect include file order in guc-file.l

2022-11-02 Thread John Naylor
gt; there. > > Agreed, it's apparently an oversight in dac048f71eb. +1 for the patch. Yeah. +1, thanks. -- John Naylor EDB: http://www.enterprisedb.com

remap the .text segment into huge pages at run time

2022-11-02 Thread John Naylor
] https://www.cs.rochester.edu/u/sandhya/papers/ispass19.pdf (paper: "On the Impact of Instruction Address Translation Overhead") [2] https://twitter.com/AndresFreundTec/status/1214305610172289024 [3] https://github.com/intel/iodlr -- John Naylor EDB: http://www.enterprisedb.com From

Re: Documentation for building with meson

2022-10-27 Thread John Naylor
a naive user -- it would be good to try running everything documented here. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-26 Thread John Naylor
info_elem rt_node_kind_info[RT_NODE_KIND_COUNT] = { In the assembly, I see it checks if there is room in the node by doing a runtime lookup in this array, which is not constant. This might not be important just yet, because I want to base the check on the proposed node capacity instead, but I menti

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-26 Thread John Naylor
This would have to change when we implement shrinking of nodes, but might still be useful. + if (!rt_node_search_leaf(node, key, RT_ACTION_FIND, value_p)) + return false; + + return true; Maybe just "return rt_node_search_leaf(...)" ? -- John Naylor EDB: http://www.enterprisedb.com

Re: fixing typo in comment for restriction_is_or_clause

2022-10-25 Thread John Naylor
On Tue, Oct 25, 2022 at 9:48 AM Richard Guo wrote: > > > On Tue, Oct 25, 2022 at 10:05 AM John Naylor wrote: >> >> It's perfectly clear and simple now, even if it doesn't win at "code golf". > > > Agree with your point. Do you think we can furt

Re: fixing typo in comment for restriction_is_or_clause

2022-10-24 Thread John Naylor
at the behavior is. > I also took the chance to simplify the code a little bit. It's perfectly clear and simple now, even if it doesn't win at "code golf". -- John Naylor EDB: http://www.enterprisedb.com

Re: Reducing the chunk header sizes on all memory context types

2022-10-24 Thread John Naylor
On Thu, Oct 20, 2022 at 1:55 AM Andres Freund wrote: > > Hi, > > On 2022-10-11 10:21:17 +0700, John Naylor wrote: > > On Tue, Oct 11, 2022 at 5:31 AM David Rowley wrote: > > > > > > The proposed patches in [1] do aim to make additional usages of the > &

Re: meson PGXS compatibility

2022-10-12 Thread John Naylor
nds on SSE4.2. We can't assume those currently. -- John Naylor EDB: http://www.enterprisedb.com

Re: introduce optimized linear search functions that return index of matching element

2022-10-11 Thread John Naylor
On Sat, Sep 17, 2022 at 12:29 PM Nathan Bossart wrote: > > On Fri, Sep 16, 2022 at 02:54:14PM +0700, John Naylor wrote: > > v6 demonstrates why this should have been put off towards the end. (more below) > > Since the SIMD code is fresh in my mind, I wanted to offer my

Re: Reducing the chunk header sizes on all memory context types

2022-10-10 Thread John Naylor
I have proposed in that thread a scheme to squeeze things into power-of-two sizes without wasting quite as much space. That's not a done deal, of course, but it could work today without adding memory management code. [1] https://www.postgresql.org/message-id/20220704220038.at2ane5xkymzzssb%40awork3.anarazel.de

Re: Remove unnecessary commas for goto labels

2022-10-10 Thread John Naylor
> You mean semi-colon? +1, and a quick regex later I don't see any other > occurrence. Interestingly, I did find that in C, some statement is needed after a label, even an empty one, otherwise it won't compile. That's not the case here, though, so pushed. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-09 Thread John Naylor
On Mon, Oct 10, 2022 at 12:16 PM John Naylor wrote: > Thanks for that! Now I can show clear results on some aspects in a simple way. The attached patches (apply on top of v6) Forgot the patchset... -- John Naylor EDB: http://www.enterprisedb.com radix-v6-addendum-jcn1.tar.gz Descript

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-09 Thread John Naylor
ple: we don't shrink at all now. :-) -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-06 Thread John Naylor
array_serach_ms -+--+-++---+--+- 100 | 10199040 | 18000 |171 | 107 | 827 | 1400 -- John Naylor EDB: http://www.enterprisedb.com From 43a50a385930ee340d0a3b003910c704a0ff342

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-06 Thread John Naylor
On Thu, Oct 6, 2022 at 2:53 PM Masahiko Sawada wrote: > > On Wed, Oct 5, 2022 at 6:40 PM John Naylor wrote: > > > > This wasn't the focus of your current email, but while experimenting with v6 I had another thought about local allocation: If we use the default slab block

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-10-05 Thread John Naylor
On Wed, Oct 5, 2022 at 1:46 PM Masahiko Sawada wrote: > > On Wed, Sep 28, 2022 at 12:49 PM Masahiko Sawada wrote: > > > > On Fri, Sep 23, 2022 at 12:11 AM John Naylor > > wrote: > > Yeah, node31 and node256 are bloated. We probably could use slab for > &g

Re: [RFC] building postgres with meson - v13

2022-09-29 Thread John Naylor
On Tue, Sep 27, 2022 at 2:41 PM John Naylor wrote: > > On Tue, Sep 27, 2022 at 2:06 AM Andres Freund wrote: > > > > On 2022-09-26 15:18:29 +0700, John Naylor wrote: > > Yea, it's /usr/local on x86-64, based on what was required to make macos CI > > work. I updat

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-28 Thread John Naylor
On Wed, Sep 28, 2022 at 1:18 PM John Naylor wrote: > [stuff about size classes] I kind of buried the lede here on one thing: If we only have 4 kinds regardless of the number of size classes, we can use 2 bits of the pointer for dispatch, which would only require 4-byte alignment. That sho

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-28 Thread John Naylor
air, but it's usually small compared to array size. Somewhat unrelated, we could still implement Andres' idea [1] to dispense with the isset array in inner nodes of the indirect array type (now node128), since we can just test if the pointer is null. [1] https://www.postgresql.org/message-id/20220704

Re: Insertion Sort Improvements

2022-09-27 Thread John Naylor
e serious about trying to improve insertion sort performance, the simple idea we discussed at the start of the thread is a much more modest step that has a good chance of justifying the time put into it. That is not to say it's easy, however, because testing is a non-trivial amount of work. --

Re: [RFC] building postgres with meson - v13

2022-09-27 Thread John Naylor
On Tue, Sep 27, 2022 at 2:06 AM Andres Freund wrote: > > On 2022-09-26 15:18:29 +0700, John Naylor wrote: > > Either way it doesn't exist on this machine. I was able to get a working > > build with > > > > /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig > &

Re: [RFC] building postgres with meson - v13

2022-09-26 Thread John Naylor
to get a working build with /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig (My homebrew install doesn't seem to have anything relevant for extra_include_dirs or extra_lib_dirs.) 2) Also, "ninja -v install" has the same line count as "ninja install" -- are there versions t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 11:46 AM John Naylor wrote: > One thing I want to try soon is storing fewer than 16/32 etc entries, so that the whole node fits comfortably inside a power-of-two allocation. That would allow us to use aset without wasting space for the smaller nodes, which would be fas

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 7:52 PM John Naylor wrote: > > > On Thu, Sep 22, 2022 at 1:26 PM Masahiko Sawada wrote: > > Good point. While keeping the chunks in the small nodes in sorted > > order is useful for visiting all keys in sorted order, additional > > branches and

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-22 Thread John Naylor
On Thu, Sep 22, 2022 at 1:26 PM Masahiko Sawada wrote: > > On Thu, Sep 22, 2022 at 1:46 PM John Naylor > wrote: > > While on the subject, I wonder how important it is to keep the chunks in the small nodes in sorted order. That adds branches and memmove calls, and is t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread John Naylor
On Thu, Sep 22, 2022 at 1:01 AM Nathan Bossart wrote: > > On Wed, Sep 21, 2022 at 01:17:21PM +0700, John Naylor wrote: > > > In short, this code needs to be lower level so that we still have full > > control while being portable. I will work on this, and also the rela

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-21 Thread John Naylor
On Tue, Sep 20, 2022 at 3:19 PM Masahiko Sawada wrote: > > On Fri, Sep 16, 2022 at 4:54 PM John Naylor > wrote: > > Here again, I'd rather put this off and focus on getting the "large > > details" in good enough shape so we can got towards integrating

Re: [RFC] building postgres with meson - v13

2022-09-20 Thread John Naylor
homebrew section is still labeled with "find MacPorts libraries". -- John Naylor EDB: http://www.enterprisedb.com

Re: Support for Rust

2022-09-19 Thread John Naylor
ndamental rethinking of memory management, error handling, > and some other things". That's pretty much a non-starter. Added "Rewrite the code in a different language" to "Features We Do Not Want" section of Wiki, referencing the two threads that came up: htt

Re: Typo in xact.c

2022-09-18 Thread John Naylor
On Fri, Sep 16, 2022 at 10:51 AM John Naylor wrote: > > On Fri, Sep 16, 2022 at 10:11 AM Kyotaro Horiguchi > wrote: > > > > The patch seems to me covering all occurances of PG_PROC as PGPROC. > > +1 since this hinders grep-ability. Pushed this. > >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-09-16 Thread John Naylor
On Fri, Sep 16, 2022 at 1:01 PM Masahiko Sawada wrote: > > On Mon, Aug 15, 2022 at 10:39 PM John Naylor > wrote: > > > > bool, buth = and <=. Should be pretty close. Also, i believe if you > > left this for last as a possible refactoring, it might save some work. v

Re: Typo in xact.c

2022-09-15 Thread John Naylor
overing all occurances of PG_PROC as PGPROC. +1 since this hinders grep-ability. > I found several uses of PG_PROC as (pg_catalog.)pg_proc, which is > quite confusing, too.. It's pretty obvious to me what that refers to in primnodes.h, although the capitalization of (some, but not all) catalo

Re: Inconsistencies in error messages

2022-09-15 Thread John Naylor
On Wed, Sep 14, 2022 at 5:25 PM John Naylor wrote: > Will commit this way unless there are objections. I forgot to mention yesterday, but this is done. -- John Naylor EDB: http://www.enterprisedb.com

Re: Cleaning up historical portability baggage

2022-09-15 Thread John Naylor
AM Thomas Munro wrote: > Remaining things from this thread: > * removing --disable-thread-safety > * removing those vestigial HAVE_XXX macros (one by one analysis and patches) > * making Unix sockets secure for Windows in tests -- John Naylor EDB: http://www.enterprisedb.com

Re: New strategies for freezing, advancing relfrozenxid early

2022-09-15 Thread John Naylor
On Wed, Sep 14, 2022 at 11:33 PM Peter Geoghegan wrote: > > On Wed, Sep 14, 2022 at 3:18 AM John Naylor > > Furthermore, it doesn't have to anticipate the maximum size, so there > > is no up front calculation assuming max-tuples-per-page, so it > > automatically u

Re: Inconsistencies in error messages

2022-09-14 Thread John Naylor
rker without shared memory access are not supported", + errmsg("background worker \"%s\": background workers without shared memory access are not supported", is a grammar error so worth backpatching, but the rest are cosmetic. Will commit this way unless there are objections. -- John Naylor EDB: http://www.enterprisedb.com

Re: New strategies for freezing, advancing relfrozenxid early

2022-09-14 Thread John Naylor
m size, so there is no up front calculation assuming max-tuples-per-page, so it automatically uses less memory for less demanding tables. (But +1 for changing that calculation for as long as we do have the single array.) -- John Naylor EDB: http://www.enterprisedb.com

Re: minimum perl version

2022-09-13 Thread John Naylor
On Wed, Sep 14, 2022 at 10:46 AM Tom Lane wrote: > > John Naylor writes: > > On Wed, Sep 14, 2022 at 6:47 AM Tom Lane wrote: > >> I've just switched longfin to use built-from-source perl 5.14.0. > > > In that case, here is a quick update with commit message. N

Re: failing to build preproc.c on solaris with sun studio

2022-09-13 Thread John Naylor
fair, our grammar changes much more often. One other possible deal-breaker of that is that it makes it more painful for forks to maintain additional syntax. -- John Naylor EDB: http://www.enterprisedb.com

Re: build remaining Flex files standalone

2022-09-13 Thread John Naylor
On Wed, Sep 14, 2022 at 6:10 AM Andres Freund wrote: > > On 2022-09-12 14:49:50 +0700, John Naylor wrote: > > CI builds fine. For v2 I only adjusted the commit message. I'll push > > in a couple days unless there are objections. > > Makes sense to me. Thanks for work

Re: minimum perl version

2022-09-13 Thread John Naylor
older Windows and Visual Studio versions, maybe it's low-enough risk defer the check to the Meson conversion? I understand our MSVC process will then go away much more quickly than autoconf... -- John Naylor EDB: http://www.enterprisedb.com From d8e241968395c552d667c1eb393cbaea12f0df6c Mon Sep 17 00

Re: minimum perl version

2022-09-13 Thread John Naylor
On Tue, Sep 13, 2022 at 5:53 PM John Naylor wrote: > > Until such time as that happens, here is a draft to require 5.14.2. As soon as I hit send, it occurred to me that we don't check the perl version on Windows, since (I seem to recall) 5.8.3 was too old to be an option on that platform.

Re: minimum perl version

2022-09-13 Thread John Naylor
so we can say > "5.14" in the docs without fine print. Until such time as that happens, here is a draft to require 5.14.2. -- John Naylor EDB: http://www.enterprisedb.com config/perl.m4 | 4 ++-- configure| 6 +++--- doc/src/sgm

Re: broken table formatting in psql

2022-09-13 Thread John Naylor
On Thu, Sep 8, 2022 at 12:39 PM John Naylor wrote: > > On Fri, Sep 2, 2022 at 3:19 PM Kyotaro Horiguchi > wrote: > > > > At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor > > wrote in > > > If there is any doubt about including all of Cf, we could also just &

Re: proposal: possibility to read dumped table's name from file

2022-09-13 Thread John Naylor
> readable. Although for this case, a handy written parser was trivial too. If the hand-written version is trivial, then we should prefer it. -- John Naylor EDB: http://www.enterprisedb.com

Re: broken table formatting in psql

2022-09-12 Thread John Naylor
he actual logic/data change is low-risk. The patch renames two files, which seems a bit much this late in the cycle. Maybe that's okay, but I'd like someone else to opine before doing so. -- John Naylor EDB: http://www.enterprisedb.com

Re: build remaining Flex files standalone

2022-09-12 Thread John Naylor
On Fri, Sep 9, 2022 at 12:18 PM John Naylor wrote: > > It seems gramparse.h isn't installed now? In any case, here's a patch > to move gramparse to the backend dir and stop symlinking/ installing > gram.h. Looking more closely at src/include/Makefile, this is incorrect -- all file

Re: broken table formatting in psql

2022-09-11 Thread John Naylor
. > > > I think it can be fixed just in v15 and master. This issue has no impact on > SQL. Well, if the regressions from v11 are not important enough to backpatch, there is not as much of a case to backpatch the full fix to v15 either. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Clarify the comments about varlena header encoding

2022-09-11 Thread John Naylor
ader is at a kind of "local maximum" -- minor adjustments are more likely to make it worse. To significantly improve clarity might require a larger rewriting, but I'm not personally interested in taking part in that. -- John Naylor EDB: http://www.enterprisedb.com

Re: proposal: possibility to read dumped table's name from file

2022-09-09 Thread John Naylor
uce derivation Filters ↳ 2: Filters Filter ↳ 1: ε • ↳ 5: C_EXCLUDE exclude_object pattern -- John Naylor EDB: http://www.enterprisedb.com

Re: Minimum bison and flex versions

2022-09-08 Thread John Naylor
On Fri, Sep 9, 2022 at 12:07 AM Andres Freund wrote: > > Hi, > > On 2022-09-06 13:32:32 +0700, John Naylor wrote: > > Here are autoconf-only patches to that effect. > > Looks like you did actually include src/tools/msvc as well :) Ah, in my head I meant "no patches f

Re: build remaining Flex files standalone

2022-09-08 Thread John Naylor
> > I found in my notes: > > * maybe gram.h and gramparse.h should not be installed > > So, yeah. ;-) It seems gramparse.h isn't installed now? In any case, here's a patch to move gramparse to the backend dir and stop symlinking/ installing gram.h. Confusingly, MSVC didn't seem to copy gram.h t

Re: [RFC] building postgres with meson - v12

2022-09-08 Thread John Naylor
On Wed, Sep 7, 2022 at 3:36 PM Alvaro Herrera wrote: > > On 2022-Sep-06, John Naylor wrote: > > > Note that the indentation hasn't changed. My thought there: perltidy > > will be run again next year, at which time it will be part of a listed > > whitespace-only co

Re: broken table formatting in psql

2022-09-07 Thread John Naylor
On Fri, Sep 2, 2022 at 3:19 PM Kyotaro Horiguchi wrote: > > At Fri, 2 Sep 2022 13:43:50 +0700, John Naylor > wrote in > > If there is any doubt about including all of Cf, we could also just > > add a branch in wchar.c to hard-code the 200B-200F range. > > If every wa

Re: failing to build preproc.c on solaris with sun studio

2022-09-07 Thread John Naylor
.) > > The approach with the make targets is better than a configure option, > because it allows you to build a narrow set of things during development > and then build everything for final confirmation, without having to > re-run configure. Also, it's less confusing for packagers. Another point is that the --with-FOO options are intended for building and linking with external library FOO. -- John Naylor EDB: http://www.enterprisedb.com

<    1   2   3   4   5   6   7   8   9   10   >