Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread John Naylor
thread? Aside from that, I plan to commit this week unless there is further bikeshedding. -- John Naylor EDB: http://www.enterprisedb.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-08 Thread John Naylor
, as far as I'm concerned. I'm interested in using a char version for json strings. > 3) Can pg_lfind32 return the index of the key found, for instance to > use it for setting/resetting the found element in the array? That was just discussed. It's slightly faster not to return an index.

Re: NAMEDATALEN increase because of non-latin languages

2022-08-10 Thread John Naylor
o varlen fields. I imagine keeping track of the attributes length will come up for some types and might be tricky. And before that, the catalog machinery will need some preprocessor tricks to declare pointers in the structs. -- John Naylor EDB: http://www.enterprisedb.com From 20fba44412e8ef1bb4cd5b051b

Re: [RFC] building postgres with meson

2022-08-10 Thread John Naylor
.c With a bit of work, we could probably get rid of those includes. See 27199058d98ef7f for one example. -- John Naylor EDB: http://www.enterprisedb.com

Re: use SSE2 for is_valid_ascii

2022-08-10 Thread John Naylor
ormance. Creating a zeroed register is just FOO PXOR FOO, which should get hoisted out of the (unrolled in this case) loop, and which a recent CPU will just map to a hard-coded zero in the register file, in which case the execution latency is 0 cycles. :-) -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2022-08-10 Thread John Naylor
On Thu, Aug 11, 2022 at 10:37 AM Tom Lane wrote: > > John Naylor writes: > > With a bit of work, we could probably get rid of those includes. See > > 27199058d98ef7f for one example. > > Yeah --- it would mean creating gram.h files for all the bison grammars > not j

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-10 Thread John Naylor
On Thu, Aug 11, 2022 at 4:46 AM Nathan Bossart wrote: > > On Wed, Aug 10, 2022 at 10:50:02AM +0700, John Naylor wrote: > > LGTM, let's see what the buildfarm thinks of 0001. > > Thanks! I haven't noticed any related buildfarm failures yet. I was waiting for all the Windows

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-08-15 Thread John Naylor
On Mon, Aug 15, 2022 at 12:39 PM Masahiko Sawada wrote: > > On Fri, Jul 22, 2022 at 10:43 AM Masahiko Sawada > wrote: > > > > On Tue, Jul 19, 2022 at 1:30 PM John Naylor > > wrote: > > > > > > > > > > > > On Tue, Jul 19, 2022 a

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-15 Thread John Naylor
ext AS with long as ( select repeat(description, 11) from pg_description ) select (select json_agg(row_to_json(long))::text as t from long) from generate_series(1, 100); VACUUM FREEZE long_json_as_text; select 1 from long_json_as_text where t::json is null; -- from Andrew upthread

Re: fix typos

2022-08-15 Thread John Naylor
On Fri, Aug 12, 2022 at 8:55 PM Tom Lane wrote: > > John Naylor writes: > > This is really a straw-man proposal, since I'm not volunteering to do > > the work, or suggest anybody else should do the same. That being the > > case, it seems we should just go a

Re: build remaining Flex files standalone

2022-08-16 Thread John Naylor
For v3, I addressed some comments and added .h files to the headerscheck exceptions. On Tue, Aug 16, 2022 at 1:11 AM Andres Freund wrote: > On 2022-08-13 15:39:06 +0700, John Naylor wrote: > > Here are the rest. Most of it was pretty straightforward, with the > >

Re: Cleaning up historical portability baggage

2022-08-12 Thread John Naylor
in the attached. I took a quick look to see if that would enable anything useful, but nothing stuck out. Aside: The MSVC builds don't report the Bison version that I can see, but otherwise it seems now the only non-Apple pre-3.0 animals are prion (2.7) and the three Sparc animals on Debian 7 (2.5).

Re: fix typos

2022-08-12 Thread John Naylor
I wrote: > On Thu, Aug 4, 2022 at 8:41 PM Tom Lane wrote: > > > > John Naylor writes: > > > > RepOriginId is a typedef for uint16, so this can't print the wrong answer, > > > but it is inconsistent with other uses. So it seems we don't need to > > &g

build remaining Flex files standalone

2022-08-12 Thread John Naylor
ld/intractable cruft from the TODO list, but we should also be sticking small nice-but-not-necessary things in there. That said, if this idea has any bearing on the guc_internal.h idea, it might be better dealt with now. -- John Naylor EDB: http://www.enterprisedb.com From d723ba14acf56fd432e9e263db937

Re: build remaining Flex files standalone

2022-08-13 Thread John Naylor
ill welcome comments on that. -- John Naylor EDB: http://www.enterprisedb.com From da2b610b8608e6759f5ed9cc32b487ea8e750ce4 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Fri, 12 Aug 2022 17:09:45 +0700 Subject: [PATCH v201 3/9] Build repl_scanner.c standalone --- src/backend/replication/Makefile |

Re: support for SSE2 intrinsics

2022-08-04 Thread John Naylor
On Thu, Aug 4, 2022 at 12:38 PM Masahiko Sawada wrote: > > I also think it's a good start. There is a typo in the commit message: > > s/hepler/helper/ > > The rest looks good to me. Fixed, and pushed, thanks to you both! I'll polish a small patch I have that actually uses thi

Re: fix typos

2022-08-04 Thread John Naylor
;depends" to "depend" here is incorrect. Maybe we > can say something like: > > the DEPENDS ON EXTENSION > extension_name action > > (I haven't tested whether this markup works.) Makes sense, I'll go make it happen. -- John Naylor EDB: http://www.enterprisedb.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-04 Thread John Naylor
"_int" vs "_byte"?). I'm not a fan of "its" as a variable name, and I'm curious what it's intended to convey. All the __m128i vars could technically be declared const, although I think it doesn't matter -- it's just a hint to the reader. Out of curiosity do we know how

Re: fix typos

2022-08-04 Thread John Naylor
002, the whitespace issue in the top comment in inval.c, I'm inclined to just change all the out-of-place tabs in a single commit, so we can add that to the list of whitespace commits. -- John Naylor EDB: http://www.enterprisedb.com

Re: build remaining Flex files standalone

2022-08-16 Thread John Naylor
included to e.g. define a PL transform or a new operator or > ... This would be solved by the %code requires thing, right? I believe it would. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Clarify the comments about varlena header encoding

2022-08-17 Thread John Naylor
AG_EXPANDED_RO = 2, VARTAG_EXPANDED_RW = 3, VARTAG_ONDISK = 18 } vartag_external; And I don't think the new comments referring to "third case", "first two cases", etc make it easier to follow. -- John Naylor EDB: http://www.enterprisedb.com

Re: fix typos

2022-08-17 Thread John Naylor
On Tue, Aug 16, 2022 at 8:48 AM John Naylor wrote: > > On Fri, Aug 12, 2022 at 8:55 PM Tom Lane wrote: > > > > John Naylor writes: > > > This is really a straw-man proposal, since I'm not volunteering to do > > > the work, or suggest anybody else should do

Re: build remaining Flex files standalone

2022-08-18 Thread John Naylor
On Wed, Aug 17, 2022 at 8:14 AM Andres Freund wrote: > > Hi, > > On 2022-08-16 17:41:43 +0700, John Naylor wrote: > > For v3, I addressed some comments and added .h files to the > > headerscheck exceptions. > > Thanks! > > > > /* > > * NB: incl

Re: build remaining Flex files standalone

2022-08-18 Thread John Naylor
ed to be defined in a semi-public header? If we do this > > > > in > > > > multiple files we'll end up with the danger of macro redefinition > > > > warnings. > > For v4, I #defined YYSTYPE Sorry for the misfire. Continuing on, I #defined YYSTYPE in cubesca

Re: build remaining Flex files standalone

2022-08-18 Thread John Naylor
ude "postgres.h" #include "cubedata.h" +#include "cube_internal.h" #include "utils/float.h" -- John Naylor EDB: http://www.enterprisedb.com

Re: Typo in pg_db_role_setting.h

2022-08-01 Thread John Naylor
didn't have such a comment, and it's kind of obvious what they're for. -- John Naylor EDB: http://www.enterprisedb.com

Re: support for SSE2 intrinsics

2022-08-02 Thread John Naylor
ly important to keep an eye on the buildfarm when this change is > committed. Thanks for checking! Here's a concrete patch for testing. -- John Naylor EDB: http://www.enterprisedb.com From 0e01cc8f5f1c2c27631953091a1d657c5e5486be Mon Sep 17 00:00:00 2001 From: John Naylor Date: Wed, 3 Aug 2022

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-02 Thread John Naylor
of those callers will rarely have more than several elements to search in the first place, or aren't performance-sensitive. -- John Naylor EDB: http://www.enterprisedb.com

Re: Fix unmatched file identifications

2022-08-08 Thread John Naylor
On Tue, Aug 9, 2022 at 8:58 AM Masahiko Sawada wrote: > I found that there are two .c and .h files whose identification in the > header comment doesn't match its actual path. > The attached small patch fixes them. Pushed, thanks! -- John Naylor EDB: http://www.enterprisedb.com

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-09 Thread John Naylor
rst for that reason. It's simpler than the qsort/qunique/binary search that was there before, so that's nice, but I've not tried to test performance. -- John Naylor EDB: http://www.enterprisedb.com From 1c89b9b7d3c71bb1a703caaf7c01c93bc9e2515f Mon Sep 17 00:00:00 2001 From: John Naylor Date: Tue

Re: failing to build preproc.c on solaris with sun studio

2022-08-07 Thread John Naylor
mes for slower animals, even if Noah reported this didn't help the issue upthread. I suspect it wouldn't slow down production use much since the output needs to be compiled anyway. -- John Naylor EDB: http://www.enterprisedb.com

remap the .text segment into huge pages at run time

2022-11-02 Thread John Naylor
] https://www.cs.rochester.edu/u/sandhya/papers/ispass19.pdf (paper: "On the Impact of Instruction Address Translation Overhead") [2] https://twitter.com/AndresFreundTec/status/1214305610172289024 [3] https://github.com/intel/iodlr -- John Naylor EDB: http://www.enterprisedb.com From

Re: Incorrect include file order in guc-file.l

2022-11-02 Thread John Naylor
gt; there. > > Agreed, it's apparently an oversight in dac048f71eb. +1 for the patch. Yeah. +1, thanks. -- John Naylor EDB: http://www.enterprisedb.com

Re: appendBinaryStringInfo stuff

2022-12-22 Thread John Naylor
, int limit) while (limit > 0 && *mbstr) { - int l = pg_mblen(mbstr); + int l; + + if (!IS_HIGHBIT_SET(*mbstr)) + l = 1; + else + l

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-22 Thread John Naylor
On Wed, Dec 21, 2022 at 3:09 PM Masahiko Sawada wrote: > > On Tue, Dec 20, 2022 at 3:09 PM John Naylor > wrote: > > https://www.postgresql.org/message-id/20220704211822.kfxtzpcdmslzm2dy%40awork3.anarazel.de > > > > I'm guessing the hash join case can afford to be p

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-22 Thread John Naylor
e clever, and I just thought of an additional concern: We're assuming behavior of the growth in size of new DSA segments, which could possibly change. Given how allocators are typically coded, though, it seems safe to assume that they'll at most double in size. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-26 Thread John Naylor
On Tue, Dec 27, 2022 at 12:14 AM Masahiko Sawada wrote: > > On Fri, Dec 23, 2022 at 8:47 PM John Naylor > wrote: > These 4 patches make sense to me.We can merge them into 0002 patch Okay, then I'll squash them when I post my next patch. > and I'll do similar changes for fun

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-10 Thread John Naylor
rest of the work involved in growing a node. Expressions like "if (idx < 0) return false;" return to an asserts-only variable, so in production builds, I would hope that branch gets elided (I haven't checked). I'm quite keen on making the smallest node padding-free, (since we don't yet have path compression or lazy path expansion), and this seems the way to get there. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-10 Thread John Naylor
lways use additional help there. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-10 Thread John Naylor
elow, be prepared to first discuss the value of the feature. Do not assume that you can select one, code it and then expect it to be committed. " -- John Naylor EDB: http://www.enterprisedb.com

Re: cutting down the TODO list thread

2023-01-10 Thread John Naylor
So, I had intended to spend some time on this at least three times a year. I've clearly failed at that, but now is as good a time as any to pick it back up again. Over in [1], Tom opined: > John Naylor writes: > > > "WARNING for Developers: Unfortunately this list doe

Re: SQL/JSON revisited

2023-01-10 Thread John Naylor
if I forget that detail, well it's not a big deal, it happens sometimes. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-04 Thread John Naylor
ide the function. The label TSS_BUILDRUNS has a similar style and also the following comment, so I will push this patch with a similar comment added unless someone wants to make a case for doing otherwise. * Note that mergeruns sets the correct state->status. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-11 Thread John Naylor
assert that we found a valid slot. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-11 Thread John Naylor
and okay to > come up with ideas and fail. Of course it is! A key skill in engineering is to fail as quickly as possible, preferably before doing any actual work. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-09 Thread John Naylor
On Thu, Jan 5, 2023 at 8:18 AM John Naylor wrote: > > The label TSS_BUILDRUNS has a similar style and also the following comment, so I will push this patch with a similar comment added unless someone wants to make a case for doing otherwise. > > * Note that mergeruns sets the c

Re: Small miscellaneus fixes (Part II)

2023-01-11 Thread John Naylor
Np->sign != '+' && IS_PLUS(Np->Num)) 5673 0 : Np->Num->flag &= ~NUM_F_PLUS; https://coverage.postgresql.org/src/backend/utils/adt/formatting.c.gcov.html I'm inclined to turn the dead unsets into asserts. -- John Naylor EDB: http://www.enterprisedb.com

Re: Small miscellaneus fixes (Part II)

2023-01-11 Thread John Naylor
On Thu, Jan 12, 2023 at 12:34 PM Justin Pryzby wrote: > > On Thu, Jan 12, 2023 at 12:15:24PM +0700, John Naylor wrote: > > On Fri, Dec 23, 2022 at 8:08 AM Justin Pryzby wrote: > > > > > Makes sense now (in your first message, you said that the problem was > > &g

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-12 Thread John Naylor
On Thu, Jan 12, 2023 at 12:44 PM Masahiko Sawada wrote: > > On Wed, Jan 11, 2023 at 12:13 PM John Naylor > wrote: > > > > On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada wrote: > I agree to keep this as a template. Okay, I'll squash the previous patch and work on c

Re: static assert cleanup

2022-12-09 Thread John Naylor
want ItemPointerData to be exactly 6 bytes. This is rather a + * random place to check, but there is no better place. Since the assert is no longer in a random function body, it seems we can remove the second sentence. -- John Naylor EDB: http://www.enterprisedb.com

Re: slab allocator performance issues

2022-12-11 Thread John Naylor
com -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-12 Thread John Naylor
On Fri, Dec 9, 2022 at 8:33 PM Masahiko Sawada wrote: > > On Fri, Dec 9, 2022 at 5:53 PM John Naylor wrote: > > > > I don't think that'd be very controversial, but I'm also not sure why we'd need 4MB -- can you explain in more detail what exactly we'd need so that the

Re: slab allocator performance issues

2022-12-05 Thread John Naylor
On Mon, Dec 5, 2022 at 3:02 PM David Rowley wrote: > > On Fri, 11 Nov 2022 at 22:20, John Naylor wrote: > > #define SLAB_FREELIST_COUNT ((1<<3) + 1) > > index = (freecount & (SLAB_FREELIST_COUNT - 2)) + (freecount != 0); > > Doesn't this create a sort of round

move some bitmapset.c macros to bitmapset.h

2022-12-05 Thread John Naylor
versions there makes sense. But that way is not a clear win, so I didn't go that far. [1] https://www.postgresql.org/message-id/CAFBsxsHgP5LP9q%3DRq_3Q2S6Oyut67z%2BVpemux%2BKseSPXhJF7sg%40mail.gmail.com -- John Naylor EDB: http://www.enterprisedb.com src/backend/nodes/bitmapset.c | 24

Re: New strategies for freezing, advancing relfrozenxid early

2022-12-15 Thread John Naylor
're *not* doing. Many times I had to pause and compute in my head what we're *doing*, i.e. the "scanning strategy". For example, I wonder if the VM strategies would be easier to read as: VMSNAP_SKIP_ALL_VISIBLE -> VMSNAP_SCAN_LAZY VMSNAP_SKIP_ALL_FROZEN -> VMSNAP_SCAN_EAGER VMSNAP_

Re: slab allocator performance issues

2022-12-20 Thread John Naylor
I reran my test from last week and got similar results. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-19 Thread John Naylor
n https://www.postgresql.org/message-id/20220704211822.kfxtzpcdmslzm2dy%40awork3.anarazel.de I'm guessing the hash join case can afford to be precise about memory because it must spill to disk when exceeding workmem. We don't have that design constraint. -- John Naylor EDB: http://www.enterprisedb.com

Re: code cleanups

2022-12-19 Thread John Naylor
On Thu, Nov 24, 2022 at 12:53 AM Tom Lane wrote: > > Justin Pryzby writes: > > Some modest cleanups I've accumulated. > 0004: Right, somebody injected code in a poorly chosen place > (yet another victim of the "add at the end" anti-pattern). I've pushed 000

Re: Rework of collation code, extensibility

2022-12-17 Thread John Naylor
ide, we don't really have a project style preference for this case. > nul-terminate -> null-terminate NUL is a common abbreviation for the zero byte (but not for zero pointers). See the ascii manpage. -- John Naylor EDB: http://www.enterprisedb.com

Re: move some bitmapset.c macros to bitmapset.h

2022-12-05 Thread John Naylor
On Mon, Dec 5, 2022 at 9:33 PM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2022-Dec-05, John Naylor wrote: > >> -#define WORDNUM(x) ((x) / BITS_PER_BITMAPWORD) > >> -#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD) > > > In this locatio

Re: move some bitmapset.c macros to bitmapset.h

2022-12-05 Thread John Naylor
eed, but it looks like it fits the bill. -- John Naylor EDB: http://www.enterprisedb.com src/backend/nodes/bitmapset.c| 36 ++-- src/include/nodes/bitmapset.h| 16 ++-- src/include/port/pg_bitutils.h | 31 +++ src/too

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-09 Thread John Naylor
uce contention on the tidstore. We could use such an array even for a single worker (always doing the same thing is simpler anyway). When the array fills up enough so that the next heap page *could* overflow it: Stop, insert into the store, and check the store's memory usage before continuing. -- John Naylor EDB: http://www.enterprisedb.com

Re: New strategies for freezing, advancing relfrozenxid early

2022-12-13 Thread John Naylor
n heap pages) would be to start with considering table-level debt measured the same way. -- John Naylor EDB: http://www.enterprisedb.com

Re: slab allocator performance issues

2022-12-14 Thread John Naylor
On Tue, Dec 13, 2022 at 7:50 AM David Rowley wrote: > > Thanks for testing the patch. > > On Mon, 12 Dec 2022 at 20:14, John Naylor wrote: > > While allocation is markedly improved, freeing looks worse here. The proportion is surprising because only about 2% of nodes are fre

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-17 Thread John Naylor
I'm thinking of it: rt_alloc_node should be always-inlined and the memset done separately (i.e. not *AllocZero). That way the compiler should be able generate more efficient zeroing code for smaller nodes. I'll test the numbers on this sometime in the future. -- John Naylor EDB: http://www.enterpri

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-20 Thread John Naylor
On Fri, Nov 18, 2022 at 8:20 PM Masahiko Sawada wrote: > > On Thu, Nov 17, 2022 at 12:24 AM Masahiko Sawada wrote: > > > > On Wed, Nov 16, 2022 at 4:39 PM John Naylor > > wrote: > > > That means my idea for the pointer struct might have some problems, at leas

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-18 Thread John Naylor
<24)). Ugh, sign extension, brain fade on my part. Thanks, I'm glad there was a straightforward explanation. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-21 Thread John Naylor
On Mon, Nov 21, 2022 at 3:43 PM Masahiko Sawada wrote: > > On Mon, Nov 21, 2022 at 4:20 PM John Naylor > wrote: > > Assuming the smallest node is fixed size (i.e. fanout/capacity member not part of the common set, so only part of variable-sized nodes), 3 has a nice property: no

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-20 Thread John Naylor
node4: 5 + 4+(7) + 4*8 = 48 bytes node3: 5 + 3 + 3*8 = 32 -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
e open questions about the design around shared memory. To keep unnecessary churn to a minimum, perhaps we should limit size class expansion to just one (or 5 total size classes) for the near future? -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-28 Thread John Naylor
sts didn't hit this. I do still intend to replace this code with something faster, but before I do so the tests should probably exercise the deletion paths more. Since VACUUM -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-28 Thread John Naylor
UUM Oops. I meant to finish with "Since VACUUM doesn't perform deletion we didn't have an opportunity to detect this during that operation." -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-29 Thread John Naylor
There are a few things up in the air, so I'm coming back to this list to summarize and add a recent update: On Mon, Nov 14, 2022 at 7:59 PM John Naylor wrote: > > - See how much performance we actually gain from tagging the node kind. Needs a benchmark that has enough branch mispredicts

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-30 Thread John Naylor
cated by aset.c is 4-bytes aligned so these atomic > variables are not always 8-bytes aligned. Is there any way to enforce > 8-bytes aligned memory allocations in 32-bit machines? The bigger question in my mind is: Why is there an atomic variable in backend-local memory? --

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-12-01 Thread John Naylor
On Thu, Dec 1, 2022 at 3:03 PM Masahiko Sawada wrote: > > On Thu, Dec 1, 2022 at 4:00 PM John Naylor wrote: > > > > The bigger question in my mind is: Why is there an atomic variable in backend-local memory? > > Because I use the same radix_tree and radix_tree_control st

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-30 Thread John Naylor
On Wed, Nov 30, 2022 at 2:28 PM Masahiko Sawada wrote: > > On Fri, Nov 25, 2022 at 5:00 PM John Naylor > wrote: > > These hacks were my work, but I think we can improve that by having two versions of NODE_HAS_FREE_SLOT -- one for fixed- and one for variable-sized nodes. F

Re: Prefetch the next tuple's memory during seqscans

2022-11-30 Thread John Naylor
ms Testing a5 latency average = 1737.173 ms Testing a6 latency average = 1828.239 ms Testing a7 latency average = 1920.909 ms Testing a8 latency average = 2036.922 ms [1] https://www.postgresql.org/message-id/CAFBsxsHqmH_S%3D4apc5agKsJsF6xZ9f6NaH0Z83jUYv3EgySHfw%40mail.gmail.com --

Re: Improve performance of pg_strtointNN functions

2022-11-30 Thread John Naylor
imal literal patch needs to be "immediately" faster? If doing this first leads to less code churn, that's another consideration, but you haven't made that argument. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PATCH] Add native windows on arm64 support

2022-12-01 Thread John Naylor
MV8_CRC32C', false) +cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1) +have_optimized_crc = true + else That seems like a heavy-handed way to force it. Could we just use the same gating in the test program that the patch puts in the code of interest? Namely: +#ifndef _MSC_VER #include +#endif -- John Naylor EDB: http://www.enterprisedb.com

Re: initdb: Refactor PG_CMD_PUTS loops

2022-12-01 Thread John Naylor
member. In other > cases, one might want to use PG_CMD_PRINTF() instead, to parametrize a > command, but that would require breaking up the loop or using > workarounds like replace_token(). This patch unwinds all that; it's > much simpler that way. +1, I can't think of a reason to ke

Re: Optimizing Node Files Support

2022-12-02 Thread John Naylor
. > 5. Avoid useless attribution, ignoring the result of pg_strtok when it is unnecessary. Looks worse. -- John Naylor EDB: http://www.enterprisedb.com

Re: Prefetch the next tuple's memory during seqscans

2022-11-22 Thread John Naylor
On Wed, Nov 23, 2022 at 5:00 AM David Rowley wrote: > > On Thu, 3 Nov 2022 at 22:09, John Naylor wrote: > > I tried a similar test, but with text fields of random length, and there is improvement here: > > Thank you for testing that. Can you share which CPU this was on? That

Re: Non-decimal integer literals

2022-11-22 Thread John Naylor
On Tue, Nov 22, 2022 at 8:36 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > On 15.11.22 11:31, Peter Eisentraut wrote: > > On 14.11.22 08:25, John Naylor wrote: > >> Regarding the patch, it looks good overall. My only suggestion would > >>

Re: Non-decimal integer literals

2022-11-23 Thread John Naylor
Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/path/to/srcdir/src/include -I/path/to/builddir/src/include -D_GNU_SOURCE

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
On Wed, Nov 16, 2022 at 11:46 AM John Naylor wrote: > > > On Tue, Nov 15, 2022 at 11:59 AM Masahiko Sawada wrote: > > Thanks! Please let me know if there is something I can help with. > > I didn't get very far because the tests fail on 0004 in rt_verify_node: > > TRAP

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
src/backend/lib/radixtree.c", Line: 2186, PID: 18242 -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-15 Thread John Naylor
On Wed, Nov 16, 2022 at 12:33 PM Masahiko Sawada wrote: > > On Wed, Nov 16, 2022 at 1:46 PM John Naylor > wrote: > > > > > > On Tue, Nov 15, 2022 at 11:59 AM Masahiko Sawada wrote: > > > Thanks! Please let me know if there is something I can help with. &

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-11-25 Thread John Naylor
that by having two versions of NODE_HAS_FREE_SLOT -- one for fixed- and one for variable-sized nodes. For that to work, in "init-node" we'd need a branch to set fanout to zero for node256. That should be fine -- it already has to branch for memset'ing node128's indexes to 0xFF. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-16 Thread John Naylor
On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > wrote: > > > Thanks! cfbot complaints about some warnings but these are expected > (due to unused delete routines etc). But one reported error[1] might > be relevant

Re: cutting down the TODO list thread

2023-01-16 Thread John Naylor
o the todo list will probably do more to reduce confusion than language in the todo list itself. -- John Naylor EDB: http://www.enterprisedb.com

Re: Small miscellaneus fixes (Part II)

2023-01-16 Thread John Naylor
On Mon, Jan 16, 2023 at 1:28 PM John Naylor wrote: > > > I wrote: > > ...but arguably the earlier check is close enough that it's silly to assert in the "else" branch, and I'd be okay with just nuking those lines. Another thing that caught my attention is the assumption t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-17 Thread John Naylor
On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > wrote: > > + * Add Tids on a block to TidStore. The caller must ensure the offset numbers > > + * in 'offsets' are ordered in ascending order. > > >

Re: cutting down the TODO list thread

2023-01-23 Thread John Naylor
e experience has shown that there are often requirements that are not obvious, and if those are not agreed on beforehand it leads to wasted effort. In the email, discuss both the internal implementation method you plan to use, and any user-visible changes (new syntax, etc)." > On Wed, Ja

Re: [PATCH] Use 128-bit math to accelerate numeric division, when 8 < divisor digits <= 16

2023-01-22 Thread John Naylor
ht 64-bit instructions in general are as fast as 32-bit instructions, > on 64-bit platforms. According to Agner's instruction tables [1], integer division on Skylake (for example) has a latency of 26 cycles for 32-bit operands, and 42-95 cycles for 64-bit. [1] https://www.agner.org/optimize/instru

bitscan forward/reverse on Windows

2023-01-24 Thread John Naylor
the platform specific types, so some further cleanup might be needed. 0003 tries one way to reduce the duplication that arose in 0002. Maybe there is a better way. -- John Naylor EDB: http://www.enterprisedb.com From 9390a74f1712d58c88ac513afff8d878071c040c Mon Sep 17 00:00:00 2001 From: John Naylor

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-25 Thread John Naylor
On Tue, Dec 13, 2022 at 12:29 AM John Naylor > wrote: > > If the number of unfrozen heap pages is the thing we care about, perhaps that, and not the total size of the table, should be the parameter that drives freezing strategy? > > That's not the only thing we care about,

Re: Considering additional sort specialisation functions for PG16

2023-01-26 Thread John Naylor
act a preferred order between these tuples -- the second key is the tie breaker in this case. -- John Naylor EDB: http://www.enterprisedb.com

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-25 Thread John Naylor
On Wed, Jan 25, 2023 at 8:42 AM Masahiko Sawada wrote: > > On Mon, Jan 23, 2023 at 8:20 PM John Naylor > wrote: > > > > On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote: > > > > > > On Mon, Jan 16, 2023 at 2:02 PM John Naylor > > > wro

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-25 Thread John Naylor
On Tue, Jan 24, 2023 at 1:17 PM Dilip Kumar wrote: > > On Mon, Jan 23, 2023 at 6:00 PM John Naylor > wrote: > > > > Attached is a rebase to fix conflicts from recent commits. > > I have reviewed v22-0022* patch and I have some comments. > > 1. >

Re: Considering additional sort specialisation functions for PG16

2023-01-26 Thread John Naylor
On Tue, Aug 23, 2022 at 1:13 PM John Naylor wrote: > > On Tue, Aug 23, 2022 at 11:24 AM David Rowley wrote: > > > > On Tue, 23 Aug 2022 at 15:22, John Naylor wrote: > > > Did you happen to see > > > > > > https://www.postgresql.org/m

<    5   6   7   8   9   10   11   12   13   14   >