Re: pgsql: Refactor to add pg_strcoll(), pg_strxfrm(), and variants.

2023-02-23 Thread Justin Pryzby
These patches cause warnings under MSVC. Of course my patch to improve CI by warning about compiler warnings is the only one to notice. https://cirrus-ci.com/task/6199582053367808 -- Justin

Re: pgsql: Refactor to add pg_strcoll(), pg_strxfrm(), and variants.

2023-02-23 Thread Justin Pryzby
On Fri, Feb 24, 2023 at 01:56:05PM +1300, Thomas Munro wrote: > On Fri, Feb 24, 2023 at 1:20 PM Justin Pryzby wrote: > > These patches cause warnings under MSVC. > > > > Of course my patch to improve CI by warning about compiler warnings is > > the only one to not

Re: Add LZ4 compression in pg_dump

2023-02-23 Thread Justin Pryzby
On Thu, Feb 23, 2023 at 09:24:46PM +0100, Tomas Vondra wrote: > I've now pushed 0002 and 0003, after minor tweaks (a couple typos etc.), > and marked the CF entry as committed. Thanks for the patch! A big thanks from me to everyone involved. > I wonder how difficult would it be to add the zstd co

zstd compression for pg_dump

2023-02-24 Thread Justin Pryzby
001 From: Justin Pryzby Date: Sat, 7 Jan 2023 15:45:06 -0600 Subject: [PATCH 1/3] WIP: pg_dump: zstd compression Previously proposed at: 20201221194924.gi30...@telsasoft.com note-to-self: see also: private commit 36ab001fb --- src/bin/pg_dump/Makefile | 1 + src/bin/pg_dump/compr

Re: zstd compression for pg_dump

2023-02-24 Thread Justin Pryzby
On Sat, Feb 25, 2023 at 01:44:36PM +0900, Michael Paquier wrote: > On Fri, Feb 24, 2023 at 01:18:40PM -0600, Justin Pryzby wrote: > > This is a draft patch - review is welcome and would help to get this > > ready to be considererd for v16, if desired. > > > > I'm

Re: Add LZ4 compression in pg_dump

2023-02-24 Thread Justin Pryzby
Custom || archiveFormat == archDirectory) && + !user_compression_defined) + parse_compress_specification(PG_COMPRESSION_GZIP, NULL, +&compression_spec); #endif ...but that elicits a warning about "variable se

Re: Add LZ4 compression in pg_dump

2023-02-25 Thread Justin Pryzby
On Fri, Feb 24, 2023 at 11:02:14PM -0600, Justin Pryzby wrote: > I have some fixes (attached) and questions while polishing the patch for > zstd compression. The fixes are small and could be integrated with the > patch for zstd, but could be applied independently.

Re: windows/meson cfbot warnings

2023-02-25 Thread Justin Pryzby
On Sat, Feb 25, 2023 at 10:29:33AM -0800, Andres Freund wrote: > Hi, > > On 2022-12-28 17:44:47 -0800, Andres Freund wrote: > > On 2022-12-28 18:35:38 -0600, Justin Pryzby wrote: > > > Since a few days ago, the windows/meson task has been spewing messages >

Re: zstd compression for pg_dump

2023-02-25 Thread Justin Pryzby
On Fri, Feb 24, 2023 at 01:18:40PM -0600, Justin Pryzby wrote: > This is a draft patch - review is welcome and would help to get this > ready to be considererd for v16, if desired. > > I'm going to add this thread to the old CF entry. > https://commitfest.postgresql.org/31/

meson vs make: missing/inconsistent ENV

2023-02-26 Thread Justin Pryzby
I noticed warnings: Use of uninitialized value $ENV{"with_icu"} in string eq at /home/pryzbyj/src/postgres/src/bin/pg_dump/t/002_pg_dump.pl line 56. and looked through: git grep ^export '*/Makefile' and found that: src/bin/pg_dump/meson.build is missing with_icu since 396d348b0 Also, e6927270c

Re: meson vs make: missing/inconsistent ENV

2023-02-26 Thread Justin Pryzby
On Sun, Feb 26, 2023 at 03:21:04PM -0800, Andres Freund wrote: > > Is there any consideration of promoting these or other warnings to > > fatal? > > You mean the perl warnings? Yes - it'd be nice if the warnings caused an obvious failure to allow addressing the issue. I noticed the icu warning w

Re: Add LZ4 compression in pg_dump

2023-02-26 Thread Justin Pryzby
On Sat, Feb 25, 2023 at 08:05:53AM -0600, Justin Pryzby wrote: > On Fri, Feb 24, 2023 at 11:02:14PM -0600, Justin Pryzby wrote: > > I have some fixes (attached) and questions while polishing the patch for > > zstd compression. The fixes are small and could be integrated with th

Re: Memory leak from ExecutorState context?

2023-02-28 Thread Justin Pryzby
On Tue, Feb 28, 2023 at 07:06:43PM +0100, Jehan-Guillaume de Rorthais wrote: > Hello all, > > A customer is facing out of memory query which looks similar to this > situation: > > > https://www.postgresql.org/message-id/flat/12064.1555298699%40sss.pgh.pa.us#eb519865575bbc549007878a5fb7219b >

Re: Add LZ4 compression in pg_dump

2023-02-28 Thread Justin Pryzby
pg_fatal("could not initialize compression library: %s", zp->msg); - - zp->next_out = gzipcs->outbuf; - zp->avail_out = gzipcs->outsize; - } gzipcs->zp->next_in = (void *) unconstify(void *, data); gzipcs->zp->avail_in

Re: Add LZ4 compression in pg_dump

2023-03-01 Thread Justin Pryzby
On Wed, Mar 01, 2023 at 01:39:14PM +, gkokola...@pm.me wrote: > On Wednesday, March 1st, 2023 at 12:58 AM, Justin Pryzby > wrote: > > > The current function order avoids 3 lines of declarations, but it's > > obviously pretty useful to be able to run that diff com

Re: cataloguing NOT NULL constraints

2023-03-01 Thread Justin Pryzby
On Tue, Feb 28, 2023 at 08:15:37PM +0100, Alvaro Herrera wrote: > Since nobody liked the idea of listing the constraints in psql \d's > footer, I changed \d+ so that the "not null" column shows the name of > the constraint if there is one, or the string "(primary key)" if the > attnotnull marking f

Re: Add LZ4 compression in pg_dump

2023-03-02 Thread Justin Pryzby
On Wed, Mar 01, 2023 at 05:20:05PM +0100, Tomas Vondra wrote: > On 2/25/23 15:05, Justin Pryzby wrote: > > On Fri, Feb 24, 2023 at 11:02:14PM -0600, Justin Pryzby wrote: > >> I have some fixes (attached) and questions while polishing the patch for > >> zstd compressio

Re: Operation log for major operations

2023-03-02 Thread Justin Pryzby
On Thu, Mar 02, 2023 at 08:57:43PM +0300, Dmitry Koval wrote: > These changes did not interest the community. It was expected (topic is very > specifiс: vendor's technical support). So no sense to distract developers Actually, I think there is interest, but it has to be phrased in a limited sense

Re: zstd compression for pg_dump

2023-03-03 Thread Justin Pryzby
On Fri, Mar 03, 2023 at 10:32:53AM -0800, Jacob Champion wrote: > On Sat, Feb 25, 2023 at 5:22 PM Justin Pryzby wrote: > > This resolves cfbot warnings: windows and cppcheck. > > And refactors zstd routines. > > And updates docs. > > And includes some fixes for earlier

Re: zstd compression for pg_dump

2023-03-04 Thread Justin Pryzby
On Fri, Mar 03, 2023 at 01:38:05PM -0800, Jacob Champion wrote: > > > With this particular dataset, I don't see much improvement with > > > zstd:long. > > > > Yeah. I this could be because either 1) you already got very good > > comprssion without looking at more data; and/or 2) the neighboring da

Re: zstd compression for pg_dump

2023-03-05 Thread Justin Pryzby
On Sat, Feb 25, 2023 at 07:22:27PM -0600, Justin Pryzby wrote: > On Fri, Feb 24, 2023 at 01:18:40PM -0600, Justin Pryzby wrote: > > This is a draft patch - review is welcome and would help to get this > > ready to be considererd for v16, if desired. > > > > I'm goi

Re: Add LZ4 compression in pg_dump

2023-03-07 Thread Justin Pryzby
assert+comment. I (Justin) argued that the new fatal shouldn't have been introduced in a refactoring commit, so this is a compromise. Report and initial patch by Justin Pryzby, test case by Georgios Kokolatos. https://www.postgresql.org/message-id/20230228235834.GC30529%40telsasoft.com ---

Re: Add SHELL_EXIT_CODE to psql

2023-03-08 Thread Justin Pryzby
On Wed, Feb 22, 2023 at 03:04:33PM -0500, Corey Huinker wrote: > + > +/* > + * Return the POSIX exit code (0 to 255) that corresponds to the argument. > + * The argument is a return code returned by wait(2) or waitpid(2), which > + * also applies to pclose(3) and system(3). > + */ > +int > +wait_re

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-09 Thread Justin Pryzby
On Tue, Mar 07, 2023 at 10:18:44AM -0800, Andres Freund wrote: > Hi, > > On 2023-03-06 15:21:14 -0500, Melanie Plageman wrote: > > Good point. Attached is what you suggested. I committed the transaction > > before the drop table so that the statistics would be visible when we > > queried pg_stat_i

Re: Add LZ4 compression in pg_dump

2023-03-09 Thread Justin Pryzby
On Wed, Mar 01, 2023 at 05:39:54PM +0100, Tomas Vondra wrote: > On 2/27/23 05:49, Justin Pryzby wrote: > > On Sat, Feb 25, 2023 at 08:05:53AM -0600, Justin Pryzby wrote: > >> On Fri, Feb 24, 2023 at 11:02:14PM -0600, Justin Pryzby wrote: > >>> I have some fixes

Re: Improve logging when using Huge Pages

2023-03-09 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > Greetings, > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > > > On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: >

Re: Add LZ4 compression in pg_dump

2023-03-10 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 06:58:20PM +0100, Tomas Vondra wrote: > I'm a bit confused about the lz4 vs. lz4f stuff, TBH. If we switch to > lz4f, doesn't that mean it (e.g. restore) won't work on systems that > only have older lz4 version? What would/should happen if we take backup > compressed with lz

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-03-10 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 11:43:01AM -0800, Andres Freund wrote: > On 2023-03-09 06:51:31 -0600, Justin Pryzby wrote: > > On Tue, Mar 07, 2023 at 10:18:44AM -0800, Andres Freund wrote: > > > Hi, > > > > > > On 2023-03-06 15:21:14 -0500, Melanie Plageman wrote:

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-11 Thread Justin Pryzby
On Sat, Mar 11, 2023 at 09:55:33AM -0500, Melanie Plageman wrote: > Subject: [PATCH v3 2/3] use shared buffers when failsafe active > > + /* > + * Assume the caller who allocated the memory for the > + * BufferAccessStrategy object will free it. > +

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2023-03-11 Thread Justin Pryzby
On Thu, Dec 15, 2022 at 10:13:23AM -0600, Justin Pryzby wrote: > Rebased on c727f511b. Rebased on 30a53b792. With minor changes including fixes to an intermediate patch. > This patch record was "closed for lack of interest", but I think what's > actually needed is c

Re: pg_dump versus hash partitioning

2023-03-12 Thread Justin Pryzby
On Sun, Mar 12, 2023 at 03:46:52PM -0400, Tom Lane wrote: > What I propose we do about that is further tweak things so that > load-via-partition-root forces dumping via COPY. AFAIK the only > compelling use-case for dump-as-INSERTs is in transferring data > to a non-Postgres database, which is a c

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-12 Thread Justin Pryzby
On Fri, Mar 10, 2023 at 03:36:10PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > Update to address a compiler warning in the supplementary patches adding > > assertions. > > I took a look through this. It seems like basically a good solution, > but the count_leaf_

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-12 Thread Justin Pryzby
On Sun, Mar 12, 2023 at 04:14:06PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Mar 10, 2023 at 03:36:10PM -0500, Tom Lane wrote: > >> I took a look through this. It seems like basically a good solution, > >> but the count_leaf_partitions() functi

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-13 Thread Justin Pryzby
On Sun, Mar 12, 2023 at 06:25:13PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Sun, Mar 12, 2023 at 04:14:06PM -0400, Tom Lane wrote: > >> Hm. Could we get rid of count_leaf_partitions by doing the work in > >> ProcessUtilitySlow? Or at least passing th

Re: Improve logging when using Huge Pages

2023-03-13 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > * Justin Pryzby (pry...@telsasoft.com) wrote: > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > > > On Wed, Feb 15, 20

Re: CI and test improvements

2023-03-13 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: > On 03.02.23 15:26, Justin Pryzby wrote: > > rebased, and re-including a patch to show code coverage of changed > > files. > > This constant flow of patches under one subject doesn't lend itself well to

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-14 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 10:42:59AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Sun, Mar 12, 2023 at 06:25:13PM -0400, Tom Lane wrote: > >> I agree that adding such a field to IndexStmt would be a very bad idea. > >> However, adding another parameter to Defi

Re: CI and test improvements

2023-03-15 Thread Justin Pryzby
On Wed, Mar 15, 2023 at 10:58:41AM +0100, Peter Eisentraut wrote: > On 14.03.23 05:56, Justin Pryzby wrote: > > I'm soliticing feedback on those patches that I've sent recently - I've > > elided patches if they have some unresolved issue. > &g

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-15 Thread Justin Pryzby
unter. - For pre-existing indexes which are ATTACHed rather than built, DefineIndex() increments the DONE counter, unless the attached index is partitioned, in which case progress report is not updated. Author: Ilya Gladyshev Reviewed-By: Justin Pryzby, Tomas Vondra, Dean Rasheed, Alvaro He

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Justin Pryzby
On Tue, Mar 14, 2023 at 08:56:58PM -0400, Melanie Plageman wrote: > On Sat, Mar 11, 2023 at 2:16 PM Justin Pryzby wrote: > > > @@ -586,6 +587,45 @@ GetAccessStrategy(BufferAccessStrategyType btype) > > > +BufferAccessStrategy > > > +GetAccessStrategyWithSize(Buffe

Re: Add LZ4 compression in pg_dump

2023-03-15 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 10:47:12PM +0100, Tomas Vondra wrote: > > Rearrange functions to their original order allowing a cleaner diff to the > > prior code; > > OK. I wasn't very enthusiastic about this initially, but after thinking > about it a bit I think it's meaningful to make diffs clearer.

Re: zstd compression for pg_dump

2023-03-15 Thread Justin Pryzby
llowed cs->readData() > to be called more than once would subtly break this code. Note that nothing bad happens if we lose track of how big it is (well, assuming that readF doesn't *shrink* the buffer). The previous patch version didn't keep track of its new size, and the onl

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-03-16 Thread Justin Pryzby
On Thu, Mar 16, 2023 at 04:52:07PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Thu, Jan 19, 2023 at 04:47:59PM -0500, Robert Treat wrote: > >> I think all of that feedback is useful, I guess the immediate question > >> becomes if Justin wants to tr

Re: Add LZ4 compression in pg_dump

2023-03-16 Thread Justin Pryzby
On Thu, Mar 16, 2023 at 11:30:50PM +0100, Tomas Vondra wrote: > On 3/16/23 01:20, Justin Pryzby wrote: > > But try reading the diff while looking for the cause of a bug. It's the > > difference between reading 50, two-line changes, and reading a hunk that > > replaces

Re: Commitfest 2023-03 starting tomorrow!

2023-03-17 Thread Justin Pryzby
On Wed, Mar 15, 2023 at 02:29:26PM -0400, Gregory Stark (as CFM) wrote: > 1) Move it yourself to the next CF (or withdraw it) > 2) Post to the list with any pending questions asking for specific > feedback -- it's much more likely to get feedback than just a generic > "here's a patch plz review"...

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-18 Thread Justin Pryzby
> Subject: [PATCH v4 3/3] add vacuum[db] option to specify ring_size and guc > +Specifies the ring buffer size to be used for a given invocation of > +VACUUM or instance of autovacuum. This size is > +converted to a number of shared buffers which will be reused as part > o

Re: Commitfest 2023-03 starting tomorrow!

2023-03-18 Thread Justin Pryzby
On Sat, Mar 18, 2023 at 02:43:38PM -0700, Peter Geoghegan wrote: > On Sat, Mar 18, 2023 at 1:26 PM Alvaro Herrera > wrote: > > At this point, I'm going to suggest that reviewers should be open to the > > idea of applying a submitted patch to some older Git commit in order to > > review it. If we

Re: Commitfest 2023-03 starting tomorrow!

2023-03-18 Thread Justin Pryzby
On Sat, Mar 18, 2023 at 04:28:02PM -0700, Peter Geoghegan wrote: > On Sat, Mar 18, 2023 at 4:19 PM Justin Pryzby wrote: > > The only issue with this is that cfbot has squished all the commits into > > one, and lost the original commit messages (if any). I submitted > > pa

Re: proposal: possibility to read dumped table's name from file

2023-03-19 Thread Justin Pryzby
On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote: > rebase + enhancing about related option from a563c24 Thanks. It looks like this doesn't currently handle extensions, which were added at 6568cef26e. > + table_and_children: tables, works like > + -t/--table, exc

Re: Memory leak from ExecutorState context?

2023-03-19 Thread Justin Pryzby
On Fri, Mar 17, 2023 at 05:41:11PM +0100, Tomas Vondra wrote: > >> * Patch 2 is worth considering to backpatch > > I'm not quite sure what exactly are the numbered patches, as some of the > threads had a number of different patch ideas, and I'm not sure which > one was/is the most promising one.

Re: Add LZ4 compression in pg_dump

2023-03-20 Thread Justin Pryzby
On Fri, Mar 17, 2023 at 03:43:58PM +, gkokola...@pm.me wrote: > From a174cdff4ec8aad59f5bcc7e8d52218a14fe56fc Mon Sep 17 00:00:00 2001 > From: Georgios Kokolatos > Date: Fri, 17 Mar 2023 14:45:58 + > Subject: [PATCH v3 1/3] Improve type handling in pg_dump's compress file API > -int > +bo

Re: proposal: possibility to read dumped table's name from file

2023-03-21 Thread Justin Pryzby
On Mon, Mar 20, 2023 at 08:01:13AM +0100, Pavel Stehule wrote: > ne 19. 3. 2023 v 15:01 odesílatel Justin Pryzby napsal: > > > On Thu, Mar 16, 2023 at 01:05:41PM +0100, Pavel Stehule wrote: > > > rebase + enhancing about related option from a563c24 > > > > T

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-21 Thread Justin Pryzby
On Thu, Mar 16, 2023 at 07:04:16PM +0400, Ilya Gladyshev wrote: > > 16 марта 2023 г., в 04:07, Justin Pryzby написал(а): > > > > On Tue, Mar 14, 2023 at 06:58:14PM +0400, Ilya Gladyshev wrote: > >>> The only change from the current patch is (3). (1) still cal

Re: Improve logging when using Huge Pages

2023-03-21 Thread Justin Pryzby
at I could do to help progress the patch? Right now, I have no idea. Most recently, you said it'd be better implemented as a GUC to allow using -C, but then recanted because -C doesn't work for this (which is why I implemented it as a string back on 2023-02-08). Which is why I reset its

Re: Should we remove vacuum_defer_cleanup_age?

2023-03-22 Thread Justin Pryzby
On Sat, Mar 18, 2023 at 10:33:57AM +0100, Alvaro Herrera wrote: > On 2023-Mar-17, Andres Freund wrote: > > > I started writing a test for vacuum_defer_cleanup_age while working on the > > fix > > referenced above, but now I am wondering if said energy would be better > > spent > > removing vacuu

Re: Improve logging when using Huge Pages

2023-03-22 Thread Justin Pryzby
On Wed, Mar 22, 2023 at 04:37:01PM +0900, Michael Paquier wrote: > I have noted something.. For the WIN32 case, we have that: > > +++ b/src/backend/port/win32_shmem.c > @@ -327,6 +327,8 @@ retry: > Sleep(1000); > continue; > } > + > + huge_pages_active = ((fl

Re: Commitfest 2023-03 starting tomorrow!

2023-03-23 Thread Justin Pryzby
- I'm guessing this patch is too big and too late to go in this CF. > And it sounds like there's still work to be done? Should this be > marked RwF? If you look, you'll see that's it's straightforward and *also* small. As I wrote last week, it's very viable for v1

Re: Improve logging when using Huge Pages

2023-03-23 Thread Justin Pryzby
rallels all the similar, and related things without needing to allocate extra bits of shared_memory. On Thu, Mar 23, 2023 at 07:23:28AM +0900, Michael Paquier wrote: > On Wed, Mar 22, 2023 at 05:18:28PM -0500, Justin Pryzby wrote: > > Wow, good point. I think to make it work we'd need

Re: Commitfest 2023-03 starting tomorrow!

2023-03-23 Thread Justin Pryzby
On Fri, Mar 24, 2023 at 10:24:43AM +0900, Michael Paquier wrote: > >> * Check consistency of GUC defaults between .sample.conf and > >> pg_settings.boot_val > > - It looks like this was pretty active until last October and might > > have been ready to apply at least partially? But no further wor

cfbot stuck

2023-03-24 Thread Justin Pryzby
It looks like cfbot is stuck since 13h ago. https://cirrus-ci.com/github/postgresql-cfbot/postgresql

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-24 Thread Justin Pryzby
xes which are ATTACHed rather than built, DefineIndex() increments the DONE counter, unless the attached index is partitioned, in which case progress report is not updated. Author: Ilya Gladyshev, Justin Pryzby Reviewed-By: Tomas Vondra, Dean Rasheed, Alvaro Herrera, Matthias van de Meent, Tom Lan

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-25 Thread Justin Pryzby
On Sat, Mar 25, 2023 at 11:55:13AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Thu, Mar 23, 2023 at 04:35:46PM -0400, Tom Lane wrote: > >> Furthermore, as things stand it's not hard > >> for PARTITIONS_TOTAL to be zero --- there's at least one s

Re: windows/meson cfbot warnings

2023-03-25 Thread Justin Pryzby
On Sun, Feb 26, 2023 at 09:54:25AM -0800, Andres Freund wrote: > On 2023-02-25 16:45:38 -0600, Justin Pryzby wrote: > > Unrelated, but something else changed and now there's this. > > > > https://cirrus-ci.com/task/6202242768830464 > > > > [20:10:34.310] c:

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-26 Thread Justin Pryzby
On Thu, Mar 21, 2024 at 01:07:01PM +0100, Alvaro Herrera wrote: > Given that Michaël is temporarily gone, I propose to push the attached > tomorrow. Thanks. On Tue, Mar 26, 2024 at 12:05:47PM +0100, Alvaro Herrera wrote: > On 2024-Mar-26, Alexander Lakhin wrote: > > > Hello Alvaro, > > > > 21.0

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-31 Thread Justin Pryzby
On Sun, Mar 31, 2024 at 12:00:00PM +0300, Alexander Lakhin wrote: > Hello Alvaro, > > 28.03.2024 18:58, Alvaro Herrera wrote: > > Grumble. I don't like initialization at declare time, so far from the > > code that depends on the value. But the alternative would have been to > > assign right wher

Re: POC, WIP: OR-clause support for indexes

2024-04-07 Thread Justin Pryzby
On Mon, Apr 08, 2024 at 01:34:37AM +0300, Alexander Korotkov wrote: > Hi! > > On Mon, Apr 1, 2024 at 9:38 AM Andrei Lepikhov > wrote: > > On 28/3/2024 16:54, Alexander Korotkov wrote: > > > The current patch has a boolean guc enable_or_transformation. > > > However, when we have just a few ORs to

Re: CI and test improvements

2024-02-13 Thread Justin Pryzby
March cf and set it WoA, I'd appreciate > that. The patches are rebased. A couple were merged since I last rebased them ~10 months ago. The freebsd patch will probably be obsoleted by a patch of Thomas. On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: > On 03.02.23 15

Re: pg_upgrade and logical replication

2024-02-13 Thread Justin Pryzby
On Tue, Jan 02, 2024 at 03:58:25PM +0530, Amit Kapila wrote: > Pushed. pg_upgrade/t/004_subscription.pl says |my $mode = $ENV{PG_TEST_PG_UPGRADE_MODE} || '--copy'; ..but I think maybe it should not. When you try to use --link, it fails: https://cirrus-ci.com/task/4669494061170688 |Adding ".old

Re: pg_upgrade and logical replication

2024-02-15 Thread Justin Pryzby
On Wed, Feb 14, 2024 at 03:37:03AM +, Hayato Kuroda (Fujitsu) wrote: > Attached patch modified the test accordingly. Also, it contains some > optimizations. > This can pass the test on my env: What optimizations? I can't see them, and since the patch is described as rearranging test cases (a

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-02-29 Thread Justin Pryzby
On Wed, Feb 28, 2024 at 05:08:49PM +0900, Michael Paquier wrote: > On Wed, Feb 21, 2024 at 08:46:48AM +0100, Peter Eisentraut wrote: > > Yes, I think most people agreed that that would be the preferred behavior. > > Challenge accepted. As of the patch attached. Thanks for picking it up. I find

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-01 Thread Justin Pryzby
On Fri, Mar 01, 2024 at 10:56:50AM +0900, Michael Paquier wrote: > On Thu, Feb 29, 2024 at 08:51:31AM -0600, Justin Pryzby wrote: > > On Wed, Feb 28, 2024 at 05:08:49PM +0900, Michael Paquier wrote: > >> I have implemented that so as we keep the default, historical > >>

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-03-07 Thread Justin Pryzby
bf7ed8889f39a59dd99b292014b73be85342 ebfe2dbd6b624e2a428e14b7ee9322cc096f63f7 - prevent DROP AM Authors: Justin Pryzby, Soumyadeep Chakraborty --- doc/src/sgml/catalogs.sgml | 5 +- doc/src/sgml/ref/alter_table.sgml | 8 ++ doc/src/sgml/ref/create_table.sgml | 4 + sr

Re: Progress report of CREATE INDEX for nested partitioned tables

2022-12-10 Thread Justin Pryzby
ertions are okay. I imagine they may break other extensions, as with file_fdw. -- Justin >From b8077babf9a101f9d1bf41dd1ad866d2ea38b603 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 10 Dec 2022 16:17:50 -0600 Subject: [PATCH] fix progress reporting of nested, partitioned indexes.

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2022-12-11 Thread Justin Pryzby
On Sun, Dec 11, 2022 at 06:25:48PM +0900, Amit Langote wrote: > On Sun, Dec 11, 2022 at 5:17 AM Justin Pryzby wrote: > > The original code rechecks rte->checkAsUser with the rte of the parent > > rel. The patch changed to access onerel instead, but that's not updated &g

Re: Add sub-transaction overflow status in pg_stat_activity

2022-12-12 Thread Justin Pryzby
On Mon, Dec 12, 2022 at 09:33:51AM -0800, Nathan Bossart wrote: > On Mon, Dec 12, 2022 at 11:15:43AM -0500, Robert Haas wrote: > > Any strenuous objections? > > Nope. In fact, +1. Until more work is done to alleviate the performance > issues, this information will likely prove useful. The docs

Re: Progress report of CREATE INDEX for nested partitioned tables

2022-12-12 Thread Justin Pryzby
On Mon, Dec 12, 2022 at 11:39:23PM +0400, Ilya Gladyshev wrote: > > > Could you check what I've written as a counter-proposal ? > > I think that this might be a good solution to start with, it gives us the > opportunity to improve the granularity later without any surprising changes > for the e

Re: Common function for percent placeholder replacement

2022-12-14 Thread Justin Pryzby
On Wed, Dec 14, 2022 at 08:31:02AM +0100, Peter Eisentraut wrote: > + return replace_percent_placeholders(base_command, "df", (const char > *[]){target_detail, filename}); This is a "compound literal", which I gather is required by C99. But I don't think that's currently being exercised, so

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-14 Thread Justin Pryzby
On Thu, Dec 15, 2022 at 01:02:39AM +0300, Pavel Luzanov wrote: > On 14.12.2022 22:46, Jeff Davis wrote: > > The behavior is that MAINTAIN > > privileges on the partitioned table does not imply MAINTAIN privileges > > on the partitions. I believe that's fine and it's consistent with other > > privil

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2022-12-15 Thread Justin Pryzby
00:00:00 2001 From: Justin Pryzby Date: Tue, 13 Jul 2021 21:25:48 -0500 Subject: [PATCH 1/4] Add pg_am_size(), pg_namespace_size() .. See also: 358a897fa, 528ac10c7 --- src/backend/utils/adt/dbsize.c | 131 src/include/catalog/pg_proc.dat | 19 + 2

Re: New strategies for freezing, advancing relfrozenxid early

2022-12-15 Thread Justin Pryzby
The patches (003 and 005) are missing a word should use to decide whether to its eager freezing strategy. On the wiki, missing a word: builds on related added

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-15 Thread Justin Pryzby
On Thu, Dec 15, 2022 at 10:10:43AM -0800, Jeff Davis wrote: > On Thu, 2022-12-15 at 12:31 +0300, Pavel Luzanov wrote: > > I think the approach that Nathan implemented [1] for TOAST tables > > in the latest version can be used for partitioned tables as well. > > Skipping the privilege check for part

(non) translatable string splicing

2022-12-16 Thread Justin Pryzby
I was surprised to see that this has been here for a few years (since 77517ba59) without complaints or inquiries from translators. src/bin/pg_upgrade/option.c:check_required_directory(&old_cluster.bindir, "PGBINOLD", false, src/bin/pg_upgrade/option.c-

Re: pg_upgrade allows itself to be run twice

2022-12-16 Thread Justin Pryzby
On Thu, Dec 01, 2022 at 10:30:16AM +0100, Peter Eisentraut wrote: > On 01.11.22 14:07, Justin Pryzby wrote: > > On Tue, Nov 01, 2022 at 01:54:35PM +0100, Peter Eisentraut wrote: > > > On 07.07.22 08:22, Justin Pryzby wrote: > > > > > This one comes from NextOID

Re: Progress report of CREATE INDEX for nested partitioned tables

2022-12-17 Thread Justin Pryzby
On Tue, Dec 13, 2022 at 10:18:58PM +0400, Ilya Gladyshev wrote: > > > I actually think that the progress view would be better off without > > > the total number of partitions, > > > > Just curious - why ? > > We don't really know how many indexes we are going to create, unless we > have some kin

Re: Add LZ4 compression in pg_dump

2022-12-17 Thread Justin Pryzby
001: still refers to "gzip", which is correct for -Fp and -Fd but not for -Fc, for which it's more correct to say "zlib". That affects the name of the function, structures, comments, etc. I'm not sure if it's an issue to re-use the basebackup compression routines here. Maybe we should accept "-Z

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-18 Thread Justin Pryzby
On Sat, Dec 17, 2022 at 04:39:29AM -0800, Ted Yu wrote: > + List *ancestors = get_partition_ancestors(relid); > + Oid root = InvalidOid; > > nit: it would be better if the variable `root` can be aligned with variable > `ancestors`. It is aligned, but only aft

Re: Add LZ4 compression in pg_dump

2022-12-19 Thread Justin Pryzby
On Mon, Dec 19, 2022 at 05:03:21PM +, gkokola...@pm.me wrote: > > > 001 still doesn't compile on freebsd, and 002 doesn't compile on > > > windows. Have you checked test results from cirrusci on your private > > > github account ? > > There are still known gaps in 0002 and 0003, for example do

Re: Add LZ4 compression in pg_dump

2022-12-19 Thread Justin Pryzby
On Mon, Dec 19, 2022 at 01:06:00PM +0900, Michael Paquier wrote: > On Sat, Dec 17, 2022 at 05:26:15PM -0600, Justin Pryzby wrote: > > 001: still refers to "gzip", which is correct for -Fp and -Fd but not > > for -Fc, for which it's more correct to say "zlib&qu

Re: Add LZ4 compression in pg_dump

2022-12-20 Thread Justin Pryzby
On Tue, Dec 20, 2022 at 11:19:15AM +, gkokola...@pm.me wrote: > --- Original Message --- > On Monday, December 19th, 2022 at 6:27 PM, Justin Pryzby > wrote: > > On Mon, Dec 19, 2022 at 05:03:21PM +, gkokola...@pm.me wrote: > > > > > > > 001 s

Re: Small miscellaneus fixes (Part II)

2022-12-20 Thread Justin Pryzby
On Fri, Nov 25, 2022 at 06:27:04PM -0300, Ranier Vilela wrote: > 5. Use boolean operator with boolean operands > (b/src/backend/commands/tablecmds.c) tablecmds.c: right. Since 074c5cfbf pg_dump.c: right. Since b08dee24a > 4. Fix dead code (src/backend/utils/adt/formatting.c) > Np->sign == '+',

Re: Simplifications for error messages related to compression

2022-12-20 Thread Justin Pryzby
On Wed, Dec 21, 2022 at 10:43:19AM +0900, Michael Paquier wrote: > On Mon, Dec 19, 2022 at 02:42:13PM +0900, Michael Paquier wrote: > > Thoughts or objections? > > Hearing nothing, done.. - pg_fatal("not built with zlib support"); + pg_fatal("this build

Re: Simplifications for error messages related to compression

2022-12-20 Thread Justin Pryzby
On Wed, Dec 21, 2022 at 01:52:21PM +0900, Michael Paquier wrote: > On Tue, Dec 20, 2022 at 08:29:32PM -0600, Justin Pryzby wrote: > > - pg_fatal("not built with zlib support"); > > + pg_fatal("this build does not support co

Re: ExecRTCheckPerms() and many prunable partitions (checkAsUser)

2022-12-21 Thread Justin Pryzby
Alvaro could you comment on this ?

Re: Add LZ4 compression in pg_dump

2022-12-22 Thread Justin Pryzby
There's a couple of lz4 bits which shouldn't be present in 002: file extension and comments.

Re: Small miscellaneus fixes (Part II)

2022-12-22 Thread Justin Pryzby
On Thu, Dec 22, 2022 at 02:29:11PM -0300, Ranier Vilela wrote: > Em ter., 20 de dez. de 2022 às 21:51, Justin Pryzby > escreveu: > > > On Fri, Nov 25, 2022 at 06:27:04PM -0300, Ranier Vilela wrote: > > > 5. Use boolean operator with boolean operands > > > (b

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-22 Thread Justin Pryzby
On Fri, Dec 23, 2022 at 11:42:39AM +0900, Michael Paquier wrote: > Hmm. 0001 does a direct check on aclitem as data type used in an > attribute, > For now, I have fixed the most pressing part for tables to match with > the buildfarm +DO $$ + DECLARE +rec text; + col text; + BEGIN +

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-23 Thread Justin Pryzby
On Fri, Dec 23, 2022 at 05:51:28PM +0900, Michael Paquier wrote: > On Thu, Dec 22, 2022 at 09:27:24PM -0600, Justin Pryzby wrote: > > This would do a single seqscan: > > SELECT format('ALTER TABLE %I ALTER COLUMN %I TYPE TEXT', > > attrelid::regclass, attname) FROM

Re: [RFC] building postgres with meson - v13

2022-12-23 Thread Justin Pryzby
> From 680ff3f7b4da1dbf21d0c7cd87af9bb5ee8b230c Mon Sep 17 00:00:00 2001 > From: Andres Freund > Date: Wed, 21 Sep 2022 20:36:36 -0700 > Subject: [PATCH v17 01/23] meson: ci: wip: move compilerwarnings task to meson >always: > gcc_warning_script: | > - time ./configure \ > -

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-26 Thread Justin Pryzby
On Mon, Dec 26, 2022 at 04:28:52PM +0900, Michael Paquier wrote: > Comments? > + file = fopen(filename, PG_BINARY_W); > + if (!file) > + pg_fatal("could not open file \"%s\": %m", filename); > + > + if (fwrite(page, BLCKSZ, 1, file) != 1) > +

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-12-26 Thread Justin Pryzby
On Fri, Dec 09, 2022 at 01:23:56PM +0300, Alexander Korotkov wrote: > Pushed, thanks to everyone! FYI: this causes meson test running ("installcheck") fail when run twice. I guess that's expected to work, per: b62303794efd97f2afb55f1e1b82fffae2cf8a2d f3bbe81db0e84fb486c6423a234c47091b30 6928

<    5   6   7   8   9   10   11   12   13   14   >