Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-06 Thread Justin Pryzby
On Fri, Aug 07, 2020 at 01:13:51PM +0900, Amit Langote wrote: > On Fri, Aug 7, 2020 at 1:05 PM Justin Pryzby wrote: > > On Fri, Aug 07, 2020 at 12:16:11PM +0900, Amit Langote wrote: > > > Curiously, Justin mentioned upthread that the crash occurred during > > > B

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-06 Thread Justin Pryzby
On Fri, Aug 07, 2020 at 12:16:11PM +0900, Amit Langote wrote: > Curiously, Justin mentioned upthread that the crash occurred during > BIND of a prepared query, so it better had been that a custom plan was > being executed, I'm looking at how to check that ... can you give a hint ? -- Justin

Re: 回复:how to create index concurrently on partitioned table

2020-08-11 Thread Justin Pryzby
Thanks for helping with this. On Wed, Aug 12, 2020 at 01:54:38PM +0900, Michael Paquier wrote: > +++ b/src/backend/catalog/index.c > @@ -3661,20 +3662,12 @@ reindex_relation(Oid relid, int flags, int options) > + elog(ERROR, "unsupported relation kind for relation \"%s\"", > +

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-08-09 Thread Justin Pryzby
On Sun, Aug 09, 2020 at 08:02:52PM +0900, Michael Paquier wrote: > I have been looking at 0001 as a start, and your patch is incorrect on > a couple of aspects for the completion of REINDEX: > - "(" is not proposed as a completion option after the initial > REINDEX, and I think that it should.

Re: 回复:how to create index concurrently on partitioned table

2020-08-08 Thread Justin Pryzby
-- Justin >From 3c465187a190ecfe21dc24dd36e64cbf304cbd17 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 7 Jun 2020 15:59:54 -0500 Subject: [PATCH v4 1/3] Implement REINDEX of partitioned tables/indexes --- doc/src/sgml/ref/reindex.sgml | 5 - src/backend/ca

Re: 回复:how to create index concurrently on partitioned table

2020-08-09 Thread Justin Pryzby
we can rely on > that so as a failure in the middle of the operation never leaves the > a partition structure in an inconsistent state. -- Justin >From 25486d56303de512368f322c87528c2be29af0de Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 7 Jun 2020 15:59:54 -0500 Subject: [P

Re: display offset along with block number in vacuum errors

2020-08-01 Thread Justin Pryzby
On Sun, Aug 02, 2020 at 01:02:42PM +0900, Masahiko Sawada wrote: > Thank you for updating the patch! > > Here are my comments on v3 patch: > > @@ -2024,6 +2033,11 @@ lazy_check_needs_freeze(Buffer buf, bool *hastup) > if (PageIsNew(page) || PageIsEmpty(page)) > return false; > > +

FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-02 Thread Justin Pryzby
Core was generated by `postgres: telsasoft ts [local] BIND '. (gdb) bt #0 0x7f0951303387 in raise () from /lib64/libc.so.6 #1 0x7f0951304a78 in abort () from /lib64/libc.so.6 #2 0x00921005 in ExceptionalCondition

Re: expose parallel leader in CSV and log_line_prefix

2020-07-31 Thread Justin Pryzby
c5f41 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 13 Mar 2020 22:03:06 -0500 Subject: [PATCH v4] Include the leader PID in logfile See also: b025f32e0b, which added the leader PID to pg_stat_activity --- doc/src/sgml/config.sgml | 10 ++- src/backend/utils/err

Re: display offset along with block number in vacuum errors

2020-07-31 Thread Justin Pryzby
Bcc: Subject: Re: display offset along with block number in vacuum errors Reply-To: In-Reply-To: On Wed, Jul 29, 2020 at 12:35:17AM +0530, Mahendra Singh Thalor wrote: > > Here: > > > > @@ -1924,14 +1932,22 @@ lazy_vacuum_page(Relation onerel, BlockNumber > > blkno, Buffer buffer, > >

[PATCH v1] elog.c: Remove special case which avoided %*s format strings..

2020-08-02 Thread Justin Pryzby
..which should no longer be needed since it was a performance hack for specific platform snprintf, which are no longer used. >From dacde652145e65e8de0a0cb7349c9f5f10314243 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 31 Jul 2020 21:53:01 -0500 Subject: [PATCH v1] elog.c: Remove spec

pg_dump from v13 is slow

2020-08-13 Thread Justin Pryzby
I ran into this while running pg_upgrade from beta2 to beta3: $ time sudo -u postgres sh -c 'cd /var/lib/pgsql; /usr/pgsql-13/bin/pg_upgrade -b /usr/pgsql-13b2/bin/ -d ./13b2/data -D ./13/data --link' real94m18.335s This instances has many table partitions, and the production

Re: proposal: possibility to read dumped table's name from file

2020-08-13 Thread Justin Pryzby
On Mon, Jul 27, 2020 at 07:25:54AM +0200, Pavel Stehule wrote: > so 25. 7. 2020 v 15:26 odesílatel vignesh C napsal: > > > On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule > > wrote: > > >> I meant can this: > > >> printf(_(" --filter=FILENAMEread object name filter > > >>

Re: pg_dump from v13 is slow

2020-08-13 Thread Justin Pryzby
I can reproduce the issue with generated data: pryzbyj=# SELECT format('create table t%s(i int)', i) FROM generate_series(1,)i; \set ECHO errors \set QUIET \gexec $ time pg_dump --section=pre-data -d pryzbyj -h /tmp -p 5613 |wc 110015 240049 1577087 real0m50.445s $ time pg_dump

Re: pg_dump from v13 is slow

2020-08-13 Thread Justin Pryzby
On Thu, Aug 13, 2020 at 08:53:46PM -0400, Alvaro Herrera wrote: > Hmm, I wonder if you're comparing an assert-enabled pg13 build to a > non-assert-enabled pg12 build, or something like that. Great question - I thought of it myself but then forgot to look.. $ rpm -q postgresql1{2,3}-server

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Justin Pryzby
On Tue, Aug 04, 2020 at 08:12:10PM +0900, Amit Langote wrote: > It may be this commit that went into PG 12 that is causing the problem: Thanks for digging into this. > to account for partitions that were pruned by the planner for which we > decided to put 0 into relid_map, but it only considered

Re: display offset along with block number in vacuum errors

2020-08-06 Thread Justin Pryzby
On Thu, Aug 06, 2020 at 07:39:21PM +0530, Amit Kapila wrote: > On Wed, Jul 29, 2020 at 1:09 AM Justin Pryzby wrote: > > > > On Wed, Jul 29, 2020 at 12:35:17AM +0530, Mahendra Singh Thalor wrote: > > > Apart from these, I fixed comments given by Sawada and Michael i

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-08-11 Thread Justin Pryzby
src/backend/tcop/utility.c | 6 +++--- src/include/nodes/parsenodes.h | 2 +- 6 files changed, 27 insertions(+), 19 deletions(-) -- Justin >From fed84348477785f8b0b2ae3a1caf2e927e405f96 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 27 Mar 2020 17:50:46 -0500 Subjec

Re: expose parallel leader in CSV and log_line_prefix

2020-07-31 Thread Justin Pryzby
On Fri, Jul 31, 2020 at 03:04:48PM -0500, Justin Pryzby wrote: > On Tue, Jul 28, 2020 at 10:10:33AM +0900, Michael Paquier wrote: > > Except for those nits, I have tested the patch and things behave as we > > want (including padding and docs), so this looks good to me. > &

Re: proposal: possibility to read dumped table's name from file

2020-07-01 Thread Justin Pryzby
On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote: > st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby napsal: > > > + /* ignore empty rows */ > > > + if (*line != '\0') > &

"tuple concurrently updated" in pg_restore --jobs

2020-07-05 Thread Justin Pryzby
I hit this issue intermittently (roughly half the time) while working with a patch David submitted, and finally found a recipe to reproduce it on an unpatched v12 instance. I was surprised to see pg_restore -j2 is restoring ACLs in pre-data in parallel. Note different session IDs and PIDs:

Re: Missing HashAgg EXPLAIN ANALYZE details for parallel plans

2020-07-08 Thread Justin Pryzby
On Fri, Jun 19, 2020 at 02:02:29PM +1200, David Rowley wrote: > if (es->format == EXPLAIN_FORMAT_TEXT) > { > ExplainIndentText(es); > appendStringInfo(es->str, "Sort Method: %s %s: %ldkB\n", ... > As for this patch, I don't think it's unreasonable to have the 3 > possible HashAgg properties on a

Re: [PATCH v2] Allow COPY "text" to output a header and add header matching mode to COPY FROM

2020-07-08 Thread Justin Pryzby
On Wed, Jul 08, 2020 at 03:21:48PM +0200, Daniel Gustafsson wrote: > > On 8 Jul 2020, at 13:45, Rémi Lapeyre wrote: > > > > Hi, here's a new version of the patch that should apply cleanly. I'll > > monitor > > the status on http://cfbot.cputube.org/ > > Please reply to the old thread about

Re: Default setting for enable_hashagg_disk (hash_mem)

2020-07-06 Thread Justin Pryzby
On Mon, Jul 06, 2020 at 09:57:08PM -0700, Jeff Davis wrote: > > I think that we should offer something like hash_mem that can work as > > a multiple of work_mem, for the reason that Justin mentioned > > recently. > > This can be justified as something that more or less maintains some > > kind of

Re: Proposal: Automatic partition creation

2020-07-06 Thread Justin Pryzby
On Mon, Jul 06, 2020 at 01:45:52PM +0300, Anastasia Lubennikova wrote: > The previous discussion of automatic partition creation [1] has addressed > static and dynamic creation of partitions and ended up with several syntax > proposals. ... > where partition_auto_create_clause is > >

Re: proposal: possibility to read dumped table's name from file

2020-07-13 Thread Justin Pryzby
On Mon, Jul 13, 2020 at 12:04:09PM +0200, Daniel Gustafsson wrote: > Sorry for jumping in late, but thinking about this extension to pg_dump: > doesn't it make more sense to use an existing file format like JSON for this, > given that virtually all devops/cd/etc tooling know about JSON already? >

Re: proposal: possibility to read dumped table's name from file

2020-07-13 Thread Justin Pryzby
else + exit_invalid_filter_format(fp, + filename, + "invalid object type (use [tndf])", + line, + lineno); + } + + if (ferror(fp)) + fatal("could not read from file \"%s\": %m", filename); + + if (fp != stdin) + fclose(f

Re: Default setting for enable_hashagg_disk

2020-07-13 Thread Justin Pryzby
On Mon, Jul 13, 2020 at 12:47:36PM -0400, Alvaro Herrera wrote: > On 2020-Jul-13, Jeff Davis wrote: > > > On Tue, 2020-07-14 at 02:25 +1200, David Rowley wrote: > > > Updated summary: > > > * For hash_mem = Tomas [7], Justin [16] > > > * For hash_mem_multiplier with a default > 1.0 = DavidG [21]

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

2020-07-13 Thread Justin Pryzby
ALIZABLE; ANALYZE vactst; COMMIT; +-- MAIN_RELATION_CLEANUP and TOAST_TABLE_CLEANUP options +ALTER TABLE vactst ADD COLUMN t TEXT; +ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL; +VACUUM (MAIN_RELATION_CLEANUP FALSE, TOAST_TABLE_CLEANUP FALSE) vactst; +VACUUM (MAIN_RELATION_CLEANUP FALSE,

Re: DROP relation IF EXISTS Docs and Tests - Bug Fix

2020-07-14 Thread Justin Pryzby
On Tue, Jul 14, 2020 at 07:25:56AM +0200, Pavel Stehule wrote: > út 14. 7. 2020 v 0:37 odesílatel David G. Johnston > napsal: > > On Mon, Jul 13, 2020 at 2:12 AM Pavel Stehule > > wrote: > I think so now all changes are correct and valuable. I''l mark this patch > as ready for commit This is

Re: avoid bitmapOR-ing indexes with scan condition inconsistent with partition constraint

2020-07-14 Thread Justin Pryzby
Added here: https://commitfest.postgresql.org/29/2644/ And updated tests to pass following: |commit 689696c7110f148ede8004aae50d7543d05b5587 |Author: Tom Lane |Date: Tue Jul 14 18:56:49 2020 -0400 | |Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join. >From

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-07-14 Thread Justin Pryzby
On Sun, Jun 21, 2020 at 08:53:25PM -0500, Justin Pryzby wrote: > I'm still waiting to hear feedback from a commiter if this is a good idea to > put this into the system catalog. Right now, ts_debug is the only nontrivial > function. I'm still missing feedback from committers about the f

Re: pg_read_file() with virtual files returns empty string

2020-07-04 Thread Justin Pryzby
Hi Joe Thanks for addressing this. But I noticed that cfbot is now populating with failures like: https://travis-ci.org/github/postgresql-cfbot/postgresql/builds/704898559 genfile.c: In function ‘read_binary_file’: genfile.c:192:5: error: ignoring return value of ‘fread’, declared with

avoid bitmapOR-ing indexes with scan condition inconsistent with partition constraint

2020-07-03 Thread Justin Pryzby
(resending to the list) Hi All I started looking into Konstantin's 30 month old thread/patch: |Re: [HACKERS] Secondary index access optimizations https://www.postgresql.org/message-id/27516421-5afa-203c-e22a-8407e9187327%40postgrespro.ru ..to which David directed me 12 months ago: |Subject: Re:

Re: Default setting for enable_hashagg_disk (hash_mem)

2020-07-02 Thread Justin Pryzby
On Thu, Jul 02, 2020 at 07:05:48PM -0700, Peter Geoghegan wrote: > On Mon, Jun 29, 2020 at 2:46 PM Peter Geoghegan wrote: > > On Mon, Jun 29, 2020 at 2:22 PM Tomas Vondra > > wrote: > > > I agree with this, and I'm mostly OK with having hash_mem. In fact, from > > > the proposals in this thread

Re: Default setting for enable_hashagg_disk (hash_mem)

2020-07-03 Thread Justin Pryzby
On Fri, Jul 03, 2020 at 10:08:08AM -0400, Bruce Momjian wrote: > On Thu, Jul 2, 2020 at 08:35:40PM -0700, Peter Geoghegan wrote: > > But the problem isn't really the hashaggs-that-spill patch itself. > > Rather, the problem is the way that work_mem is supposed to behave in > > general, and the

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 12:45:29PM -0400, Alvaro Herrera wrote: > On 2020-Mar-18, Justin Pryzby wrote: > > > On Sun, Mar 15, 2020 at 12:49:33PM +0100, Julien Rouhaud wrote: > > > template1=# SET log_temp_files=0; explain analyze SELECT a,COUNT(1) FROM t > > a J

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Justin Pryzby
ommands. I also amended the docs. -- Justin >From 68a892d0023e915cc48aea1bb77eae5deabcecec Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 13 Mar 2020 22:03:06 -0500 Subject: [PATCH v2] Include the leader PID in logfile See also: b025f32e0b, which adds the leader PID to pg_stat

Re: proposal: possibility to read dumped table's name from file

2020-07-11 Thread Justin Pryzby
On Mon, Jul 06, 2020 at 06:34:15AM +0200, Pavel Stehule wrote: > >> > Comment support is still missing but easily added :) > >> Still missing from the latest patch. > > > > I can implement a comment support. But I am not sure about the format. The > > here is support for comment's line - first

Re: Avoid useless retrieval of defaults and check constraints in pg_dump -a

2020-07-12 Thread Justin Pryzby
On Sun, Jul 12, 2020 at 07:48:50AM +0200, Julien Rouhaud wrote: > Currently, getTableAttrs() always retrieves info about columns defaults and > check constraints, while this will never be used if --data-only option if > used. > This seems like a waste of resources, so here's a patch to skip those

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 05:36:28PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Jul 10, 2020 at 04:54:40PM -0400, Tom Lane wrote: > >> This works about 99% of the time, in fact. It falls down in the --clean > > > Note that this fails for me (some

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 04:54:40PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I hit this issue intermittently (roughly half the time) while working with a > > patch David submitted, and finally found a recipe to reproduce it on an > > unpatched v12 instance. > &g

Re: Online checksums verification in the backend

2020-07-12 Thread Justin Pryzby
A relfinode should be returned'); +isnt($stdout, '', 'A relfilenode should be returned'); my $filename = $stdout; -- 2.17.0 >From 7b9e61d640b5137dbcd0221dae0c293b705989ba Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 12 Jul 2020 12:25:34 -0500 Subject: [PATCH] fixes

Re: should INSERT SELECT use a BulkInsertState?

2020-07-12 Thread Justin Pryzby
On Thu, Jun 04, 2020 at 10:30:47AM -0700, Andres Freund wrote: > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > Well, the problem is th

Re: Don't choke on files that are removed while pg_rewind runs.

2020-07-13 Thread Justin Pryzby
commit b36805f3c54fe0e50e58bb9e6dad66daca46fbf6 Author: Heikki Linnakangas Date: Sun Jun 28 21:35:51 2015 +0300 ... |@@ -175,22 +175,31 @@ libpqProcessFileList(void) |pg_fatal("unexpected result set while fetching file list\n"); | |/* Read result to local variables */

Re: proposal: possibility to read dumped table's name from file

2020-07-05 Thread Justin Pryzby
On Wed, Jul 01, 2020 at 04:24:52PM -0500, Justin Pryzby wrote: > On Thu, Jun 11, 2020 at 09:36:18AM +0200, Pavel Stehule wrote: > > st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby > > napsal: > > > > +

Re: expose parallel leader in CSV and log_line_prefix

2020-07-09 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 11:09:40AM +0900, Michael Paquier wrote: > On Thu, Jul 09, 2020 at 01:53:39PM +0200, Julien Rouhaud wrote: > > Sure! I've been quite busy with internal work duties recently but > > I'll review this patch shortly. Thanks for the reminder! > > Hmm. In which cases would it

Re: Default setting for enable_hashagg_disk

2020-07-09 Thread Justin Pryzby
On Thu, Jul 09, 2020 at 06:58:40PM -0400, Stephen Frost wrote: > * Peter Geoghegan (p...@bowt.ie) wrote: > > On Thu, Jul 9, 2020 at 7:03 AM Stephen Frost wrote: > > It makes more sense than simply ignoring what our users will see as a > > simple regression. (Though I still lean towards fixing the

Re: Postgresql13_beta1 (could not rename temporary statistics file) Windows 64bits

2020-06-15 Thread Justin Pryzby
On Mon, Jun 15, 2020 at 11:49:33PM -0300, Ranier Vilela wrote: > I can confirm that the problem is in pgrename (dirmod.c), > something is not OK, with MoveFileEx, even with the > (MOVEFILE_REPLACE_EXISTING) flag. > > Replacing MoveFileEx, with > unlink (to); > rename (from, to); > > #if defined

Re: Missing HashAgg EXPLAIN ANALYZE details for parallel plans

2020-06-18 Thread Justin Pryzby
On Thu, Jun 18, 2020 at 03:37:21PM +1200, David Rowley wrote: > Now that HashAgg can spill to disk, we see a few more details in > EXPLAIN ANALYZE than we did previously, e.g. Peak Memory Usage, Disk > Usage. However, the new code neglected to make EXPLAIN ANALYZE show > these new details for

Re: Warn when parallel restoring a custom dump without data offsets

2020-06-22 Thread Justin Pryzby
500, Justin Pryzby wrote: > On Sun, Jun 21, 2020 at 03:18:58PM -0400, David Gilman wrote: > > Thank you for taking a stab at the perl thing. I took the question to > > StackOverflow, I haven't yet looped back to try their suggestion but I > > think there is hope by messing

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-22 Thread Justin Pryzby
On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote: > On 2020-06-22 10:35:47 +0530, Amit Kapila wrote: > > I propose to backpatch b61d161c14 [1] (Introduce vacuum errcontext to > > display additional information.). ... > I think having the additional information in the back branches

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-22 Thread Justin Pryzby
On Mon, Jun 22, 2020 at 06:15:27PM -0400, Tom Lane wrote: > Andres Freund writes: > > No, I don't think that's a solution. I think it's wrong to have > > something like olderrinfo in the first place. Using a struct with ~25 > > members to store the current state of three variables just doesn't

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-22 Thread Justin Pryzby
On Mon, Jun 22, 2020 at 01:57:12PM -0700, Andres Freund wrote: > On 2020-06-22 15:43:11 -0500, Justin Pryzby wrote: > > On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote: > > > I'm also uncomfortable with the approach of just copying all of > > > L

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-23 Thread Justin Pryzby
ted patches for consideration. I left the "struct" patch there to show what it'd look like. -- Justin >From bbfdff2483730d45c663b75de4700e50f09ab4d3 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 22 Jun 2020 18:22:58 -0500 Subject: [PATCH v2 1/5] Rename from "errcbarg"..

Re: Default setting for enable_hashagg_disk

2020-06-24 Thread Justin Pryzby
On Wed, Jun 24, 2020 at 05:06:28AM -0400, Bruce Momjian wrote: > On Wed, Jun 24, 2020 at 02:11:57PM +1200, David Rowley wrote: > > On Tue, 23 Jun 2020 at 08:24, Jeff Davis wrote: > > > Another way of looking at it is that the weird behavior is already > > > there in v12, so there are already

Re: Backpatch b61d161c14 (Introduce vacuum errcontext ...)

2020-06-25 Thread Justin Pryzby
On Thu, Jun 25, 2020 at 02:31:51PM +0530, Amit Kapila wrote: > I have looked at both the patches (using separate variables (by > Justin) and using a struct (by Andres)) and found the second one bit > better. Thanks for looking. > I have improved some comments in the code and for now, kept as two

Re: Postgresql13_beta1 (could not rename temporary statistics file) Windows 64bits

2020-06-14 Thread Justin Pryzby
On Fri, Jun 12, 2020 at 03:15:52PM -0300, Ranier Vilela wrote: > Posgres13_beta1, is consistently writing to the logs, "could not rename > temporary statistics file". > When analyzing the source that writes the log, I simplified the part that > writes the logs a little. What windows version and

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-06-21 Thread Justin Pryzby
emain to be discussed: with or without metadata, hidden files, dotdirs. As I'm still waiting for committer feedback on the first 10 patches, so not intending to add more. -- Justin >From 60593b397f03ec840a97f8004d97177dec463752 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 16 Mar 20

Re: Default setting for enable_hashagg_disk

2020-06-22 Thread Justin Pryzby
On Mon, Jun 22, 2020 at 10:52:37AM -0400, Robert Haas wrote: > On Wed, Jun 10, 2020 at 2:39 PM Jeff Davis wrote: > > The behavior in v13 master is, by default, analagous to Sort or > > anything else that adapts at runtime to spill. If we had spillable > > HashAgg the whole time, we wouldn't be

Re: Missing HashAgg EXPLAIN ANALYZE details for parallel plans

2020-06-18 Thread Justin Pryzby
On Fri, Jun 19, 2020 at 02:02:29PM +1200, David Rowley wrote: > On Fri, 19 Jun 2020 at 01:45, Justin Pryzby wrote: > > Note that "incremental sort" is also new, and splits things up more than > > sort. > > > > See in particular 6a918c3ac8a6b1d8b53cead6fcb7c

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-06-18 Thread Justin Pryzby
On Tue, Apr 07, 2020 at 08:40:30AM -0400, James Coleman wrote: > On Tue, Apr 7, 2020 at 12:25 AM Justin Pryzby wrote: > > On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: > > > I've pushed the fist part of this patch series - I've reorganized it a > > > >

Re: Missing HashAgg EXPLAIN ANALYZE details for parallel plans

2020-06-18 Thread Justin Pryzby
On Fri, Jun 19, 2020 at 03:03:41PM +1200, David Rowley wrote: > On Fri, 19 Jun 2020 at 14:20, Justin Pryzby wrote: > > Please be sure to use two spaces between each field ! > > > > See earlier discussions (and commits referenced by the Opened Items page). > > https://ww

Re: Warn when parallel restoring a custom dump without data offsets

2020-06-13 Thread Justin Pryzby
On Mon, May 25, 2020 at 01:54:29PM -0500, David Gilman wrote: > > Is it possible to dump to stdout (or pipe to cat or dd) to avoid a new > > option ? > > The underlying IPC::Run code seems to support piping in a cross-platform > way. I am not a Perl master though and after spending an evening

Re: Operator class parameters and sgml docs

2020-06-20 Thread Justin Pryzby
And a couple more in spgist.sgml (some of which were not added by this patch). >From ed7bd93c6a6953d8626331de9bcba126ddb79b54 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 20 Jun 2020 16:37:51 -0500 Subject: [PATCH v2] Language fixen for 15cb2bd27009f73a84a35c2ba60fdd105b4bf

Re: Default setting for enable_hashagg_disk

2020-06-20 Thread Justin Pryzby
On Thu, Jun 11, 2020 at 01:22:57PM -0700, Jeff Davis wrote: > > think the names you suggested quite fit, but the idea to use a more > > interesting GUC value might help express the behavior. Perhaps making > > enable_hashagg a ternary "enable_hashagg=on|off|avoid_disk"? The word > > "reject" is

Re: [HACKERS] Restricting maximum keep segments by repslots

2020-06-20 Thread Justin Pryzby
lication slots >From f675c46b25b744dfd17e6139dfb6ab89b28e558f Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 20 Jun 2020 18:03:19 -0500 Subject: [PATCH v1] Tweak user-facing doc in c6550776394e25c1620bc8258427c8f1d448080d --- doc/src/sgml/config.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) di

Re: Operator class parameters and sgml docs

2020-06-20 Thread Justin Pryzby
f options that are specific [-for-]{+to+} this operator class (optional) 7 @@ -778,7 +778,7 @@ options Defines {+a+} set of options that are specific [-for-]{+to+} this operator class (optiona

Re: Default setting for enable_hashagg_disk

2020-06-23 Thread Justin Pryzby
On Wed, Jun 24, 2020 at 02:11:57PM +1200, David Rowley wrote: > On Tue, 23 Jun 2020 at 08:24, Jeff Davis wrote: > > Another way of looking at it is that the weird behavior is already > > there in v12, so there are already users relying on this weird behavior > > as a crutch for some other planner

Re: Add A Glossary

2020-06-16 Thread Justin Pryzby
On Tue, Jun 16, 2020 at 08:09:26PM -0400, Alvaro Herrera wrote: > diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml > index 25b03f3b37..e29b55e5ac 100644 > --- a/doc/src/sgml/glossary.sgml > +++ b/doc/src/sgml/glossary.sgml > @@ -395,15 +395,15 @@ > > The base

Re: how to create index concurrently on paritioned table

2020-06-07 Thread Justin Pryzby
On Sat, Jun 06, 2020 at 09:23:32AM -0500, Justin Pryzby wrote: > On Wed, Jun 03, 2020 at 08:22:29PM +0800, 李杰(慎追) wrote: > > Partitioning is necessary for very large tables. > > However, I found that postgresql does not support create index > > concurrently

Re: proposal: possibility to read dumped table's name from file

2020-06-09 Thread Justin Pryzby
On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote: > po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby napsal: > I still wonder if a better syntax would use a unified --filter option, whose > > argument would allow including/excluding any type of object: > I tried to i

Re: proposal: possibility to read dumped table's name from file

2020-06-09 Thread Justin Pryzby
On Wed, Jun 10, 2020 at 05:03:49AM +0200, Pavel Stehule wrote: > st 10. 6. 2020 v 0:30 odesílatel Justin Pryzby napsal: > > > On Tue, Jun 09, 2020 at 11:46:24AM +0200, Pavel Stehule wrote: > > > po 8. 6. 2020 v 23:30 odesílatel Justin Pryzby > > > napsal: > &

Re: Default setting for enable_hashagg_disk

2020-06-09 Thread Justin Pryzby
On Tue, Jun 09, 2020 at 06:20:13PM -0700, Melanie Plageman wrote: > On Thu, Apr 9, 2020 at 1:02 PM Jeff Davis wrote: > > > 2. enable_groupingsets_hash_disk (default false): > > > > This is about how we choose which grouping sets to hash and which to > > sort when generating mixed mode paths. > >

Re: how to create index concurrently on paritioned table

2020-06-06 Thread Justin Pryzby
On Wed, Jun 03, 2020 at 08:22:29PM +0800, 李杰(慎追) wrote: > Partitioning is necessary for very large tables. > However, I found that postgresql does not support create index concurrently > on partitioned tables. > The document show that we need to create an index on each partition > individually

Re: should CREATE INDEX ON partitioned_table call PreventInTransactionBlock() ?

2020-06-08 Thread Justin Pryzby
On Mon, Jun 08, 2020 at 11:27:26AM -0400, Alvaro Herrera wrote: > On 2020-Jun-08, Justin Pryzby wrote: > > > This blocks writes to all partitions until commit: > > > > postgres=# begin; CREATE INDEX ON pt(i); > > BEGIN > > CREATE INDEX > > > > C

should CREATE INDEX ON partitioned_table call PreventInTransactionBlock() ?

2020-06-08 Thread Justin Pryzby
This blocks writes to all partitions until commit: postgres=# begin; CREATE INDEX ON pt(i); BEGIN CREATE INDEX Compare with CLUSTER rel1, rel2, ..., and REINDEX {SCHEMA|DATABASE|SYSTEM}, which release their locks as soon as each rel is processed. I noticed while implementing REINDEX for

Re: proposal: possibility to read dumped table's name from file

2020-06-08 Thread Justin Pryzby
On Mon, Jun 08, 2020 at 07:18:49PM +0200, Pavel Stehule wrote: > pá 29. 5. 2020 v 20:25 odesílatel Justin Pryzby napsal: > > > On Fri, May 29, 2020 at 04:21:00PM +0200, Pavel Stehule wrote: > > > one my customer has to specify dumped tables name by name. After years and >

Re: how to create index concurrently on partitioned table

2020-06-11 Thread Justin Pryzby
On Sun, Jun 07, 2020 at 01:04:48PM -0500, Justin Pryzby wrote: > On Sat, Jun 06, 2020 at 09:23:32AM -0500, Justin Pryzby wrote: > > On Wed, Jun 03, 2020 at 08:22:29PM +0800, 李杰(慎追) wrote: > > > Partitioning is necessary for very large tables. However, I found that >

pg_dump, gzwrite, and errno

2020-06-11 Thread Justin Pryzby
While testing Pavel's patch for pg_dump --filter, I got: pg_dump: error: could not write to output file: Success [pryzbyj@database postgresql]$ echo $? 1 I see we tried to fix it few years ago: https://www.postgresql.org/message-id/flat/1498120508308.9826%40infotecs.ru

Re: how to create index concurrently on partitioned table

2020-06-12 Thread Justin Pryzby
20 at 04:06:28PM +0800, 李杰(慎追) wrote: > >On Sat, Jun 06, 2020 at 09:23:32AM -0500, Justin Pryzby wrote: > > I looked at CIC now and came up with the attached. All that's needed to > > allow > > this case is to close the relation before recursing to partitions - it >

Re: doc review for v13

2020-06-11 Thread Justin Pryzby
Some new bits, And some old ones. -- Justin >From b74bef6f9d36a2fac71045ab707ed4be65730ba7 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 29 Apr 2020 13:13:29 -0500 Subject: [PATCH v5 1/5] Fix comments for WITH OIDs, removed at 578b22971 Previously mentioned here: ht

Re: Default setting for enable_hashagg_disk

2020-06-11 Thread Justin Pryzby
On Thu, Jun 11, 2020 at 01:22:57PM -0700, Jeff Davis wrote: > On Wed, 2020-06-10 at 11:39 -0700, Jeff Davis wrote: > > 1. Testing the spilling of hashed grouping sets: I'm inclined to just > > get rid of enable_groupingsets_hash_disk and use Melanie's stats- > > hacking approach instead. > >

Re: expose parallel leader in CSV and log_line_prefix

2020-07-17 Thread Justin Pryzby
On Fri, Jul 17, 2020 at 11:35:40AM -0400, Alvaro Herrera wrote: > > On Fri, Jul 17, 2020 at 7:01 AM Michael Paquier wrote: > > > > > Hmm. Knowing if a leader is actually running parallel query or not > > > requires a lookup at lockGroupMembers, that itself requires a LWLock. > > > I think that

Re: expose parallel leader in CSV and log_line_prefix

2020-07-17 Thread Justin Pryzby
On Fri, Jul 17, 2020 at 05:27:21PM -0400, Alvaro Herrera wrote: > On 2020-Jul-17, Justin Pryzby wrote: > > Ok, but should we then consider changing pg_stat_activity for consistency ? > > Probably in v13 to avoid changing it a year later. > > https://git.postgresql.org/git

13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Justin Pryzby
An internal server aborted last night while running a maintenance script. I reproduced this easily running the crashing command in a loop, and verified this is a live issue on REL_13_STABLE (dc6f2fb43). REINDEX INDEX pg_class_tblspc_relfilenode_index It looks like this crashed once before, and I

Re: shared tempfile was not removed on statement_timeout

2020-07-28 Thread Justin Pryzby
On Mon, Jul 27, 2020 at 05:39:02AM -0500, Justin Pryzby wrote: > On Mon, Jul 27, 2020 at 08:00:46PM +1200, Thomas Munro wrote: > > Why can't tuplesort_end do it? > > Because then I think the parallel workers remove their own files, with tests > failing like: > > +ERROR:

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Justin Pryzby
On Tue, Jul 28, 2020 at 11:40:14AM -0400, James Coleman wrote: > > This appears to be an issue with d2d8a229b (Incremental Sort), so I will add > > at: https://wiki.postgresql.org/wiki/PostgreSQL_13_Open_Items > > Is that assumption largely based on the incremental sort patch > refactoring

[PATCH] Tab completion for VACUUM of partitioned tables

2020-07-28 Thread Justin Pryzby
ing2(RELKIND_MATVIEW) ")", + .viscondition = "pg_catalog.pg_table_is_visible(c.oid)", + .namespace = "c.relnamespace", + .result = "pg_catalog.quote_ident(c.relname)", +}; static const SchemaQuery Query_for_list_of_constraints_with_schema = { .

Re: display offset along with block number in vacuum errors

2020-07-28 Thread Justin Pryzby
On Wed, Jul 29, 2020 at 12:35:17AM +0530, Mahendra Singh Thalor wrote: > Apart from these, I fixed comments given by Sawada and Michael in the > latest patch. Attaching v2 patch for review. Thanks. lazy_check_needs_freeze iterates over blocks and this patch changes it to update vacrelstats. I

Re: display offset along with block number in vacuum errors

2020-07-27 Thread Justin Pryzby
On Fri, Jul 24, 2020 at 11:18:43PM +0530, Mahendra Singh Thalor wrote: > Hi hackers, > We discussed in another email thread[1], that it will be helpful if we can > display offset along with block number in vacuum error. Here, proposing a > patch to add offset along with block number in vacuum

Re: shared tempfile was not removed on statement_timeout

2020-07-27 Thread Justin Pryzby
On Mon, Jul 27, 2020 at 08:00:46PM +1200, Thomas Munro wrote: > On Tue, Jul 21, 2020 at 4:33 PM Justin Pryzby wrote: > > /* > > * clean up a spool structure and its substructures. > > */ > > static void > > _bt_spooldestroy(BTSpool *btspool)

Re: proposal: possibility to read dumped table's name from file

2020-07-26 Thread Justin Pryzby
On Sat, Jul 25, 2020 at 06:56:31PM +0530, vignesh C wrote: > On Tue, Jul 14, 2020 at 12:03 PM Pavel Stehule > wrote: > >> I meant can this: > >> printf(_(" --filter=FILENAMEread object name filter > >> expressions from file\n")); > >> be changed to: > >> printf(_("

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-26 Thread Justin Pryzby
On Mon, Jul 27, 2020 at 10:48:45AM +1200, David Rowley wrote: > On Wed, 1 Jul 2020 at 18:46, Jeff Davis wrote: > > > > On Tue, Jun 30, 2020, 7:04 PM David Rowley wrote: > >> > >> Does anyone have any objections to that being changed? > > > > That's OK with me. By the way, I'm on vacation and

Re: expose parallel leader in CSV and log_line_prefix

2020-07-26 Thread Justin Pryzby
varo, and applied the patch > down to 13. Now let's see about the original item of this thread.. Updated with updated wording to avoid "null", per Tom. -- Justin >From 665c9ea8827d4ad94ea0100c5ba93dbf05db5943 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 13 Mar 2020

printing oid with %d

2020-07-27 Thread Justin Pryzby
+JsonEncodeDateTime(char *buf, Datum value, Oid typid) ... + elog(ERROR, "unknown jsonb value datetime type oid %d", typid); I think this should be %u. commit cc4feded0a31d2b732d4ea68613115cb720e624e Author: Andrew Dunstan Date: Tue Jan 16 19:07:13 2018 -0500

Re: Online checksums patch - once again

2020-07-27 Thread Justin Pryzby
On Thu, Jun 25, 2020 at 11:43:00AM +0200, Daniel Gustafsson wrote: > The attached v19 fixes a few doc issues I had missed. + They can also be enabled or disabled at a later timne, either as an offline => time +* awaiting shutdown, but we can continue turning off checksums anyway => a

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-27 Thread Justin Pryzby
On Tue, Jul 28, 2020 at 12:54:35PM +1200, David Rowley wrote: > On Mon, 27 Jul 2020 at 14:54, Justin Pryzby wrote: > > It's unrelated to hashAgg vs hashJoin, but I also noticed that this is shown > > only conditionally: > > > > if (es->

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-27 Thread Justin Pryzby
On Mon, Jul 27, 2020 at 08:20:45PM -0700, Peter Geoghegan wrote: > On Mon, Jul 27, 2020 at 5:54 PM David Rowley wrote: > > hmm. I'm not sure. I think this should follow the same logic as what > > "Disk Usage" follows, and right now we don't show Disk Usage unless we > > spill. Since we only use

Re: [HACKERS] [PATCH] Generic type subscripting

2020-07-19 Thread Justin Pryzby
ce for +* subscripting it's custom code who should define types. should maybe say: "its custom code should define types." (no apostrophe is "possessive") -- Justin >From 6c3fad421619526765127571570e0bcbc7fe1679 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Dat

<    3   4   5   6   7   8   9   10   11   12   >