Re: ToDo: show size of partitioned table

2019-04-17 Thread Justin Pryzby
A reminder about these. Also, I suggest renaming "On Table" to "Table", for consistency with \di. Justin >From e275a0958f0f2cd826e9683fb24b6f757d0fe6c7 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 7 Apr 2019 18:24:22 -0500 Subject: [PATCH v2] not necessary t

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-04-11 Thread Justin Pryzby
On Fri, Apr 12, 2019 at 02:01:39PM +1200, David Rowley wrote: > On Thu, 11 Apr 2019 at 17:40, Justin Pryzby wrote: > > I tweaked this patch some more (sorry): > > - remove "especially"; > > I think that likely needs to be kept for the PG11 version. I was >

make \d pg_toast.foo show its indices

2019-04-22 Thread Justin Pryzby
It's deliberate that \dt doesn't show toast tables. \d shows them, but doesn't show their indices. It seems to me that their indices should be shown, without having to think and know to query pg_index. postgres=# \d pg_toast.pg_toast_2600 TOAST table "pg_toast.pg_toast_2600" Column | Type

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions

2019-07-16 Thread Justin Pryzby
I realized that the test added to show-childs patch was listing partitioned tables not indices..fixed. >From 237f0bb2a048aa71726eff2580d01404ae3a98b4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subject: [PATCH v5] print table associated with given TO

Re: Zedstore - compressed in-core columnar storage

2019-08-18 Thread Justin Pryzby
On Thu, Aug 15, 2019 at 01:05:49PM +0300, Heikki Linnakangas wrote: > We've continued hacking on Zedstore, here's a new patch version against > current PostgreSQL master (commit f1bf619acdf). If you want to follow the > development in real-time, we're working on this branch: >

Re: Zedstore - compressed in-core columnar storage

2019-08-19 Thread Justin Pryzby
On Mon, Aug 19, 2019 at 04:15:30PM -0700, Alexandra Wang wrote: > On Sun, Aug 18, 2019 at 12:35 PM Justin Pryzby wrote: > > > . I was missing a way to check for compression ratio; > > Here are the ways to check compression ratio for zedstore: > > Table level: >

pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-24 Thread Justin Pryzby
Core was generated by `postgres: telsasoft ts 10.100.2.162(33500) SELECT '. Program terminated with signal 6, Aborted. #0 0x0039ff6325e5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); Missing separate

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-26 Thread Justin Pryzby
On Mon, Aug 26, 2019 at 12:45:24PM -0400, Tom Lane wrote: > However ... there is some pretty interesting info at > https://bugzilla.redhat.com/show_bug.cgi?id=1338673 > suggesting that compiling with a late-model gcc against older RHEL6 > headers could result in bad code. I wonder whether the

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-25 Thread Justin Pryzby
Unfortunately that tells us > very little. > > On Sun, Aug 25, 2019 at 2:25 PM Justin Pryzby wrote: > > #4 0x0039ff678dd0 in _int_free (av=0x39ff98e120, p=0x1d40b090, > > have_lock=0) at malloc.c:4846 > > #5 0x006269e5 in ExecHashJoinNewBatch (pst

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-25 Thread Justin Pryzby
I'm not sure but maybe this is useful ? |(gdb) p VfdCache[2397] |$9 = {fd = -1, fdstate = 0, resowner = 0x24f93e0, nextFree = 2393, lruMoreRecently = 0, lruLessRecently = 2360, seekPos = 73016512, fileSize = 0, fileName = 0x0, fileFlags = 2, fileMode = 384} Knowing this report, very possibly

crash 11.5~

2019-08-07 Thread Justin Pryzby
A daily report crashed repeatedly this morning running pg11.4. I compiled 11.5 and reproduced it there, too, so I'm including backtrace with -O0. I'm trying to dig further into it, but it seems to be crashing under load, but not when I try to narrow down to a single report, which seem to run to

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
Just found this, although I'm not sure what to do about it. If it's corrupt table data, I can restore from backup. ts=# VACUUM FREEZE VERBOSE child.huawei_umts_ucell_201908; INFO: 0: aggressively vacuuming "child.huawei_umts_ucell_201908" LOCATION: lazy_scan_heap, vacuumlazy.c:502 ERROR:

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
I checked this still happens with max_parallel_workers_per_gather=0. Now, I just reproduced using SELECT * FROM that table: (gdb) p thisatt->attrelid $4 = 2015128626 ts=# SELECT 2015128626::regclass; regclass | child.huawei_umts_ucell_201908 (gdb) p thisatt->attnum $1 = 2 (gdb) p attnum # For

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
On Wed, Aug 07, 2019 at 04:51:54PM -0700, Andres Freund wrote: > Hi, > > On 2019-08-07 14:29:28 -0500, Justin Pryzby wrote: > > Just found this, although I'm not sure what to do about it. If it's corrupt > > table data, I can restore from backup. In the meantime, I'v

Re: stress test for parallel workers

2019-07-23 Thread Justin Pryzby
On Wed, Jul 24, 2019 at 10:46:42AM +1200, Thomas Munro wrote: > On Wed, Jul 24, 2019 at 10:42 AM Justin Pryzby wrote: > > On Wed, Jul 24, 2019 at 10:03:25AM +1200, Thomas Munro wrote: > > > On Wed, Jul 24, 2019 at 5:42 AM Justin Pryzby > > > wrote: > > > &

Re: stress test for parallel workers

2019-07-23 Thread Justin Pryzby
On Wed, Jul 24, 2019 at 10:03:25AM +1200, Thomas Munro wrote: > On Wed, Jul 24, 2019 at 5:42 AM Justin Pryzby wrote: > > #2 0x0085ddff in errfinish (dummy=) at > > elog.c:555 > > edata = > > If you have that core, it might be interesting to go to

Re: stress test for parallel workers

2019-07-23 Thread Justin Pryzby
On Wed, Jul 24, 2019 at 11:32:30AM +1200, Thomas Munro wrote: > On Wed, Jul 24, 2019 at 11:04 AM Justin Pryzby wrote: > > I ought to have remembered that it *was* in fact out of space this AM when > > this > > core was dumped (due to having not touched it since

stress test for parallel workers

2019-07-23 Thread Justin Pryzby
Does anyone have a stress test for parallel workers ? On a customer's new VM, I got this several times while (trying to) migrate their DB: < 2019-07-23 10:33:51.552 CDT postgres >FATAL: postmaster exited during a parallel transaction < 2019-07-23 10:33:51.552 CDT postgres >STATEMENT: CREATE

Re: stress test for parallel workers

2019-07-23 Thread Justin Pryzby
On Tue, Jul 23, 2019 at 01:28:47PM -0400, Tom Lane wrote: > ... you'd think an OOM kill would show up in the kernel log. > (Not necessarily in dmesg, though. Did you try syslog?) Nothing in /var/log/messages (nor dmesg ring). I enabled abrtd while trying to reproduce it last week. Since you

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions

2019-07-17 Thread Justin Pryzby
Find attached updated patches which also work against old servers. 1) avoid ::regnamespace; 2) don't PQgetvalue() fields which don't exist and then crash. >From 16b31dc1e4142ed6d0f5f7ed6d65c6184f546a3c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subj

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions (and tablespace)

2019-07-15 Thread Justin Pryzby
r simular entries, so probably this is bad but okay:-) Leaving this for another commit-day. Thanks for testing. Justin >From 237f0bb2a048aa71726eff2580d01404ae3a98b4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subject: [PATCH v5] print table associated wit

default partitions can be partitioned and have default partitions?

2019-09-28 Thread Justin Pryzby
postgres=# CREATE TABLE t(i int)PARTITION BY RANGE(i); CREATE TABLE postgres=# CREATE TABLE t0 PARTITION OF t DEFAULT PARTITION BY RANGE(i); CREATE TABLE postgres=# CREATE TABLE t00 PARTITION OF t0 DEFAULT; -- oh yes CREATE TABLE ... Not sure how it could be useful to partition default into

v12 relnotes: alter system tables

2019-09-27 Thread Justin Pryzby
https://www.postgresql.org/docs/12/release-12.html |Allow modifications of system catalogs' options using ALTER TABLE (Peter Eisentraut) |Modifications of catalogs' reloptions and autovacuum settings are now supported. I wonder if that should say: "... WHEN ALLOW_SYSTEM_TABLE_MODS IS ENABLED."

tab complete for explain SETTINGS

2019-09-26 Thread Justin Pryzby
Here's to hoping this is the worst omission in v12. Justin >From e21f58504e5006de9766fe586550b59167e00ffd Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 26 Sep 2019 21:12:26 -0500 Subject: [PATCH v1] tab completion for explain (SETTINGS) mis

format of pg_upgrade loadable_libraries warning

2019-10-02 Thread Justin Pryzby
e: postgres |Database: too I think the list of databases should be formatted to indicate its association with the preceding error by indentation and verbage, or larger refactoring to present in a list, like: "Databases with library which failed to load: %s: %s", PQerrorMessage(conn), li

typo: postGER

2019-09-29 Thread Justin Pryzby
$ git grep Postger src/backend/po/tr.po:"Bu durum, sistemin semaphore set (SEMMNI) veya semaphore (SEMMNS) sayı sınırlaması aşmasında meydana gelmektedir. Belirtilen parametrelerin değerleri yükseltmelisiniz. Başka seçeneğiniz ise PostgerSQL sisteminin semaphore tütekitimini max_connections

Re: doc: improve PG 12 to_timestamp()/to_date() wording

2019-07-06 Thread Justin Pryzby
On Tue, Apr 30, 2019 at 07:14:04PM -0500, Justin Pryzby wrote: > On Tue, Apr 30, 2019 at 09:48:14PM +0300, Alexander Korotkov wrote: > > I'd like to add couple of comments from my side. > > > > - returns an error because the second template string spa

Re: bitmaps and correlation

2019-11-02 Thread Justin Pryzby
Attached is a fixed and rebasified patch for cfbot. Included inline for conceptual review. >From f3055a5696924427dda280da702c41d2d2796a24 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 1 Jan 2019 16:17:28 -0600 Subject: [PATCH v2] Use correlation statistic in costing bitmap sc

psql \d for wide tables / pattern for individual columns

2019-11-10 Thread Justin Pryzby
ttrelid, attname) "pg_attribute_relid_attnum_index" UNIQUE, btree (attrelid, attnum) >From fdcde33f93af544eb1be0f327ffa49a133397da3 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 10 Nov 2019 15:02:00 -0600 Subject: [PATCH v1] psql: Allow filtering columns shown by \

Re: shared tempfile was not removed on statement_timeout (unreproducible)

2019-12-12 Thread Justin Pryzby
On Fri, Dec 13, 2019 at 03:03:47PM +1300, Thomas Munro wrote: > On Fri, Dec 13, 2019 at 7:05 AM Justin Pryzby wrote: > > I have a nagios check on ancient tempfiles, intended to catch debris left by > > crashed processes. But triggered on this file: > > > > $ sudo find

Re: error context for vacuum to include block number

2019-12-12 Thread Justin Pryzby
On Wed, Dec 11, 2019 at 12:33:53PM -0300, Alvaro Herrera wrote: > On 2019-Dec-11, Justin Pryzby wrote: > > + cbarg.blkno = 0; /* Not known yet */ > Shouldn't you use InvalidBlockNumber for this initialization? .. > > pgstat_progress_update_param(PROGRESS_VACUUM

pg_ls_tmpdir to show shared filesets

2019-12-12 Thread Justin Pryzby
On Thu, Dec 12, 2019, Justin Pryzby wrote in 20191212180506.gr2...@telsasoft.com: > Actually, I tried using pg_ls_tmpdir(), but it unconditionally masks > non-regular files and thus shared filesets. Maybe that's worth discussion on > a > new thread ? > > src/backend/u

Re: error context for vacuum to include block number

2019-12-13 Thread Justin Pryzby
ts); > vacrelstats->num_dead_tuples = 0; > - vacrelstats->num_index_scans++; > You are moving this comment within lazy_vacuum_heap_index, but it > applies to num_dead_tuples and not num_index_scans, no? You should > keep the incrementation of num_index_scans within the routine thoug

Re: pg_ls_tmpdir to show shared filesets

2019-12-14 Thread Justin Pryzby
On Thu, Dec 12, 2019 at 11:39:31PM -0600, Justin Pryzby wrote: > I suggested that pg_ls_tmpdir should show shared filesets like > > 169347 5492 -rw-r- 1 postgres postgres 5619712 Dec 7 01:35 > > /var/lib/pgsql/12/data/base/pgsql_tmp/pgsql_tmp11025.0.sharedfileset/0.0 ..

Re: error context for vacuum to include block number

2019-12-15 Thread Justin Pryzby
On Sun, Dec 15, 2019 at 10:07:08PM +0900, Michael Paquier wrote: > On Fri, Dec 13, 2019 at 04:47:35PM -0600, Justin Pryzby wrote: > > It's related code which I cleaned up before adding new stuff. Not > > essential, > > thus separate (0002 should be backpatched). > >

Re: error context for vacuum to include block number

2019-12-11 Thread Justin Pryzby
On Wed, Dec 11, 2019 at 09:15:07PM +0900, Michael Paquier wrote: > On Fri, Dec 06, 2019 at 10:23:25AM -0600, Justin Pryzby wrote: > > Find attached updated patch: > > . Use structure to include relation name. > > . Split into a separate patch rename of "StringInfoData

shared tempfile was not removed on statement_timeout (unreproducible)

2019-12-12 Thread Justin Pryzby
I have a nagios check on ancient tempfiles, intended to catch debris left by crashed processes. But triggered on this file: $ sudo find /var/lib/pgsql/12/data/base/pgsql_tmp -ls 1429774 drwxr-x--- 3 postgres postgres 4096 Dec 12 11:32 /var/lib/pgsql/12/data/base/pgsql_tmp 1698684

Re: error context for vacuum to include block number

2019-12-06 Thread Justin Pryzby
relation t 2019-11-27 20:04:53.640 CST [14244] STATEMENT: vacuum t; I tried to use BufferGetTag() to avoid using a 2ndary structure, but fails if the buffer is not pinned. >From 41e1d6d118346e84aac7cfe68424f7452c7dcb8d Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 20 Nov 2019 14:53:20

verbose cost estimate

2019-12-07 Thread Justin Pryzby
Jeff said: https://www.postgresql.org/message-id/CAMkU%3D1zBJNVo2DGYBgLJqpu8fyjCE_ys%2Bmsr6pOEoiwA7y5jrA%40mail.gmail.com |What would I find very useful is a verbosity option to get the cost |estimates expressed as a multiplier of each *_cost parameter, rather than |just as a scalar. I guess the

ERROR: could not resize shared memory segment...No space left on device

2019-12-16 Thread Justin Pryzby
A customer's report query hit this error. ERROR: could not resize shared memory segment "/PostgreSQL.2011322019" to 134217728 bytes: No space left on device I found: https://www.postgresql.org/message-id/flat/CAEepm%3D2D_JGb8X%3DLa-0PX9C8dBX9%3Dj9wY%2By1-zDWkcJu0%3DBQbA%40mail.gmail.com

update ALTER TABLE with ATTACH PARTITION lock mode

2019-10-27 Thread Justin Pryzby
n the future.. >From c820a81fba0a6c2388ec58fc0204ca833523e81e Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 27 Oct 2019 18:54:24 -0500 Subject: [PATCH v1 1/2] Mention reduced locking strength of ATTACH PARTITION.. See commit 898e5e32 --- doc/src/sgml/ref/alter_table.sgml | 7 +++ 1 file changed

Re: v12 and pg_restore -f-

2019-10-16 Thread Justin Pryzby
On Wed, Oct 16, 2019 at 03:04:52PM -0400, Stephen Frost wrote: > > On Wed, Oct 16, 2019 at 01:21:48PM -0400, Stephen Frost wrote: > > > [...] if they actually need to work with both concurrently (which strikes > > > me > > > as already at least relatively uncommon...). > > > > I doubt it's

Re: v12 and pg_restore -f-

2019-10-16 Thread Justin Pryzby
On Sun, Oct 06, 2019 at 04:43:13PM -0400, Tom Lane wrote: > Nobody is going to wish that to mean "write to a file named '-'", so Probably right, but it occurs to me that someone could make a named pipe called that, possibly to make "dump to stdout" work with scripts that don't support dumping to

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-16 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 04:18:34PM -0300, Alvaro Herrera wrote: > (FWIW I expect the crash is possible not just in reindex but also in > CREATE INDEX CONCURRENTLY.) FWIW, for sake of list archives, and for anyone running v12 hoping to avoid crashing, I believe we hit this for DROP INDEX

Re: BRIN index which is much faster never chosen by planner

2019-10-15 Thread Justin Pryzby
This reminds me of an issue I reported several years ago where Btree index scans were chosen over seq scan of a large, INSERT-only table due to very high correlation, but performed poorly. I concluded that use of the the high "large scale" correlation on a large 50+GB table caused the planner to

Re: v12.0 ERROR: trying to store a heap tuple into wrong type of slot

2019-10-15 Thread Justin Pryzby
On Tue, Oct 15, 2019 at 01:50:09PM -0700, Andres Freund wrote: > On 2019-10-13 07:51:06 -0700, Andres Freund wrote: > > On 2019-10-11 16:03:20 -0500, Justin Pryzby wrote: > > > ts=# CLUSTER huawei_m2000_config_enodebcell_enodeb USING > > > huawei_m2000_config_eno

v12.0: interrupt reindex CONCURRENTLY: ccold: ERROR: could not find tuple for parent of relation ...

2019-10-15 Thread Justin Pryzby
On a badly-overloaded VM, we hit the previously-reported segfault in progress reporting. This left around some *ccold indices. I tried to drop them but: sentinel=# DROP INDEX child.alarms_null_alarm_id_idx1_ccold; -- child.alarms_null_alarm_time_idx_ccold; -- alarms_null_alarm_id_idx_ccold;

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 03:10:21PM -0300, Alvaro Herrera wrote: > On 2019-Oct-13, Justin Pryzby wrote: > > > Looks like it's a race condition and dereferencing *holder=NULL. The first > > crash was probably the same bug, due to report query running during "reindex > >

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 01:30:29PM -0500, Justin Pryzby wrote: > BTW it probably should've been documented as an "Open Item" for v12. https://commitfest.postgresql.org/25/2278/ I realized possibly people were thinking of that as a "feature" and not a bugfix for backp

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 02:06:02PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Oct 11, 2019 at 10:48:37AM -0400, Tom Lane wrote: > >> Could you provide a self-contained test case please? > > > [ test case ] > > Oh, this is the same is

Re: update ALTER TABLE with ATTACH PARTITION lock mode (docs)

2019-10-28 Thread Justin Pryzby
On Mon, Oct 28, 2019 at 12:06:44PM -0300, Alvaro Herrera wrote: > On 2019-Oct-28, Michael Paquier wrote: > > > On Sun, Oct 27, 2019 at 07:12:07PM -0500, Justin Pryzby wrote: > > > commit #898e5e32 (Allow ATTACH PARTITION with only > > > ShareUpdateExclusi

Re: stress test for parallel workers

2019-10-22 Thread Justin Pryzby
database system is ready to accept connections On Tue, Jul 23, 2019 at 11:27:03AM -0500, Justin Pryzby wrote: > Does anyone have a stress test for parallel workers ? > > On a customer's new VM, I got this several times while (trying to) migrate > their DB: > > < 2019-07-23

Re: v12.0: reindex CONCURRENTLY: lock ShareUpdateExclusiveLock on object 14185/39327/0 is already held

2019-10-23 Thread Justin Pryzby
On Thu, Oct 24, 2019 at 11:42:04AM +0900, Michael Paquier wrote: > Please see the attached. Justin, does it fix your problems regarding > the locks? Confirmed. Thanks, Justin

Re: v12.0: reindex CONCURRENTLY: lock ShareUpdateExclusiveLock on object 14185/39327/0 is already held

2019-10-18 Thread Justin Pryzby
Checking if anybody is working on either of these https://www.postgresql.org/message-id/20191013025145.GC4475%40telsasoft.com https://www.postgresql.org/message-id/20191015164047.GA22729%40telsasoft.com On Sat, Oct 12, 2019 at 09:51:45PM -0500, Justin Pryzby wrote: > I ran into this while try

Re: v12 and pg_restore -f-

2019-10-17 Thread Justin Pryzby
ty to correct myself, before someone else does: On Wed, Oct 16, 2019 at 02:28:40PM -0500, Justin Pryzby wrote: > And vendors (something like pgadmin) will end up "having to" write to a file > to be portable, or else check the full version, not just the major version. I take back tha

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-19 Thread Justin Pryzby
On Tue, Nov 19, 2019 at 04:49:10PM -0600, Justin Pryzby wrote: > On Wed, Nov 20, 2019 at 09:26:53AM +1300, Thomas Munro wrote: > > Perhaps we should not panic if we failed to open (not fsync) the file, > > but it's not the root problem here which is that somehow we though

checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-19 Thread Justin Pryzby
I (finally) noticed this morning on a server running PG12.1: < 2019-11-15 22:16:07.098 EST >PANIC: could not fsync file "base/16491/1731839470.2": No such file or directory < 2019-11-15 22:16:08.751 EST >LOG: checkpointer process (PID 27388) was terminated by signal 6: Aborted /dev/vdb on

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-26 Thread Justin Pryzby
This same crash occured on a 2nd server. Also qemu/KVM, but this time on a 2ndary ZFS tablespaces which (fails to) include the missing relfilenode. Linux database7 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux This is

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-25 Thread Justin Pryzby
I looked and found a new "hint". On Tue, Nov 19, 2019 at 05:57:59AM -0600, Justin Pryzby wrote: > < 2019-11-15 22:16:07.098 EST >PANIC: could not fsync file > "base/16491/1731839470.2": No such file or directory > < 2019-11-15 22:16:08.751 EST

Re: bitmaps and correlation

2019-12-01 Thread Justin Pryzby
On Sun, Dec 01, 2019 at 12:34:37PM +0900, Michael Paquier wrote: > On Sat, Nov 02, 2019 at 03:26:17PM -0500, Justin Pryzby wrote: > > Attached is a fixed and rebasified patch for cfbot. > > Included inline for conceptual review. > > Your patch still causes two regression

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-28 Thread Justin Pryzby
On Fri, Nov 29, 2019 at 10:50:36AM +1300, Thomas Munro wrote: > On Fri, Nov 29, 2019 at 3:13 AM Thomas Munro wrote: > > On Wed, Nov 27, 2019 at 7:53 PM Justin Pryzby wrote: > > > 2019-11-26 23:41:50.009-05 | could not fsync file > > > "pg_tblspc/16401/PG

Re: update ALTER TABLE with ATTACH PARTITION lock mode (docs)

2019-11-01 Thread Justin Pryzby
On Thu, Oct 31, 2019 at 06:07:34PM +0900, Michael Paquier wrote: > On Mon, Oct 28, 2019 at 10:56:33PM -0500, Justin Pryzby wrote: > > I suppose it should something other than partition(ed), since partitions > > can be > > partitioned, too... > > > > Attac

Re: update ALTER TABLE with ATTACH PARTITION lock mode (docs)

2019-11-01 Thread Justin Pryzby
On Fri, Nov 01, 2019 at 11:01:22PM +0900, Michael Paquier wrote: > On Fri, Nov 01, 2019 at 08:59:48AM -0500, Justin Pryzby wrote: > > I guess you mean because it's not a child until after the ALTER. Yes, that > > makes sense. > > Yes, perhaps you have another idea than

planner support functions: handle GROUP BY estimates ?

2019-11-19 Thread Justin Pryzby
Tom implemented "Planner support functions": https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a391ff3c3d418e404a2c6e4ff0865a107752827b https://www.postgresql.org/docs/12/xfunc-optimization.html I wondered whether there was any consideration to extend that to allow providing

error context for vacuum to include block number

2019-11-20 Thread Justin Pryzby
-20 14:52:49.521 CST [6319] STATEMENT: vacuum t; Justin >From 2aac5cdc16c222a053c02818ea2b3a6a5adfb89a Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 20 Nov 2019 14:53:20 -0600 Subject: [PATCH v1] vacuum errcontext to show block being processed As requested here. https://www.po

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-20 Thread Justin Pryzby
On Tue, Nov 19, 2019 at 07:22:26PM -0600, Justin Pryzby wrote: > I was trying to reproduce what was happening: > set -x; psql postgres -txc "DROP TABLE IF EXISTS t" -c "CREATE TABLE t(i int > unique); INSERT INTO t SELECT generate_series(1,99)"; echo "

Re: checkpointer: PANIC: could not fsync file: No such file or directory

2019-11-19 Thread Justin Pryzby
On Wed, Nov 20, 2019 at 09:26:53AM +1300, Thomas Munro wrote: > Perhaps we should not panic if we failed to open (not fsync) the file, > but it's not the root problem here which is that somehow we thought we > should be fsyncing a file that had apparently been removed already > (due to CLUSTER,

consider including server_version in explain(settings)

2019-10-03 Thread Justin Pryzby
explain(SETTINGS) was implemented to show relevant settings for which an odd value could affect a query but could be forgotten during troubleshooting. This is a "concept" patch to show the version, which is frequently requested on -performance list and other support requests. If someone sends

Re: format of pg_upgrade loadable_libraries warning

2019-10-04 Thread Justin Pryzby
On Fri, Oct 04, 2019 at 05:37:46PM -0400, Bruce Momjian wrote: > On Wed, Oct 2, 2019 at 12:23:37PM -0500, Justin Pryzby wrote: > > Regarding the previous thread and commit here: > > https://www.postgresql.org/message-id/flat/20180713162815.GA3835%40momjian.us > > https://git.

v12.0 ERROR: trying to store a heap tuple into wrong type of slot

2019-10-11 Thread Justin Pryzby
I'm not sure why we have that index, and my script probably should have known to choose a better one to cluster on, but still.. ts=# CLUSTER huawei_m2000_config_enodebcell_enodeb USING huawei_m2000_config_enodebcell_enodeb_coalesce_idx ; DEBUG: 0: building index "pg_toast_1840151315_index"

v12.0: ERROR: could not find pathkey item to sort

2019-10-11 Thread Justin Pryzby
I've reduced the failing query as much as possible to this: -- This is necessary to fail: SET enable_nestloop=off; SELECT * FROM (SELECT start_time, t1.site_id FROM pgw_kpi_view t1 -- Apparently the where clause is necessary to fail... WHERE

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-11 Thread Justin Pryzby
On Fri, Oct 11, 2019 at 10:48:37AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > The view is actually a join of two relkind=p partitioned tables (which I > > will acknowledge probably performs poorly). > > Could you provide a self-contained test case please? Working

v12.0: segfault in reindex CONCURRENTLY

2019-10-11 Thread Justin Pryzby
One of our servers crashed last night like this: < 2019-10-10 22:31:02.186 EDT postgres >STATEMENT: REINDEX INDEX CONCURRENTLY child.eric_umts_rnc_utrancell_hsdsch_eul_201910_site_idx < 2019-10-10 22:31:02.399 EDT >LOG: server process (PID 29857) was terminated by signal 11: Segmentation

Re: v12.0: ERROR: could not find pathkey item to sort

2019-10-12 Thread Justin Pryzby
On Fri, Oct 11, 2019 at 10:48:37AM -0400, Tom Lane wrote: > Could you provide a self-contained test case please? SET enable_partitionwise_aggregate = 'on'; SET enable_partitionwise_join = 'on'; SET max_parallel_workers_per_gather=0; -- maybe not important but explain(settings) suggests I should

v12.0: reindex CONCURRENTLY: lock ShareUpdateExclusiveLock on object 14185/39327/0 is already held

2019-10-12 Thread Justin Pryzby
I ran into this while trying to trigger the previously-reported segfault. CREATE TABLE t(i) AS SELECT * FROM generate_series(1,9); CREATE INDEX ON t(i); [pryzbyj@database ~]$ for i in `seq 1 9`; do PGOPTIONS='-cstatement_timeout=9' psql postgres --host /tmp --port 5678 -c "REINDEX INDEX

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-13 Thread Justin Pryzby
Resending this message, which didn't make it to the list when I sent it earlier. (And, notified -www). On Sun, Oct 13, 2019 at 06:06:43PM +0900, Michael Paquier wrote: > On Fri, Oct 11, 2019 at 07:44:46PM -0500, Justin Pryzby wrote: > > Unfortunately, there was no core file, and I'm sti

Re: d25ea01275 and partitionwise join

2019-10-13 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 03:02:17PM -0500, Justin Pryzby wrote: > On Thu, Sep 19, 2019 at 05:15:37PM +0900, Amit Langote wrote: > > Please find attached updated patches. > > Tom pointed me to this thread, since we hit it in 12.0 > https://www.postgresql.org/message-id/fla

Re: d25ea01275 and partitionwise join

2019-10-14 Thread Justin Pryzby
crashes in check-world (possibly due to misapplied hunks). -- Justin Pryzby System Administrator Telsasoft +1-952-707-8581

Re: v12.0: segfault in reindex CONCURRENTLY

2019-10-14 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 06:06:43PM +0900, Michael Paquier wrote: > On Fri, Oct 11, 2019 at 07:44:46PM -0500, Justin Pryzby wrote: > > Unfortunately, there was no core file, and I'm still trying to reproduce it. > > Forgot to set ulimit -c? Having a backtrace would surely hel

Re: planner support functions: handle GROUP BY estimates ?

2019-12-22 Thread Justin Pryzby
On Tue, Nov 19, 2019 at 01:34:21PM -0600, Justin Pryzby wrote: > Tom implemented "Planner support functions": > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a391ff3c3d418e404a2c6e4ff0865a107752827b > https://www.postgresql.org/docs/12/xfunc-optimization.h

vacuum verbose detail logs are unclear (show debug lines at *start* of each stage?)

2019-12-20 Thread Justin Pryzby
This is a usability complaint. If one knows enough about vacuum and/or logging, I'm sure there's no issue. Right now vacuum shows: | 1 postgres=# VACUUM t; | 2 DEBUG: vacuuming "public.t" | 3 DEBUG: scanned index "t_i_key" to remove 999 row versions | 4 DETAIL: CPU: user: 0.00 s,

assert pg_class.relnatts is consistent

2020-02-12 Thread Justin Pryzby
Pryzby wrote: > From 7eea0a17e495fe13379ffd589b551f2f145f5672 Mon Sep 17 00:00:00 2001 > From: Justin Pryzby > Date: Thu, 6 Feb 2020 21:48:13 -0600 > Subject: [PATCH v1 1/3] Update comment obsolete since b9b8831a > > --- > src/backend/commands/cluster.c | 6 +++--- > 1 fi

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-02-28 Thread Justin Pryzby
On Fri, Feb 28, 2020 at 06:26:04PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > I think the attached is 80% complete (I didn't touch pg_dump). > > One objection to this change would be that all relations (including indices) > > end up with relclustered fields, and

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-02-29 Thread Justin Pryzby
VACUUM FULL and REINDEX to change tablespace on the fly On 2020-02-11 19:48, Justin Pryzby wrote: > For your v7 patch, which handles REINDEX to a new tablespace, I have a > few > minor comments: > > + * the relation will be rebuilt. If InvalidOid is used, the default > > =>

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-02-29 Thread Justin Pryzby
On Fri, Feb 28, 2020 at 08:42:02PM -0600, Justin Pryzby wrote: > On Fri, Feb 28, 2020 at 06:26:04PM -0500, Tom Lane wrote: > > Justin Pryzby writes: > > > I think the attached is 80% complete (I didn't touch pg_dump). > > > One objection to this change would be th

Re: vacuum verbose detail logs are unclear; log at *start* of each stage

2020-02-29 Thread Justin Pryzby
On Thu, Feb 27, 2020 at 10:10:57AM +0100, Peter Eisentraut wrote: > On 2020-01-26 06:36, Justin Pryzby wrote: > >Subject: [PATCH v3 1/4] Remove gettext erronously readded at 580ddce > > > >-appendStringInfo(, _("%s."), pg_rusage_show()); > >+app

Re: explain HashAggregate to report bucket and memory stats

2020-03-01 Thread Justin Pryzby
Updated for new tests in 58c47ccfff20b8c125903482725c1dbfd30beade and rebased. >From 3e1904c6c36ee3ff4f56a2808c2400a3b2d2a0e5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 31 Dec 2019 18:49:41 -0600 Subject: [PATCH v6 1/7] explain to show tuplehash bucket and memory stats.. N

Re: vacuum verbose detail logs are unclear; log at *start* of each stage; show allvisible/frozen/hintbits

2020-01-21 Thread Justin Pryzby
Rebased against 40d964ec997f64227bc0ff5e058dc4a5770a70a9 I added to March CF https://commitfest.postgresql.org/27/2425/ >From 83407b81dfc1ed2dfaa6f115dc6c4a276efb07fc Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 27 Nov 2019 20:07:10 -0600 Subject: [PATCH v2 1/6] Rename buf to av

Re: error context for vacuum to include block number

2020-03-04 Thread Justin Pryzby
"reltuples". -- Justin >From a5d981a5ecd867bb46f51a8fb1b153d203df03ac Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 12 Dec 2019 20:54:37 -0600 Subject: [PATCH v24 1/4] vacuum errcontext to show block being processed Discussion: https://www.postgresql.org/message-id/20191120210600.gc30..

Re: error context for vacuum to include block number

2020-03-04 Thread Justin Pryzby
On Tue, Mar 03, 2020 at 04:49:00PM -0300, Alvaro Herrera wrote: > On 2020-Mar-03, Justin Pryzby wrote: > > On Thu, Feb 27, 2020 at 09:09:42PM -0300, Alvaro Herrera wrote: > > > > + case PROGRESS_VACUUM_PHASE_VACUUM_HEAP: > > > > +

Re: error context for vacuum to include block number

2020-03-03 Thread Justin Pryzby
t intended. I was hacked around that by setting ->previous=NULL, but your way in parallel main() seems better. -- Justin >From ca15c197328eb3feb851ec1c3b6ca7e0f1973e93 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 12 Dec 2019 20:54:37 -0600 Subject: [PATCH v23 1/3] vacuum errcontext to show blo

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-03-03 Thread Justin Pryzby
On Tue, Mar 03, 2020 at 02:51:54PM -0500, David Steele wrote: > Hi Fabien, > > On 1/16/20 9:38 AM, Justin Pryzby wrote: > >On Thu, Jan 16, 2020 at 09:34:32AM +0100, Fabien COELHO wrote: > >>Also, I'm not fully sure why ".*" files should be skipped, maybe it sh

Re: error context for vacuum to include block number

2020-02-27 Thread Justin Pryzby
On Thu, Feb 20, 2020 at 02:02:36PM -0300, Alvaro Herrera wrote: > On 2020-Feb-19, Justin Pryzby wrote: > > > Also, I was thinking that lazy_scan_heap doesn't needs to do this: > > > > + /* Pop the error context stack while calling vacuum */ > > +

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-05 Thread Justin Pryzby
On Tue, Mar 03, 2020 at 05:23:13PM -0300, Alvaro Herrera wrote: > On 2020-Mar-03, Justin Pryzby wrote: > > > But I don't think it makes sense to go through more implementation/review > > cycles without some agreement from a larger group regarding the > > desired/int

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-05 Thread Justin Pryzby
On Thu, Mar 05, 2020 at 03:27:31PM +0100, Laurenz Albe wrote: > On Thu, 2020-03-05 at 19:40 +1300, David Rowley wrote: > > 1. I'd go for 2 new GUCs and reloptions. > > autovacuum_vacuum_insert_scale_factor and these should work exactly > > I disagree about the scale_factor (and have not added it

Re: explain HashAggregate to report bucket and memory stats

2020-02-24 Thread Justin Pryzby
hat instead. I also fixed wrong output and wrong non-text formatting for grouping sets, tweaked output for subplan, and broke style rules less often. -- Justin >From 4edb6652f8e8923e0ae7f044817a30b9024b3f49 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 31 Dec 2019 18:49:41 -

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-03-02 Thread Justin Pryzby
@cfbot: resending with only Amit's 0001, since Michael pushed a variation on 0002. -- Justin >From 865fc2713ad29d0f8c0f63609a7c15c83cfa5cfe Mon Sep 17 00:00:00 2001 From: Amit Langote Date: Thu, 6 Feb 2020 18:14:16 +0900 Subject: [PATCH v5] ALTER tbl rewrite loses CLUSTER ON index ---

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-03-02 Thread Justin Pryzby
On Mon, Mar 02, 2020 at 12:28:18PM +0900, Michael Paquier wrote: > > +SELECT indexrelid::regclass FROM pg_index WHERE > > indrelid='concur_clustered'::regclass; > > This test should check after indisclustered. Except that, the patch > is fine so I'll apply it if there are no objections. Oops -

Re: ALTER INDEX fails on partitioned index

2020-02-27 Thread Justin Pryzby
a partitioned table?, STATISTICS, ...). The first patch makes a prettier message, per Robert's suggestion. -- Justin >From e5bb363f514d768a4f540d9c82ad5745944b1486 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 30 Dec 2019 09:31:03 -0600 Subject: [PATCH v2 1/2] More specific error messag

Re: BUG #16109: Postgres planning time is high across version (Expose buffer usage during planning in EXPLAIN)

2020-01-23 Thread Justin Pryzby
On Wed, Nov 13, 2019 at 11:39:04AM +0100, Julien Rouhaud wrote: > (moved to -hackers) > > On Tue, Nov 12, 2019 at 9:55 PM Andres Freund wrote: > > > > This last point is more oriented towards other PG developers: I wonder > > if we ought to display buffer statistics for plan time, for EXPLAIN >

<    1   2   3   4   5   6   7   8   9   10   >