Re: pgbench - add pseudo-random permutation function

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-02, Fabien COELHO wrote: > > I'm planning to mark this patch RwF on April 8 and I suggest you > > resubmit if you are able to get some consensus. > > People interested in non-uniform benchmarks would see the point. Why many > people would be happy with uniform benchmarks only while li

Re: Some problems of recovery conflict wait events

2020-04-02 Thread Fujii Masao
On 2020/04/02 15:54, Masahiko Sawada wrote: On Thu, 2 Apr 2020 at 15:34, Fujii Masao wrote: On 2020/04/02 14:25, Masahiko Sawada wrote: On Wed, 1 Apr 2020 at 22:32, Fujii Masao wrote: On 2020/03/30 20:10, Masahiko Sawada wrote: On Fri, 27 Mar 2020 at 17:54, Fujii Masao wrote:

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-04-02 Thread Michael Paquier
On Thu, Apr 02, 2020 at 01:52:09AM -0500, Justin Pryzby wrote: > Sounds right. Or else get_index_isclustered() could be redefined to take a > boolean "do_elog" flag, and if syscache fails and that's false, then return > false instead of ERROR. Not sure if that's completely right to do either. Fo

Re: color by default

2020-04-02 Thread Michael Paquier
On Wed, Apr 01, 2020 at 03:52:17PM +0200, Peter Eisentraut wrote: > On 2020-03-30 10:08, Michael Paquier wrote: >> Cannot you add a link to the page for color support in each one of >> them? That seems more user-friendly to me. > > Do you have a specific phrasing or look in mind? I actually do.

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-02, Michael Paquier wrote: > Now, regarding patch 0002, note that you have a problem for this part: > -tuple = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexOid)); > -if (!HeapTupleIsValid(tuple))/* probably can't happen */ > -{ > -

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-04-02 Thread Michael Paquier
On Thu, Apr 02, 2020 at 04:24:03PM +0900, Michael Paquier wrote: > On Thu, Apr 02, 2020 at 01:52:09AM -0500, Justin Pryzby wrote: >> Sounds right. Or else get_index_isclustered() could be redefined to take a >> boolean "do_elog" flag, and if syscache fails and that's false, then return >> false in

Re: Unicode normalization SQL functions

2020-04-02 Thread Peter Eisentraut
On 2020-03-26 08:25, Peter Eisentraut wrote: On 2020-03-24 10:20, Peter Eisentraut wrote: Now I have some concerns about the size of the new table in unicode_normprops_table.h, and the resulting binary size. At the very least, we should probably make that #ifndef FRONTEND or something like that

Re: Unicode normalization SQL functions

2020-04-02 Thread Peter Eisentraut
On 2020-03-26 18:41, John Naylor wrote: We don't have a trie implementation in Postgres, but we do have a perfect hash implementation. Doing that would bring the tables back to 64 bits per entry, but would likely be noticeably faster than binary search. Since v4 has left out the biggest tables en

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread movead...@highgo.ca
>Sorry about that, attached is the changed patch for PG13, and the one >for older branches will send sooner. A little update for the patch, and patches for all stable avilable. Regards, Highgo Software (Canada/China/Pakistan) URL : www.highgo.ca EMAIL: mailto:movead(dot)li(at)highgo(dot)ca <>

Re: pgbench - add \aset to store results of a combined query

2020-04-02 Thread Michael Paquier
On Thu, Apr 02, 2020 at 08:08:08AM +0200, Fabien COELHO wrote: > ISTM that I submitted a patch to test whether a variable exists in pgbench, > like available in psql (:{?var} I think), but AFAICS it did not pass. Maybe > I should resurect it as it would allow to test simply whether an empty > resul

EINTR while resizing dsm segment.

2020-04-02 Thread Kyotaro Horiguchi
I provided the subject, and added -hackers. > Hello, > I am running postgres 11.5 and we were having issues with shared segments. > So I increased the max_connection as suggested by you guys and reduced my > work_mem to 600M. > > Right now instead, it is the second time I see this error : > > ER

Re: WAL usage calculation patch

2020-04-02 Thread Julien Rouhaud
On Thu, Apr 02, 2020 at 11:07:29AM +0530, Amit Kapila wrote: > On Wed, Apr 1, 2020 at 8:00 PM Julien Rouhaud wrote: > > > > On Wed, Apr 01, 2020 at 04:29:16PM +0530, Amit Kapila wrote: > > > 3. Doing some testing with and without parallelism to ensure WAL usage > > > data is correct would be great

Re: Add A Glossary

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-01, Corey Huinker wrote: > > I propose we define "planner" and make "optimizer" a entry. > > I have no objection to more entries, or edits to entries, but am concerned > that the process leads to someone having to manually merge several > start-from-scratch patches, with no clear sen

Re: [Proposal] Global temporary tables

2020-04-02 Thread Prabhat Sahu
Hi All, I have noted down few behavioral difference in our GTT implementation in PG as compared to Oracle DB: As per my understanding, the behavior of DROP TABLE in case of "Normal table and GTT" in Oracle DB are as below: 1. Any tables(Normal table / GTT) without having data in a session, we

Re: [Proposal] Global temporary tables

2020-04-02 Thread Pavel Stehule
čt 2. 4. 2020 v 10:45 odesílatel Prabhat Sahu napsal: > Hi All, > > I have noted down few behavioral difference in our GTT implementation in > PG as compared to Oracle DB: > As per my understanding, the behavior of DROP TABLE in case of "Normal > table and GTT" in Oracle DB are as below: > >1

Re: WAL usage calculation patch

2020-04-02 Thread Amit Kapila
On Thu, Apr 2, 2020 at 2:00 PM Julien Rouhaud wrote: > > On Thu, Apr 02, 2020 at 11:07:29AM +0530, Amit Kapila wrote: > > On Wed, Apr 1, 2020 at 8:00 PM Julien Rouhaud wrote: > > > > > > On Wed, Apr 01, 2020 at 04:29:16PM +0530, Amit Kapila wrote: > > > > 3. Doing some testing with and without pa

Re: truncating timestamps on arbitrary intervals

2020-04-02 Thread John Naylor
On Tue, Mar 31, 2020 at 4:34 PM Artur Zakirov wrote: > Thank you for new version of the patch. Thanks for taking a look! Attached is v8, which addresses your points, adds tests and fixes some bugs. There are still some WIP detritus in the timezone code, so I'm not claiming it's ready, but it's mu

Re: proposal \gcsv

2020-04-02 Thread Daniel Verite
Alvaro Herrera wrote: > Can we fix that by adding some syntax to allow command aliases? > So you could add to your .psqlrc something like > > \alias \gcsv \pset push all \; \cbuf; \; \pset pop > > where the \cbuf is a hypothetical "function" that expands to the current > query buffer. T

Re: control max length of parameter values logged

2020-04-02 Thread Alexey Bashtanov
Hi, Please see the new version attached. +If greater than zero, bind parameter values reported in non-error +statement-logging messages are trimmed to no more than this many bytes. +If this value is specified without units, it is taken as bytes. +Zero disables log

Re: WIP/PoC for parallel backup

2020-04-02 Thread Rajkumar Raghuwanshi
Hi Asif, My colleague Kashif Zeeshan reported an issue off-list, posting here, please take a look. When executing two backups at the same time, getting FATAL error due to max_wal_senders and instead of exit Backup got completed And when tried to start the server from the backup cluster, getting

Re: WIP/PoC for parallel backup

2020-04-02 Thread Kashif Zeeshan
Hi Asif The backup failed with errors "error: could not connect to server: could not look up local user ID 1000: Too many open files" when the max_wal_senders was set to 2000. The errors generated for the workers starting from backup worke=1017. Please note that the backup directory was also not c

Re: WIP/PoC for parallel backup

2020-04-02 Thread Robert Haas
On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman wrote: > Yes, we are fetching a single file. However, SEND_FILES is still capable of > fetching multiple files in one > go, that's why the name. I don't see why it should work that way. If we're fetching individual files, why have an unused capability

Re: WIP/PoC for parallel backup

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan < kashif.zees...@enterprisedb.com> wrote: > The backup failed with errors "error: could not connect to server: could > not look up local user ID 1000: Too many open files" when the > max_wal_senders was set to 2000. > The errors generated for the worke

Re: WIP/PoC for parallel backup

2020-04-02 Thread Kashif Zeeshan
On Thu, Apr 2, 2020 at 4:48 PM Robert Haas wrote: > On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan < > kashif.zees...@enterprisedb.com> wrote: > >> The backup failed with errors "error: could not connect to server: could >> not look up local user ID 1000: Too many open files" when the >> max_wal_s

Re: adding partitioned tables to publications

2020-04-02 Thread Peter Eisentraut
On 2020-03-30 17:42, Amit Langote wrote: I have updated the comments in apply_handle_tuple_routing() (see 0002) to better explain what's going on with UPDATE handling. I also rearranged the tests a bit for clarity. Attached updated patches. Test coverage for 0002 is still a bit lacking. Plea

Re: SLRU statistics

2020-04-02 Thread Tomas Vondra
On Thu, Apr 02, 2020 at 02:04:10AM +, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: Hi, Thank you for developing great features. The attached patch is a small fix to the committed documentation for the data type name of blks_hit column. Thank you for the patch, pushed. regards -

Re: Add A Glossary

2020-04-02 Thread Jürgen Purtz
+1 and many thanks to Alvaros edits. Kind regards Jürgen Purtz

Re: WAL usage calculation patch

2020-04-02 Thread Julien Rouhaud
On Thu, Apr 02, 2020 at 02:32:07PM +0530, Amit Kapila wrote: > On Thu, Apr 2, 2020 at 2:00 PM Julien Rouhaud wrote: > > > > On Thu, Apr 02, 2020 at 11:07:29AM +0530, Amit Kapila wrote: > > > On Wed, Apr 1, 2020 at 8:00 PM Julien Rouhaud wrote: > > > > > > > > On Wed, Apr 01, 2020 at 04:29:16PM +0

Re: Berserk Autovacuum (let's save next Mandrill)

2020-04-02 Thread Tomas Vondra
On Wed, Apr 01, 2020 at 11:13:12PM -0400, Tom Lane wrote: Dean Rasheed writes: Yeah, that makes sense. I still can't see what might be causing those failures. The tests that were doing an ALTER COLUMN and then expecting to see the results of a non-analysed table ought to be fixed by 0936d1b6f,

Re: WAL usage calculation patch

2020-04-02 Thread Amit Kapila
On Thu, Apr 2, 2020 at 6:18 PM Julien Rouhaud wrote: > > =# select query, calls, wal_bytes, wal_records, wal_num_fpw from > pg_stat_statements where query ilike '%create index%'; > query | calls | wal_bytes | > wal_records | wal_num_fpw > -

Re: Ltree syntax improvement

2020-04-02 Thread Nikita Glukhov
On 02.04.2020 2:46, Tom Lane wrote: Nikita Glukhov writes: [ latest version of ltree syntax extension ] This is going to need another rebase after all the other ltree hacking that just got done. However, I did include 0001 (use a switch) in the commit I just pushed, so you don't need to worr

Re: Resolving the python 2 -> python 3 mess

2020-04-02 Thread Tom Lane
I wrote: > [ a couple patches ] Ping? I wish somebody would review this. I'm not wedded to any of the details, but it would be an embarrassment for us to ship v13 without any response to the fact that Python 2 is EOL. regards, tom lane

Re: WIP/PoC for parallel backup

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan wrote: > Thanks alot Robert, > In this case the backup folder was not being emptied as the backup was > failed, the cleanup should be done in this case too. Does it fail to clean up the backup folder in all cases where the backup failed, or just in t

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-04-02 Thread Julien Rouhaud
New conflict, rebased v9 attached. >From 26b98194d8add282158c65f6ac46c721ba80f498 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 18 Mar 2019 18:55:50 +0100 Subject: [PATCH v9 1/2] Expose queryid in pg_stat_activity and log_line_prefix Similarly to other fields in pg_stat_activity, only

Re: WIP/PoC for parallel backup

2020-04-02 Thread Kashif Zeeshan
On Thu, Apr 2, 2020 at 6:23 PM Robert Haas wrote: > On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan > wrote: > > Thanks alot Robert, > > In this case the backup folder was not being emptied as the backup was > failed, the cleanup should be done in this case too. > > Does it fail to clean up the ba

Re: pgbench - add \aset to store results of a combined query

2020-04-02 Thread Fabien COELHO
Michaël, ISTM that I submitted a patch to test whether a variable exists in pgbench, like available in psql (:{?var} I think), Not sure if improving the readability of the tests is a reason for this patch. So I would suggest to just live with relying on debug() for now to check that a variab

Re: WIP/PoC for parallel backup

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 9:46 AM Kashif Zeeshan < kashif.zees...@enterprisedb.com> wrote: > Does it fail to clean up the backup folder in all cases where the >> backup failed, or just in this case? >> > The cleanup is done in the cases I have seen so far with base > pg_basebackup functionality (not

Re: WAL usage calculation patch

2020-04-02 Thread Dilip Kumar
On Thu, Apr 2, 2020 at 6:41 PM Amit Kapila wrote: > > On Thu, Apr 2, 2020 at 6:18 PM Julien Rouhaud wrote: > > > > =# select query, calls, wal_bytes, wal_records, wal_num_fpw from > > pg_stat_statements where query ilike '%create index%'; > > query | calls

Re: WAL usage calculation patch

2020-04-02 Thread Julien Rouhaud
On Thu, Apr 02, 2020 at 06:40:51PM +0530, Amit Kapila wrote: > On Thu, Apr 2, 2020 at 6:18 PM Julien Rouhaud wrote: > > > > =# select query, calls, wal_bytes, wal_records, wal_num_fpw from > > pg_stat_statements where query ilike '%create index%'; > > query

Re: Berserk Autovacuum (let's save next Mandrill)

2020-04-02 Thread Tom Lane
David Rowley writes: > On Thu, 2 Apr 2020 at 16:13, Tom Lane wrote: >> Quite :-(. While it's too early to declare victory, we've seen no >> more failures of this ilk since 0936d1b6f, so it's sure looking like >> autovacuum did have something to do with it. > How about [1]? It seems related to m

Re: Proposal: Expose oldest xmin as SQL function for monitoring

2020-04-02 Thread James Coleman
On Thu, Apr 2, 2020 at 12:13 AM Craig Ringer wrote: > > > > > On Thu, 2 Apr 2020 at 07:57, Tom Lane wrote: >> >> Alvaro Herrera writes: >> > On 2020-Apr-01, Tom Lane wrote: >> >> The fact that I had to use max(age(...)) in that sample query >> >> hints at one reason: it's really hard to do arith

Re: WIP/PoC for parallel backup

2020-04-02 Thread Asif Rehman
On Thu, Apr 2, 2020 at 4:47 PM Robert Haas wrote: > On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman > wrote: > > Yes, we are fetching a single file. However, SEND_FILES is still capable > of fetching multiple files in one > > go, that's why the name. > > I don't see why it should work that way. If w

Re: [HACKERS] WAL logging problem in 9.4.3?

2020-04-02 Thread Robert Haas
On Wed, Apr 1, 2020 at 11:51 PM Noah Misch wrote: > I've translated the non-vote comments into estimated votes of -0.3, -0.6, > -0.4, +0.5, and -0.3. Hence, I revoke the plan to back-patch. FWIW, I also think that it would be better not to back-patch. The risk of back-patching is that this will

Re: WIP/PoC for parallel backup

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman wrote: >> Why would you need to do that? As long as the process where >> STOP_BACKUP can do the check, that seems good enough. > > Yes, but the user will get the error only after the STOP_BACKUP, not while > the backup is > in progress. So if the backup

Re: Berserk Autovacuum (let's save next Mandrill)

2020-04-02 Thread Tom Lane
I wrote: > I'd be inclined to undo what you did in favor of initializing the > test tables to contain significantly different numbers of rows, > because that would (a) achieve plan stability more directly, > and (b) demonstrate that the planner is actually ordering the > tables by cost correctly.

Re: WAL usage calculation patch

2020-04-02 Thread Dilip Kumar
On Thu, Apr 2, 2020 at 6:41 PM Amit Kapila wrote: > > On Thu, Apr 2, 2020 at 6:18 PM Julien Rouhaud wrote: > > > > =# select query, calls, wal_bytes, wal_records, wal_num_fpw from > > pg_stat_statements where query ilike '%create index%'; > > query | calls

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Kevin Grittner
On Wed, Apr 1, 2020 at 6:59 PM Andres Freund wrote: > index fetches will never even try to > detect that tuples it needs actually have already been pruned away. > I looked at this flavor of problem today and from what I saw: (1) This has been a problem all the way back to 9.6.0. (2) The behavio

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Mark Dilger
> On Apr 1, 2020, at 8:21 PM, Thomas Munro wrote: > > On Sat, Mar 21, 2020 at 11:14 AM Thomas Munro wrote: >> * updated OIDs to avoid collisions >> * added btequalimage to btree/xid8_ops > > Here's the version I'm planning to commit tomorrow, if no one objects. > Changes: > > * txid.c ren

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Mark Dilger
> On Apr 2, 2020, at 9:06 AM, Mark Dilger wrote: > > ("xid8_current" is not exercised by name anywhere in the regression suite, > that I can see.) I spoke too soon. That's exercised in the new xid.sql test file. It didn't show up in my 'git diff', because it's new. Sorry about that. — Ma

Re: Ltree syntax improvement

2020-04-02 Thread Tom Lane
Nikita Glukhov writes: > Rebased patch attached. Thanks for rebasing! The cfbot's not very happy though: 4842ltxtquery_io.c: In function ‘makepol’: 4843ltxtquery_io.c:188:13: error: ‘escaped’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 4844 if (lenval - escaped <=

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Kevin Grittner
On Wed, Apr 1, 2020 at 7:17 PM Andres Freund wrote: > FWIW, with autovacuum=off the query does not get killed until a manual > vacuum, nor if fewer rows are deleted and the table has previously been > vacuumed. > > The vacuum in the second session isn't required. There just needs to be > somethin

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Andres Freund
Hi, On April 2, 2020 9:36:32 AM PDT, Kevin Grittner wrote: >On Wed, Apr 1, 2020 at 7:17 PM Andres Freund >wrote: > >> FWIW, with autovacuum=off the query does not get killed until a >manual >> vacuum, nor if fewer rows are deleted and the table has previously >been >> vacuumed. >> >> The vacuum

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread Daniel Verite
movead...@highgo.ca wrote: > A little update for the patch, and patches for all stable avilable. Some comments about the set_bit/get_bit parts. I'm reading long_bytea_string_bug_fix_ver6_pg10.patch, but that applies probably to the other files meant for the existing releases (I think you

Re: control max length of parameter values logged

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-02, Alexey Bashtanov wrote: > > > + if > > > (log_parameter_max_length_on_error > 0) > > > + { > > > + /* > > > +

Re: backup manifests

2020-04-02 Thread Andres Freund
Hi, On 2020-04-02 13:04:45 -0400, Robert Haas wrote: > And here's another new patch set. After some experimentation, I was > able to manually test the timeline-switch-during-a-base-backup case > and found that it had bugs in both pg_validatebackup and the code I > added to the backend's basebackup

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-02, Thomas Munro wrote: > On Sat, Mar 21, 2020 at 11:14 AM Thomas Munro wrote: > > * updated OIDs to avoid collisions > > * added btequalimage to btree/xid8_ops > > Here's the version I'm planning to commit tomorrow, if no one objects. > Changes: > > * txid.c renamed to xid8funcs.

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-04-02 Thread Alex Malek
On Wed, Feb 19, 2020 at 4:35 PM Alex Malek wrote: > > Hello Postgres Hackers - > > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > "incorrect resource manager data checksum in record at ..." > This has been occurring on average once every 1

Re: Proposal: Expose oldest xmin as SQL function for monitoring

2020-04-02 Thread Andres Freund
Hi, On 2020-04-01 19:57:32 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > On 2020-Apr-01, Tom Lane wrote: > >> The fact that I had to use max(age(...)) in that sample query > >> hints at one reason: it's really hard to do arithmetic correctly > >> on raw XIDs. Dealing with wraparound is a p

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Andres Freund
Hi, On 2020-04-02 14:33:18 -0300, Alvaro Herrera wrote: > On 2020-Apr-02, Thomas Munro wrote: > > > On Sat, Mar 21, 2020 at 11:14 AM Thomas Munro > > wrote: > > > * updated OIDs to avoid collisions > > > * added btequalimage to btree/xid8_ops > > > > Here's the version I'm planning to commit t

Re: Allow continuations in "pg_hba.conf" files

2020-04-02 Thread David Zhang
On 2020-04-01 10:25 p.m., Fabien COELHO wrote: Hello, FWIW, I don't think so. Generally a trailing backspace is an escape character for the following newline.  And '\ ' is a escaped space, which is usualy menas a space itself. In this case escape character doesn't work generally and I think

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-04-02 Thread Andres Freund
Hi, On 2020-02-19 16:35:53 -0500, Alex Malek wrote: > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > "incorrect resource manager data checksum in record at ..." Could you show the *exact* log output please? Because this could temporarily oc

Re: backup manifests

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 1:23 PM Andres Freund wrote: > I suspect its possible to control the timing by preventing the > checkpoint at the end of recovery from completing within a relevant > timeframe. I think configuring a large checkpoint_timeout and using a > non-fast base backup ought to do the

Re: backup manifests

2020-04-02 Thread Andres Freund
Hi, On 2020-04-02 14:16:27 -0400, Robert Haas wrote: > On Thu, Apr 2, 2020 at 1:23 PM Andres Freund wrote: > > I suspect its possible to control the timing by preventing the > > checkpoint at the end of recovery from completing within a relevant > > timeframe. I think configuring a large checkpoi

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Peter Geoghegan
On Tue, Mar 31, 2020 at 11:40 PM Andres Freund wrote: > The problem, as far as I can tell, is that > oldSnapshotControl->head_timestamp appears to be intended to be the > oldest value in the ring. But we update it unconditionally in the "need > a new bucket, but it might not be the very next one"

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Mark Dilger
> On Apr 2, 2020, at 11:01 AM, Andres Freund wrote: > >> >> Hmm, for some reason I had it in my head that we would make these use an >> "epoch/val" output format rather than raw uint64 values. > > Why would we do that? IMO the goal should be to reduce awareness of the > 32bitness of normal x

Re: control max length of parameter values logged

2020-04-02 Thread Tom Lane
Alvaro Herrera writes: > More or less. If you don't add these chars, mbcliplen doesn't think > there's character there, so it ends up not adding the ellipsis. (I > don't remember why it has to be two chars rather than just one.) I think the idea is to be sure that there's a full multibyte chara

Re: backup manifests

2020-04-02 Thread Andres Freund
On 2020-04-02 14:55:19 -0400, Robert Haas wrote: > > Yes, I am asking for something to be changed: I'd like the code that > > read()s the file when computing the checksum to add up how many bytes > > were read, and compare that to the size in the manifest. And if there's > > a difference report an

Re: control max length of parameter values logged

2020-04-02 Thread Tom Lane
Alexey Bashtanov writes: > Please see the new version attached. Pushed with a bit of editorialization. regards, tom lane

Re: Proposal: Expose oldest xmin as SQL function for monitoring

2020-04-02 Thread Tom Lane
Andres Freund writes: > On 2020-04-01 19:57:32 -0400, Tom Lane wrote: >> Agreed, but just knowing what the oldest xmin is doesn't help you >> find *where* it is. Maybe what we need is a view showing all of >> these potential sources of an old xmin. > +1. This would be extermely useful. It's a v

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Tom Lane
Andres Freund writes: > On 2020-04-02 14:33:18 -0300, Alvaro Herrera wrote: >> Hmm, for some reason I had it in my head that we would make these use an >> "epoch/val" output format rather than raw uint64 values. > Why would we do that? IMO the goal should be to reduce awareness of the > 32bitness

Re: backup manifests

2020-04-02 Thread David Steele
On 4/2/20 1:04 PM, Robert Haas wrote: > There are still some things that not everybody is happy about. In particular, Stephen and David are unhappy about using CRC-32C as the default algorithm, but Andres and Noah both think it's a reasonable choice, even if not as robust as everybody will want.

Re: Add A Glossary

2020-04-02 Thread Corey Huinker
On Thu, Apr 2, 2020 at 8:44 AM Jürgen Purtz wrote: > +1 and many thanks to Alvaros edits. > > I did some of the grunt work Alvaro alluded to in v6, and the results are attached and they build, which means there are no invalid links. Notes: * no definition wordings were changed * added a linkend

Re: backup manifests

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 3:26 PM David Steele wrote: > So, with the addition of the 0004 patch down-thread this looks > committable to me. Glad to hear it. Thank you. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: backup manifests

2020-04-02 Thread Robert Haas
On Thu, Apr 2, 2020 at 2:55 PM Robert Haas wrote: > On Thu, Apr 2, 2020 at 2:23 PM Andres Freund wrote: > > > That might make the window fairly wide on normal systems, but I'm not > > > sure about Raspberry Pi BF members or things running > > > CLOBBER_CACHE_ALWAYS/RECURSIVELY. I guess I could tr

Re: backup manifests

2020-04-02 Thread Andres Freund
On 2020-04-02 15:42:48 -0400, Robert Haas wrote: > I suspect I'm not doing quite what you had in mind here... thoughts? I have some ideas, but I think it's complicated enough that I'd not put it in the "pre commit path" for now.

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread Tom Lane
"Daniel Verite" writes: > These 2 tests need to allocate big chunks of contiguous memory, so they > might fail for lack of memory on tiny machines, and even when not failing, > they're pretty slow to run. Are they worth the trouble? Yeah, I'd noticed those on previous readings of the patch. They

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-04-02 Thread legrand legrand
Fujii Masao-4 wrote > On 2020/04/01 18:19, Fujii Masao wrote: > > Finally I pushed the patch! > Many thanks for all involved in this patch! > > As a remaining TODO item, I'm thinking that the document would need to > be improved. For example, previously the query was not stored in pgss > when it

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Peter Geoghegan
On Thu, Apr 2, 2020 at 11:28 AM Peter Geoghegan wrote: > In conclusion, I share Andres' concerns here. There are glaring > problems with how we manipulate the data structure that controls the > effective horizon for pruning. Maybe they can be fixed while leaving > the code that manages the OldSnap

Re: backup manifests

2020-04-02 Thread David Steele
On 4/2/20 3:47 PM, Andres Freund wrote: On 2020-04-02 15:42:48 -0400, Robert Haas wrote: I suspect I'm not doing quite what you had in mind here... thoughts? I have some ideas, but I think it's complicated enough that I'd not put it in the "pre commit path" for now. +1. These would be great

Re: Berserk Autovacuum (let's save next Mandrill)

2020-04-02 Thread David Rowley
On Fri, 3 Apr 2020 at 04:46, Tom Lane wrote: > > I wrote: > > I'd be inclined to undo what you did in favor of initializing the > > test tables to contain significantly different numbers of rows, > > because that would (a) achieve plan stability more directly, > > and (b) demonstrate that the plan

Re: control max length of parameter values logged

2020-04-02 Thread Alexey Bashtanov
Pushed with a bit of editorialization. Great, and thanks for the fixes! Best, Alex

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread James Coleman
On Thu, Apr 2, 2020 at 2:47 PM Mark Dilger wrote: > > > > > On Apr 2, 2020, at 11:01 AM, Andres Freund wrote: > > > >> > >> Hmm, for some reason I had it in my head that we would make these use an > >> "epoch/val" output format rather than raw uint64 values. > > > > Why would we do that? IMO the

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Andres Freund
Hi, On 2020-04-02 11:47:32 -0700, Mark Dilger wrote: > I agree with transitioning to 64-bit xids with 32 bit xid/epoch pairs > as an internal implementation and storage detail only, but we still > have user facing views that don't treat it that way. Note that epochs are not really a thing interna

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Mark Dilger
> On Apr 2, 2020, at 2:13 PM, Andres Freund wrote: > > Hi, > > On 2020-04-02 11:47:32 -0700, Mark Dilger wrote: >> I agree with transitioning to 64-bit xids with 32 bit xid/epoch pairs >> as an internal implementation and storage detail only, but we still >> have user facing views that don't

Re: Should we add xid_current() or a int8->xid cast?

2020-04-02 Thread Andres Freund
Hi, On 2020-04-02 14:26:41 -0700, Mark Dilger wrote: > Since Thomas's patch is really just focused on transitioning from txid > -> xid8, I think this conversation is a bit beyond scope for this > patch, except that "xid8" sounds an awful lot like the new type that > all user facing xid output will

Re: Nicer error when connecting to standby with hot_standby=off

2020-04-02 Thread David Zhang
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: not tested Documentation:not tested I applied the patch to the latest master branch and run a test below. The

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > "Daniel Verite" writes: >> These 2 tests need to allocate big chunks of contiguous memory, so they >> might fail for lack of memory on tiny machines, and even when not failing, >> they're pretty slow to run. Are they worth the trouble? > > Yeah, I'd noticed those on previous r

Re: Add A Glossary

2020-04-02 Thread Alvaro Herrera
On 2020-Apr-02, Corey Huinker wrote: > On Thu, Apr 2, 2020 at 8:44 AM Jürgen Purtz wrote: > > > +1 and many thanks to Alvaros edits. > > > > > I did some of the grunt work Alvaro alluded to in v6, and the results are > attached and they build, which means there are no invalid links. Thank you!

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread Tom Lane
ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > Tom Lane writes: >> Yeah, I'd noticed those on previous readings of the patch. They'd almost >> certainly fail on some of our older/smaller buildfarm members, so they're >> not getting committed, even if they didn't require m

Re: Add A Glossary

2020-04-02 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 07:09:32PM -0300, Alvaro Herrera wrote: > "partition" instead). If you (or anybody) have suggestions for the > definition of "client" and "session", I'm all ears. We already have Session: A Connection to the Database. I propose: Client: A host (or a process o

Datum values consistency within one query

2020-04-02 Thread Paul Ramsey
Imagine a function that was going to take a table of, say, images, and so something to them over and over, like: SELECT pixel_count(img), clr_name, img_name FROM images img CROSS JOIN colors clr When you run this function, you find that a great amount of time is being spend in the decompres

Re: [PATCH] Fix for slow GIN index queries when "gin_fuzzy_search_limit" setting is relatively small for large tables

2020-04-02 Thread Tom Lane
=?UTF-8?B?QWTDqQ==?= writes: >> It seems like what you're actually trying >> to accomplish is to ensure that entryLoadMoreItems's "stepright" path >> is taken, instead of re-descending the index from the root. > What I was primarily trying to do is make sure that when entryLoadMoreItems > is call

Re: Datum values consistency within one query

2020-04-02 Thread Tom Lane
Paul Ramsey writes: > Getting the datum value is really fast, so I can have a cache that > keeps the latest detoasted object around, and update it when the datum > changes, and store the cache information in the parent context. Like > so: Jeez, no, not like that. You're just testing a pointer.

Re: Berserk Autovacuum (let's save next Mandrill)

2020-04-02 Thread Tom Lane
David Rowley writes: > On Fri, 3 Apr 2020 at 04:46, Tom Lane wrote: >> Concretely, I suggest the attached, which replaces the autovac disables >> with adjusting partition boundaries so that the partitions contain >> different numbers of rows. > I've looked over this and I agree that it's a bette

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Andres Freund
Hi, I just spend a good bit more time improving my snapshot patch, so it could work well with a fixed version of the old_snapshot_threshold feature. Mostly so there's no unnecessary dependency on the resolution of the issues in that patch. When testing my changes, for quite a while, I could not

Re: snapshot too old issues, first around wraparound and then more.

2020-04-02 Thread Andres Freund
Hi, On 2020-04-01 12:02:18 -0400, Robert Haas wrote: > I have no objection to the idea that *if* the feature is hopelessly > broken, it should be removed. I don't think we have a real choice here at this point, at least for the back branches. Just about nothing around old_snapshot_threshold work

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-02 Thread Tomas Vondra
Hi, Thanks, the v52 looks almost ready. I've been looking at the two or three things I mentioned, and I have a couple of comments. 1) /* XXX comparison_cost shouldn't be 0? */ I'm not worried about this, because this is not really intriduced by this patch - create_sort_path has the same commen

src/test/regress/standby_schedule

2020-04-02 Thread Thomas Munro
Hello, I guess no human or machine ever runs $SUBJECT, because when I tried it while hunting down users of txid_XXX functions, it failed (see end). To run it, you need a primary/standby pair, here 5441/5442, and then: PGPORT=5441 psql postgres -f sql/hs_primary_setup.sql PGPORT=5442 ./pg_regress

Re: src/test/regress/standby_schedule

2020-04-02 Thread Tom Lane
Thomas Munro writes: > I guess no human or machine ever runs $SUBJECT, because when I tried > it while hunting down users of txid_XXX functions, it failed (see > end). To run it, you need a primary/standby pair, here 5441/5442, and > then: > PGPORT=5441 psql postgres -f sql/hs_primary_setup.sql

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-02 Thread James Coleman
On Thu, Apr 2, 2020 at 8:20 PM Tomas Vondra wrote: > > Hi, > > Thanks, the v52 looks almost ready. I've been looking at the two or > three things I mentioned, and I have a couple of comments. > > > 1) /* XXX comparison_cost shouldn't be 0? */ > > I'm not worried about this, because this is not rea

  1   2   >