Re: reindex concurrently and two toast indexes

2020-03-04 Thread Julien Rouhaud
On Wed, Mar 4, 2020 at 6:15 AM Michael Paquier wrote: > > On Tue, Mar 03, 2020 at 06:25:51PM +0900, Michael Paquier wrote: > > Or actually, a more simple solution is to abuse of the two existing > > routines so as the dependency switch is done the other way around, > > from the new index to the

Re: Identifying user-created objects

2020-03-04 Thread Masahiko Sawada
On Wed, 4 Mar 2020 at 16:43, Fujii Masao wrote: > > > > On 2020/02/05 20:26, Masahiko Sawada wrote: > > Hi, > > > > User can create database objects such as functions into pg_catalog. > > But if I'm not missing something, currently there is no > > straightforward way to identify if the object is

Re: [PATCH] Add object names to partition errors

2020-03-04 Thread Chris Bandy
On 3/3/20 11:18 PM, Chris Bandy wrote: > On 3/3/20 10:08 AM, Alvaro Herrera wrote: >> I don't suppose you mean to >> test that every single ereport() call that includes errtable() contains >> a TABLE NAME item. > > Correct. I intend only to test the few calls I'm touching in this > It might be

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-04 Thread Andy Fan
> > >> * There are some changes in existing regression cases that aren't >> visibly related to the stated purpose of the patch, eg it now >> notices that "select distinct max(unique2) from tenk1" doesn't >> require an explicit DISTINCT step. That's not wrong, but I wonder >> if maybe you should

Re: replay pause vs. standby promotion

2020-03-04 Thread Jehan-Guillaume de Rorthais
On Wed, 04 Mar 2020 15:00:54 +0300 Sergei Kornilov wrote: > Hello > > > I want to start this discussion because this is related to the patch > > (propoesd at the thread [1]) that I'm reviewing. It does that partially, > > i.e., prefers the promotion only when the pause is requested by > >

Re: Symbolic names for the values of typalign and typstorage

2020-03-04 Thread Tom Lane
Michael Paquier writes: > On Tue, Mar 03, 2020 at 04:45:51PM -0500, Tom Lane wrote: >> Yeah, that works out very nicely: there's now only one place besides >> tupmacs.h that needs a new #include. >> I did a little more polishing, and consider the attached committable, >> unless anyone has

Re: Minor issues in .pgpass

2020-03-04 Thread Fujii Masao
On 2020/03/04 20:39, Hamid Akhtar wrote: On Tue, Mar 3, 2020 at 8:57 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>> wrote: On 2020/03/03 21:38, Hamid Akhtar wrote: > > > On Mon, Mar 2, 2020 at 6:07 PM Fujii Masao mailto:masao.fu...@oss.nttdata.com>

Re: Minor issues in .pgpass

2020-03-04 Thread Hamid Akhtar
On Wed, Mar 4, 2020 at 4:54 PM Fujii Masao wrote: > > > On 2020/03/04 20:39, Hamid Akhtar wrote: > > > > > > On Tue, Mar 3, 2020 at 8:57 PM Fujii Masao > wrote: > > > > > > > > On 2020/03/03 21:38, Hamid Akhtar wrote: > > > > > > > > > On

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 11:31:27AM +0100, Fabien COELHO wrote: > No objection. I did not know there was such a thing… > > Maybe a more detailed explanation about PG_COLOR could be stored somewhere, > and all affected tools could link to it? Or not. One argument against that position is that each

Re: PG14 target version?

2020-03-04 Thread David Steele
On 3/4/20 10:43 AM, Alvaro Herrera wrote: On 2020-Mar-03, David Steele wrote: Anybody know how to add 14 to the "Target version" dropdown in the CF app? I haven't needed it yet but I'd like it to be there when I do. Done. I see it, thanks! -- -David da...@pgmasters.net

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 10:12:23AM +0100, Daniel Gustafsson wrote: > + color in diagnostics messages. Possible values are > + always, auto, > + never. > > Not being a native english speaker, I might have it backwards, but I find > lists > of values in a sentence like this to be easier to

Re: Refactor compile-time assertion checks for C/C++

2020-03-04 Thread Georgios Kokolatos
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested In my humble opinion the patch improves readability, hence gets my +1. No

Re: logical copy_replication_slot issues

2020-03-04 Thread Arseny Sher
Masahiko Sawada writes: > I've attached the updated version patch that incorporated your > comments. I believe we're going in the right direction for fixing this > bug. I'll register this item to the next commit fest so as not to > forget. I've moved confirmed_flush check to the second lookup

replay pause vs. standby promotion

2020-03-04 Thread Fujii Masao
Hi, Currently if pg_wal_replay_pause() is called after the standby promotion is triggerred before the promotion has successfully finished, WAL replay is paused. That is, the replay pause is preferred than the promotion. Is this desiderable behavior? ISTM that most users including me want the

Re: replay pause vs. standby promotion

2020-03-04 Thread Sergei Kornilov
Hello > I want to start this discussion because this is related to the patch > (propoesd at the thread [1]) that I'm reviewing. It does that partially, > i.e., prefers the promotion only when the pause is requested by > recovery_target_action=pause. But I think that it's reasonable and > more

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-04 Thread David Steele
On 3/4/20 5:36 AM, Fujii Masao wrote: On Tue, Mar 13, 2018 at 10:06 PM David Steele wrote: On 3/6/18 9:25 PM, Michael Paquier wrote: On Tue, Mar 06, 2018 at 02:24:24PM +0300, Ivan Kartyshov wrote: Hello, I now is preparing the patch over syntax that Simon offered. And in few day I will

Re: backup manifests

2020-03-04 Thread tushar
Hi, There is a scenario in which i add something inside the pg_tablespace directory , i am getting an error like- pg_validatebackup: * manifest_checksum = 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04 pg_validatebackup: error: "pg_tblspc/16385/*PG_13_202002271*/test" is

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-04 Thread Laurenz Albe
On Tue, 2020-03-03 at 16:28 +0100, Laurenz Albe wrote: > As a more substantial base for discussion, here is a patch that: > > - introduces a GUC and reloption "autovacuum_vacuum_insert_limit", > default 1000 > > - introduces a statistics counter "inserts_since_vacuum" per table > that

Re: PG14 target version?

2020-03-04 Thread Alvaro Herrera
On 2020-Mar-03, David Steele wrote: > Anybody know how to add 14 to the "Target version" dropdown in the CF app? > > I haven't needed it yet but I'd like it to be there when I do. Done. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: jsonpath syntax extensions

2020-03-04 Thread David Steele
Hi Nikita, On 2/27/20 10:57 AM, Nikita Glukhov wrote: Attached patches implement several useful jsonpath syntax extensions. I already published them two years ago in the original SQL/JSON thread, but then after creation of separate threads for SQL/JSON functions and JSON_TABLE I forgot about

Re: [PATCH] kNN for btree

2020-03-04 Thread Alexander Korotkov
On Wed, Mar 4, 2020 at 4:58 AM Peter Geoghegan wrote: > On Mon, Mar 2, 2020 at 1:27 PM Alexander Korotkov > wrote: > > I've rebased the patchset to the current master and made some > > refactoring. I hope it would be possible to bring it to committable > > shape during this CF. This need more

Re: Minor issues in .pgpass

2020-03-04 Thread Hamid Akhtar
On Tue, Mar 3, 2020 at 8:57 PM Fujii Masao wrote: > > > On 2020/03/03 21:38, Hamid Akhtar wrote: > > > > > > On Mon, Mar 2, 2020 at 6:07 PM Fujii Masao > wrote: > > > > > > > > On 2020/02/29 0:46, Hamid Akhtar wrote: > > > The following review has

Re: Identifying user-created objects

2020-03-04 Thread Fujii Masao
On 2020/03/04 19:14, Masahiko Sawada wrote: On Wed, 4 Mar 2020 at 18:57, Fujii Masao wrote: On 2020/03/04 18:36, Masahiko Sawada wrote: On Wed, 4 Mar 2020 at 18:02, Fujii Masao wrote: On 2020/03/04 17:05, Masahiko Sawada wrote: On Wed, 4 Mar 2020 at 16:43, Fujii Masao wrote:

Re: Cast to uint16 in pg_checksum_page()

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 07:02:43AM -0500, David Steele wrote: > Yes, this is being included in an external project. Previously we have used > a highly marked-up version but we are now trying to pull in the header more > or less verbatim. > > Since this header is specifically designated as

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 10:22:26AM +0100, Juan José Santamaría Flecha wrote: > I think there are a couple tools missing: pg_archivecleanup, pg_ctl, > pg_test_fsync and pg_upgrade. pg_regress also, but there is nothing to do > in the documentation with it. Indeed, true for pg_archivecleanup and

Re: Asynchronous Append on postgres_fdw nodes.

2020-03-04 Thread David Steele
On 2/28/20 3:06 AM, Kyotaro Horiguchi wrote: Hello, this is a follow-on of [1] and [2]. Currently the executor visits execution nodes one-by-one. Considering sharding, Append on multiple postgres_fdw nodes can work simultaneously and that can largely shorten the respons of the whole query.

Re: WAL usage calculation patch

2020-03-04 Thread Julien Rouhaud
On Thu, Feb 20, 2020 at 06:56:27PM +0300, Kirill Bychik wrote: > > вт, 18 февр. 2020 г. в 06:23, Thomas Munro : > > > On Mon, Feb 10, 2020 at 8:20 PM Craig Ringer > > > wrote: > > > > On Wed, 5 Feb 2020 at 21:36, Kirill Bychik > > > > wrote: > > > > > Patch is separated in two parts: core

Re: Cast to uint16 in pg_checksum_page()

2020-03-04 Thread David Steele
On 3/4/20 1:05 AM, Tom Lane wrote: Michael Paquier writes: On Tue, Mar 03, 2020 at 06:37:36PM -0500, David Steele wrote: It seems like an explicit cast to uint16 would be better? Attempting to compile the backend code with -Wconversion leads to many warnings, still there has been at least

Re: Improving connection scalability: GetSnapshotData()

2020-03-04 Thread David Steele
On 3/1/20 3:36 AM, Andres Freund wrote: I think these are pretty nice results. Indeed they are. Is the target version PG13 or PG14? It seems like a pretty big patch to go in the last commitfest for PG13. Regards, -- -David da...@pgmasters.net

Re: Some improvements to numeric sqrt() and ln()

2020-03-04 Thread David Steele
Hi Dean, On 2/28/20 3:15 AM, Dean Rasheed wrote: Attached is a WIP patch to improve the performance of numeric sqrt() and ln(), which also makes a couple of related improvements to div_var_fast(), all of which have knock-on benefits for other numeric functions. The actual impact varies greatly

Re: [Proposal] Global temporary tables

2020-03-04 Thread tushar
On 3/3/20 2:10 PM, 曾文旌(义从) wrote: I fixed in global_temporary_table_v16-pg13.patch. Thanks Wenjing. The reported  issue is fixed now  but  there is an another similar  scenario - if we enable 'on_commit_delete_rows' to true using alter command then getting same issue i.e rows are not removing

Re: Minor issues in .pgpass

2020-03-04 Thread Hamid Akhtar
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Tested and looks fine to me. The new status of this patch

Re: ALTER TEXT SEARCH DICTIONARY tab completion

2020-03-04 Thread Georgios Kokolatos
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested It looks good and does what it says on the tin. One minor nitpick I feel I

Re: HAVE_WORKING_LINK still needed?

2020-03-04 Thread Peter Eisentraut
On 2020-02-28 19:44, Alvaro Herrera wrote: On 2020-Feb-28, Tom Lane wrote: Also +1 for s/durable_link_or_rename/durable_link/. Actually, it's not *that* either, because what the function does is link followed by unlink. So it's more a variation of durable_rename with slightly different

Re: Some improvements to numeric sqrt() and ln()

2020-03-04 Thread Dean Rasheed
On Wed, 4 Mar 2020 at 14:41, David Steele wrote: > > Are these improvements targeted at PG13 or PG14? This seems a pretty > big change for the last CF of PG13. > Well of course that's not entirely up to me, but I was hoping to commit it for PG13. It's very well covered by a large number of

Re: Bug in pg_restore with EventTrigger in parallel mode

2020-03-04 Thread vignesh C
On Fri, Feb 21, 2020 at 12:06 AM Fabrízio de Royes Mello wrote: > > > > On Thu, Feb 20, 2020 at 4:52 AM Michael Paquier wrote: > > > > That sounds right, as event triggers could interact with GRANT and > > REFRESH of matviews, so they should be logically last. Looking at the > > recent commit

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-04 Thread Alexey Kondratov
On 04.03.2020 10:45, Michael Paquier wrote: On Mon, Mar 02, 2020 at 08:59:49PM +0300, Alexey Kondratov wrote: All other remarks look clear for me, so I fix them in the next patch version, thanks. Already done as per the attached, with a new routine named getRestoreCommand() and more done.

Re: Option to dump foreign data in pg_dump

2020-03-04 Thread Ashutosh Bapat
I am just responding on the latest mail on this thread. But the question is about functionality. The proposal is to add a single flag --include-foreign-data which controls whether or not data is dumped for all the foreign tables in a database. That may not serve the purpose. A foreign table may

Re: pgbench: option delaying queries till connections establishment?

2020-03-04 Thread Andres Freund
Hi, On 2020-03-01 22:16:06 +0100, Fabien COELHO wrote: > > Hello Andres, > > > FWIW, leaving windows, error handling, and other annoyances aside, this > > can be implemented fairly simply. See below. > > Attached an attempt at improving things. Awesome! > I've put 2 barriers: one so that all

Re: Portal->commandTag as an enum

2020-03-04 Thread Mark Dilger
> On Mar 2, 2020, at 1:57 PM, Alvaro Herrera wrote: > > I pushed it now. Thanks again! While rebasing some other work on top, I noticed one of your comments is out of date: --- a/src/include/tcop/cmdtaglist.h +++ b/src/include/tcop/cmdtaglist.h @@ -23,7 +23,7 @@ * textual name, so that

Re: Allowing ALTER TYPE to change storage strategy

2020-03-04 Thread Tom Lane
Tomas Vondra writes: > On Mon, Mar 02, 2020 at 02:11:10PM -0500, Tom Lane wrote: >> In the v3 patch below, I've ripped out the ALTER DOMAIN syntax on >> the assumption that we'd do the latter; but I've not written the >> cascade recursion logic, because that seemed like a lot of work >> to do in

Re: jsonpath syntax extensions

2020-03-04 Thread Nikita Glukhov
On 04.03.2020 19:13, David Steele wrote: Hi Nikita, On 2/27/20 10:57 AM, Nikita Glukhov wrote: Attached patches implement several useful jsonpath syntax extensions. I already published them two years ago in the original SQL/JSON thread, but then after creation of separate threads for

Re: Setting ACL

2020-03-04 Thread Stephen Frost
Greetings, * Vik Fearing (v...@postgresfriends.org) wrote: > So I have to manually do a diff of the two acls and generate > GRANT/REVOKE statements? That's not encouraging. :( Not sure if it's helpful to you, but pg_dump has code that generates SQL to do more-or-less exactly this. Thanks,

useless RangeIOData->typiofunc

2020-03-04 Thread Alvaro Herrera
I noticed while going over the multirange types patch that it adds a pointless typiofunc cached OID to a struct used for I/O functions' fn_extra. It seems to go completely unused, so I checked range types (which this was cribbed from) and indeed, it is completely unused there either. My guess is

Re: Allowing ALTER TYPE to change storage strategy

2020-03-04 Thread David G. Johnston
On Wed, Mar 4, 2020 at 4:15 PM Tom Lane wrote: > I wrote: > > I think this is committable --- how about you? > > ... or not. I just noticed that the typcache tracks each type's > typstorage setting, and there's no provision for flushing/reloading > that. > > As far as I can find, there is only

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-03-04 Thread Magnus Hagander
On Mon, Mar 2, 2020 at 10:03 PM Fujii Masao wrote: > > > > On 2020/03/03 14:37, Shinoda, Noriyoshi (PN Japan A Delivery) wrote: > > Hi, > > > > Thank you for developing good features. > > The attached patch is a small fix to the committed documentation. This > > patch fixes the description

Re: [PATCH] distinct aggregates within a window function WIP

2020-03-04 Thread Krasiyan Andreev
I have currently suspended development of this patch, based on it's review, but I will continue development of the other Oliver Ford's work about adding support of respect/ignore nulls for lag(),lead(),first_value(),last_value() and nth_value() and from first/last for nth_value() patch, but I am

Re: Minor issues in .pgpass

2020-03-04 Thread Fujii Masao
On 2020/03/04 23:01, Hamid Akhtar wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Tested and

Re: backup manifests

2020-03-04 Thread Suraj Kharage
On Wed, Mar 4, 2020 at 7:21 PM tushar wrote: > Hi, > > There is a scenario in which i add something inside the pg_tablespace > directory , i am getting an error like- > > pg_validatebackup: * manifest_checksum = > 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04 >

[PATCH] respect/ignore nulls for lag,lead,first_value,last_value and nth_value and from first/last for nth_value

2020-03-04 Thread Krasiyan Andreev
Hi hackers, I want to continue development of Oliver Ford's respect/ignore nulls for lag,lead,first_value,last_value and nth_value and from first/last for nth_value patch, but I am not sure how to proceed with it and any feedback will be very useful. I have dropped support of from first/last for

kill_prior_tuple and index scan costing

2020-03-04 Thread Peter Geoghegan
If I run the regression tests so that the "tenk1" table is available, and then create an index on tenk1.twothousand, I notice that simple "where twothousand = ?" queries have query plans that look like the following sample plan: pg@regression:5432 [17755]=# explain (analyze, buffers, costs off)

Re: Crash by targetted recovery

2020-03-04 Thread Kyotaro Horiguchi
At Mon, 2 Mar 2020 20:54:04 +0900, Fujii Masao wrote in > > And random access during StandbyMode ususally (always?) lets RecPtr go > > back. I'm not sure WaitForWALToBecomeAvailable works correctly if we > > don't have a file in pg_wal and the REDO point is far back by more > > than a segment

Re: Identifying user-created objects

2020-03-04 Thread Kyotaro Horiguchi
At Wed, 4 Mar 2020 21:07:05 +0900, Fujii Masao wrote in > >> The function that you are proposing is really enough for this use > >> case? > >> What if malicious users directly change the oid of function > >> to < FirstNormalObjectId? Or you're assuming that malicious users will

Re: backup manifests

2020-03-04 Thread Suraj Kharage
On Wed, Mar 4, 2020 at 3:51 PM tushar wrote: > Another scenario, in which if we modify Manifest-Checksum" value from > backup_manifest file , we are not getting an error > > [centos@tushar-ldap-docker bin]$ ./pg_validatebackup data/ > pg_validatebackup: * manifest_checksum = >

Re: reindex concurrently and two toast indexes

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 09:21:45AM +0100, Julien Rouhaud wrote: > Thanks for the patch! I started to look at it during the weekend, but > I got interrupted and unfortunately didn't had time to look at it > since. No problem, thanks for looking at it. I have looked at it again this morning, and

Re: Identifying user-created objects

2020-03-04 Thread Fujii Masao
On 2020/03/05 12:32, Kyotaro Horiguchi wrote: At Wed, 4 Mar 2020 21:07:05 +0900, Fujii Masao wrote in The function that you are proposing is really enough for this use case? What if malicious users directly change the oid of function to < FirstNormalObjectId? Or you're assuming that

Re: [PATCH] Incremental sort

2020-03-04 Thread James Coleman
On Tue, Mar 3, 2020 at 1:43 PM Tomas Vondra wrote: > On Tue, Mar 03, 2020 at 12:17:22PM -0500, David Steele wrote: > >James and Tomas, > > > >On 1/21/20 10:03 AM, James Coleman wrote: > >>On Tue, Jan 21, 2020 at 9:58 AM Tomas Vondra > >> wrote: > >>> > >>>On Tue, Jan 21, 2020 at 09:37:01AM

Re: PATCH: add support for IN and @> in functional-dependency statistics use

2020-03-04 Thread Tomas Vondra
Hi Pierre, I've looked at this patch series, hoping to get it close to committable. Here is a somewhat improved version of the patch series, split into 5 pieces. The first 4 parts are about applying functional dependencies to ScalarArrayOpExpr clauses. The last part is about doing the same thing

Command statistics system (cmdstats)

2020-03-04 Thread Mark Dilger
Hackers, as mentioned in [1], I have created an implementation of command counter statistics very similar in purpose to the one already pending in the commitfest going by the name "pg_stat_sql". I don't really care if this implementation is seen as building on that one or as separate, but I

Re: Cast to uint16 in pg_checksum_page()

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 09:52:08PM +0900, Michael Paquier wrote: > This sounds like a sensible argument, similar to the ones raised on > the other thread, so no objections from me to improve things here. I > can look at that tomorrow, except if somebody else beats me to it. And done. -- Michael

Re: Identifying user-created objects

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 06:57:00PM +0900, Fujii Masao wrote: > Yes. But I'm sure that DBA has already considered the measures > againt such threads. Otherwise malicious users can do anything > more malicious rather than changing oid. A superuser is by definition able to do anything on the system

Re: Asynchronous Append on postgres_fdw nodes.

2020-03-04 Thread Thomas Munro
On Fri, Feb 28, 2020 at 9:08 PM Kyotaro Horiguchi wrote: > - v2-0001-Allow-wait-event-set-to-be-regsitered-to-resoure.patch > The async feature uses WaitEvent, and it needs to be released on > error. This patch makes it possible to register WaitEvent to > resowner to handle that case.. +1

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-03-04 Thread Fujii Masao
On 2020/03/05 9:31, Magnus Hagander wrote: On Mon, Mar 2, 2020 at 10:03 PM Fujii Masao wrote: On 2020/03/03 14:37, Shinoda, Noriyoshi (PN Japan A Delivery) wrote: Hi, Thank you for developing good features. The attached patch is a small fix to the committed documentation. This patch

Re: Change atoi to strtol in same place

2020-03-04 Thread Joe Nelson
Daniel Gustafsson wrote: > > On 11 Feb 2020, at 17:54, Alvaro Herrera wrote: > > > > This patch doesn't currently apply; it has conflicts with at least > > 01368e5d9da7 and 7e735035f208; even in 7e735035f208^ it applies with > > fuzz. Please post an updated version so that it can move forward.

Re: range_agg

2020-03-04 Thread Paul Jungwirth
Thanks for looking at this again! On 3/4/20 1:33 PM, Alvaro Herrera wrote: I came across an interesting thing, namely multirange_canonicalize()'s use of qsort_arg with a callback of range_compare(). range_compare() calls range_deserialize() (non-trivial parsing) for each input range;

Re: Allowing ALTER TYPE to change storage strategy

2020-03-04 Thread Tom Lane
I wrote: > 3. Drop the ability for ALTER TYPE to promote from PLAIN to not-PLAIN > typstorage, and adjust the typcache so that it only remembers boolean > toastability not the specific toasting strategy. Then the cache is > still immutable so no need for update logic. > > I'm kind of liking #3,

Re: [PATCH] Finally split StdRdOptions into HeapOptions and ToastOptions

2020-03-04 Thread Nikolay Shaplov
В письме от понедельник, 9 декабря 2019 г. 12:11:17 MSK пользователь Michael Paquier написал: > On Fri, Dec 06, 2019 at 06:18:48PM +0300, Nikolay Shaplov wrote: > > In the thread > > https://www.postgresql.org/message-id/2620882.s52SJui4ql@x200m > > I've suggested to split one big StdRdOption

Re: error context for vacuum to include block number

2020-03-04 Thread Justin Pryzby
On Wed, Mar 04, 2020 at 04:21:06PM +0900, Masahiko Sawada wrote: > Thank you for updating the patch. But we have two more places where we > do fsm vacuum. Oops, thanks. I realized that vacuum_page is called not only from lazy_vacuum_heap, but also directly from lazy_scan_heap, which failed to

Re: New SQL counter statistics view (pg_stat_sql)

2020-03-04 Thread Mark Dilger
> On Mar 3, 2020, at 6:50 PM, Alvaro Herrera wrote: > > On 2019-Nov-13, Smith, Peter wrote: > >> From: Thomas Munro Sent: Monday, 4 November 2019 >> 1:43 PM >> >>> No comment on the patch but I noticed that the documentation changes don't >>> build. Please make sure you can "make docs"

Re: error context for vacuum to include block number

2020-03-04 Thread Justin Pryzby
On Tue, Mar 03, 2020 at 04:49:00PM -0300, Alvaro Herrera wrote: > On 2020-Mar-03, Justin Pryzby wrote: > > On Thu, Feb 27, 2020 at 09:09:42PM -0300, Alvaro Herrera wrote: > > > > + case PROGRESS_VACUUM_PHASE_VACUUM_HEAP: > > > > + if

Re: Ought to use heap_multi_insert() for pg_attribute/depend insertions?

2020-03-04 Thread Daniel Gustafsson
> On 2 Mar 2020, at 03:06, Michael Paquier wrote: Thanks a lot for another round of review, much appreciated! > On Fri, Feb 28, 2020 at 05:24:29PM +0900, Michael Paquier wrote: >> + /* >> +* CONTAINS_NEW_TUPLE will always be set unless the multi_insert was >> +* performed for a

Re: Allowing ALTER TYPE to change storage strategy

2020-03-04 Thread Tom Lane
I wrote: > I think this is committable --- how about you? ... or not. I just noticed that the typcache tracks each type's typstorage setting, and there's no provision for flushing/reloading that. As far as I can find, there is only one place where the cached value is used, and that's in

Re: Add LogicalTapeSetExtend() to logtape.c

2020-03-04 Thread Jeff Davis
On Wed, 2020-03-04 at 11:57 +0800, Adam Lee wrote: > Master(e537aed61d): 13342.844 ms 13195.982 ms 13271.023 ms > With my patch(a pointer): 13020.029 ms 13008.158 ms 13063.658 ms > With your patch(flexible array): 12870.117 ms 12814.725 ms 13119.255 > ms I tracked the problem down. When we

Re: range_agg

2020-03-04 Thread Alvaro Herrera
I came across an interesting thing, namely multirange_canonicalize()'s use of qsort_arg with a callback of range_compare(). range_compare() calls range_deserialize() (non-trivial parsing) for each input range; multirange_canonicalize() later does a few extra deserialize calls of its own. Call me

Re: useless RangeIOData->typiofunc

2020-03-04 Thread Paul Jungwirth
On 3/4/20 1:57 PM, Alvaro Herrera wrote: I noticed while going over the multirange types patch that it adds a pointless typiofunc cached OID to a struct used for I/O functions' fn_extra. It seems to go completely unused, so I checked range types (which this was cribbed from) and indeed, it is

Re: Asynchronous Append on postgres_fdw nodes.

2020-03-04 Thread Kyotaro Horiguchi
At Wed, 4 Mar 2020 09:56:55 -0500, David Steele wrote in > On 2/28/20 3:06 AM, Kyotaro Horiguchi wrote: > > Hello, this is a follow-on of [1] and [2]. > > Currently the executor visits execution nodes one-by-one. Considering > > sharding, Append on multiple postgres_fdw nodes can work > >

Re: useless RangeIOData->typiofunc

2020-03-04 Thread Tom Lane
Alvaro Herrera writes: > I noticed while going over the multirange types patch that it adds a > pointless typiofunc cached OID to a struct used for I/O functions' > fn_extra. It seems to go completely unused, so I checked range types > (which this was cribbed from) and indeed, it is completely

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 08:14:20PM +0300, Alexey Kondratov wrote: > On 04.03.2020 10:45, Michael Paquier wrote: > - *        Functions for finding and validating executable files > + *        Functions for finding and validating from executables files > > There is probably something missing here.

Re: pg_stat_progress_basebackup - progress reporting for pg_basebackup, in the server side

2020-03-04 Thread Peter Eisentraut
On 2020-03-05 05:53, Fujii Masao wrote: Or, as another approach, it might be worth considering to make the server always estimate the total backup size whether --progress is specified or not, as Amit argued upthread. If the time required to estimate the backup size is negligible compared to

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 10:05:30PM +0900, Michael Paquier wrote: > On Wed, Mar 04, 2020 at 11:31:27AM +0100, Fabien COELHO wrote: >> For "pgbench", you could also add the standard sentence that it uses libpq >> environment variables, as it is also missing? > > Yeah, that's true. Let's fix this

Re: backup manifests

2020-03-04 Thread Rajkumar Raghuwanshi
Hi, In a negative test scenario, if I changed size to -1 in backup_manifest, pg_validatebackup giving error with a random size number. [edb@localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold --manifest-checksum 'SHA256' [edb@localhost bin]$ ./pg_validatebackup /tmp/bold pg_validatebackup:

Re: Make mesage at end-of-recovery less scary.

2020-03-04 Thread Kyotaro Horiguchi
Hello. I changed the condition from randAccess to fetching_ckpt considering the discussion in another thread [1]. Then I moved the block that shows the new messages to more appropriate place. At Fri, 28 Feb 2020 17:28:06 +0900 (JST), Kyotaro Horiguchi wrote in > > > > Have you considered an

Re: Identifying user-created objects

2020-03-04 Thread Kyotaro Horiguchi
At Thu, 5 Mar 2020 15:21:49 +0900, Masahiko Sawada wrote in > > > I don't come up with another use cases but, anyway, I think we need to > > > clarify the scope of the feature. > > > > Agreed. Also we would need to consider that the existing approach > > (e.g., checking whether the object is

Re: Berserk Autovacuum (let's save next Mandrill)

2020-03-04 Thread David Rowley
On Thu, 5 Mar 2020 at 04:15, Laurenz Albe wrote: > I just realized that the exercise is pointless unless that > autovacuum also runs with FREEZE on. I think we need to move forward with doing something to cope with INSERT-only tables not being auto-vacuumed. I think the patch you have is

Re: [HACKERS] make async slave to wait for lsn to be replayed

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 07:17:31AM -0500, David Steele wrote: > On 3/4/20 5:36 AM, Fujii Masao wrote: >> There are no updates from about two years before, but this patch >> has been registered in CF 2020-03. Not sure why. It should be marked >> as Returned with Feedback again? > > Worse, it was

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-04 Thread Amit Kapila
On Wed, Mar 4, 2020 at 11:45 AM Mahendra Singh Thalor wrote: > > On Mon, 24 Feb 2020 at 15:39, Amit Kapila wrote: > > > > On Thu, Feb 20, 2020 at 8:06 AM Andres Freund wrote: > > > > > > What I'm advocating is that extension locks should continue to go > > > through lock.c. And yes, that

Re: Identifying user-created objects

2020-03-04 Thread Masahiko Sawada
On Thu, 5 Mar 2020 at 13:23, Fujii Masao wrote: > > > > On 2020/03/05 12:32, Kyotaro Horiguchi wrote: > > At Wed, 4 Mar 2020 21:07:05 +0900, Fujii Masao > > wrote in > The function that you are proposing is really enough for this use > case? > What if malicious users

Re: WAL usage calculation patch

2020-03-04 Thread Michael Paquier
On Wed, Mar 04, 2020 at 05:02:25PM +0100, Julien Rouhaud wrote: > I'm quite worried about the stability of those counters for regression tests. > Wouldn't a checkpoint happening during the test change them? Yep. One way to go through that would be to test if this output is non-zero still I

Re: Identifying user-created objects

2020-03-04 Thread Masahiko Sawada
On Wed, 4 Mar 2020 at 15:28, vignesh C wrote: > > On Wed, Mar 4, 2020 at 9:02 AM Masahiko Sawada > wrote: > > > > On Tue, 3 Mar 2020 at 23:33, vignesh C wrote: > > > > > > Should we add some check if object exists or not here: > > > +Datum > > > +pg_is_user_object(PG_FUNCTION_ARGS) > > > +{ > >

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-03-04 Thread Mahendra Singh Thalor
On Wed, 4 Mar 2020 at 12:03, Dilip Kumar wrote: > > On Wed, Mar 4, 2020 at 11:45 AM Mahendra Singh Thalor > wrote: > > > > On Mon, 24 Feb 2020 at 15:39, Amit Kapila wrote: > > > > > > On Thu, Feb 20, 2020 at 8:06 AM Andres Freund wrote: > > > > > > > > Hi, > > > > > > > > On 2020-02-19

Re: PATCH: Add uri percent-encoding for binary data

2020-03-04 Thread Daniel Gustafsson
> On 20 Feb 2020, at 23:27, Alvaro Herrera wrote: > > On 2019-Oct-07, Anders Åstrand wrote: > >> Attached is a patch for adding uri as an encoding option for >> encode/decode. It uses what's called "percent-encoding" in rfc3986 >> (https://tools.ietf.org/html/rfc3986#section-2.1). > > Thanks.

Re: Use compiler intrinsics for bit ops in hash

2020-03-04 Thread John Naylor
On Thu, Feb 27, 2020 at 1:56 PM David Fetter wrote: > > [v6 patch set] Here I'm only looking at 0001. It needs rebasing, but it's trivial to see what it does. I noticed in some places, you've replaced "long" with uint64, but many are int64. I started making a list, but it got too long, and I had

Re: Identifying user-created objects

2020-03-04 Thread Fujii Masao
On 2020/03/04 17:05, Masahiko Sawada wrote: On Wed, 4 Mar 2020 at 16:43, Fujii Masao wrote: On 2020/02/05 20:26, Masahiko Sawada wrote: Hi, User can create database objects such as functions into pg_catalog. But if I'm not missing something, currently there is no straightforward way to

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-03-04 Thread Amit Kapila
On Wed, Mar 4, 2020 at 10:28 AM Amit Kapila wrote: > > On Wed, Mar 4, 2020 at 3:16 AM Tomas Vondra > wrote: > > > > The first thing I realized that WAL-logging of assignments in v12 does > > both the "old" logging (using dedicated message) and "new" with > > toplevel-XID embedded in the first

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-03-04 Thread Dilip Kumar
On Wed, Mar 4, 2020 at 2:33 PM Amit Kapila wrote: > > On Wed, Mar 4, 2020 at 10:28 AM Amit Kapila wrote: > > > > On Wed, Mar 4, 2020 at 3:16 AM Tomas Vondra > > wrote: > > > > > > The first thing I realized that WAL-logging of assignments in v12 does > > > both the "old" logging (using

Re: PG14 target version?

2020-03-04 Thread Daniel Gustafsson
> On 4 Mar 2020, at 06:28, Michael Paquier wrote: > > On Tue, Mar 03, 2020 at 01:10:26PM -0500, David Steele wrote: >> Anybody know how to add 14 to the "Target version" dropdown in the CF app? > > The only person knowing that stuff is I think Magnus. I don't have an > access to that. Magnus,

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Daniel Gustafsson
> On 4 Mar 2020, at 08:54, Michael Paquier wrote: > All the tools mentioned in $subject have been switched recently to use > the central logging infrastructure, which means that they have gained > coloring output. However we (mostly I) forgot to update the docs. +1 on updating the docs with

Re: PG_COLOR not mentioned in docs of vacuumlo, oid2name and pgbench

2020-03-04 Thread Juan José Santamaría Flecha
On Wed, Mar 4, 2020 at 8:54 AM Michael Paquier wrote: > > Attached is a patch to fix this issue. Please let me know if there > are comments and/or objections. > I think there are a couple tools missing: pg_archivecleanup, pg_ctl, pg_test_fsync and pg_upgrade. pg_regress also, but there is

Re: PG14 target version?

2020-03-04 Thread Dave Page
On Wed, Mar 4, 2020 at 9:14 AM Daniel Gustafsson wrote: > > On 4 Mar 2020, at 06:28, Michael Paquier wrote: > > > > On Tue, Mar 03, 2020 at 01:10:26PM -0500, David Steele wrote: > >> Anybody know how to add 14 to the "Target version" dropdown in the CF > app? > > > > The only person knowing

Re: Use compiler intrinsics for bit ops in hash

2020-03-04 Thread John Naylor
On Tue, Mar 3, 2020 at 4:46 AM Jesse Zhang wrote: > The math in the new (from v4 to v6) patch is wrong: it yields > ceil_log2(1) = 1 or next_power_of_2(1) = 2. I think you're right. > I can see that you lifted > the restriction of "num greater than one" for ceil_log2() in this patch > set, but

  1   2   >