Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Heikki Linnakangas
On 07/03/2019 15:41, Heikki Linnakangas wrote: On 07/03/2019 14:54, Peter Geoghegan wrote: On Wed, Mar 6, 2019 at 10:15 PM Heikki Linnakangas wrote: After staring at the first patch for bit longer, a few things started to bother me: * The new struct is called BTScanInsert, but it's used for

Re: pg_basebackup against older server versions

2019-03-06 Thread Sergei Kornilov
Hi > No problem. Thanks for the patch, the logic looks good and I made > some adjustments as attached. Does that look fine to you? Looks fine, thanks. I tested against HEAD and v11.2 with and without -R in both plain and tar formats. regards, Sergei

Re: Re: [HACKERS] Custom compression methods

2019-03-06 Thread Alexander Korotkov
On Thu, Mar 7, 2019 at 10:43 AM David Steele wrote: > On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: > > > there are another set of patches. > > Only rebased to current master. > > > > Also I will change status on commitfest to 'Needs review'. > > This patch has seen periodic rebases but no code

Re: insensitive collations

2019-03-06 Thread Peter Eisentraut
On 2019-03-05 18:48, Daniel Verite wrote: >> Older ICU versions (<54) don't support all the locale customization >> options, so many of my new tests in collate.icu.utf8.sql will fail on >> older systems. What should we do about that? Have another extra test file? > Maybe stick to the old-style

Re: Re: [HACKERS] Custom compression methods

2019-03-06 Thread David Steele
On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote: there are another set of patches. Only rebased to current master. Also I will change status on commitfest to 'Needs review'. This patch has seen periodic rebases but no code review that I can see since last January 2018. As Andres noted in

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Heikki Linnakangas
On 07/03/2019 14:54, Peter Geoghegan wrote: On Wed, Mar 6, 2019 at 10:15 PM Heikki Linnakangas wrote: After staring at the first patch for bit longer, a few things started to bother me: * The new struct is called BTScanInsert, but it's used for searches, too. It makes sense when you read the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-06 Thread amul sul
Thanks Rajkumar, I am looking into this. Regards, Amul On Thu, Mar 7, 2019 at 11:54 AM Rajkumar Raghuwanshi < rajkumar.raghuwan...@enterprisedb.com> wrote: > > > On Tue, Mar 5, 2019 at 3:45 PM amul sul wrote: > >> Attached is the rebased atop of the latest master head(35bc0ec7c8). >> > thanks

Re: pg_basebackup against older server versions

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 01:42:16PM +0300, Sergei Kornilov wrote: > My fault. I thought pg_basebackup works only with same major version, sorry. > How about attached patch? No problem. Thanks for the patch, the logic looks good and I made some adjustments as attached. Does that look fine to you?

Re: Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-06 Thread David Steele
On 3/6/19 5:38 PM, Andrey Borodin wrote: 20 февр. 2019 г., в 17:06, Alexey Kondratov написал(а): The new patch is much smaller (less than 400 lines) and works as advertised. There's a typo "retreive" there. These lines look a little suspicious: char

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Peter Geoghegan
On Wed, Mar 6, 2019 at 10:54 PM Peter Geoghegan wrote: > It will also have to store heapkeyspace, of course. And minusinfkey. > BTW, I would like to hear what you think of the idea of minusinfkey > (and the !minusinfkey optimization) specifically. > I'm not sure that that's an improvement.

Re: Re: Reviving the "Stopping logical replication protocol" patch from Vladimir Gordichuk

2019-03-06 Thread David Steele
On 2/16/19 10:38 PM, Dave Cramer wrote: Thanks for looking at this. FYI, I did not originally write this, rather the original author has not replied to requests. JDBC could use this, I assume others could as well. That said I'm certainly open to suggestions on how to do this. Craig, do you

Re: ECPG regression with DECLARE STATEMENT support

2019-03-06 Thread Rushabh Lathia
On Thu, Mar 7, 2019 at 9:56 AM Michael Meskes wrote: > Hi, > > > Commit bd7c95f0c1a38becffceb3ea7234d57167f6d4bf add DECLARE > > STATEMENT support to ECPG. This introduced the new rule > > for EXEC SQL CLOSE cur and with that it gets transformed into > > ECPGclose(). > > > > Now prior to the

Re: Refactoring the checkpointer's fsync request queue

2019-03-06 Thread Thomas Munro
On Wed, Mar 6, 2019 at 6:16 AM Shawn Debnath wrote: > On Tue, Mar 05, 2019 at 11:53:16AM -0500, Tom Lane wrote: > > Thomas Munro writes: > > > Why do we need to include fmgr.h in md.h? > > > > More generally, any massive increase in an include file's inclusions > > is probably a sign that you

Re: Re: proposal: plpgsql pragma statement

2019-03-06 Thread David Steele
On 2/4/19 8:12 PM, Pavel Stehule wrote:  attached rebased patch This patch has gone through a few iterations but I don't think there's any agreement on what it should look like. There's been no code review that I can see. I think this should be pushed to PG13 at the least, perhaps

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Peter Geoghegan
On Wed, Mar 6, 2019 at 10:15 PM Heikki Linnakangas wrote: > After staring at the first patch for bit longer, a few things started to > bother me: > > * The new struct is called BTScanInsert, but it's used for searches, > too. It makes sense when you read the README, which explains the >

Re: Re: [HACKERS] proposal: schema variables

2019-03-06 Thread David Steele
On 3/3/19 10:27 PM, Pavel Stehule wrote: rebase and fix compilation due changes related pg_dump This patch hasn't receive any review in a while and I'm not sure if that's because nobody is interested or the reviewers think it does not need any more review. It seems to me that this patch

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-06 Thread Rajkumar Raghuwanshi
On Tue, Mar 5, 2019 at 3:45 PM amul sul wrote: > Attached is the rebased atop of the latest master head(35bc0ec7c8). > thanks Amul, patches applied cleanly on PG head. While testing this I got a server crash with below test case. CREATE TABLE plt1 (a int, b int, c varchar) PARTITION BY

Re: Prevent extension creation in temporary schemas

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 09:33:55AM +0100, Chris Travers wrote: > Ok so at present I see three distinct issues here, where maybe three > different patches over time might be needed. > > The issues are: > > 1. create extension pgcrypto with schema pg_temp; fails because there is > no schema

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Heikki Linnakangas
On 06/03/2019 04:03, Peter Geoghegan wrote: On Tue, Mar 5, 2019 at 3:37 AM Heikki Linnakangas wrote: I'm looking at the first patch in the series now. I'd suggest that you commit that very soon. It's useful on its own, and seems pretty much ready to be committed already. I don't think it will

Re: New vacuum option to do only freezing

2019-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2019 at 3:55 AM Robert Haas wrote: > > On Tue, Mar 5, 2019 at 11:29 PM Masahiko Sawada wrote: > > Attached updated patch incorporated all of comments. Also I've added > > new reloption vacuum_index_cleanup as per discussion on the "reloption > > to prevent VACUUM from truncating

Re: ECPG regression with DECLARE STATEMENT support

2019-03-06 Thread Michael Meskes
Hi, > Commit bd7c95f0c1a38becffceb3ea7234d57167f6d4bf add DECLARE > STATEMENT support to ECPG. This introduced the new rule > for EXEC SQL CLOSE cur and with that it gets transformed into > ECPGclose(). > > Now prior to the above commit, someone can declare the > cursor in the SQL statement and

Re: SQL statement PREPARE does not work in ECPG

2019-03-06 Thread Michael Meskes
Hi all, > ... > But it doesn't allow to use host variable in parameter clause of > EXECUTE statement like the following. > I'm afraid that it's not usefull. I will research the standard and > other RDBMS. > If you have some information, please adivise to me. This also seems to be conflicting

Re: [HACKERS] Block level parallel vacuum

2019-03-06 Thread Masahiko Sawada
On Thu, Mar 7, 2019 at 2:54 AM Robert Haas wrote: > > On Wed, Mar 6, 2019 at 1:26 AM Masahiko Sawada wrote: > > Okay, attached the latest version of patch set. I've incorporated all > > comments I got and separated the patch for making vacuum options a > > Node (0001 patch). And the patch

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2019-03-06 Thread Noah Misch
On Mon, Mar 04, 2019 at 06:04:20PM +0900, Kyotaro HORIGUCHI wrote: > I found that I have 65(h) segments left alone on my environment:p Did patched PostgreSQL create those, or did unpatched PostgreSQL create them? > At Sat, 11 Aug 2018 23:48:15 -0700, Noah Misch wrote in >

Re: Drop type "smgr"?

2019-03-06 Thread Thomas Munro
On Thu, Feb 28, 2019 at 7:02 PM Thomas Munro wrote: > The type smgr has only one value 'magnetic disk'. ~15 years ago it > also had a value 'main memory', and in Berkeley POSTGRES 4.2 there was > a third value 'sony jukebox'. Back then, all tables had an associated > block storage manager, and

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Andres Freund
On 2019-03-07 15:25:34 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 11:37, Andres Freund wrote: > > > > On 2019-03-07 11:31:15 +1300, David Rowley wrote: > > > Do you think it's fine to reword the docs to make this point more > > > clear, or do you see this as a fundamental problem with the

Re: Batch insert in CTAS/MatView code

2019-03-06 Thread Michael Paquier
Hi, On Wed, Mar 06, 2019 at 10:06:27PM +0800, Paul Guo wrote: > The copy code has used batch insert with function heap_multi_insert() to > speed up. It seems that Create Table As or Materialized View could leverage > that code also to boost the performance also. Attached is a patch to > implement

Re: pgsql: tableam: introduce table AM infrastructure.

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 03:03:44PM -0300, Alvaro Herrera wrote: > On 2019-Mar-06, Andres Freund wrote: >> tableam: introduce table AM infrastructure. > > Thanks for doing this!! +1. -- Michael signature.asc Description: PGP signature

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 11:37, Andres Freund wrote: > > On 2019-03-07 11:31:15 +1300, David Rowley wrote: > > Do you think it's fine to reword the docs to make this point more > > clear, or do you see this as a fundamental problem with the patch? > > Hm, both? I mean I wouldn't necessarily

Re: few more wait events to add to docs

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 11:08:12AM -0800, Jeremy Schneider wrote: > LWLock order in documentation: > 1) CamelCase LWLocks: individually named - see lwlocknames.txt > 2) lowercase LWLocks: tranches > 2a) SLRUs - see SimpleLruInit() callers on doxygen > 2b) Shared Buffer (buffer_content, buffer_io)

Re: [bug fix] Produce a crash dump before main() on Windows

2019-03-06 Thread Kyotaro HORIGUCHI
Hello. At Tue, 5 Mar 2019 16:45:53 +1100, Haribabu Kommi wrote in > > I don't have an idea about which is better behavior, but does > > this work for you? > > > > > > https://docs.microsoft.com/en-us/windows/desktop/wer/collecting-user-mode-dumps > > > > No, this option is not generating

Re: Online verification of checksums

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 08:53:57PM +0100, Tomas Vondra wrote: > Not sure. AFAICS that would to require a single transaction, and if we > happen to add some sort of throttling (which is a feature request I'd > expect pretty soon to make it usable on live clusters) that might be > quite

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 02:54:52PM +, Georgios Kokolatos wrote: > Overall the patch looks good and according to the previous > discussion fulfils its purpose. > > It might be worthwhile to also check for errors on close in > SaveSlotToPath(). Thanks for the feedback, added. I have spent

RE: speeding up planning with partitions

2019-03-06 Thread Imai, Yoshikazu
Amit-san, On Wed, Mar 6, 2019 at 5:38 AM, Amit Langote wrote: > On 2019/03/06 11:09, Imai, Yoshikazu wrote: > > [0004 or 0005] > > There are redundant process in add_appendrel_other_rels (or > expand_xxx_rtentry()?). > > I modified add_appendrel_other_rels like below and it actually worked. > > >

Re: pgsql: Removed unused variable, openLogOff.

2019-03-06 Thread Michael Paquier
On Wed, Mar 06, 2019 at 02:47:16PM +, Robert Haas wrote: > Removed unused variable, openLogOff. Is that right for the report if data is written in chunks? The same patch has been proposed a couple of weeks ago, and I commented about it as follows:

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Peter Geoghegan
On Wed, Mar 6, 2019 at 1:37 PM Robert Haas wrote: > I know I'm stating the obvious here, but we don't have many weeks left > at this point. I have not reviewed any code, but I have been > following this thread and I'd really like to see this work go into > PostgreSQL 12, assuming it's in good

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-03-06 Thread Bruce Momjian
On Wed, Mar 6, 2019 at 10:49:17AM -0800, Jeremy Schneider wrote: > Might it make sense to generalize a little bit to secret management? It > would be *great* if PostgreSQL could have a standard "secrets" API which > could then use plugins or extensions to provide an internal > implementation

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Tomas Vondra
On 3/6/19 9:17 PM, Tom Lane wrote: > Robert Haas writes: >> OK, so this is getting simpler, but I'm wondering why we need >> dlist_move_tail() at all. It is a well-known fact that maintaining >> LRU ordering is expensive and it seems to be unnecessary for our >> purposes here. > > Yeah ... LRU

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Jerry Jelinek
On Wed, Mar 6, 2019 at 11:02 AM Alvaro Herrera wrote: > On 2019-Mar-06, Robert Haas wrote: > > > On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera > wrote: > > > I want your dictating software. > > > > I'm afraid this is just me and a keyboard, but sadly for me you're not > > the first person to

Re: Pluggable Storage - Andres's take

2019-03-06 Thread Andres Freund
Hi, On 2019-03-07 11:56:57 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > > Here's a cleaned up version of that patch. David, Alvaro, you also > > played in that area, any objections? I think this makes that part of the > > code easier to read actually. Robert,

Re: Pluggable Storage - Andres's take

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 08:33, Andres Freund wrote: > Here's a cleaned up version of that patch. David, Alvaro, you also > played in that area, any objections? I think this makes that part of the > code easier to read actually. Robert, thanks for looking at that patch > already. I only had a

Re: shared-memory based stats collector

2019-03-06 Thread Andres Freund
Hi, > From 88740269660d00d548910c2f3aa631878c7cf0d4 Mon Sep 17 00:00:00 2001 > From: Kyotaro Horiguchi > Date: Thu, 21 Feb 2019 12:42:07 +0900 > Subject: [PATCH 4/6] Allow dsm to use on postmaster. > > DSM is inhibited to be used on postmaster. Shared memory baesd stats > collector needs it to

Re: Patch to document base64 encoding

2019-03-06 Thread Karl O. Pinc
On Wed, 6 Mar 2019 19:30:16 +0100 (CET) Fabien COELHO wrote: > "... section 6.8" -> "... Section 6.8" (capital S). Fixed. > "The string and the binary encode and decode functions..." sentence > looks strange to me, especially with the English article that I do > not really master, so maybe it

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Andres Freund
Hi, On 2019-03-07 11:31:15 +1300, David Rowley wrote: > On Thu, 7 Mar 2019 at 05:17, Andres Freund wrote: > > I'm also concerned that the the current catalog representation isn't > > right. As I said: > > > > > I also find it far from clear that: > > > > > > > > > The

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 05:17, Andres Freund wrote: > I'm also concerned that the the current catalog representation isn't > right. As I said: > > > I also find it far from clear that: > > > > > > The tablespace_name is > > the name > > of the tablespace in which the new

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 03:36, Tom Lane wrote: > > David Rowley writes: > > As far as I can see, the biggest fundamental difference with doing > > things this way will be that the column order of partitions will be > > preserved, where before it would inherit the order of the partitioned > >

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread David Rowley
On Thu, 7 Mar 2019 at 08:54, Andrew Dunstan wrote: > > > On 3/6/19 1:38 PM, Jeremy Schneider wrote: > > On 3/5/19 14:14, Andrew Dunstan wrote: > >> This patch is tiny, seems perfectly reasonable, and has plenty of > >> support. I'm going to commit it shortly unless there are last minute > >>

Re: shared-memory based stats collector

2019-03-06 Thread Robert Haas
On Sun, Feb 24, 2019 at 11:53 PM Kyotaro HORIGUCHI wrote: > The two aboves are fixed in the attached v17. Andres just drew my attention to patch 0004 in this series, which is definitely not OK. That patch allows the postmaster to use dynamic shared memory, claiming: "Shared memory baesd stats

Re: Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 16:46:07 -0500, Andrew Dunstan wrote: > > On 3/6/19 3:41 PM, Andres Freund wrote: > > Hi, > > > > After my tableam patch Andrew's buildfarm animal started failing in the > > cross-version upgrades: > >

Re: Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andrew Dunstan
On 3/6/19 3:41 PM, Andres Freund wrote: > Hi, > > After my tableam patch Andrew's buildfarm animal started failing in the > cross-version upgrades: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake=2019-03-06%2019%3A32%3A24 Incidentally, I just fixed a bug that was preventing the

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 3:03 PM Peter Geoghegan wrote: > I agree that the parts covered by the first patch in the series are > very unlikely to need changes, but I hesitate to commit it weeks ahead > of the other patches. I know I'm stating the obvious here, but we don't have many weeks left at

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Robert Haas
On Fri, Mar 1, 2019 at 5:06 PM Joe Conway wrote: > Seems like it would be better to modify the arguments to > CloseTransientFile() to include the filename being closed, errorlevel, > and fail_on_error or something similar. Then all the repeated ereport > stanzas could be eliminated. Hmm. I'm

Binary upgrade from <12 to 12 creates toast table for partitioned tables

2019-03-06 Thread Andres Freund
Hi, After my tableam patch Andrew's buildfarm animal started failing in the cross-version upgrades: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake=2019-03-06%2019%3A32%3A24 But I actually don't think that't really fault of the tableam patch. The reason for the assertion is that we

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread Tomas Vondra
On 3/6/19 12:10 AM, David Rowley wrote: > Thanks for chipping in on this. > > On Wed, 6 Mar 2019 at 01:53, Tomas Vondra > wrote: >> But on the other hand it feels a bit weird that we increase this one >> value and leave all the other (also very conservative) defaults alone. > > Which others

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Tom Lane
Robert Haas writes: > OK, so this is getting simpler, but I'm wondering why we need > dlist_move_tail() at all. It is a well-known fact that maintaining > LRU ordering is expensive and it seems to be unnecessary for our > purposes here. Yeah ... LRU maintenance was another thing that used to be

Re: [HACKERS] Incomplete startup packet errors

2019-03-06 Thread Andrew Dunstan
On 3/6/19 12:12 PM, Robert Haas wrote: > On Tue, Mar 5, 2019 at 5:35 PM Andrew Dunstan > wrote: >> OK, I think we have agreement on Tom's patch. Do we want to backpatch >> it? It's a change in behaviour, but I find it hard to believe anyone >> relies on the existence of these annoying messages,

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 8:41 PM, Andres Freund wrote: > Hi, > > On 2019-03-06 20:37:39 +0100, Tomas Vondra wrote: >> Not sure how to integrate it into the CLI tool, though. Perhaps we it >> could require connection info so that it can execute a function, when >> executed in online mode? > > To me the right

Re: Should we increase the default vacuum_cost_limit?

2019-03-06 Thread Andrew Dunstan
On 3/6/19 1:38 PM, Jeremy Schneider wrote: > On 3/5/19 14:14, Andrew Dunstan wrote: >> This patch is tiny, seems perfectly reasonable, and has plenty of >> support. I'm going to commit it shortly unless there are last minute >> objections. > +1 > done. cheers andrew -- Andrew Dunstan

Re: Optimization of some jsonb functions

2019-03-06 Thread Andrew Dunstan
On 3/5/19 5:24 AM, David Steele wrote: > On 2/22/19 2:05 AM, Nikita Glukhov wrote: >> Attached set of patches with some jsonb optimizations that were made >> during >> comparison of performance of ordinal jsonb operators and jsonpath >> operators. > > This patch was submitted just before the

Re: Make drop database safer

2019-03-06 Thread Ashwin Agrawal
On Wed, Mar 6, 2019 at 7:56 AM Tomas Vondra wrote: > > > On 2/12/19 12:55 AM, Ashwin Agrawal wrote: > > > > Thanks for the response and inputs. > > > > On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > > wrote: > > > > Hi, > > > > On 2019-02-08 16:36:13 -0800,

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Tomas Vondra
On 3/6/19 8:04 PM, Robert Haas wrote: > On Wed, Mar 6, 2019 at 1:53 PM Alvaro Herrera > wrote: >> On 2019-Feb-08, Tomas Vondra wrote: >>> I'm wondering if we should just get rid of all such optimizations, and >>> make the unowned list doubly-linked (WIP patch attached, needs fixing >>> the

Re: Online verification of checksums

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 20:37:39 +0100, Tomas Vondra wrote: > Not sure how to integrate it into the CLI tool, though. Perhaps we it > could require connection info so that it can execute a function, when > executed in online mode? To me the right fix would be to simply have this run as part of the

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 6:42 PM, Andres Freund wrote: > On 2019-03-06 12:33:49 -0500, Robert Haas wrote: >> On Sat, Mar 2, 2019 at 5:45 AM Michael Banck >> wrote: >>> Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: On Tue, Sep 18, 2018 at 10:37 AM Michael Banck wrote: > I have

Re: Pluggable Storage - Andres's take

2019-03-06 Thread Andres Freund
Hi, On 2019-03-05 23:07:21 -0800, Andres Freund wrote: > My next steps are: > - final polish & push the basic DDL and pg_dump patches Done and pushed. Some collation dependent fallout, I'm hoping I've just fixed that. > - cleanup & polish the ON CONFLICT refactoring Here's a cleaned up version

Re: Online verification of checksums

2019-03-06 Thread Tomas Vondra
On 3/6/19 6:26 PM, Robert Haas wrote: > On Sat, Mar 2, 2019 at 4:38 PM Tomas Vondra > wrote: >> FWIW I don't think this qualifies as torn page - i.e. it's not a full >> read with a mix of old and new data. This is partial write, most likely >> because we read the blocks one by one, and when we

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Tomas Vondra
On 3/6/19 7:52 PM, Alvaro Herrera wrote: > On 2019-Feb-08, Tomas Vondra wrote: > >> I'm wondering if we should just get rid of all such optimizations, and >> make the unowned list doubly-linked (WIP patch attached, needs fixing >> the comments etc.). > > +1 for that approach. > > Did you

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:53 PM Alvaro Herrera wrote: > On 2019-Feb-08, Tomas Vondra wrote: > > I'm wondering if we should just get rid of all such optimizations, and > > make the unowned list doubly-linked (WIP patch attached, needs fixing > > the comments etc.). > > +1 for that approach. +1 for

Re: Tighten error control for OpenTransientFile/CloseTransientFile

2019-03-06 Thread Georgios Kokolatos
Overall the patch looks good and according to the previous discussion fulfils its purpose. It might be worthwhile to also check for errors on close in SaveSlotToPath(). pgstat_report_wait_end(); CloseTransientFile(fd); /* rename to permanent file, fsync file and

Re: New vacuum option to do only freezing

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 11:29 PM Masahiko Sawada wrote: > Attached updated patch incorporated all of comments. Also I've added > new reloption vacuum_index_cleanup as per discussion on the "reloption > to prevent VACUUM from truncating empty pages at the end of relation" > thread. Autovacuums also

Re: performance issue in remove_from_unowned_list()

2019-03-06 Thread Alvaro Herrera
On 2019-Feb-08, Tomas Vondra wrote: > I'm wondering if we should just get rid of all such optimizations, and > make the unowned list doubly-linked (WIP patch attached, needs fixing > the comments etc.). +1 for that approach. Did you consider using a dlist? -- Álvaro Herrera

Re: Patch to document base64 encoding

2019-03-06 Thread Fabien COELHO
Attached: doc_base64_v7.patch Patch applies cleanly, doc compiles, navigation tested and ok. "... section 6.8" -> "... Section 6.8" (capital S). "The string and the binary encode and decode functions..." sentence looks strange to me, especially with the English article that I do not

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:02 PM Alvaro Herrera wrote: > Well, I don't have a problem reading long texts; my problem is that I'm > unable to argue as quickly. That's my secret weapon... except that it's not much of a secret. > I do buy your argument, though (if reluctantly); in particular I was >

Re: Protect syscache from bloating with negative cache entries

2019-03-06 Thread Robert Haas
On Fri, Mar 1, 2019 at 3:33 AM Kyotaro HORIGUCHI wrote: > > > It is artificial (or acutually wont't be repeatedly executed in a > > > session) but anyway what can get benefit from > > > catalog_cache_memory_target would be a kind of extreme. > > > > I agree. So then let's not have it. > > Ah...

Re: pgsql: tableam: introduce table AM infrastructure.

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Andres Freund wrote: > tableam: introduce table AM infrastructure. Thanks for doing this!! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera > wrote: > > I want your dictating software. > > I'm afraid this is just me and a keyboard, but sadly for me you're not > the first person to accuse me of producing giant walls of text. Well, I don't have a

Re: [HACKERS] Block level parallel vacuum

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 1:26 AM Masahiko Sawada wrote: > Okay, attached the latest version of patch set. I've incorporated all > comments I got and separated the patch for making vacuum options a > Node (0001 patch). And the patch doesn't use parallel_workers. It > might be proposed in the another

Re: Server Crash in logical decoding if used inside --single mode

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, tushar wrote: > backend> select * from pg_logical_slot_get_changes('m7',null,null); [...] > TRAP: FailedAssertion("!(slot != ((void *)0) && slot->active_pid != 0)", > File: "slot.c", Line: 428) > Aborted (core dumped) See argumentation in

Re: Online verification of checksums

2019-03-06 Thread Andres Freund
On 2019-03-06 12:33:49 -0500, Robert Haas wrote: > On Sat, Mar 2, 2019 at 5:45 AM Michael Banck > wrote: > > Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: > > > On Tue, Sep 18, 2018 at 10:37 AM Michael Banck > > > wrote: > > > > I have added a retry for this as well now, without

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Pavel Stehule
st 6. 3. 2019 v 16:24 odesílatel Chapman Flack napsal: > On 3/6/19 10:12 AM, Andrew Dunstan wrote: > > > Having reviewed the thread, I'm with Andres and Tom. Maybe though we > > should have a note somewhere to the effect that you can't use VARIADIC > > with these. > > Perhaps such a note belongs

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera wrote: > I want your dictating software. I'm afraid this is just me and a keyboard, but sadly for me you're not the first person to accuse me of producing giant walls of text. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: Online verification of checksums

2019-03-06 Thread Robert Haas
On Sat, Mar 2, 2019 at 5:45 AM Michael Banck wrote: > Am Freitag, den 01.03.2019, 18:03 -0500 schrieb Robert Haas: > > On Tue, Sep 18, 2018 at 10:37 AM Michael Banck > > wrote: > > > I have added a retry for this as well now, without a pg_sleep() as well. > > > This catches around 80% of the

Re: Online verification of checksums

2019-03-06 Thread Robert Haas
On Sat, Mar 2, 2019 at 4:38 PM Tomas Vondra wrote: > FWIW I don't think this qualifies as torn page - i.e. it's not a full > read with a mix of old and new data. This is partial write, most likely > because we read the blocks one by one, and when we hit the last page > while the table is being

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
I want your dictating software. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Incomplete startup packet errors

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 5:35 PM Andrew Dunstan wrote: > OK, I think we have agreement on Tom's patch. Do we want to backpatch > it? It's a change in behaviour, but I find it hard to believe anyone > relies on the existence of these annoying messages, so my vote would be > to backpatch it. I don't

Re: [PATCH] kNN for btree

2019-03-06 Thread Nikita Glukhov
Attached 9th version of the patches. On 03.03.2019 12:46, Anastasia Lubennikova wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 11:41 AM Alvaro Herrera wrote: > I can understand this argument. Is there really a reason to change > those two behaviors separately? See my previous rely to Andrew, but also, I think you're putting the burden of proof in the wrong place. You could equally well ask "Is

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 11:37 AM Andrew Dunstan wrote: > Well, let's put the question another way. Is there any reason to allow > skipping zero filling if we are recycling? That seems possibly > dangerous. I can imagine turning off recycling but leaving on > zero-filling, although I don't have a

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera > wrote: > > I think the idea of it being a generic tunable for assorted behavior > > changes, rather than specific to WAL recycling, is a good one. I'm > > unsure about your proposed name -- maybe

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Andrew Dunstan
On 3/6/19 11:30 AM, Robert Haas wrote: > On Wed, Mar 6, 2019 at 10:55 AM Andrew Dunstan > wrote: >>> I *really* dislike this. For one thing, it means that users don't >>> have control over the behaviors individually. For another, the >>> documentation is now quite imprecise about what the

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Mar 6, 2019 at 10:55 AM Andrew Dunstan wrote: > > I *really* dislike this. For one thing, it means that users don't > > have control over the behaviors individually. For another, the > > documentation is now quite imprecise about what the option actually > > does, while expecting users

Re: pg_dump is broken for partition tablespaces

2019-03-06 Thread Andres Freund
Hi, On 2019-03-06 19:45:06 +1300, David Rowley wrote: > Over on [1] Andres pointed out that the pg_dump support for the new to > PG12 tablespace inheritance feature is broken. This is the feature > added in ca4103025dfe26 to allow a partitioned table to have a > tablespace that acts as the

Re: PostgreSQL vs SQL/XML Standards

2019-03-06 Thread Chapman Flack
This CF entry shows Pavel and me as reviewers, but the included patches were also produced by one or the other of us, so additional review by someone who isn't us seems appropriate. :) Would it make sense to remove one or both of us from the 'reviewers' field in the app, to make it more obviously

Re: Make drop database safer

2019-03-06 Thread Tomas Vondra
On 2/12/19 12:55 AM, Ashwin Agrawal wrote: Thanks for the response and inputs. On Sat, Feb 9, 2019 at 4:51 AM Andres Freund > wrote: Hi, On 2019-02-08 16:36:13 -0800, Alexandra Wang wrote: > Current sequence of operations for drop database

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Andrew Dunstan
On 3/6/19 10:38 AM, Robert Haas wrote: > On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera > wrote: >> I think the idea of it being a generic tunable for assorted behavior >> changes, rather than specific to WAL recycling, is a good one. I'm >> unsure about your proposed name -- maybe

Re: A separate table level option to control compression

2019-03-06 Thread Robert Haas
On Tue, Mar 5, 2019 at 5:30 PM Andrew Dunstan wrote: > This is a nice idea, and I'm a bit surprised it hasn't got more > attention. The patch itself seems very simple and straightforward, > although it could probably do with having several sets of eyeballs on it. I haven't needed this for

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Robert Haas
On Wed, Feb 27, 2019 at 6:12 PM Alvaro Herrera wrote: > I think the idea of it being a generic tunable for assorted behavior > changes, rather than specific to WAL recycling, is a good one. I'm > unsure about your proposed name -- maybe "wal_cow_filesystem" is better? I *really* dislike this.

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-06 Thread Andrey Borodin
Hi! > 20 февр. 2019 г., в 17:06, Alexey Kondratov > написал(а): > >> >> I will work out this one with postgres -C and come back till the next >> commitfest. I found that something similar is already used in pg_ctl and >> there is a mechanism for finding valid executables in exec.c. So it

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Chapman Flack
On 3/6/19 10:12 AM, Andrew Dunstan wrote: > Having reviewed the thread, I'm with Andres and Tom. Maybe though we > should have a note somewhere to the effect that you can't use VARIADIC > with these. Perhaps such a note belongs hoisted into the functions-conditional section of the manual, making

Re: [HACKERS] CLUSTER command progress monitor

2019-03-06 Thread Alvaro Herrera
On 2019-Mar-06, Robert Haas wrote: > On Tue, Mar 5, 2019 at 8:03 PM Alvaro Herrera > wrote: > > One, err, small issue with that idea is that we need the param numbers > > not to conflict for any "progress update providers" that are to be used > > simultaneously by any command. > > Is that

Re: proposal: variadic argument support for least, greatest function

2019-03-06 Thread Andrew Dunstan
On 3/4/19 9:39 AM, David Steele wrote: > On 3/1/19 3:59 AM, Chapman Flack wrote: >> The following review has been posted through the commitfest application: >> make installcheck-world:  tested, passed >> Implements feature:   tested, passed >> Spec compliant:   not tested >>

  1   2   >