Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Michael Paquier
On Tue, Apr 23, 2019 at 01:33:39PM +0900, Kyotaro HORIGUCHI wrote: > I think this is rahter a testing or debugging feature. This can > be apply to all paths, so the variable might be "path_prefix" or > something more generic than tablespace_chroot. > > Does it make sense? A GUC which enforces

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread David Rowley
On Tue, 23 Apr 2019 at 13:49, Amit Langote wrote: > > On 2019/04/23 7:51, Alvaro Herrera wrote: > > To me, it sounds > > unintuitive to accept partitions that don't exactly match the order of > > the parent table; but it's been supported all along. > > You might know it already, but even though

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Paul Guo
Hi Kyotaro, ignoring the MakePGDirectory() failure will fix this database create redo error, but I suspect some other kind of redo, which depends on the files under the directory (they are not copied since the directory is not created) and also cannot be covered by the invalid page mechanism,

Re: Symbol referencing errors

2019-04-22 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: >> ... >> ld: warning: symbol referencing errors Tom> Yeah, our SmartOS buildfarm members show those warnings too, eg Tom>

Re: [PATCH v1] Show whether tables are logged in \dt+

2019-04-22 Thread Fabien COELHO
Hello David, I noticed that there wasn't a bulk way to see table logged-ness in psql, so I made it part of \dt+. Applies, compiles, works for me. ISTM That temporary-ness is not shown either. Maybe the persistence column should be shown as is? Also I'd suggest that the column should be

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 11:34:38 +0900, Michael Paquier wrote in <20190423023438.gh2...@paquier.xyz> > On Mon, Apr 22, 2019 at 09:19:33PM +0900, Kyotaro HORIGUCHI wrote: > > The attached exercises this sequence, needing some changes in > > PostgresNode.pm and RecursiveCopy.pm to allow tablespaces. >

Re: REINDEX INDEX results in a crash for an index of pg_class since 9.6

2019-04-22 Thread Michael Paquier
On Thu, Apr 18, 2019 at 10:14:30AM +0900, Michael Paquier wrote: > Doing a REINDEX TABLE directly on pg_class proves to work correctly, > and CONCURRENTLY is not supported for catalog tables. > > Bisecting my way through it, the first commit causing the breakage is > that: > commit:

Re: display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Amit Langote
On 2019/04/23 0:58, Tom Lane wrote: > BTW, now that I look at this, I think the reason why I didn't make > tlist printouts pay attention to VERBOSE for this purpose is that > you don't get them at all if not verbose: > > regression=# explain select * from tt; > QUERY PLAN

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 13:33:39 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20190423.19.113770648.horiguchi.kyot...@lab.ntt.co.jp> > At Tue, 23 Apr 2019 11:27:06 +0900, Michael Paquier > wrote in <20190423022706.gg2...@paquier.xyz> > > On Mon, Apr 22, 2019 at 03:52:59PM +0530,

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kyotaro HORIGUCHI
At Tue, 23 Apr 2019 11:27:06 +0900, Michael Paquier wrote in <20190423022706.gg2...@paquier.xyz> > On Mon, Apr 22, 2019 at 03:52:59PM +0530, Kuntal Ghosh wrote: > > I accept that configuring master-standby on the same machine for this > > test is not okay. But, can we avoid the PANIC somehow?

Re: speeding up planning with partitions

2019-04-22 Thread Amit Langote
On 2019/04/23 7:08, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/02 2:34, Tom Lane wrote: >>> I'm not at all clear on what we think the interaction between >>> enable_partition_pruning and constraint_exclusion ought to be, >>> so I'm not sure what the appropriate resolution is here.

Re: Symbol referencing errors

2019-04-22 Thread Li Japin
On 4/23/19 12:09 PM, Tom Lane wrote: > AFAICT they're harmless, so my advice is just ignore them. > > If you're sufficiently annoyed by them to find the cause > and try to fix it, go ahead, but I haven't heard anyone > else worried about it. It might be that SmartOS wants > something like what

Re: Symbol referencing errors

2019-04-22 Thread Tom Lane
Li Japin writes: > When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: > ... > ld: warning: symbol referencing errors Yeah, our SmartOS buildfarm members show those warnings too, eg

Symbol referencing errors

2019-04-22 Thread Li Japin
Hi, When I compile PostgreSQL-11.2 on SmartOS, I find the following errors: Undefined            first referenced  symbol                  in file per_MultiFuncCall   adminpack.o end_MultiFuncCall   adminpack.o BuildTupleFromCStrings  adminpack.o

Re: clean up docs for v12

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 12:19:55PM -0400, Alvaro Herrera wrote: > On 2019-Apr-22, Andres Freund wrote: >> I think it'd be better just to fix s/the all/that all/. > > (and s/if's/if it's/) FWIW, I have noticed that part when gathering all the pieces for what became 148266f, still the full

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 09:19:33PM +0900, Kyotaro HORIGUCHI wrote: > The attached exercises this sequence, needing some changes in > PostgresNode.pm and RecursiveCopy.pm to allow tablespaces. +# Check for symlink -- needed only on source dir +# (note: this will fall through quietly if

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Michael Paquier
On Mon, Apr 22, 2019 at 03:52:59PM +0530, Kuntal Ghosh wrote: > I accept that configuring master-standby on the same machine for this > test is not okay. But, can we avoid the PANIC somehow? Or, is this > intentional and I should not include testtablespace in this case? Well, it is a bit more

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 7:04 PM Tomas Vondra wrote: > Some time ago there was a discussion about prefetching blocks during > recovery on a standby, and that's a great example of a use case that > benefit from this - look which blocks where modified in the next chunk > of WAL, prefetch them. But

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Amit Langote
On 2019/04/23 7:51, Alvaro Herrera wrote: > On 2019-Mar-06, Tom Lane wrote: >> David Rowley writes: >>> As far as I can see, the biggest fundamental difference with doing >>> things this way will be that the column order of partitions will be >>> preserved, where before it would inherit the order

Re: change password_encryption default to scram-sha-256?

2019-04-22 Thread Jonathan S. Katz
On 4/8/19 6:10 PM, Jonathan S. Katz wrote: > On 4/8/19 4:20 PM, Alvaro Herrera wrote: >> On 2019-Apr-08, Jonathan S. Katz wrote: >> >>> On 4/8/19 4:10 PM, Alvaro Herrera wrote: >> I wonder why we have two pages https://wiki.postgresql.org/wiki/Client_Libraries

Re: memory leak checking

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 20:29:17 -0400, Tom Lane wrote: >> I would not call the timezone data a "leak", since it's still useful, and >> accessible from static pointers, right up to exit. A true leak for this >> purpose is memory that's allocated but not usefully accessible, and I'd

Re: memory leak checking

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 20:29:17 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: > >> What is the standard memory leak checking policy for the PostgreSQL > >> codebase? I know there is some support for valgrind -- is the test suite > >> being run

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 08:52:11PM -0400, Bruce Momjian wrote: > Well, the interesting question is whether the server will generate a > single modblock file for all WAL in pg_wal only right before we are > ready to expire some WAL, or whether modblock files will be generated > offline, perhaps

Re: bug in update tuple routing with foreign partitions

2019-04-22 Thread Amit Langote
Fujita-san, On 2019/04/22 20:00, Etsuro Fujita wrote: > (2019/04/19 14:39), Etsuro Fujita wrote: >> (2019/04/19 13:00), Amit Langote wrote: >>> I agree that we can move to fix the other issue right away as the fix you >>> outlined above seems reasonable, but I wonder if it wouldn't be better to

[PATCH v1] Show whether tables are logged in \dt+

2019-04-22 Thread David Fetter
Folks, I noticed that there wasn't a bulk way to see table logged-ness in psql, so I made it part of \dt+. What say? Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate >From

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Tue, Apr 23, 2019 at 02:13:29AM +0200, Tomas Vondra wrote: > Well, I understand that concern - all I'm saying is that makes this > useless for some use cases (that may or may not be important enough). > > However, it seems to me those files are guaranteed to be much smaller > than the WAL

Re: memory leak checking

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: >> What is the standard memory leak checking policy for the PostgreSQL >> codebase? I know there is some support for valgrind -- is the test suite >> being run continuously with valgrind on the build farm? > Leaks are

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Mon, Apr 22, 2019 at 07:44:45PM -0400, Bruce Momjian wrote: On Tue, Apr 23, 2019 at 01:21:27AM +0200, Tomas Vondra wrote: On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: > On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > > > Oh. Well, I already explained my algorithm

Re: memory leak checking

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: > What is the standard memory leak checking policy for the PostgreSQL > codebase? I know there is some support for valgrind -- is the test suite > being run continuously with valgrind on the build farm? There's continuous use of valgrind on

memory leak checking

2019-04-22 Thread Mikhail Bautin
Hello PostgreSQL Hackers, What is the standard memory leak checking policy for the PostgreSQL codebase? I know there is some support for valgrind -- is the test suite being run continuously with valgrind on the build farm? Is there any plan to support clang's AddressSanitizer? I've seen a

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Tue, Apr 23, 2019 at 01:21:27AM +0200, Tomas Vondra wrote: > On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: > > On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > > > > Oh. Well, I already explained my algorithm for doing that upthread, > > > > which I believe would be

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > Oh. Well, I already explained my algorithm for doing that upthread, > which I believe would be quite cheap. > > 1. When you generate the .modblock files, stick all the block >

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > Now that I re-read this complaint once again, I wonder if a mismatching > column order in partitions isn't a thing we ought to preserve anyhow. > Robert, Amit -- is it by design that pg_dump loses the original column > order for partitions, when not in binary-upgrade

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Apr 22, 2019 at 3:12 PM Melanie Plageman > wrote: >> PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I >> was >> wondering if there is a reason not to call it inside BufFileCreateTemp. > The best answer I can think of is that a

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Mon, Apr 22, 2019 at 01:15:49PM -0400, Bruce Momjian wrote: On Mon, Apr 22, 2019 at 01:11:22PM -0400, Robert Haas wrote: On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > I assumed the modblock files would be stored in the WAL archive so some > external tools could generate

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Thu, Apr 18, 2019 at 04:25:24PM -0400, Robert Haas wrote: On Thu, Apr 18, 2019 at 3:51 PM Bruce Momjian wrote: How would you choose the STARTLSN/ENDLSN? If you could do it per checkpoint, rather than per-WAL, I think that would be great. I thought of that too. It seems appealing,

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Mar-06, Tom Lane wrote: > David Rowley writes: > > As far as I can see, the biggest fundamental difference with doing > > things this way will be that the column order of partitions will be > > preserved, where before it would inherit the order of the partitioned > > table. I'm a little

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 3:12 PM Melanie Plageman wrote: > PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I > was > wondering if there is a reason not to call it inside BufFileCreateTemp. The best answer I can think of is that a BufFileCreateTemp() caller might not

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 4:43 PM Alvaro Herrera wrote: > Well, frequently when people discuss ideas on this list, others discuss > and provide further ideas to try help to find a working solution, rather > than throw every roadblock they can think of (though roadblocks are > indeed thrown now and

Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Melanie Plageman
Hi, PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I was wondering if there is a reason not to call it inside BufFileCreateTemp. As a developer using BufFileCreateTemp to write code that will create spill files, it was easy to forget the extra step of checking the

Re: speeding up planning with partitions

2019-04-22 Thread Tom Lane
Amit Langote writes: > On 2019/04/02 2:34, Tom Lane wrote: >> I'm not at all clear on what we think the interaction between >> enable_partition_pruning and constraint_exclusion ought to be, >> so I'm not sure what the appropriate resolution is here. Thoughts? > Prior to 428b260f87 (that is, in

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Juan José Santamaría Flecha
Actually, I tried to show my findings with the tr_TR regression test, but you can reproduce the same issue with other locales and non-ASCII characters, as Tom has pointed out. For exampe: de_DE ISO-8859-1: March es_ES ISO-8859-1: Wednesday fr_FR ISO-8859-1: February Regards, Juan José

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 22:56:20 +0200, Laurenz Albe wrote: > On Mon, 2019-04-22 at 13:27 -0700, Andres Freund wrote: > > On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > > > Commit 3d956d956a introduced support for foreign tables as partitions > > > and COPY FROM on foreign tables. > > > > > > If

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 13:27 -0700, Andres Freund wrote: > On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > > Commit 3d956d956a introduced support for foreign tables as partitions > > and COPY FROM on foreign tables. > > > > If a foreign data wrapper supports data modifications, but either has

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 16:24 -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 3:37 PM Laurenz Albe wrote: > > Sure, it is not hard to modify a FDW to continue working with v11. > > > > My point is that this should not be necessary. > > I'm not sure whether this proposal is a good idea or a

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Robert Haas wrote: > On Mon, Apr 22, 2019 at 3:08 PM Alvaro Herrera > wrote: > > On 2019-Apr-22, Andres Freund wrote: > > > Why is the obvious answer is to not just remove the whole tablespace > > > inheritance behaviour? > > > > Because it was requested by many, and there were

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > Subject: [PATCH] Foreign table COPY FROM and tuple routing requires > BeginForeignInsert > > Commit 3d956d956a introduced support for foreign tables as partitions > and COPY FROM on foreign tables. > > If a foreign data wrapper supports

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 3:37 PM Laurenz Albe wrote: > Sure, it is not hard to modify a FDW to continue working with v11. > > My point is that this should not be necessary. I'm not sure whether this proposal is a good idea or a bad idea, but I think that it's inevitable that FDWs are going to

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 3:08 PM Alvaro Herrera wrote: > On 2019-Apr-22, Andres Freund wrote: > > Why is the obvious answer is to not just remove the whole tablespace > > inheritance behaviour? > > Because it was requested by many, and there were plenty of people > surprised that things didn't

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2019-04-22 Thread Alexander Korotkov
On Thu, Nov 29, 2018 at 3:44 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Aug 24, 2018 at 5:53 PM Alexander Korotkov > > wrote: > > > > Given I've no feedback on this idea yet, I'll try to implement a PoC > > patch for that. It doesn't look to be difficult. And we'll see how >

translatability tweaks

2019-04-22 Thread Alvaro Herrera
Here's a small number of translatability tweaks to the error messages in the backend. I found these while updating the Spanish translation over the past weekend, a task I had neglected for two or three years, so they might involve some older messages. However, I won't backpatch this -- only

Re: block-level incremental backup

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 2:26 PM Stephen Frost wrote: > There was basically zero discussion about what things would look like at > a protocol level (I went back and skimmed over the thread before sending > my last email to specifically see if I was going to get this response > back..). I get the

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 21:45 +0900, Etsuro Fujita wrote: Thanks for looking into this! > (2019/04/20 20:53), Laurenz Albe wrote: > > On Fri, 2018-04-06 at 23:24 +, Robert Haas wrote: > > > Allow insert and update tuple routing and COPY for foreign tables. > > > > > > Also enable this for

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-22, Andres Freund wrote: >> Why is the obvious answer is to not just remove the whole tablespace >> inheritance behaviour? > Because it was requested by many, and there were plenty of people > surprised that things didn't work that way. There are lots of

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Apr 21, 2019 at 6:26 AM Andrew Dunstan > wrote: >> How does one do that? Just set a Turkish locale? > tr_TR is, in a sense, special among locales: > http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html > The Turkish dotless i has apparently

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Tom Lane wrote: > Yeah, that's where I'm at as well. Alvaro's proposal could be made > to work perhaps, but I think it would still end up with some odd > corner-case behaviors. One example is that "TABLESPACE X" would > be allowed if the database's default tablespace is Y, but

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-04-22 14:26:40 -0400, Stephen Frost wrote: > > I'm disappointed that the concerns about the trouble that end users are > > likely to have with this didn't garner more discussion. > > My impression is that endusers are having a lot

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Andres Freund wrote: > Why is the obvious answer is to not just remove the whole tablespace > inheritance behaviour? Because it was requested by many, and there were plenty of people surprised that things didn't work that way. -- Álvaro Herrera

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread Tom Lane
=?UTF-8?Q?Micha=c5=82_=22phoe=22_Herda?= writes: > My reasoning in this case is - if we allow the any[] type to only be > passed to other functions that accept any[], and disallow any kind of > other operations on this array (such as retrieving its elements or > modifying it), I do not yet see

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Peter Geoghegan
On Sun, Apr 21, 2019 at 6:26 AM Andrew Dunstan wrote: > How does one do that? Just set a Turkish locale? tr_TR is, in a sense, special among locales: http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html The Turkish dotless i has apparently been implicated in all kinds of

Re: block-level incremental backup

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:26:40 -0400, Stephen Frost wrote: > I'm disappointed that the concerns about the trouble that end users are > likely to have with this didn't garner more discussion. My impression is that endusers are having a lot more trouble due to important backup/restore features not

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 14:16:28 -0400, Alvaro Herrera wrote: >> I think we can get out of this whole class of problems by forbidding the >> TABLESPACE clause for partitioned rels from mentioning the database >> tablespace -- that is, users either mention some *other* tablespace, or

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Apr 22, 2019 at 1:08 PM Stephen Frost wrote: > > > One could instead just do a straightforward extension > > > to the existing BASE_BACKUP command to enable incremental backup. > > > > Ok, how do you envision that? As I mentioned

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 13:18:18 -0400, Tom Lane wrote: > Andres Freund writes: > >> (Possibly I'd not think this if I weren't fresh off a couple of days > >> with my nose in the ALTER TABLE SET NOT NULL code. But right now, > >> I think that believing that that code does not and never will have > >>

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:17:48 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-04-22 13:27:17 -0400, Tom Lane wrote: > >> I wonder > >> also if it wouldn't be smart to explicitly check that the "guaranteeing" > >> column is not attisdropped. > > > Yea, that probably would be smart. I

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:16:28 -0400, Alvaro Herrera wrote: > On 2019-Apr-22, Robert Haas wrote: > > > PostgreSQL has historically and very deliberately *not made a > > distinction* between "this object is in the default tablespace" and > > "this object is in tablespace X which happens to be the

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 13:27:17 -0400, Tom Lane wrote: >> I wonder >> also if it wouldn't be smart to explicitly check that the "guaranteeing" >> column is not attisdropped. > Yea, that probably would be smart. I don't think there's an active > problem, because we remove NOT NULL

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Robert Haas wrote: > PostgreSQL has historically and very deliberately *not made a > distinction* between "this object is in the default tablespace" and > "this object is in tablespace X which happens to be the default." I > think that it's too late to invent such a distinction

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Tom Lane
Andrey Lepikhov writes: > It is possible that custom_scan_tlist is designed too nontrivially, and > it is possible that it needs some comments describing in more detail how > to use it. I totally buy the argument that the custom scan stuff is underdocumented :-(. FWIW, if we did have a

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-04-19 20:04:41 -0400, Stephen Frost wrote: > > I agree that we don't want another implementation and that there's a lot > > that we want to do to improve replay performance. We've already got > > frontend tools which work with

Re: Patch: doc for pg_logical_emit_message()

2019-04-22 Thread Fujii Masao
On Fri, Apr 19, 2019 at 3:21 PM Matsumura, Ryo wrote: > > Hi, Hackers > > pg_logical_emit_message() can be used by any user, > but the following document says that it can be used by only superuser. > > > Table 9.88. Replication SQL Functions > > Use of these functions is restricted to superusers.

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 13:27:17 -0400, Tom Lane wrote: > Andres Freund writes: > > The computation of that variable above has: > > > * If the column is possibly missing, we can't rely on its (or > > * subsequent) NOT NULL constraints to indicate minimum > > attributes in >

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread Pavel Stehule
Hi po 22. 4. 2019 v 19:20 odesílatel Michał "phoe" Herda napsal: > Hey! > > OK - thank you for the update and the explanation. > > My reasoning in this case is - if we allow the any[] type to only be > passed to other functions that accept any[], and disallow any kind of other > operations on

Re: [PATCH v1] Add \echo_stderr to psql

2019-04-22 Thread Fabien COELHO
\warn ... \warning ... These two seem about the best to me, drawing from the perl warn command. Yep, I was thinking of perl & gmake. Maybe the 4 letter option is better because its the same length as "echo". I suppose we could go the bash &2 route here, but I don't want to. I

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread phoe
Hey! OK - thank you for the update and the explanation. My reasoning in this case is - if we allow the any[] type to only be passed to other functions that accept any[], and disallow any kind of other operations on this array (such as retrieving its elements or modifying it), I do not yet see

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 10:32 AM Stephen Frost wrote: > Yes, global indexes for partitioned tables could potentially be simpler > than the logical row identifiers, but maybe it'd be useful to just have > one implementation based around logical row identifiers which ends up > working for global

Re: block-level incremental backup

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 1:08 PM Stephen Frost wrote: > > I think we're getting closer to a meeting of the minds here, but I > > don't think it's intrinsically necessary to rewrite the whole method > > of operation of pg_basebackup to implement incremental backup in a > > sensible way. > > It

Re: Trouble with FETCH_COUNT and combined queries in psql

2019-04-22 Thread Fabien COELHO
Bonjour Daniel, When FETCH_COUNT is set, queries combined in a single request don't work as expected: \set FETCH_COUNT 10 select pg_sleep(2) \; select 1; No result is displayed, the pg_sleep(2) is not run, and no error is shown. That's disconcerting. Indeed. Does anyone have thoughts

Re: block-level incremental backup

2019-04-22 Thread Andres Freund
Hi, On 2019-04-19 20:04:41 -0400, Stephen Frost wrote: > I agree that we don't want another implementation and that there's a lot > that we want to do to improve replay performance. We've already got > frontend tools which work with multiple execution threads, so I'm not > sure I get the "not

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > > This seems like it would be helpful for global indexes as well, wouldn't > > it? > > Yes, though that should probably work by reusing what we already do > with heap TID (use standard

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > The computation of that variable above has: >* If the column is possibly missing, we can't rely on its (or >* subsequent) NOT NULL constraints to indicate minimum > attributes in >* the tuple, so stop here. >

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 9:35 AM Andres Freund wrote: > I, more generally, wonder if there's not a case to squeeze out more > padding than "just" what you describe (since we IIRC don't frequently > keep pointers into such tuples anyway, and definitely don't for byval > attrs). But that's very

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Andrey Lepikhov
On 22/04/2019 18:40, Robert Haas wrote: On Fri, Apr 19, 2019 at 12:45 AM Tom Lane wrote: I don't buy this for a minute. Where do you think projection is going to happen? There isn't any existing node type that *couldn't* support projection if we insisted that that be done

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 12:33:24 -0400, Tom Lane wrote: >> But TBH, now that I look at the code, I think the entire optimization >> is a bad idea and should be removed. Am I right in thinking that the >> presence of a wrong attnotnull marker could cause the generated code to >>

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > This seems like it would be helpful for global indexes as well, wouldn't > it? Yes, though that should probably work by reusing what we already do with heap TID (use standard IndexTuple fields on the leaf level for heap TID), plus an

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 01:11:22PM -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > > I assumed the modblock files would be stored in the WAL archive so some > > external tools could generate incremental backups using just the WAL > > files. I assumed they

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > I assumed the modblock files would be stored in the WAL archive so some > external tools could generate incremental backups using just the WAL > files. I assumed they would also be sent to standby servers so > incremental backups could be

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Apr 20, 2019 at 4:32 PM Stephen Frost wrote: > > Having been around for a while working on backup-related things, if I > > was to implement the protocol for pg_basebackup today, I'd definitely > > implement "give me a list" and

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 18:49:44 +0530, Amit Kapila wrote: > Attached is a hacky and work-in-progress patch to move fsm map to > relcache. This will give you some idea. I think we need to see if > there is a need to invalidate the relcache due to this patch. I think > some other comments of Andres

Trouble with FETCH_COUNT and combined queries in psql

2019-04-22 Thread Daniel Verite
Hi, When FETCH_COUNT is set, queries combined in a single request don't work as expected: \set FETCH_COUNT 10 select pg_sleep(2) \; select 1; No result is displayed, the pg_sleep(2) is not run, and no error is shown. That's disconcerting. The sequence that is sent under the hood is: #1

Optimizer items in the release notes

2019-04-22 Thread Bruce Momjian
We had a discussion in October about adding more optimizer items to the release notes: https://www.postgresql.org/message-id/flat/20181010220601.GA7807%40momjian.us#11d805ea0b0fcd0552dfa99251417cc1 There was no agreement on a change, but if people want to propose a change, please post

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 09:43:56 -0700, Andres Freund wrote: > On 2019-04-22 12:33:24 -0400, Tom Lane wrote: > > ISTM that Michael's proposed wording change shows that the existing > > comment is easily misinterpreted. I don't think these minor grammatical > > fixes will avoid the misinterpretation

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 12:33:24 -0400, Tom Lane wrote: > ISTM that Michael's proposed wording change shows that the existing > comment is easily misinterpreted. I don't think these minor grammatical > fixes will avoid the misinterpretation problem, and so some more-extensive > rewording is called

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Andres Freund
Hi, On 2019-04-21 17:46:09 -0700, Peter Geoghegan wrote: > Andres has suggested that I work on teaching nbtree to accommodate > variable-width, logical table identifiers, such as those required for > indirect indexes, or clustered indexes, where secondary indexes must > use a logical primary key

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 12:15:32PM -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 11:48 AM Bruce Momjian wrote: > > My point is that you would normally only remove the modblock file when 4 > > is removed because this modblock files is useful for incremental backups > > from base backups that

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-22, Andres Freund wrote: >> On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: >>> /* >>> -* Check if's guaranteed the all the desired attributes are available in >>> -* tuple. If so, we can start deforming. If not, need to make sure to >>> -*

Re: clean up docs for v12

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Andres Freund wrote: > On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: > > /* > > -* Check if's guaranteed the all the desired attributes are available in > > -* tuple. If so, we can start deforming. If not, need to make sure to > > -* fetch the missing

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 11:48 AM Bruce Momjian wrote: > My point is that most WAL archive tools will order and remove files > based on their lexical ordering, so if you put the start first, the file > will normally be removed when it should be kept, e.g., if you have WAL > files like: > >

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: > On Fri, Apr 19, 2019 at 09:43:01AM -0500, Justin Pryzby wrote: > > Thanks for committing those portions. > > I have done an extra pass on your patch set to make sure that I am > missing nothing, and the last two remaining places which

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Sun, Apr 21, 2019 at 10:21 PM Michael Paquier wrote: > If you create the extra file when a segment is finished and we switch > to a new one, then the extra work would happen for a random backend, > and it is going to be more costly to scan a 1GB segment than a 16MB > segment as a one-time

  1   2   >