Re: Should we warn against using too many partitions?

2019-05-23 Thread Amit Langote
On 2019/05/24 13:37, David Rowley wrote: > I've attached 3 patches of what I think should go into master, pg11, and pg10. Thanks for the updated patches. In pg11 and pg10 patches, I see this text: + Whether using table inheritance or native partitioning, hierarchies Maybe, it would better

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-05-23 Thread Amit Langote
Hi, On 2019/05/23 4:15, Andreas Seltenreich wrote: > …but when doing it on the parent relation, even 100 statements are > enough to exceed the limit: > > , > | $ psql -c "$(yes update t set c=c where c=6 \; | head -n 100)" > | FEHLER: Speicher aufgebraucht > | DETAIL: Failed on request of

Re: Should we warn against using too many partitions?

2019-05-23 Thread David Rowley
On Fri, 24 May 2019 at 14:04, Amit Langote wrote: > The latest patch on the thread linked from this CF entry (a modified > version of your patch sent by Justin Pryzby) looks good to me. Why not > post it on this thread and link this one to the CF entry? I'm not much of a fan of that patch: +

Re: Remove page-read callback from XLogReaderState.

2019-05-23 Thread Kyotaro HORIGUCHI
Thank you for looking this, Antonin. At Wed, 22 May 2019 13:53:23 +0200, Antonin Houska wrote in <25494.1558526...@spoje.net> > Kyotaro HORIGUCHI wrote: > > > Hello. Thank you for looking this. > > ... > > Yeah, I'll register this, maybe the week after next week. > > I've checked the new

Re: Top-N sorts in EXPLAIN, row count estimates, and parallelism

2019-05-23 Thread David Rowley
On Fri, 24 May 2019 at 10:44, Peter Geoghegan wrote: > > On Thu, May 23, 2019 at 3:31 PM Tom Lane wrote: > > Given the way that's implemented, I doubt that we can report it > > reliably in EXPLAIN. > > Does it have to be totally reliable? > > cost_sort() costs sorts as top-N heapsorts. While we

Re: Zedstore - compressed in-core columnar storage

2019-05-23 Thread Ajin Cherian
Hi Ashwin, - how to pass the "column projection list" to table AM? (as stated in initial email, currently we have modified table am API to pass the projection to AM) We were working on a similar columnar storage using pluggable APIs; one idea that we thought of was to modify the scan slot

Re: initdb recommendations

2019-05-23 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > "Jonathan S. Katz" writes: > > For now I have left in the password based method to be scram-sha-256 as > > I am optimistic about the support across client drivers[1] (and FWIW I > > have an implementation for crystal-pg ~60% done). > > >

Re: initdb recommendations

2019-05-23 Thread Tom Lane
"Jonathan S. Katz" writes: > For now I have left in the password based method to be scram-sha-256 as > I am optimistic about the support across client drivers[1] (and FWIW I > have an implementation for crystal-pg ~60% done). > However, this probably means we would need to set the default

Re: Minor typos and copyright year slippage

2019-05-23 Thread Tom Lane
Thomas Munro writes: > Thanks, pushed. There are also a few 2018 copyright messages in .po > files but I understand that those are managed with a different > workflow. Right. I'm not sure what the copyright-maintenance process is for the .po files, but in any case the .po files in our

Re: Should we warn against using too many partitions?

2019-05-23 Thread Amit Langote
Hi David, On 2019/05/23 18:02, David Rowley wrote: > Over on [1] I raised a concern about the lack of any warning in our > documents to inform users that they might not want to use thousands of > partitions. More recently there's [2], also suffering from OOM using > 100 partitions. Perhaps

Re: nitpick about poor style in MergeAttributes

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 5:24 PM Michael Paquier wrote: > > On Thu, May 23, 2019 at 08:27:19AM -0700, Mark Dilger wrote: > > On Thu, May 23, 2019 at 7:54 AM Tom Lane wrote: > >> Are we sure that's not just a newly-introduced bug, ie it has not > >> been tested in cases where the tlist could

Re: Read-only access to temp tables for 2PC transactions

2019-05-23 Thread Michael Paquier
On Thu, May 23, 2019 at 08:54:59AM -0700, Andres Freund wrote: > On 2019-05-23 12:36:09 +0100, Simon Riggs wrote: >> The ONLY case where this matters is if someone does a PREPARE and then >> starts doing other work on the session. Which makes no sense in the normal >> workflow of a session. I'm

Re: nitpick about poor style in MergeAttributes

2019-05-23 Thread Michael Paquier
On Thu, May 23, 2019 at 08:27:19AM -0700, Mark Dilger wrote: > On Thu, May 23, 2019 at 7:54 AM Tom Lane wrote: >> Are we sure that's not just a newly-introduced bug, ie it has not >> been tested in cases where the tlist could become empty? My first >> thought would be to assign the list pointer

Re: initdb recommendations

2019-05-23 Thread Jonathan S. Katz
On 5/23/19 6:47 PM, Jonathan S. Katz wrote: > On 5/23/19 12:54 PM, Peter Eisentraut wrote: >> On 2019-04-06 20:08, Noah Misch wrote: > I think we should just change the defaults. There is a risk of warning > fatigue. initdb does warn about this, so anyone who cared could have >

Re: Minor typos and copyright year slippage

2019-05-23 Thread Thomas Munro
On Thu, May 23, 2019 at 1:55 PM Michael Paquier wrote: > On Thu, May 23, 2019 at 01:28:45PM +1200, Thomas Munro wrote: > > Here are some tiny things I noticed in passing. > > Good catches. And you have spotted all the blank spots for the > copyright notices. Thanks, pushed. There are also a

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-23 Thread Andres Freund
Hi, On 2019-05-24 08:32:29 +0900, Michael Paquier wrote: > On Thu, May 23, 2019 at 03:31:30PM -0700, Andres Freund wrote: > > Well, I think the approach of duplicating code all over is a bad idea, > > and the fix is many times too big. But it's better than not fixing. > > Well, I can see why the

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-23 Thread Michael Paquier
On Thu, May 23, 2019 at 03:31:30PM -0700, Andres Freund wrote: > Well, I think the approach of duplicating code all over is a bad idea, > and the fix is many times too big. But it's better than not fixing. Well, I can see why the current solution is not perfect, but we have been doing that for

Re: Inconsistency between table am callback and table function names

2019-05-23 Thread Andres Freund
Hi, On 2019-05-14 12:11:46 -0700, Ashwin Agrawal wrote: > Thank you. Please find the patch to rename the agreed functions. It would > be good to make all consistent instead of applying exception to three > functions but seems no consensus on it. Given table_ api are new, we could > modify them

No mention of no CIC support for partitioned index in docs

2019-05-23 Thread David Rowley
It appears there is no mention of lack of support for CREATE INDEX CONCURRENTLY on partitioned index in the documents. Added in the attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: initdb recommendations

2019-05-23 Thread Jonathan S. Katz
On 5/23/19 12:54 PM, Peter Eisentraut wrote: > On 2019-04-06 20:08, Noah Misch wrote: I think we should just change the defaults. There is a risk of warning fatigue. initdb does warn about this, so anyone who cared could have gotten the information. >>> >>> I've been

Re: Question about BarrierAttach spinlock

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 3:44 PM Thomas Munro wrote: > > On Fri, May 24, 2019 at 4:10 AM Mark Dilger wrote: > > In src/backend/storage/ipc/barrier.c, BarrierAttach > > goes to the bother of storing the phase before > > releasing the spinlock, and then returns the phase. > > > > In nodeHash.c,

Re: ClosePipeStream failure ignored in pg_import_system_collations

2019-05-23 Thread Tom Lane
Mark Dilger writes: > On Thu, May 23, 2019 at 3:23 PM Tom Lane wrote: >> The concrete case where that's an issue, I think, is that "locale -a" >> fails, possibly after outputting a few locale names. The only report >> we get about that is a failure indication from ClosePipeStream. >> As things

Re: Question about BarrierAttach spinlock

2019-05-23 Thread Thomas Munro
On Fri, May 24, 2019 at 4:10 AM Mark Dilger wrote: > In src/backend/storage/ipc/barrier.c, BarrierAttach > goes to the bother of storing the phase before > releasing the spinlock, and then returns the phase. > > In nodeHash.c, ExecHashTableCreate ignores the > phase returned by BarrierAttach, and

Re: Top-N sorts in EXPLAIN, row count estimates, and parallelism

2019-05-23 Thread Peter Geoghegan
On Thu, May 23, 2019 at 3:31 PM Tom Lane wrote: > Given the way that's implemented, I doubt that we can report it > reliably in EXPLAIN. Does it have to be totally reliable? cost_sort() costs sorts as top-N heapsorts. While we cannot make an iron-clad guarantee that it will work out that way

Re: ClosePipeStream failure ignored in pg_import_system_collations

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 3:23 PM Tom Lane wrote: > > Mark Dilger writes: > > I only see three invocations of ClosePipeStream in the sources. > > In two of them, the return value is checked and an error is raised > > if it failed. In the third, the error (if any) is squashed. > > > I don't know

Re: Top-N sorts in EXPLAIN, row count estimates, and parallelism

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 18:31:43 -0400, Tom Lane wrote: > Andres Freund writes: > > It's also noticable that we preposterously assume that the sort actually > > will return exactly the number of rows in the table, despite being a > > top-n style sort. > > In general, we report nodes below LIMIT with

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Tom Lane
Donald Dong writes: > Perhaps the cheapest-total-cost should not be the best/only choice > for fitness? Well, really the GEQO code should be thrown out and rewritten from the ground up ... but that hasn't quite gotten done yet. regards, tom lane

Re: pg_dump throwing "column number -1 is out of range 0..36" on HEAD

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 09:11:33 +0900, Michael Paquier wrote: > On Wed, May 22, 2019 at 02:39:54PM -0400, Tom Lane wrote: > > Andres Freund writes: > >> Well, if we explicitly have to check for -1, it's not really an error of > >> omission for everything. Yes, we could forget returning the amname in

Re: Top-N sorts in EXPLAIN, row count estimates, and parallelism

2019-05-23 Thread Tom Lane
Andres Freund writes: > Right now we don't indicate that a top-n sort is going to be used in > EXPLAIN, just EXPLAIN ANALYZE. Given the way that's implemented, I doubt that we can report it reliably in EXPLAIN. > It's also noticable that we preposterously assume that the sort actually > will

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Tom Lane
Andres Freund writes: > On 2019-05-23 14:05:19 -0400, Tom Lane wrote: >> I think you're vastly overstating the case for refusing support for this. >> Adding "#ifndef FRONTEND" to relevant headers isn't a huge amount of work >> --- it's certainly far less of a problem than the Microsoft-droppings

Re: ClosePipeStream failure ignored in pg_import_system_collations

2019-05-23 Thread Tom Lane
Mark Dilger writes: > I only see three invocations of ClosePipeStream in the sources. > In two of them, the return value is checked and an error is raised > if it failed. In the third, the error (if any) is squashed. > I don't know if a pipe stream over "locale -a" could ever fail to > close,

Re: Remove useless associativity/precedence from parsers

2019-05-23 Thread Andres Freund
Hi, On 2019-05-22 17:25:31 -0400, Tom Lane wrote: > The easiest method is to fire up some client code that repeatedly > does whatever you want to test, and then look at perf or oprofile > or local equivalent to see where the time is going in the backend > process. > > For the particular case of

Re: SQL-spec incompatibilities in similar_escape() and related stuff

2019-05-23 Thread Tom Lane
I wrote: > I propose therefore that we leave similar_escape in place with its > current behavior, as a compatibility measure for cases like this. > Intead, invent two new strict functions, say > similar_to_escape(pattern) > similar_to_escape(pattern, escape) > and change the parser and

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Donald Dong
On May 23, 2019, at 10:43 AM, Tom Lane wrote: > Donald Dong writes: >> On May 23, 2019, at 9:02 AM, Tom Lane wrote: >>> (2) the paths you show do not correspond to the finally selected >>> plans --- they aren't even the same shape. (The Gathers are in >>> different places, to start with.) I'm

Top-N sorts in EXPLAIN, row count estimates, and parallelism

2019-05-23 Thread Andres Freund
Hi, Right now we don't indicate that a top-n sort is going to be used in EXPLAIN, just EXPLAIN ANALYZE. That's imo suboptimal, because one quite legitimately might want to know that before actually executing (as it will make a huge amount of difference in the actual resource intensity of the

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 14:05:19 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-23 13:46:15 -0400, Tom Lane wrote: > >> I'm not really excited about adopting a position that PG will only > >> build on GCC and clones thereof. > > > That's not what I said though? Not supporting one

ClosePipeStream failure ignored in pg_import_system_collations

2019-05-23 Thread Mark Dilger
Hackers, I only see three invocations of ClosePipeStream in the sources. In two of them, the return value is checked and an error is raised if it failed. In the third, the error (if any) is squashed. I don't know if a pipe stream over "locale -a" could ever fail to close, but it seems sensible

Re: fsync failure in durable_unlink ignored in xlog.c?

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 14:06:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-23 10:46:02 -0700, Mark Dilger wrote: > >> Is this code safe against fsync failures? If so, can I get an explanation > >> that I might put into a code comment patch? > > > What's the danger you're thinking

Re: fsync failure in durable_unlink ignored in xlog.c?

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 11:07 AM Tom Lane wrote: > > Andres Freund writes: > > On 2019-05-23 10:46:02 -0700, Mark Dilger wrote: > >> Is this code safe against fsync failures? If so, can I get an explanation > >> that I might put into a code comment patch? > > > What's the danger you're thinking

Re: fsync failure in durable_unlink ignored in xlog.c?

2019-05-23 Thread Tom Lane
Andres Freund writes: > On 2019-05-23 10:46:02 -0700, Mark Dilger wrote: >> Is this code safe against fsync failures? If so, can I get an explanation >> that I might put into a code comment patch? > What's the danger you're thinking of here? The issue with ignoring fsync > failures is that it

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Tom Lane
Andres Freund writes: > On 2019-05-23 13:46:15 -0400, Tom Lane wrote: >> I'm not really excited about adopting a position that PG will only >> build on GCC and clones thereof. > That's not what I said though? Not supporting one compiler, on an OS > that's effectively not being developed anymore,

Re: fsync failure in durable_unlink ignored in xlog.c?

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 10:46:02 -0700, Mark Dilger wrote: > I have seen other lengthy discussions about fsync semantics, and if this > question is being addressed there, this question might not be relevant. > > Two calls to durable_unlink at log level DEBUG1 are ignoring the > return value. Other

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 13:46:15 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-22 16:13:02 -0400, Tom Lane wrote: > >> It doesn't sound like "use a newer compiler" is going to be a helpful > >> answer there. > > > Well, GCC is available on solaris, and IIRC not that hard to install >

Re: Minimal logical decoding on standbys

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 23:08:55 +0530, Amit Khandekar wrote: > On Thu, 23 May 2019 at 21:29, Andres Freund wrote: > > On 2019-05-23 17:39:21 +0530, Amit Khandekar wrote: > > > On Tue, 21 May 2019 at 21:49, Andres Freund wrote: > > > Yeah, I agree we should add such checks to minimize the possibility

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Tom Lane
Andres Freund writes: > On 2019-05-22 16:13:02 -0400, Tom Lane wrote: >> It doesn't sound like "use a newer compiler" is going to be a helpful >> answer there. > Well, GCC is available on solaris, and IIRC not that hard to install > (isn't it just a 'pkg install gcc' or such?). Don't think we

fsync failure in durable_unlink ignored in xlog.c?

2019-05-23 Thread Mark Dilger
Hackers, I have seen other lengthy discussions about fsync semantics, and if this question is being addressed there, this question might not be relevant. Two calls to durable_unlink at log level DEBUG1 are ignoring the return value. Other calls at ERROR and FATAL are likewise ignoring the

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Tom Lane
Donald Dong writes: > On May 23, 2019, at 9:02 AM, Tom Lane wrote: >> (2) the paths you show do not correspond to the finally selected >> plans --- they aren't even the same shape. (The Gathers are in >> different places, to start with.) I'm not sure where you were >> capturing the path data,

Re: Minimal logical decoding on standbys

2019-05-23 Thread Amit Khandekar
On Thu, 23 May 2019 at 21:29, Andres Freund wrote: > > Hi, > > On 2019-05-23 17:39:21 +0530, Amit Khandekar wrote: > > On Tue, 21 May 2019 at 21:49, Andres Freund wrote: > > Yeah, I agree we should add such checks to minimize the possibility of > > reading logical records from a master that has

Fix link for v12

2019-05-23 Thread Euler Taveira
Hi, I noticed that v12 release notes is referencing the wrong GUC. It should be recovery_target_timeline instead of recovery_target_time. Patch attached. -- Euler Taveira Timbira - http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento,

Re: FullTransactionId changes are causing portability issues

2019-05-23 Thread Andres Freund
Hi, On 2019-05-22 16:13:02 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-22 15:55:50 -0400, Tom Lane wrote: > Per Bjorn's report: > >> The compiler used in Sun Studio 12u1, very old and and I can try to > >> upgrade and see if that helps. > > I tried Sun Studio 12u2 and then a

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Donald Dong
On May 23, 2019, at 9:02 AM, Tom Lane wrote: > > Donald Dong writes: >> On May 22, 2019, at 11:42 AM, Tom Lane wrote: >>> You're still asking us to answer hypothetical questions unsupported >>> by evidence. In what case does that really happen? > >> I attached the query plan and

Re: initdb recommendations

2019-05-23 Thread Magnus Hagander
On Thu, May 23, 2019, 18:54 Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-04-06 20:08, Noah Misch wrote: > >>> I think we should just change the defaults. There is a risk of warning > >>> fatigue. initdb does warn about this, so anyone who cared could have > >>> gotten

Re: initdb recommendations

2019-05-23 Thread Peter Eisentraut
On 2019-04-06 20:08, Noah Misch wrote: >>> I think we should just change the defaults. There is a risk of warning >>> fatigue. initdb does warn about this, so anyone who cared could have >>> gotten the information. >>> >> >> I've been suggesting that for years, so definite strong +1 for doing

Re: nitpick about useless floating point division in gimme_edge_table

2019-05-23 Thread Tom Lane
Mark Dilger writes: > Hackers, > The return value of gimme_edge_table is not used anywhere in the > core code, so far as I can see. But the value is computed as > /* return average number of edges per index */ > return ((float) (edge_total * 2) / (float) num_gene); > which involves some

Re: Is it safe to ignore the return value of SPI_finish and SPI_execute?

2019-05-23 Thread Tom Lane
I wrote: > It's not entirely clear to me whether we ought to change the return > convention to be "returns void" or make it "always returns SPI_OK" > for those functions where the return code becomes trivial. The > latter would avoid churn for external modules, but it seems not to > have much

Question about BarrierAttach spinlock

2019-05-23 Thread Mark Dilger
Hackers, In src/backend/storage/ipc/barrier.c, BarrierAttach goes to the bother of storing the phase before releasing the spinlock, and then returns the phase. In nodeHash.c, ExecHashTableCreate ignores the phase returned by BarrierAttach, and then immediately calls BarrierPhase to get the phase

Re: Memory bug in dsnowball_lexize

2019-05-23 Thread Tom Lane
Mark Dilger writes: > On Thu, May 23, 2019 at 8:46 AM Tom Lane wrote: >> Mark Dilger writes: >>> In src/backend/snowball/dict_snowball.c, 'dsnowball_lexize' >>> calls 'SN_set_current' and ignores the return value, thereby >>> failing to notice the error, if any. >> Hm. This seems like

Re: Memory bug in dsnowball_lexize

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 8:46 AM Tom Lane wrote: > > Mark Dilger writes: > > In src/backend/snowball/libstemmer/utilities.c, 'create_s' uses > > malloc (not palloc) to allocate memory, and on memory exhaustion > > returns NULL rather than throwing an exception. > > Actually not, see macros in

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Tom Lane
Donald Dong writes: > On May 22, 2019, at 11:42 AM, Tom Lane wrote: >> You're still asking us to answer hypothetical questions unsupported >> by evidence. In what case does that really happen? > I attached the query plan and debug_print_rel output for GEQO and > standard_join_search. >

Re: Minimal logical decoding on standbys

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 09:37:50 -0400, Robert Haas wrote: > On Thu, May 23, 2019 at 9:30 AM Sergei Kornilov wrote: > > > wal_level is PGC_POSTMASTER. > > > > But primary can be restarted without restart on standby. We require > > wal_level replica or highter (currently only logical) on standby. So

Re: Minimal logical decoding on standbys

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 17:39:21 +0530, Amit Khandekar wrote: > On Tue, 21 May 2019 at 21:49, Andres Freund wrote: > Yeah, I agree we should add such checks to minimize the possibility of > reading logical records from a master that has insufficient wal_level. > So to summarize : > a.

Re: with oids option not removed in pg_dumpall

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 16:26:38 +0300, Surafel Temesgen wrote: > Thank you for applying > > regards > Surafel > > On Thu, May 23, 2019 at 3:43 AM Michael Paquier wrote: > > > On Tue, May 21, 2019 at 05:24:57PM +0900, Michael Paquier wrote: > > > Good catch. Your cleanup looks correct to me.

Re: crash testing suggestions for 12 beta 1

2019-05-23 Thread Peter Geoghegan
On Thu, May 23, 2019 at 8:24 AM Jeff Janes wrote: > Now that beta is out, I wanted to do some crash-recovery testing where I > inject PANIC-inducing faults and see if it recovers correctly. Thank you for doing this. It's important work. > Making the ctid be tie-breakers in btree index is also

Re: Read-only access to temp tables for 2PC transactions

2019-05-23 Thread Andres Freund
Hi, On 2019-05-23 12:36:09 +0100, Simon Riggs wrote: > The ONLY case where this matters is if someone does a PREPARE and then > starts doing other work on the session. Which makes no sense in the normal > workflow of a session. I'm sure there are tests that do that, but those > tests are

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-23 Thread Pantelis Theodosiou
On Thu, May 23, 2019 at 4:36 PM Pantelis Theodosiou wrote: > > On Thu, May 23, 2019 at 1:01 PM Jonathan S. Katz > wrote: > >> On 5/23/19 1:45 AM, David Rowley wrote: >> > On Thu, 23 May 2019 at 15:31, Jonathan S. Katz >> wrote: >> >> Attached is >> >> v3 of the patch, along with a diff. >> >

Re: Memory bug in dsnowball_lexize

2019-05-23 Thread Tom Lane
Mark Dilger writes: > In src/backend/snowball/libstemmer/utilities.c, 'create_s' uses > malloc (not palloc) to allocate memory, and on memory exhaustion > returns NULL rather than throwing an exception. Actually not, see macros in src/include/snowball/header.h. > In

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-23 Thread Pantelis Theodosiou
On Thu, May 23, 2019 at 1:01 PM Jonathan S. Katz wrote: > On 5/23/19 1:45 AM, David Rowley wrote: > > On Thu, 23 May 2019 at 15:31, Jonathan S. Katz > wrote: > >> Attached is > >> v3 of the patch, along with a diff. > > > > Minor details, but this query is not valid: > > > >> WITH c AS

Re: nitpick about poor style in MergeAttributes

2019-05-23 Thread Mark Dilger
On Thu, May 23, 2019 at 7:54 AM Tom Lane wrote: > > Michael Paquier writes: > > On Wed, May 22, 2019 at 06:20:01PM -0700, Mark Dilger wrote: > >> What to do about this is harder to say. In the following > >> patch, I'm just doing what I think is standard for callers > >> of list_delete_cell,

crash testing suggestions for 12 beta 1

2019-05-23 Thread Jeff Janes
Now that beta is out, I wanted to do some crash-recovery testing where I inject PANIC-inducing faults and see if it recovers correctly. A long-lived Perl process keeps track of what it should find after the crash, and verifies that it finds it. You will probably be familiar with the general theme

Re: refactoring - share str2*int64 functions

2019-05-23 Thread Fabien COELHO
Although I agree it is not worth a lot of trouble, and even if I don't do Windows, I think it valuable that the behavior is the same on all platform. The attached match shares pg_str2*int64 functions between frontend and backend by moving them to "common/", which avoids some code duplication.

Memory bug in dsnowball_lexize

2019-05-23 Thread Mark Dilger
Hackers, In src/backend/snowball/libstemmer/utilities.c, 'create_s' uses malloc (not palloc) to allocate memory, and on memory exhaustion returns NULL rather than throwing an exception. In this same file, 'replace_s' calls 'create_s' and if it gets back NULL, returns the error code -1.

RE: psql - add SHOW_ALL_RESULTS option

2019-05-23 Thread Fabien COELHO
Here is a v3 which fixes \errverbose, hopefully. V5 is a rebase and an slightly improved documentation. -- Fabien.diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index b86764003d..4a1b562f24 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++

Re: nitpick about poor style in MergeAttributes

2019-05-23 Thread Tom Lane
Michael Paquier writes: > On Wed, May 22, 2019 at 06:20:01PM -0700, Mark Dilger wrote: >> What to do about this is harder to say. In the following >> patch, I'm just doing what I think is standard for callers >> of list_delete_cell, and assigning the return value back >> to the list (similar to

Re: ACL dump ordering broken as well for tablespaces

2019-05-23 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Wed, May 22, 2019 at 06:35:31PM +, Bossart, Nathan wrote: > > A bit of digging led me to the commit that removed databases and > > tablespaces from pg_init_privs [0] and to a related thread [1]. IIUC > > the problem is that using

Re: pgbench - add pseudo-random permutation function

2019-05-23 Thread Fabien COELHO
Hello Hironobu-san, Here is a v15 which is a rebase, plus a large simplification of the modmul function if an int128 type is available, which is probably always… Could you have a look and possibly revalidate? Sorry for the late reply. I reviewed this patch. Function nbits(), which was

Re: "long" type is not appropriate for counting tuples

2019-05-23 Thread Tom Lane
Peter Eisentraut writes: > On 2019-05-23 15:52, Robert Haas wrote: >> On Thu, May 23, 2019 at 5:31 AM Peter Eisentraut >> wrote: >>> Another option is that in cases where it doesn't affect storage layouts, >>> like the counting tuples case that started this thread, code could just >>> use long

Re: pglz performance

2019-05-23 Thread Binguo Bao
Hi hackers! I am a student participating in GSoC 2019. I am looking forward to working with you all and learning from you. My project would aim to provide the ability to de-TOAST a fully TOAST'd and compressed field using an iterator. For more details, please take a look at my proposal[0]. Any

Re: "long" type is not appropriate for counting tuples

2019-05-23 Thread Peter Eisentraut
On 2019-05-23 15:52, Robert Haas wrote: > On Thu, May 23, 2019 at 5:31 AM Peter Eisentraut > wrote: >> Another option is that in cases where it doesn't affect storage layouts, >> like the counting tuples case that started this thread, code could just >> use long long int directly instead of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2019-05-23 Thread Fabien COELHO
V11 is just a rebase after the reindentation patches. Indeed, yet again, I forgot the attachement:-( I stared at the new test case for a while, and I must say it looks very cryptic. It's not exactly this patch's fault - all the pgbench tests are cryptic - Perl is cryptic. Regexprs are

Re: pgbench - add \aset to store results of a combined query

2019-05-23 Thread Fabien COELHO
V2 is a rebase. A long time ago I submitted a pgbench \into command to store results of queries into variables independently of the query being processed, which got turn into \gset (;) and \cset (\;), which got committed, then \cset was removed because it was not "up to standard", as it

Re: "long" type is not appropriate for counting tuples

2019-05-23 Thread Robert Haas
On Thu, May 23, 2019 at 5:31 AM Peter Eisentraut wrote: > Another option is that in cases where it doesn't affect storage layouts, > like the counting tuples case that started this thread, code could just > use long long int directly instead of int64. Then if someone wants to > make it 128 bits

Re: Minimal logical decoding on standbys

2019-05-23 Thread Robert Haas
On Thu, May 23, 2019 at 9:30 AM Sergei Kornilov wrote: > > wal_level is PGC_POSTMASTER. > > But primary can be restarted without restart on standby. We require wal_level > replica or highter (currently only logical) on standby. So online change from > logical to replica wal_level is possible on

Re: Minimal logical decoding on standbys

2019-05-23 Thread Sergei Kornilov
Hello > wal_level is PGC_POSTMASTER. But primary can be restarted without restart on standby. We require wal_level replica or highter (currently only logical) on standby. So online change from logical to replica wal_level is possible on standby's controlfile. regards, Sergei

Re: with oids option not removed in pg_dumpall

2019-05-23 Thread Surafel Temesgen
Thank you for applying regards Surafel On Thu, May 23, 2019 at 3:43 AM Michael Paquier wrote: > On Tue, May 21, 2019 at 05:24:57PM +0900, Michael Paquier wrote: > > Good catch. Your cleanup looks correct to me. Andres, perhaps you > > would prefer doing the cleanup yourself? > > As I am

Re: nitpick about poor style in MergeAttributes

2019-05-23 Thread Mark Dilger
On Wed, May 22, 2019 at 10:21 PM Michael Paquier wrote: > > On Wed, May 22, 2019 at 06:20:01PM -0700, Mark Dilger wrote: > > What to do about this is harder to say. In the following > > patch, I'm just doing what I think is standard for callers > > of list_delete_cell, and assigning the return

Re: Minimal logical decoding on standbys

2019-05-23 Thread Robert Haas
On Thu, May 23, 2019 at 8:08 AM Amit Khandekar wrote: > This made me think more of the race conditions. For instance, in > pg_create_logical_replication_slot(), just after > CheckLogicalDecodingRequirements and before actually creating the > slot, suppose concurrently Controlfile->wal_level is

Re: [HACKERS] Unlogged tables cleanup

2019-05-23 Thread Robert Haas
On Thu, May 23, 2019 at 2:43 AM Michael Paquier wrote: > On Tue, May 21, 2019 at 08:39:18AM -0400, Robert Haas wrote: > > Yes. I thought I had described it. You create an unlogged table, > > with an index of a type that does not smgrimmedsync(), your > > transaction commits, and then the system

Re: Fuzzy thinking in is_publishable_class

2019-05-23 Thread Peter Eisentraut
On 2019-05-09 15:41, Tom Lane wrote: >> I think we can get rid of the ability to reload the information_schema >> after initdb. That was interesting in the early phase of its >> development, but now it just creates complications. > We've relied on that more than once to allow minor-release

Re: Remove useless associativity/precedence from parsers

2019-05-23 Thread Robert Haas
On Thu, May 23, 2019 at 12:00 AM Tom Lane wrote: > > One possible > > idea is a way to mark a rule %weak, meaning that it should only be > > used if no non-%weak rule could apply. I'm not sure if that would > > really be the best way, but it's one idea. A more general version > > would be some

Re: Minimal logical decoding on standbys

2019-05-23 Thread Amit Khandekar
On Tue, 21 May 2019 at 21:49, Andres Freund wrote: > > Hi, > > Sorry for the late response. > > On 2019-04-16 12:27:46 +0530, Amit Khandekar wrote: > > On Sat, 13 Apr 2019 at 00:57, Andres Freund wrote: > > > > Not sure why this is happening. On slave, wal_level is logical, so > > > > logical

Re: PostgreSQL 12 Beta 1 press release draft

2019-05-23 Thread Jonathan S. Katz
On 5/23/19 1:45 AM, David Rowley wrote: > On Thu, 23 May 2019 at 15:31, Jonathan S. Katz wrote: >> Attached is >> v3 of the patch, along with a diff. > > Minor details, but this query is not valid: > >> WITH c AS MATERIALIZED ( >> SELECT * FROM a WHERE a.x % 4 >> ) >> SELECT * FROM c JOIN d

Re: Read-only access to temp tables for 2PC transactions

2019-05-23 Thread Simon Riggs
On Tue, 14 May 2019 at 10:53, Stas Kelvich wrote: > One of the problems regarding the use of temporary tables in prepared > transactions > is that such transaction will hold locks for a temporary table after being > prepared. > That locks will prevent the backend from exiting since it will fail

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-05-23 Thread Julian Schauder
Hey David, > "multiple statements in a single query", did you mean to write > session > or maybe transaction there? Maybe the wording isn't perfect. It is required that the querys are sent as a single batch. Try the exact bash-script Andreas used for updating the parent. > Which version?

Re: Patch to fix write after end of array in hashed agg initialization

2019-05-23 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: Andrew> My inclination is to fix this in the planner rather than the Andrew> executor; there seems no good reason to actually hash a Andrew> duplicate column more than once. I take this back; I don't believe it's possible to eliminate duplicates in all

Re: Create function using quote_literal issues

2019-05-23 Thread Mohamed DIA
I found the solution by defining r as record and using FOR r in EXECUTE v_select Thanks On Thu, May 23, 2019 at 9:49 AM Mohamed DIA wrote: > Hi, > I am trying to use a create function in order to update some values in a > table (see below code). > However, when I run the function, it never

Re: Excessive memory usage in multi-statement queries w/ partitioning

2019-05-23 Thread David Rowley
On Thu, 23 May 2019 at 21:19, Julian Schauder wrote: > > "multiple statements in a single query", did you mean to write > > session > > or maybe transaction there? > > Maybe the wording isn't perfect. It is required that the querys are > sent as a single batch. Try the exact bash-script Andreas

Create function using quote_literal issues

2019-05-23 Thread Mohamed DIA
Hi, I am trying to use a create function in order to update some values in a table (see below code). However, when I run the function, it never enters into the following loop *FOR r IN SELECT * FROM immatriculationemployeursucctemp2 where succursale = quote_literal(s.succursale) order by

Re: "long" type is not appropriate for counting tuples

2019-05-23 Thread Peter Eisentraut
On 2019-05-22 21:21, Tom Lane wrote: > Peter Eisentraut writes: >> Could we define int64 to be long long int on all platforms and just >> always use %lld? > > Hmmm ... maybe. Once upon a time we had to cope with compilers > that didn't have "long long", but perhaps that time is past. It's

Should we warn against using too many partitions?

2019-05-23 Thread David Rowley
Over on [1] I raised a concern about the lack of any warning in our documents to inform users that they might not want to use thousands of partitions. More recently there's [2], also suffering from OOM using 100 partitions. Perhaps there's more too this, but the planner using a lot of memory

Re: Why could GEQO produce plans with lower costs than the standard_join_search?

2019-05-23 Thread Andrew Gierth
> "Finnerty" == Finnerty, Jim writes: Finnerty> planstate-> total_cost cheapest_total_path Finnerty> GEQO 54190.1354239.03 Finnerty> STD 54179.0254273.73 These differences aren't significant - the standard join search

  1   2   >