pgsql: Track identical top vs nested queries independently in pg_stat_s

2021-04-08 Thread Magnus Hagander
Track identical top vs nested queries independently in pg_stat_statements Changing pg_stat_statements.track between 'all' and 'top' would control if pg_stat_statements tracked just top level statements or also statements inside functions, but when tracking all it would not differentiate between th

pgsql: doc: Prefer explicit JOIN syntax over old implicit syntax in tut

2021-04-08 Thread Peter Eisentraut
doc: Prefer explicit JOIN syntax over old implicit syntax in tutorial Update src/tutorial/basics.source to match. Author: Jürgen Purtz Reviewed-by: Thomas Munro Reviewed-by: "David G. Johnston" Discussion: https://www.postgresql.org/message-id/flat/158996922318.7035.10603922579567326...@wrigl

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Amit Kapila
On Thu, Apr 8, 2021 at 12:18 PM Julien Rouhaud wrote: > > On Thu, Apr 08, 2021 at 12:08:02PM +0530, Amit Kapila wrote: > > On Thu, Apr 8, 2021 at 11:40 AM Julien Rouhaud wrote: > > > > > > That was indeed the problem. I think the best is to entirely ignore > > > parallel > > > workers in pg_sta

pgsql: Add functions to wait for backend termination

2021-04-08 Thread Magnus Hagander
Add functions to wait for backend termination This adds a function, pg_wait_for_backend_termination(), and a new timeout argument to pg_terminate_backend(), which will wait for the backend to actually terminate (with or without signaling it to do so depending on which function is called). The defa

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Julien Rouhaud
On Thu, Apr 08, 2021 at 02:52:26PM +0530, Amit Kapila wrote: > > Yeah, we do pass instrumentation flags from leader to worker but not > the one allocated via pg_stat_staments (pgss_ExecutorStart {... > queryDesc->totaltime ...}. Ah indeed, I now remember that problem that I had to deal with in so

pgsql: Add ORDER BY to some regression test queries

2021-04-08 Thread Peter Eisentraut
Add ORDER BY to some regression test queries Apparently, an unrelated patch introduced some variation on the build farm. Reported-by: Magnus Hagander Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7e3c54168d9ec058cb3c9d47f8105b1d32dc8ceb Modified Files -

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Amit Kapila
On Thu, Apr 8, 2021 at 3:11 PM Julien Rouhaud wrote: > > On Thu, Apr 08, 2021 at 02:52:26PM +0530, Amit Kapila wrote: > > > > Yeah, we do pass instrumentation flags from leader to worker but not > > the one allocated via pg_stat_staments (pgss_ExecutorStart {... > > queryDesc->totaltime ...}. > >

pgsql: Cleanup partition pruning step generation

2021-04-08 Thread David Rowley
Cleanup partition pruning step generation There was some code in gen_prune_steps_from_opexps that needlessly checked a list was not empty when it clearly had to contain at least one item. This prompted a further cleanup operation in partprune.c. Additionally, the previous code could end up adding

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Julien Rouhaud
On Thu, Apr 08, 2021 at 03:52:02PM +0530, Amit Kapila wrote: > On Thu, Apr 8, 2021 at 3:11 PM Julien Rouhaud wrote: > > > > On Thu, Apr 08, 2021 at 02:52:26PM +0530, Amit Kapila wrote: > > > > > > Yeah, we do pass instrumentation flags from leader to worker but not > > > the one allocated via pg_s

pgsql: Add circular WAL decoding buffer.

2021-04-08 Thread Thomas Munro
Add circular WAL decoding buffer. Teach xlogreader.c to decode its output into a circular buffer, to support optimizations based on looking ahead. * XLogReadRecord() works as before, consuming records one by one, and allowing them to be examined via the traditional XLogRecGetXXX() macros.

pgsql: Remove read_page callback from XLogReader.

2021-04-08 Thread Thomas Munro
Remove read_page callback from XLogReader. Previously, the XLogReader module would fetch new input data using a callback function. Redesign the interface so that it tells the caller to insert more data with a special return value instead. This API suits later patches for prefetching, encryption

pgsql: Optionally prefetch referenced data in recovery.

2021-04-08 Thread Thomas Munro
Optionally prefetch referenced data in recovery. Introduce a new GUC recovery_prefetch, disabled by default. When enabled, look ahead in the WAL and try to initiate asynchronous reading of referenced data blocks that are not yet cached in our buffer pool. For now, this is done with posix_fadvise(

pgsql: Speedup ScalarArrayOpExpr evaluation

2021-04-08 Thread David Rowley
Speedup ScalarArrayOpExpr evaluation ScalarArrayOpExprs with "useOr=true" and a set of Consts on the righthand side have traditionally been evaluated by using a linear search over the array. When these arrays contain large numbers of elements then this linear search could become a significant par

pgsql: Allow TRUNCATE command to truncate foreign tables.

2021-04-08 Thread Fujii Masao
Allow TRUNCATE command to truncate foreign tables. This commit introduces new foreign data wrapper API for TRUNCATE. It extends TRUNCATE command so that it accepts foreign tables as the targets to truncate and invokes that API. Also it extends postgres_fdw so that it can issue TRUNCATE command to

Re: pgsql: SQL-standard function body

2021-04-08 Thread Robert Haas
On Thu, Apr 8, 2021 at 2:25 AM Peter Eisentraut wrote: > On 08.04.21 01:06, Tom Lane wrote: > > Andres Freund writes: > >> On 2021-04-07 19:53:35 +, Peter Eisentraut wrote: > >>> SQL-standard function body > > > >> This is turning the BF red: > >> https://buildfarm.postgresql.org/cgi-bin/show

pgsql: Remove duplicate typedef.

2021-04-08 Thread Thomas Munro
Remove duplicate typedef. Thinko in commit 323cbe7c, per complaint from BF animal locust's older GCC compiler. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/34399a670a1c559ef8a355ed25d090d73e400ad4 Modified Files -- src/include/replication/logical.h |

pgsql: Merge v1.10 of pg_stat_statements into v1.9

2021-04-08 Thread Magnus Hagander
Merge v1.10 of pg_stat_statements into v1.9 v1.9 is already new in this version of PostgreSQL, so turn it into just one change. Author: Julien Rohaud Discussion: https://postgr.es/m/20210408120505.7zinijtdexbyghvb@nol Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

Re: pgsql: SQL-standard function body

2021-04-08 Thread Andrew Dunstan
On 4/7/21 9:50 PM, Andres Freund wrote: > Hi, > > On 2021-04-08 13:33:18 +1200, David Rowley wrote: >> On Thu, 8 Apr 2021 at 11:06, Tom Lane wrote: >>> Andres Freund writes: >>> Might be force_parallel_mode=regress related. >>> Yeah. On my machine, it's fine without force_parallel_mode an

Re: pgsql: SQL-standard function body

2021-04-08 Thread Tom Lane
Andrew Dunstan writes: > On 4/7/21 9:50 PM, Andres Freund wrote: >> I've wondered about that too. Perhaps we could reuse the pg_upgrade run? > Honestly I'd prefer it if we could get rid of the rerun of 'make check' > by pg_upgrade's test.sh and instead upgrade the data directory made by > the ear

Re: pgsql: SQL-standard function body

2021-04-08 Thread Andres Freund
Hi, On Thu, Apr 8, 2021, at 07:52, Tom Lane wrote: > Andrew Dunstan writes: > > On 4/7/21 9:50 PM, Andres Freund wrote: > >> I've wondered about that too. Perhaps we could reuse the pg_upgrade run? > > > Honestly I'd prefer it if we could get rid of the rerun of 'make check' > > by pg_upgrade's

pgsql: Fixes for query_id feature

2021-04-08 Thread Bruce Momjian
Fixes for query_id feature Ignore parallel workers in pg_stat_statements Oversight in 4f0b0966c8 which exposed queryid in parallel workers. Counters are aggregated by the main backend process so parallel workers would report duplicated activity, and could also report activity for the wrong

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Bruce Momjian
On Thu, Apr 8, 2021 at 02:11:02PM +0800, Julien Rouhaud wrote: > > oh, I think it's because parallel workers now have the queryid of the main > > query. Probably ignoring parallel workers in the executor end hook will fix > > the problem. I'll look at it as soon as I will be back home. > > That w

Re: pgsql: Move pg_stat_statements query jumbling to core.

2021-04-08 Thread Julien Rouhaud
On Thu, Apr 08, 2021 at 11:16:24AM -0400, Bruce Momjian wrote: > On Thu, Apr 8, 2021 at 02:11:02PM +0800, Julien Rouhaud wrote: > > > oh, I think it's because parallel workers now have the queryid of the main > > > query. Probably ignoring parallel workers in the executor end hook will > > > fix

pgsql: Suppress uninitialized-variable warning.

2021-04-08 Thread Tom Lane
Suppress uninitialized-variable warning. Several buildfarm critters that don't usually produce such warnings are complaining about e717a9a18. I think it's actually safe, but move initialization to silence the warning. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

pgsql: Add support for tab-completion of type arguments in \df, \do.

2021-04-08 Thread Tom Lane
Add support for tab-completion of type arguments in \df, \do. Oversight in commit a3027e1e7. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d1fcbde579d440c35023baa0de7ebf27f644a314 Modified Files -- src/bin/psql/tab-complete.c | 5 + 1 file changed,

pgsql: Silence another _bt_check_unique compiler warning.

2021-04-08 Thread Peter Geoghegan
Silence another _bt_check_unique compiler warning. Per complaint from Tom Lane Discussion: https://postgr.es/m/1922884.1617909...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/796092fb84c08162ae803e83a13aa8bd6d9b23d0 Modified Files -- s

Re: pgsql: SQL-standard function body

2021-04-08 Thread Michael Paquier
On Thu, Apr 08, 2021 at 07:56:10AM -0700, Andres Freund wrote: > IIRC we made it run with fsync explicitly enabled. Which obviously > will change timing somewhat substantially... Yeah. I think that this kind of things pretty much boils down into making the tests of pg_upgrade into TAP. We have a

Re: pgsql: Add some information about authenticated identity via log_connec

2021-04-08 Thread Michael Paquier
On Thu, Apr 08, 2021 at 07:37:56AM +0900, Michael Paquier wrote: > I have applied the fix for fairywren now, to close the loop. I have been keeping an eye on fairywren since yesterday to see if this failure is completely taken care of, but it has not reported yet. Andrew, is this animal alive? --

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- mas

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- REL

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- REL

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- REL

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- REL

pgsql: Fix typos and grammar in documentation and code comments

2021-04-08 Thread Michael Paquier
Fix typos and grammar in documentation and code comments Comment fixes are applied on HEAD, and documentation improvements are applied on back-branches where needed. Author: Justin Pryzby Discussion: https://postgr.es/m/20210408164008.gj6...@telsasoft.com Backpatch-through: 9.6 Branch -- REL

Re: pgsql: Add a new GUC and a reloption to enable inserts in parallel-mode

2021-04-08 Thread Amit Kapila
On Fri, Mar 26, 2021 at 11:07 AM Amit Kapila wrote: > > On Wed, Mar 24, 2021 at 12:14 AM Andres Freund wrote: > > > > > > Shouldn't we have this in IndexOptInfo already? > > > > No, because I think we don't build IndexOptInfo for target tables > (tables in which insert is performed). It is only b