pgsql: Minor edits to catalog files and scripts

2017-12-21 Thread Alvaro Herrera
Minor edits to catalog files and scripts This fixes a few typos and small mistakes; it also cleans a few minor stylistic issues. The biggest functional change is that Gen_fmgrtab.pl no longer knows the OID of language 'internal'. Author: John Naylor Discussion: https://postgr.es/m/cajvsvgxakwbk

pgsql: Adjust assertion in GetCurrentCommandId.

2017-12-21 Thread Robert Haas
Adjust assertion in GetCurrentCommandId. currentCommandIdUsed is only used to skip redundant increments of the command counter, and CommandCounterIncrement() is categorically denied under parallelism anyway. Therefore, it's OK for GetCurrentCommandId() to mark the counter value used, as long as i

pgsql: Rearrange execution of PARAM_EXTERN Params for plpgsql's benefit

2017-12-21 Thread Tom Lane
Rearrange execution of PARAM_EXTERN Params for plpgsql's benefit. This patch does three interrelated things: * Create a new expression execution step type EEOP_PARAM_CALLBACK and add the infrastructure needed for add-on modules to generate that. As discussed, the best control mechanism for that s

pgsql: Get rid of copy_partition_key

2017-12-21 Thread Alvaro Herrera
Get rid of copy_partition_key That function currently exists to avoid leaking memory in CacheMemoryContext in case of trouble while the partition key is being built, but there's a better way: allocate everything in a memcxt that goes away if the current (sub)transaction fails, and once the partiti

pgsql: Fix typo

2017-12-21 Thread Alvaro Herrera
Fix typo Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9ef6aba1d3513829d9f77a4a91ca52f2e5719aef Modified Files -- src/backend/storage/ipc/procarray.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Tom Lane
Andres Freund writes: > On 2017-12-21 08:49:46 +, Andres Freund wrote: >> Add parallel-aware hash joins. > There's to relatively mundane failures: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-12-21%2008%3A48%3A12 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl

pgsql: Avoid putting build-location-dependent strings into generated fi

2017-12-21 Thread Tom Lane
Avoid putting build-location-dependent strings into generated files. Various Perl scripts we use to generate files were in the habit of printing things like "generated by $0" into their output files. That looks like a fine idea at first glance, but it results in non-reproducible output, because in

Re: condition variable cleanup and subtransactions

2017-12-21 Thread Robert Haas
On Thu, Dec 21, 2017 at 6:46 AM, Thomas Munro wrote: > On Fri, Dec 22, 2017 at 12:00 AM, Andres Freund wrote: >> Oh. This seems to be a condition variable bug independent of PHJ. The >> problem is that the DSM segment etc all get cleaned up in >> *subtransaction* abort. >> >> Afaict it's a bug th

pgsql: Cancel CV sleep during subtransaction abort.

2017-12-21 Thread Robert Haas
Cancel CV sleep during subtransaction abort. Generally, error recovery paths that need to do things like LWLockReleaseAll and pgstat_report_wait_end also need to call ConditionVariableCancelSleep, but AbortSubTransaction was missed. Since subtransaction abort might destroy up the DSM segment that

pgsql: Cancel CV sleep during subtransaction abort.

2017-12-21 Thread Robert Haas
Cancel CV sleep during subtransaction abort. Generally, error recovery paths that need to do things like LWLockReleaseAll and pgstat_report_wait_end also need to call ConditionVariableCancelSleep, but AbortSubTransaction was missed. Since subtransaction abort might destroy up the DSM segment that

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Thomas Munro
On Thu, Dec 21, 2017 at 10:55 PM, Andres Freund wrote: > Thomas, I wonder if the problem is that PHJ_GROW_BATCHES_ELECTING > updates, via ExecParallelHashJoinSetUpBatches(), HashJoinTable->nbatch, > while other backends also access ->nbatch in > ExecParallelHashCloseBatchAccessors(). Both happens

Re: condition variable cleanup and subtransactions

2017-12-21 Thread Thomas Munro
On Fri, Dec 22, 2017 at 12:00 AM, Andres Freund wrote: > Oh. This seems to be a condition variable bug independent of PHJ. The > problem is that the DSM segment etc all get cleaned up in > *subtransaction* abort. > > Afaict it's a bug that AbortTransaction() does > ConditionVariableCancelSleep() b

condition variable cleanup and subtransactions

2017-12-21 Thread Andres Freund
Hi, On 2017-12-21 02:42:25 -0800, Andres Freund wrote: > Trying to debug this I found another issue. I'd placed a sleep(10) in > ExecParallelHashCloseBatchAccessors() and then ctrl-c'ed the server for > some reason. Segfault time: > > #0 0x55bfbac42539 in tas (lock=0x7fcd82ae14ac memory at

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Andres Freund
On 2017-12-21 01:55:50 -0800, Andres Freund wrote: > On 2017-12-21 01:29:40 -0800, Andres Freund wrote: > > On 2017-12-21 08:49:46 +, Andres Freund wrote: > > > Add parallel-aware hash joins. > > > > There's to relatively mundane failures: > > https://buildfarm.postgresql.org/cgi-bin/show_log.p

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Andres Freund
On 2017-12-21 01:29:40 -0800, Andres Freund wrote: > On 2017-12-21 08:49:46 +, Andres Freund wrote: > > Add parallel-aware hash joins. > > There's to relatively mundane failures: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-12-21%2008%3A48%3A12 > https://buildfarm.pos

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Thomas Munro
On Thu, Dec 21, 2017 at 10:29 PM, Andres Freund wrote: > On 2017-12-21 08:49:46 +, Andres Freund wrote: >> Add parallel-aware hash joins. > > There's to relatively mundane failures: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-12-21%2008%3A48%3A12 > https://buildfarm.

Re: pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Andres Freund
On 2017-12-21 08:49:46 +, Andres Freund wrote: > Add parallel-aware hash joins. There's to relatively mundane failures: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-12-21%2008%3A48%3A12 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=termite&dt=2017-12-21%2008%3

pgsql: Add parallel-aware hash joins.

2017-12-21 Thread Andres Freund
Add parallel-aware hash joins. Introduce parallel-aware hash joins that appear in EXPLAIN plans as Parallel Hash Join with Parallel Hash. While hash joins could already appear in parallel queries, they were previously always parallel-oblivious and had a partial subplan only on the outer side, mea