Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

On Mon, Jul 3, 2017 at 8:57 PM, Simon Riggs wrote: > On 30 June 2017 at 05:14, Amit Kapila wrote: > >> This is explained in section 15.2 [1], refer below para: >> "The query might be suspended during execution. In any situation in >> which the

Re: [HACKERS] More race conditions in logical replication

On Sun, Jul 02, 2017 at 07:54:48PM -0400, Tom Lane wrote: > I noticed a recent failure that looked suspiciously like a race condition: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet=2017-07-02%2018%3A02%3A07 > > The critical bit in the log file is > > error running SQL:

Re: [HACKERS] Race conditions with WAL sender PID lookups

On Mon, Jul 03, 2017 at 12:14:55PM -0400, Alvaro Herrera wrote: > Michael Paquier wrote: > > > In passing, clean up some leftover braces which were used to create > > > unconditional blocks. Once upon a time these were used for > > > volatile-izing accesses to those shmem structs, which is no

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Mon, Jul 3, 2017 at 6:15 PM, Amit Kapila wrote: > On Thu, Mar 23, 2017 at 1:18 PM, Ashutosh Sharma > wrote: >> >> Conclusion: >> As seen from the test results mentioned above, there is some performance >> improvement with 3 SP(s), with 5 SP(s)

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Fri, Jun 30, 2017 at 2:53 PM, Rafia Sabih wrote: > > > On Mon, May 22, 2017 at 12:02 PM, Ashutosh Bapat > wrote: >> >> >> Here's set of patches rebased on latest head. > > > In an attempt to test this set of patches, I found that

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

Thanks for the review. On 2017/07/03 20:13, Ashutosh Bapat wrote: > Thanks for working on the previous comments. The code really looks good now. > On Fri, Jun 23, 2017 at 2:29 PM, Amit Langote > wrote: >> >>> Don't we need an exclusive lock to >>> make sure that

Re: [HACKERS] asynchronous execution

Thank you for the thought. This is at PoC level so I'd be grateful for this kind of fundamental comments. At Wed, 28 Jun 2017 20:22:24 +0200, Antonin Houska wrote in <392.1498674144@localhost> > Kyotaro HORIGUCHI wrote: > > > The patch got

[HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

I was reminded by bug #14729 that we are not very good on corner cases involving inequality operators, ie < <= > >=. Historically, the selectivity functions have simply not distinguished < from <= or > from >=, arguing that the fraction of the population that satisfies the "=" aspect can be

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

On Tue, Jul 4, 2017 at 12:21 PM, Peter Eisentraut wrote: > On 6/25/17 06:35, Petr Jelinek wrote: >> - Do LockSharedObject in ALTER SUBSCRIPTION, DROP SUBSCRIPTION (this one >> is preexisting but mentioning it for context), SetSubscriptionRelState, >>

Re: [HACKERS] [BUGS] BUG #14699: Statement trigger and logical replication

On 6/17/17 08:29, Euler Taveira wrote: > 2017-06-16 22:08 GMT-03:00 Peter Eisentraut > >: > > > The issue is that the logical replication initial data copy fires a > statement trigger for INSERT, because it's

Re: [HACKERS] subscription worker signalling wal writer too much

On Sat, Jun 24, 2017 at 5:09 PM, Andres Freund wrote: > Hi, > > On 2017-06-15 15:06:43 -0700, Jeff Janes wrote: > > > It looks like only limited consolidation was happening, the number of > kills > > invoked was more than half of the number of SetLatch. I think the reason >

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

On 6/25/17 06:35, Petr Jelinek wrote: > - Do LockSharedObject in ALTER SUBSCRIPTION, DROP SUBSCRIPTION (this one > is preexisting but mentioning it for context), SetSubscriptionRelState, > AddSubscriptionRelState, and in the logicalrep_worker_launch. This means > we use granular per object locks

Re: [HACKERS] Error while copying a large file in pg_rewind

On Tue, Jul 4, 2017 at 4:27 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On 7/3/17 09:53, Tom Lane wrote: >>> Hm. Before we add a bunch of code to deal with that, are we sure we >>> *want* it to copy such files? Seems like that's

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

On Mon, Jul 3, 2017 at 11:00 PM, Tom Lane wrote: > Michael Paquier writes: >> On Mon, Jul 3, 2017 at 9:25 PM, Greg Stark wrote: >>> On 2 July 2017 at 18:33, Tom Lane wrote: system("cp -a ...") call in favor

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

Mark Rofail wrote: > On Mon, Jun 26, 2017 at 6:44 PM, Alexander Korotkov > wrote: > > > Have you met any particular problem here? Or is it just a lot of > > mechanical work? > > > > Just A LOT of mechanictal work, thankfully. The patch is now rebased and > all regress

Re: [HACKERS] Revisiting NAMEDATALEN

Emrul writes: > Is this something that can be revisited for an upcoming release? Also, are > there any technical problems that would be created by increasing this > attribute? This has been discussed before, eg here:

Re: [HACKERS] Error while copying a large file in pg_rewind

Peter Eisentraut writes: > On 7/3/17 09:53, Tom Lane wrote: >> Hm. Before we add a bunch of code to deal with that, are we sure we >> *want* it to copy such files? Seems like that's expending a lot of >> data-transfer work for zero added value --- consider e.g.

Re: [HACKERS] Error while copying a large file in pg_rewind

On 7/3/17 09:53, Tom Lane wrote: > Hm. Before we add a bunch of code to deal with that, are we sure we > *want* it to copy such files? Seems like that's expending a lot of > data-transfer work for zero added value --- consider e.g. a server > with a bunch of old core files laying about in

Re: [HACKERS] Revisiting NAMEDATALEN

Yes, for the example given in the Reddit post I would tend to agree. This is one of those issues where for the most part the solution is better naming conventions but for the few instances where this isn't possible it is a right pain. -- View this message in context:

Re: [HACKERS] Revisiting NAMEDATALEN

On 07/03/2017 11:31 AM, Emrul wrote: Hi hackers, This question came up again on Reddit: https://www.reddit.com/r/PostgreSQL/comments/6kyyev/i_have_hit_the_table_name_length_limit_a_number/ and I thought I'd echo it here. I totally am on board with short, descriptive names and a good

[HACKERS] Revisiting NAMEDATALEN

Hi hackers, This question came up again on Reddit: https://www.reddit.com/r/PostgreSQL/comments/6kyyev/i_have_hit_the_table_name_length_limit_a_number/ and I thought I'd echo it here. I totally am on board with short, descriptive names and a good convention. However, there are just so many

Re: [HACKERS] Race conditions with WAL sender PID lookups

Michael Paquier wrote: > Thanks Álvaro for pushing the patch. I had a second look and the > result looks good to me. > > - SpinLockAcquire(>mutex); > + } > + pid = walsnd->pid; > The WAL receiver code used a cast to (int) in > pg_stat_get_wal_receiver(). I don't recall adding

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

On 07/03/2017 09:39 AM, Heikki Linnakangas wrote: > Hmm. That's not the problem, though. Imagine that instead of the loop > above, you do just: > > WALInsertLockUpdateInsertingAt(CurrPos); > AdvanceXLInsertBuffer(EndPos, false); > > AdvanceXLInsertBuffer() will call XLogWrite(), to flush out

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

On 30 June 2017 at 05:14, Amit Kapila wrote: > This is explained in section 15.2 [1], refer below para: > "The query might be suspended during execution. In any situation in > which the system thinks that partial or incremental execution might > occur, no parallel plan

Re: [HACKERS] Dumping database creation options and ACLs

On 06/29/2017 06:30 PM, Adrien Nayrat wrote: > As reported by Ronan there's no other option than using pg_dumpall to restore > database options and ACLs. > > So, we use this trick to stop pg_dumpall before \connect and then use > pg_restore: > > pg_dumpall -s | sed -rn '/^\\connect/{q}; p' >

Re: [HACKERS] Postgres process invoking exit resulting in sh-QUIT core

Hi Craig, Thanks for the response. Scenario tried here is restart of the system multiple times. sh-QUIT core is generated when Postgres is invoking the shell to exit and may not be due to kernel or file system issues. I will try to reproduce the issue with dmesg output being printed.

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

On Fri, Jun 30, 2017 at 9:32 AM, Mark Dilger wrote: >> On Jun 29, 2017, at 8:55 PM, Mark Dilger wrote: >> Changing myfunc to create a temporary table, to execute the sql to populate >> that temporary table, and to then loop through the temporary

Re: [HACKERS] Dumping database creation options and ACLs

On Thu, Jun 29, 2017 at 12:30 PM, Adrien Nayrat wrote: > As reported by Ronan there's no other option than using pg_dumpall to restore > database options and ACLs. > > So, we use this trick to stop pg_dumpall before \connect and then use > pg_restore: > > pg_dumpall -s

Re: [HACKERS] Default Partition for Range

On Fri, Jun 30, 2017 at 11:56 AM, Beena Emerson wrote: > Hello Dilip, > > On Wed, Jun 21, 2017 at 6:27 PM, Dilip Kumar wrote: >> On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar wrote: >>> This is basically crashing in

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

On 07/03/2017 09:39 AM, Heikki Linnakangas wrote: > The most straightforward solution would be to just clear each page with > memset() in the loop. It's a bit wasteful to clear the page again, just > after AdvanceXLInsertBuffer() has initialized it, but this isn't > performance-critical. An in

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

Michael Paquier writes: > On Mon, Jul 3, 2017 at 9:25 PM, Greg Stark wrote: >> On 2 July 2017 at 18:33, Tom Lane wrote: >>> system("cp -a ...") call in favor of something more portable. >> If we're ok with using Perl there's

Re: [HACKERS] Error while copying a large file in pg_rewind

Kuntal Ghosh writes: > pg_rewind throws the following error when there is a file of large > size available in the Slave server's data directory. Hm. Before we add a bunch of code to deal with that, are we sure we *want* it to copy such files? Seems like that's

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

On 06/26/2017 04:20 AM, Chapman Flack wrote: I notice CopyXLogRecordToWAL contains this loop (in the case where the record being copied is a switch): while (CurrPos < EndPos) { /* initialize the next page (if not initialized already) */ WALInsertLockUpdateInsertingAt(CurrPos);

Re: [HACKERS] user-based query white list

2017-07-03 3:11 GMT-03:00 Tim Burgan : > > Since then, is it now possible to configure a user to only be able to > execute a limited white-listing of queries? Is this something that could > now be implemented through extensions? > Since pg_stat_statements infrastructure, it

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

On Mon, Jul 3, 2017 at 10:02 PM, Alvaro Herrera wrote: > Tom Lane wrote: > >> (Other unfinished work: teaching the MSVC scripts to use this, >> and teaching pg_upgrade's test script to use it.) > > Maybe it'd be simpler to rewrite pg_upgrade tests using PostgresNode >

Re: [HACKERS] Error while copying a large file in pg_rewind

On Mon, Jul 3, 2017 at 8:22 PM, Kuntal Ghosh wrote: > pg_rewind throws the following error when there is a file of large > size available in the Slave server's data directory. Oops. > I guess we've to change the data type to bigint. Also, we need some >

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

Tom Lane wrote: > (Other unfinished work: teaching the MSVC scripts to use this, > and teaching pg_upgrade's test script to use it.) Maybe it'd be simpler to rewrite pg_upgrade tests using PostgresNode instead? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

The number of retries and maybe failures should be counted, maybe with some adjustable maximum, as suggested. If we fix the maximum number of attempts the maximum number of failures for one script execution will be bounded above (number_of_transactions_in_script *

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

On Mon, Jul 3, 2017 at 9:25 PM, Greg Stark wrote: > On 2 July 2017 at 18:33, Tom Lane wrote: >> system("cp -a ...") call in favor of something more portable. > > If we're ok with using Perl there's File::Copy::Recursive::dircopy() > which does exactly that.

Re: [HACKERS] Error-like LOG when connecting with SSL for password authentication

On Mon, Jul 3, 2017 at 9:02 PM, Heikki Linnakangas wrote: > Thanks, I've pushed the backend read part of this patch. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Mar 23, 2017 at 1:18 PM, Ashutosh Sharma wrote: > > *Conclusion:* > As seen from the test results mentioned above, there is some performance > improvement with 3 SP(s), with 5 SP(s) the results with patch is slightly > better than HEAD, with 7 and 10 SP(s) we do see

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

On 2 July 2017 at 18:33, Tom Lane wrote: > system("cp -a ...") call in favor of something more portable. If we're ok with using Perl there's File::Copy::Recursive::dircopy() which does exactly that. -- greg -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

The current error handling is either "close connection" or maybe in some cases even "exit". If this is changed, then the client may continue execution in some unforseen state and behave unexpectedly. We'll see. Thanks, now I understand this. ISTM that the retry implementation should be

Re: [HACKERS] Error-like LOG when connecting with SSL for password authentication

On 05/24/2017 05:29 PM, Michael Paquier wrote: Attached is an updated patch. Thanks, I've pushed the backend read part of this patch. That's enough to fix the original complaint with password authentication. I think the rest was a bit dubious, and I'm hesitant to commit that (or at least to

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

Hi, On Mon, Jul 3, 2017 at 4:10 PM, Amit Kapila wrote: > > On Mon, Jul 3, 2017 at 3:24 PM, Ashutosh Sharma wrote: > > On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: > >> The basic issue was that the WAL logging for

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

Hello Marina, I agree that improving the error handling ability of pgbench is a good thing, although I'm not sure about the implications... Could you tell a little bit more exactly.. What implications are you worried about? The current error handling is either "close connection" or maybe

[HACKERS] Error while copying a large file in pg_rewind

Hello all, pg_rewind throws the following error when there is a file of large size available in the Slave server's data directory. unexpected result while sending file list: ERROR: value "214800" is out of range for type integer CONTEXT: COPY fetchchunks, line 2402, column begin:

Re: [BUGS] [HACKERS] Segmentation fault in libpq

On 07/03/2017 04:58 AM, Craig Ringer wrote: On 3 July 2017 at 03:12, Andres Freund wrote: Hi, On 2017-07-02 20:58:52 +0200, Michal Novotný wrote: thank you all for your advice. I've been investigating this a little more and finally it turned out it's not a bug in libpq

Re: [BUGS] [HACKERS] Segmentation fault in libpq

On 07/02/2017 09:12 PM, Andres Freund wrote: Hi, On 2017-07-02 20:58:52 +0200, Michal Novotný wrote: thank you all for your advice. I've been investigating this a little more and finally it turned out it's not a bug in libpq although I got confused by going deep as several libpq functions.

[HACKERS] user-based query white list

This old thread on "user-based query white list" is now nearly 10 years old! http://grokbase.com/t/postgresql/pgsql-hackers/08c6zh42fa/user-based-query-white-list Since then, is it now possible to configure a user to only be able to execute a limited white-listing of queries? Is this something

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

Thanks for working on the previous comments. The code really looks good now. On Fri, Jun 23, 2017 at 2:29 PM, Amit Langote wrote: > >> Don't we need an exclusive lock to >> make sure that the constraints are not changed while we are validating those? > > If I

Re: [HACKERS] [POC] hash partitioning

On Fri, Jun 23, 2017 at 11:19 AM, Yugo Nagata wrote: > On Fri, 23 Jun 2017 13:41:15 +0900 > Yugo Nagata wrote: > >> On Tue, 6 Jun 2017 13:03:58 +0530 >> amul sul wrote: >> >> >> > Updated patch attached. >> >> I looked into the latest

Re: [HACKERS] [POC] hash partitioning

On Fri, Jun 23, 2017 at 10:11 AM, Yugo Nagata wrote: > On Tue, 6 Jun 2017 13:03:58 +0530 > amul sul wrote: > > >> Updated patch attached. > > I looked into the latest patch (v13) and have some comments > althogh they might be trivial. > Thanks for your

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

On Mon, Jul 3, 2017 at 3:24 PM, Ashutosh Sharma wrote: > On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: >> The basic issue was that the WAL logging for Create Index operation >> was oblivion of the fact that for unlogged tables only INIT forks

Re: [HACKERS] Proposal : For Auto-Prewarm.

On Fri, Jun 23, 2017 at 3:22 AM, Robert Haas wrote: > On Thu, Jun 15, 2017 at 12:35 AM, Mithun Cy > wrote: > > * Instead of creating our own buffering system via buffer_file_write() > and buffer_file_flush(), why not just use the facilities

Re: [HACKERS] Proposal : For Auto-Prewarm.

On Sun, Jul 2, 2017 at 10:32 PM, Mithun Cy wrote: > On Tue, Jun 27, 2017 at 11:41 AM, Mithun Cy > wrote: >> On Fri, Jun 23, 2017 at 5:45 AM, Thom Brown wrote: >>> >>> Also, I find it a bit messy that

Re: [HACKERS] UPDATE of partition key

On 2017/07/02 20:10, Robert Haas wrote: > On Fri, Jun 30, 2017 at 4:20 PM, Robert Haas wrote: >> I don't think the approach of building a hash table to figure out >> which result rels have already been created is a good one. That too >> feels like something that the

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

Hi, On Mon, Jul 3, 2017 at 9:12 AM, Amit Kapila wrote: > While discussing the behavior of hash indexes with Bruce in the nearby > thread [1], it has been noticed that hash index on unlogged tables > doesn't behave as expected. Prior to 10, it has the different set of >

Re: [HACKERS] Multi column range partition table

Hi Dean, On 2017/07/03 17:36, Dean Rasheed wrote: > On 3 July 2017 at 06:00, Amit Langote wrote: >> On 2017/07/03 2:15, Dean Rasheed wrote: >>> My first thought was UNBOUNDED ABOVE/BELOW, because that matches the >>> terminology already in use of upper and lower

Re: [HACKERS] asynchronous execution

Hi, I've returned. At Thu, 29 Jun 2017 14:08:27 +0900, Amit Langote wrote in <63a5a01c-2967-83e0-8bbf-c981404f5...@lab.ntt.co.jp> > Hi, > > On 2017/06/29 13:45, Kyotaro HORIGUCHI wrote: > > Thank you for looking this. > > > > At Wed, 28 Jun 2017 10:23:54 +0200,

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

Hello Marina, Hello, Fabien! A few comments about the submitted patches. Thank you very much for them! I agree that improving the error handling ability of pgbench is a good thing, although I'm not sure about the implications... Could you tell a little bit more exactly.. What

Re: [HACKERS] Multi column range partition table

On Mon, Jul 3, 2017 at 2:06 PM, Dean Rasheed wrote: > On 3 July 2017 at 06:00, Amit Langote wrote: >> On 2017/07/03 2:15, Dean Rasheed wrote: >>> My first thought was UNBOUNDED ABOVE/BELOW, because that matches the >>> terminology already

Re: [HACKERS] Multi column range partition table

On 3 July 2017 at 06:00, Amit Langote wrote: > On 2017/07/03 2:15, Dean Rasheed wrote: >> My first thought was UNBOUNDED ABOVE/BELOW, because that matches the >> terminology already in use of upper and lower bounds. > > I was starting to like the Ashutosh's

Re: [HACKERS] Macros bundling RELKIND_* conditions

Forgot to attach the patch with the earlier mail. On Mon, Jul 3, 2017 at 1:22 PM, Ashutosh Bapat wrote: > On Mon, May 29, 2017 at 12:55 PM, Ashutosh Bapat > wrote: >> >> -- >> I noticed, that >> after we introduced

Re: [HACKERS] Macros bundling RELKIND_* conditions

On Mon, May 29, 2017 at 12:55 PM, Ashutosh Bapat wrote: > > -- > I noticed, that > after we introduced RELKIND_PARTITIONED_TABLE, we required to change a > number of conditions to include this relkind. We missed some places in > initial commits and fixed those

Re: [HACKERS] Proposal : For Auto-Prewarm.

On Sun, Jul 2, 2017 at 10:32 PM, Mithun Cy wrote: > On Tue, Jun 27, 2017 at 11:41 AM, Mithun Cy > wrote: >> On Fri, Jun 23, 2017 at 5:45 AM, Thom Brown wrote: >>> >>> Also, I find it a bit messy that

Re: [HACKERS] Multi column range partition table

On 2017/07/03 14:00, Amit Langote wrote: > On 2017/07/03 2:15, Dean Rasheed wrote: >> On 30 June 2017 at 10:04, Ashutosh Bapat >> wrote: >>> On Fri, Jun 30, 2017 at 1:36 PM, Amit Langote >>> wrote: Alright, I spent some