[COMMITTERS] pgsql: Add a C API for parallel heap scans.

2015-10-16 Thread Robert Haas
Add a C API for parallel heap scans. Using this API, one backend can set up a ParallelHeapScanDesc to which multiple backends can then attach. Each tuple in the relation will be returned to exactly one of the scanning backends. Only forward scans are supported, and rescans must be carefully coor

[COMMITTERS] pgsql: Allow a parallel context to relaunch workers.

2015-10-16 Thread Robert Haas
Allow a parallel context to relaunch workers. This may allow some callers to avoid the overhead involved in tearing down a parallel context and then setting up a new one, which means releasing the DSM and then allocating and populating a new one. I suspect we'll want to revise the Gather node to

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Improve performance of pullback/pushfwd in regular-expression co

2015-10-16 Thread Tom Lane
Improve performance of pullback/pushfwd in regular-expression compiler. The previous coding would create a new intermediate state every time it wanted to interchange the ordering of two constraint arcs. Certain regex features such as \Y can generate large numbers of parallel constraint arcs, and

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Miscellaneous cleanup of regular-expression compiler.

2015-10-16 Thread Tom Lane
Miscellaneous cleanup of regular-expression compiler. Revert our previous addition of "all" flags to copyins() and copyouts(); they're no longer needed, and were never anything but an unsightly hack. Improve a couple of infelicities in the REG_DEBUG code for dumping the NFA data structure, includ

[COMMITTERS] pgsql: Fix regular-expression compiler to handle loops of constraint ar

2015-10-16 Thread Tom Lane
Fix regular-expression compiler to handle loops of constraint arcs. It's possible to construct regular expressions that contain loops of constraint arcs (that is, ^ $ AHEAD BEHIND or LACON arcs). There's no use in fully traversing such a loop at execution, since you'd just end up in the same NFA

[COMMITTERS] pgsql: Improve memory-usage accounting in regular-expression compiler.

2015-10-16 Thread Tom Lane
Improve memory-usage accounting in regular-expression compiler. This code previously counted the number of NFA states it created, and complained if a limit was exceeded, so as to prevent bizarre regex patterns from consuming unreasonable time or memory. That's fine as far as it went, but the code

[COMMITTERS] pgsql: Improve performance of fixempties() pass in regular-expression c

2015-10-16 Thread Tom Lane
Improve performance of fixempties() pass in regular-expression compiler. The previous coding took something like O(N^4) time to fully process a chain of N EMPTY arcs. We can't really do much better than O(N^2) because we have to insert about that many arcs, but we can do lots better than what's t

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Fix O(N^2) performance problems in regular-expression compiler.

2015-10-16 Thread Tom Lane
Fix O(N^2) performance problems in regular-expression compiler. Change the singly-linked in-arc and out-arc lists to be doubly-linked, so that arc deletion is constant time rather than having worst-case time proportional to the number of other arcs on the connected states. Modify the bulk arc tra

[COMMITTERS] pgsql: Remove volatile qualifiers from proc.c and procarray.c

2015-10-16 Thread Robert Haas
Remove volatile qualifiers from proc.c and procarray.c Prior to commit 0709b7ee72e4bc71ad07b7120acd117265ab51d0, access to variables within a spinlock-protected critical section had to be done through a volatile pointer, but that should no longer be necessary. Michael Paquier Branch -- maste

[COMMITTERS] pgsql: Remove volatile qualifiers from dynahash.c, shmem.c, and sinvala

2015-10-16 Thread Robert Haas
Remove volatile qualifiers from dynahash.c, shmem.c, and sinvaladt.c Prior to commit 0709b7ee72e4bc71ad07b7120acd117265ab51d0, access to variables within a spinlock-protected critical section had to be done through a volatile pointer, but that should no longer be necessary. Thomas Munro Branch -

[COMMITTERS] pgsql: Remove cautions about using volatile from spin.h.

2015-10-16 Thread Robert Haas
Remove cautions about using volatile from spin.h. Commit 0709b7ee72e4bc71ad07b7120acd117265ab51d0 obsoleted this comment but neglected to update it. Thomas Munro Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/78652a3332128c89ae369e03698f7212ca73d022 Modified Files

[COMMITTERS] pgsql: Remove cautions about using volatile from spin.h.

2015-10-16 Thread Robert Haas
Remove cautions about using volatile from spin.h. Commit 0709b7ee72e4bc71ad07b7120acd117265ab51d0 obsoleted this comment but neglected to update it. Thomas Munro Branch -- REL9_5_STABLE Details --- http://git.postgresql.org/pg/commitdiff/22884414cbac345c9143738634123f76e61ca343 Modifie

[COMMITTERS] pgsql: Prohibit parallel query when the isolation level is serializable

2015-10-16 Thread Robert Haas
Prohibit parallel query when the isolation level is serializable. In order for this to be safe, the code which hands true serializability will need to taught that the SIRead locks taken by a parallel worker pertain to the same transaction as those taken by the parallel leader. Some further changes

[COMMITTERS] pgsql: Rewrite interaction of parallel mode with parallel executor supp

2015-10-16 Thread Robert Haas
Rewrite interaction of parallel mode with parallel executor support. In the previous coding, before returning from ExecutorRun, we'd shut down all parallel workers. This was dead wrong if ExecutorRun was called with a non-zero tuple count; it had the effect of truncating the query output. To fix

[COMMITTERS] pgsql: Mark more functions parallel-restricted or parallel-unsafe.

2015-10-16 Thread Robert Haas
Mark more functions parallel-restricted or parallel-unsafe. Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b was overoptimistic about the degree of safety associated with running various functions in parallel mode. Functions that take a table name or OID as an argument are at least parallel-restri

[COMMITTERS] pgsql: Fix a problem with parallel workers being unable to restore role

2015-10-16 Thread Robert Haas
Fix a problem with parallel workers being unable to restore role. check_role() tries to verify that the user has permission to become the requested role, but this is inappropriate in a parallel worker, which needs to exactly recreate the master's authorization settings. So skip the check in that

[COMMITTERS] pgsql: Fix a problem with parallel workers being unable to restore role

2015-10-16 Thread Robert Haas
Fix a problem with parallel workers being unable to restore role. check_role() tries to verify that the user has permission to become the requested role, but this is inappropriate in a parallel worker, which needs to exactly recreate the master's authorization settings. So skip the check in that

[COMMITTERS] pgsql: Invalidate caches after cranking up a parallel worker transactio

2015-10-16 Thread Robert Haas
Invalidate caches after cranking up a parallel worker transaction. Starting a parallel worker transaction changes our notion of which XIDs are in-progress or committed, and our notion of the current command counter ID. Therefore, our view of these caches prior to starting this transaction may no

[COMMITTERS] pgsql: Invalidate caches after cranking up a parallel worker transactio

2015-10-16 Thread Robert Haas
Invalidate caches after cranking up a parallel worker transaction. Starting a parallel worker transaction changes our notion of which XIDs are in-progress or committed, and our notion of the current command counter ID. Therefore, our view of these caches prior to starting this transaction may no

[COMMITTERS] pgsql: Fix order of arguments in ecpg generated typedef command.

2015-10-16 Thread Michael Meskes
Fix order of arguments in ecpg generated typedef command. Branch -- master Details --- http://git.postgresql.org/pg/commitdiff/d07fea542f18f3d4bc5413a5fc23d69b9fcb631c Modified Files -- src/interfaces/ecpg/preproc/ecpg.trailer |2 +- 1 file changed, 1 insertion(+), 1 delet

[COMMITTERS] pgsql: Tighten up application of parallel mode checks.

2015-10-16 Thread Robert Haas
Tighten up application of parallel mode checks. Commit 924bcf4f16d54c55310b28f77686608684734f42 failed to enforce parallel mode checks during the commit of a parallel worker, because we exited parallel mode prior to ending the transaction so that we could pop the active snapshot. Re-establish par

[COMMITTERS] pgsql: Tighten up application of parallel mode checks.

2015-10-16 Thread Robert Haas
Tighten up application of parallel mode checks. Commit 924bcf4f16d54c55310b28f77686608684734f42 failed to enforce parallel mode checks during the commit of a parallel worker, because we exited parallel mode prior to ending the transaction so that we could pop the active snapshot. Re-establish par

[COMMITTERS] pgsql: Transfer current command counter ID to parallel workers.

2015-10-16 Thread Robert Haas
Transfer current command counter ID to parallel workers. Commit 924bcf4f16d54c55310b28f77686608684734f42 correctly forbade parallel workers to modify the command counter while in parallel mode, but it inexplicably neglected to actually transfer the current command counter from leader to workers.

[COMMITTERS] pgsql: Transfer current command counter ID to parallel workers.

2015-10-16 Thread Robert Haas
Transfer current command counter ID to parallel workers. Commit 924bcf4f16d54c55310b28f77686608684734f42 correctly forbade parallel workers to modify the command counter while in parallel mode, but it inexplicably neglected to actually transfer the current command counter from leader to workers.

Re: [COMMITTERS] pgsql: Don't send protocol messages to a shm_mq that no longer exists.

2015-10-16 Thread Robert Haas
On Fri, Oct 16, 2015 at 9:50 AM, Robert Haas wrote: > Don't send protocol messages to a shm_mq that no longer exists. > > Commit 2bd9e412f92bc6a68f3e8bcb18e04955cc35001d introduced a mechanism > for relaying protocol messages from a background worker to another > backend via a shm_mq. However, th

[COMMITTERS] pgsql: Don't send protocol messages to a shm_mq that no longer exists.

2015-10-16 Thread Robert Haas
Don't send protocol messages to a shm_mq that no longer exists. Commit 2bd9e412f92bc6a68f3e8bcb18e04955cc35001d introduced a mechanism for relaying protocol messages from a background worker to another backend via a shm_mq. However, there was no provision for shutting down the communication chann

[COMMITTERS] pgsql: Don't send protocol messages to a shm_mq that no longer exists.

2015-10-16 Thread Robert Haas
Don't send protocol messages to a shm_mq that no longer exists. Commit 2bd9e412f92bc6a68f3e8bcb18e04955cc35001d introduced a mechanism for relaying protocol messages from a background worker to another backend via a shm_mq. However, there was no provision for shutting down the communication chann