Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-11 Thread Sokolov Yura
В Thu, 10 Aug 2017 18:12:34 +0300 Alexander Korotkov пишет: > On Thu, Aug 10, 2017 at 3:30 PM, Sokolov Yura > wrote: > > > On 2017-07-31 18:56, Sokolov Yura wrote: > > > >> Good day, every one. > >> > >> In attempt to improve

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-11 Thread Robert Haas
On Fri, Aug 11, 2017 at 5:36 AM, Rushabh Lathia wrote: > Please find attach patch with the changes. I found the way that you had the logic structured in flagInhTables() to be quite hard to follow, so I rewrote it in the attached version. This version also has a bunch of

Re: [HACKERS] Arrays of domains

2017-08-11 Thread Tom Lane
I wrote: > Probably a better answer is to start supporting arrays over domain > types. That was left unimplemented in the original domains patch, > but AFAICS not for any better reason than lack of round tuits. Attached is a patch series that allows us to create arrays of domain types. I

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-11 Thread Robert Haas
On Thu, Aug 10, 2017 at 11:12 AM, Alexander Korotkov wrote: > These results look very cool! > I think CSN is eventually inevitable, but it's a long distance feature. > Thus, this optimization could make us a good serve before we would have CSN. > Do you expect there are

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-11 Thread Rushabh Lathia
On Thu, Aug 10, 2017 at 8:26 PM, Robert Haas wrote: > On Thu, Aug 10, 2017 at 3:47 AM, Rushabh Lathia > wrote: > >> (1) seems like a pretty arbitrary restriction, so I don't like that > >> option. (2) would hurt performance in some use cases.

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Álvaro Hernández Tortosa
On 11/08/17 03:57, Peter Eisentraut wrote: The SCRAM protocol documentation (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) states "To avoid confusion, the client should use pg_same_as_startup_message as the username in the client-first-message." However, the client

Re: [HACKERS] Foreign tables privileges not shown in information_schema.table_privileges

2017-08-11 Thread Ashutosh Bapat
On Thu, Aug 10, 2017 at 6:30 PM, Nicolas Thauvin wrote: > Hello, > > The information_schema.table_privileges view filters on regular tables > and views. Foreign tables are not shown in this view but they are in > other views of the information_schema like tables or

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-11 Thread Thomas Munro
Hi, Please find attached a new patch series. I apologise in advance for 0001 and note that the patchset now weighs in at ~75kB compressed. Here are my in-line replies to your two reviews: On Tue, Jul 25, 2017 at 10:09 PM, Andres Freund wrote: > It does concern me that we're

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-11 Thread Augustine, Jobin
Appears that patch is not helping. Errors like below are still appearing in the log === 2017-08-11 12:22:35 UTC [2840]: [1-1] user=,db=,app=,client= FATAL: could not connect to the primary server: could not send data to server: Socket is not connected

Re: [HACKERS] Funny WAL corruption issue

2017-08-11 Thread Chris Travers
On Fri, Aug 11, 2017 at 1:33 PM, Greg Stark wrote: > On 10 August 2017 at 15:26, Chris Travers wrote: > > > > > > The bitwise comparison is interesting. Remember the error was: > > > > pg_xlogdump: FATAL: error in WAL record at 1E39C/E1117FB8:

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Michael Paquier
On Fri, Aug 11, 2017 at 9:31 PM, Álvaro Hernández Tortosa wrote: > On 11/08/17 13:18, Michael Paquier wrote: >> On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa >> wrote: Relatedly, the SCRAM specification doesn't appear to allow omitting the

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Álvaro Hernández Tortosa
On 11/08/17 15:00, Michael Paquier wrote: On Fri, Aug 11, 2017 at 9:31 PM, Álvaro Hernández Tortosa wrote: On 11/08/17 13:18, Michael Paquier wrote: On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa wrote: Relatedly, the SCRAM specification

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-11 Thread Ashutosh Sharma
> > Comments on the latest patch: Thank you once again for reviewing my patches. > > 1. > /* > + * We remember prev and next block number along with current block > + * number so that if fetching the tuples using cursor we know the > + * page from where to begin. This is for the case where we

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Peter Eisentraut
On 8/11/17 07:18, Michael Paquier wrote: > The problem is where a username includes characters as a comma or '=', > which can be avoided if the string is in UTF-8 as the username is > prepared with SASLprep before being used in the SASL exchange, but we > have no way now to be sure now that the

Re: [HACKERS] Thoughts on unit testing?

2017-08-11 Thread Peter Eisentraut
On 8/10/17 17:53, Thomas Munro wrote: > The current regression tests, isolation tests and TAP tests are very > good (though I admit my experience with TAP is limited), but IMHO we > are lacking support for C-level unit testing. Complicated, fiddly > things with many states, interactions, edge

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Peter Eisentraut
On 8/11/17 09:06, Álvaro Hernández Tortosa wrote: > Strictly speaking the RFC assumes that the username is at least 1 > character. I understand this was precisely Peter's original comment. Well, my main point was that the documentation, the code, and the code comments all say slightly

Re: [HACKERS] Funny WAL corruption issue

2017-08-11 Thread Chris Travers
On Fri, Aug 11, 2017 at 1:33 PM, Greg Stark wrote: > On 10 August 2017 at 15:26, Chris Travers wrote: > > > > > > The bitwise comparison is interesting. Remember the error was: > > > > pg_xlogdump: FATAL: error in WAL record at 1E39C/E1117FB8:

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Michael Paquier
On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa wrote: > On 11/08/17 03:57, Peter Eisentraut wrote: >> The SCRAM protocol documentation >> (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) >> states >> >> "To avoid confusion, the client should use

Re: [HACKERS] Funny WAL corruption issue

2017-08-11 Thread Greg Stark
On 10 August 2017 at 15:26, Chris Travers wrote: > > > The bitwise comparison is interesting. Remember the error was: > > pg_xlogdump: FATAL: error in WAL record at 1E39C/E1117FB8: unexpected > pageaddr 1E375/61118000 in log segment 0001E39C00E1, offset >

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Álvaro Hernández Tortosa
On 11/08/17 13:18, Michael Paquier wrote: On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa wrote: On 11/08/17 03:57, Peter Eisentraut wrote: The SCRAM protocol documentation (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) states "To avoid

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-08-11 Thread Andres Freund
Hi, On 2017-07-21 19:32:02 +0300, Marina Polyakova wrote: > Here is the third version of the patch for pgbench thanks to Fabien Coelho > comments. As in the previous one, transactions with serialization and > deadlock failures are rolled back and retried until they end successfully or > their

[HACKERS] additional contrib test suites

2017-08-11 Thread Peter Eisentraut
Here are some small test suites for some contrib modules as well as pg_archivecleanup that didn't have one previously, as well as one patch to improve code coverage in a module. Will add to commit fest. Testing on different platforms and with different build configurations would be useful. --

[HACKERS] pg_stat_statements query normalization, and the 'in' operator

2017-08-11 Thread unixway . drive
Hello there, Given the following list of queries: create table foo (id serial, bar integer); select * from foo where id in (1); select * from foo where id in (2,3); select * from foo where id in (1,3,5); select * from foo where id in (select id from foo); would it be possible to have

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-11 Thread Andres Freund
Hi, On 2017-08-11 20:39:13 +1200, Thomas Munro wrote: > Please find attached a new patch series. I apologise in advance for > 0001 and note that the patchset now weighs in at ~75kB compressed. > Here are my in-line replies to your two reviews: Replying to a few points here, then I'll do a pass

Re: [HACKERS] WIP: Failover Slots

2017-08-11 Thread Andres Freund
On 2017-08-02 16:35:17 -0400, Robert Haas wrote: > I actually think failover slots are quite desirable, especially now > that we've got logical replication in core. In a review of this > thread I don't see anyone saying otherwise. The debate has really > been about the right way of implementing

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-11 Thread Andres Freund
On 2017-08-11 11:14:44 -0400, Robert Haas wrote: > On Fri, Aug 11, 2017 at 4:39 AM, Thomas Munro > wrote: > > OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic > > shared". Better? If we were to do other data structures in DSA > > memory they could

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-11 Thread Noah Misch
On Thu, Aug 10, 2017 at 09:59:40PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: > >> I don't think I can usefully contribute to this. Could someone else > >> take it? This PostgreSQL 10 open item is past due

Re: [HACKERS] Pluggable storage

2017-08-11 Thread Haribabu Kommi
On Tue, Aug 8, 2017 at 2:21 PM, Amit Kapila wrote: > On Tue, Jun 13, 2017 at 7:20 AM, Haribabu Kommi > wrote: > > > > > > On Fri, Oct 14, 2016 at 7:26 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > >> > >> I have sent the partial

Re: [HACKERS] Pluggable storage

2017-08-11 Thread Haribabu Kommi
On Mon, Aug 7, 2017 at 11:12 PM, Amit Kapila wrote: > On Tue, Aug 1, 2017 at 1:56 PM, Haribabu Kommi > wrote: > > > > > > On Sun, Jul 23, 2017 at 4:10 PM, Amit Kapila > > wrote: > >> > > > >> > >> > 1. Design an API

Re: [HACKERS] Thoughts on unit testing?

2017-08-11 Thread Tom Lane
Peter Eisentraut writes: > On 8/10/17 17:53, Thomas Munro wrote: >> The current regression tests, isolation tests and TAP tests are very >> good (though I admit my experience with TAP is limited), but IMHO we >> are lacking support for C-level unit testing.

Re: [HACKERS] Parallel COPY FROM execution

2017-08-11 Thread Robert Haas
On Fri, Aug 11, 2017 at 9:55 AM, Alex K wrote: > - I have used both Latch and ConditionalVariable for the same > purpose–wait until some signal > occurs–and for me as an end user they perform quite similar. I > looked into the condition_variable.c > code and it

[HACKERS] Patches I'm thinking of pushing shortly

2017-08-11 Thread Tom Lane
I have some patches sitting around in my workspace that I think are non-controversial, and so I was considering just pushing them once the tree opens for v11 development. If anyone thinks they need further review, I'll put them into the September commitfest, but otherwise we might as well skip

Re: [HACKERS] scan on inheritance parent with no children in current session

2017-08-11 Thread Robert Haas
On Sun, Aug 6, 2017 at 10:56 PM, Amit Langote wrote: > Good catch. I agree that getting an Append node after that commit is > unintentional and we should fix so that we don't get an Append. So, +1 to > your patch. I looked at the patch and the code fix seems to

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-11 Thread Robert Haas
On Fri, Aug 11, 2017 at 11:24 AM, Tom Lane wrote: > 3. remove-pgbench-option-ordering-constraint.patch from > https://www.postgresql.org/message-id/20559.1501703...@sss.pgh.pa.us > > That one was already informally reviewed by two people, so I don't > think it needs another

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

2017-08-11 Thread Robert Haas
On Thu, Aug 10, 2017 at 6:21 PM, Mark Dilger wrote: > That misses the point I was making. I was suggesting a syntax where > the caller promises to use all rows without stopping short, and the > database performance problems of having a bunch of parallel workers >

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-11 Thread Tom Lane
Robert Haas writes: > On Fri, Aug 11, 2017 at 11:24 AM, Tom Lane wrote: >> 3. remove-pgbench-option-ordering-constraint.patch from >> https://www.postgresql.org/message-id/20559.1501703...@sss.pgh.pa.us >> >> That one was already informally reviewed by

Re: [HACKERS] Parallel COPY FROM execution

2017-08-11 Thread Alex K
Greetings pgsql-hackers, I have completed the general infrastructure for parallel COPY FROM execution, consisting of Main (master) process and multiple BGWorkers connected with master using a personal message query (shm_mq). Master process does: - Dynamic shared memory allocation with parallel

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-11 Thread Robert Haas
On Fri, Aug 11, 2017 at 4:39 AM, Thomas Munro wrote: > OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic > shared". Better? If we were to do other data structures in DSA > memory they could follow that style: ds_red_black_tree.c, ds_vector.c, >