Re: [HACKERS] Thoughts on unit testing?

2017-08-13 Thread Peter Geoghegan
On Thu, Aug 10, 2017 at 2:53 PM, Thomas Munro wrote: > The current regression tests, isolation tests and TAP tests are very > good (though I admit my experience with TAP is limited), but IMHO we > are lacking support for C-level unit testing. Complicated, fiddly >

Re: [HACKERS] Pluggable storage

2017-08-13 Thread Haribabu Kommi
On Sun, Aug 13, 2017 at 5:21 PM, Amit Kapila wrote: > On Sat, Aug 12, 2017 at 10:34 AM, Haribabu Kommi > wrote: > > > > On Tue, Aug 8, 2017 at 2:21 PM, Amit Kapila > wrote: > >> > >> +typedef struct StorageAmRoutine >

Re: [HACKERS] initdb failure on Debian sid/mips64el in EventTriggerEndCompleteQuery

2017-08-13 Thread Tom Lane
Christoph Berg writes: > 10beta3 and 9.6.4 are both failing during initdb on mips64el on > Debian/sid (unstable): > All other architectures have succeeded, as well as the 9.6.4 build for > Debian/stretch (stable) on mips64el. The difference might be the > compiler version (6.3.0

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-08-13 Thread Fabien COELHO
Hello Alik, Now “a” does not have upper bound, that’s why on using iterative algorithm with a >= 1 program will stuck on infinite loop because of following line of code: double b = pow(2.0, s - 1.0); Because after overflow “b” becomes “+Inf”. Yep, overflow can happen. So should upper

Re: [HACKERS] initdb failure on Debian sid/mips64el in EventTriggerEndCompleteQuery

2017-08-13 Thread Tom Lane
Christoph Berg writes: > Seems to be a gcc-7 problem affecting several packages on mips64el: > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=871514 Hm, unless there is a use of sigsetjmp earlier in that clamav routine, I would not assume that that's the same issue. The bug

[HACKERS] initdb failure on Debian sid/mips64el in EventTriggerEndCompleteQuery

2017-08-13 Thread Christoph Berg
10beta3 and 9.6.4 are both failing during initdb on mips64el on Debian/sid (unstable): https://buildd.debian.org/status/fetch.php?pkg=postgresql-9.6=mips64el=9.6.4-1=1502374949=0 https://buildd.debian.org/status/fetch.php?pkg=postgresql-10=mips64el=10%7Ebeta3-1=1502535836=0 All other

Re: [HACKERS] parallelize queries containing initplans

2017-08-13 Thread Haribabu Kommi
On Fri, Aug 11, 2017 at 1:18 AM, Amit Kapila wrote: > On Wed, Aug 9, 2017 at 6:51 PM, Haribabu Kommi > wrote: > > > > > > + if (IsA(plan, Gather)) > > + ((Gather *) plan)->initParam = bms_intersect(plan->lefttree->extParam, > > initSetParam); >

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-13 Thread Andreas Seltenreich
Tom Lane writes: > I wonder if Andreas would be interested in trying the randomly-timed- > SIGTERM thing with sqlsmith. So far, most of the core dumps generated are Jeevan's assertion failing with backtraces through SearchCatCacheList. The rest is failing this assertion: TRAP:

Re: [HACKERS] pgbench - allow to store select results into variables

2017-08-13 Thread Fabien COELHO
Here is a v11. It is basically a simple rebase after Tom committed the "pgbench -M order" patch. It interfered because the compound command management also needs to delay part of the SQL command initialization. Some patch are luckier than others:-) Here is a v10: - does not talk about

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-13 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 8, 2017 at 10:48 PM, Tom Lane wrote: >> Robert Haas writes: >>> On Tue, Aug 8, 2017 at 4:34 PM, Tom Lane wrote: In the meantime, I think my vote would be to remove

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-08-13 Thread Alik Khilazhev
Hello Fabien, > > I think that this method should be used for a>1, and the other very rough one > can be kept for parameter a in [0, 1), a case which does not make much sense > to a mathematician as it diverges if unbounded. Now “a” does not have upper bound, that’s why on using iterative

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-13 Thread Tom Lane
Andreas Seltenreich writes: > Tom Lane writes: >> I wonder if Andreas would be interested in trying the randomly-timed- >> SIGTERM thing with sqlsmith. > So far, most of the core dumps generated are Jeevan's assertion failing > with backtraces through SearchCatCacheList. The

Re: [HACKERS] Pluggable storage

2017-08-13 Thread Amit Kapila
On Sat, Aug 12, 2017 at 10:31 AM, Haribabu Kommi wrote: >> >> Why do we need to store handler function in TupleDesc? As of now, the >> above patch series has it available in RelationData and >> TupleTableSlot, I am not sure if instead of that keeping it in >> TupleDesc

Re: [HACKERS] Pluggable storage

2017-08-13 Thread Amit Kapila
On Sat, Aug 12, 2017 at 10:34 AM, Haribabu Kommi wrote: > > On Tue, Aug 8, 2017 at 2:21 PM, Amit Kapila wrote: >> >> On Tue, Jun 13, 2017 at 7:20 AM, Haribabu Kommi >> wrote: >> > >> > >> > On Fri, Oct 14, 2016 at 7:26

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Andres Freund
On 2017-08-13 16:55:33 -0400, Tom Lane wrote: > Peter Geoghegan writes: > > I think that it's useful for these things to be handled in an > > adversarial manner, in the same way that litigation is adversarial in > > a common law court. I doubt that Noah actually set out to

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Tom Lane
Andres Freund writes: > As a measure of last restart we could add a libpq workaround that forces > a pqSocketCheck() at the right moment, while still establishing a > connection. That's not good from an interruptability perspective, but > better than blocking for the entire

Re: [HACKERS] Comment in snapbuild.c file

2017-08-13 Thread Masahiko Sawada
On Sun, Aug 13, 2017 at 12:28 PM, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> Masahiko Sawada wrote: >> > Hi all, >> > >> > In snapbuild.c file, there is a comment as follows. >> > >> >* NB: Because of that xmax can be lower than xmin, because we only >> >*

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Peter Geoghegan
On Sun, Aug 13, 2017 at 12:57 PM, Andres Freund wrote: > FWIW, I'm personally quite demotivated by this style of handling > issues. You're essentially saying that any code change, even if it just > increases exposure of a preexisting bug, needs to be handled by the > committer

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-13 Thread Robert Haas
On Sun, Aug 13, 2017 at 3:05 PM, Tom Lane wrote: > Not having heard anyone arguing against that, I'll go make it so, > ie AtEOXact_CatCache is toast in all branches. Great, thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-13 Thread Andres Freund
On 2017-08-13 17:43:10 -0400, Robert Haas wrote: > On Sun, Aug 13, 2017 at 5:24 PM, Tom Lane wrote: > >> I'd vote for including this in v10. There doesn't seem to be any > >> downside to this: it's a no brainer to avoid our exploding hash table > >> case when we can see it

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-13 Thread Amit Kapila
On Mon, Aug 14, 2017 at 6:10 AM, AP wrote: > On Fri, Aug 11, 2017 at 07:33:51AM +0530, Amit Kapila wrote: >> On Thu, Aug 10, 2017 at 4:11 PM, AP wrote: >> > mdstash=# select * from pgstathashindex('link_datum_id_idx'); >> > version | bucket_pages |

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-13 Thread Robert Haas
On Sun, Aug 13, 2017 at 5:24 PM, Tom Lane wrote: >> I'd vote for including this in v10. There doesn't seem to be any >> downside to this: it's a no brainer to avoid our exploding hash table >> case when we can see it coming. > > Anybody else want to vote that way? For myself

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Peter Geoghegan
On Sun, Aug 13, 2017 at 2:22 PM, Andres Freund wrote: > On 2017-08-13 16:55:33 -0400, Tom Lane wrote: >> Peter Geoghegan writes: >> > I think that it's useful for these things to be handled in an >> > adversarial manner, in the same way that litigation is

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Andres Freund
Hi, On 2017-08-11 18:11:03 +0530, Augustine, Jobin wrote: > Appears that patch is not helping. > Errors like below are still appearing in the log > === > 2017-08-11 12:22:35 UTC [2840]: [1-1] user=,db=,app=,client= FATAL: could > not connect to the primary server:

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-13 Thread Andres Freund
Hi, On 2017-08-11 20:39:13 +1200, Thomas Munro wrote: > Please find attached a new patch series. Review for 0001: I think you made a few long lines even longer, like: @@ -1106,11 +1106,11 @@ pltcl_trigger_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,

[HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-13 Thread Andres Freund
Hi, Since we're getting a bit into the weeds of a different topic, and since I think it's an interesting feature, I'm detaching this into a separate thread. On 2017-08-12 23:37:27 -0400, Tom Lane wrote: > >> On 2017-08-12 22:52:57 -0400, Robert Haas wrote: > >>> I think it'd be pretty

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Noah Misch
On Fri, Aug 11, 2017 at 08:56:22PM -0700, Noah Misch wrote: > On Thu, Aug 10, 2017 at 09:59:40PM -0400, Tom Lane wrote: > > Noah Misch writes: > > > On Tue, Aug 08, 2017 at 07:25:37PM -0400, Peter Eisentraut wrote: > > >> I don't think I can usefully contribute to this. Could

Re: [HACKERS] pgbench - allow to store select results into variables

2017-08-13 Thread Pavel Stehule
Hi 2017-08-13 20:33 GMT+02:00 Fabien COELHO : > > Here is a v11. > > It is basically a simple rebase after Tom committed the "pgbench -M order" > patch. It interfered because the compound command management also needs > to delay part of the SQL command initialization. Some

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Andres Freund
On 2017-08-11 20:56:22 -0700, Noah Misch wrote: > > > If nobody volunteers, you could always resolve this by reverting 1e8a850 > > > and > > > successors. > > > > I think you're blaming the victim. Our current theory about the cause > > of this is that on Windows, WaitLatchOrSocket cannot be

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-13 Thread Tom Lane
Thomas Munro writes: > On Sat, Aug 12, 2017 at 3:24 AM, Tom Lane wrote: >> 1. check-hash-bucket-size-against-work_mem-2.patch from >> https://www.postgresql.org/message-id/13698.1487283...@sss.pgh.pa.us > +1 > I'd vote for including this in

Re: [HACKERS] shared memory based stat collector (was: Sharing record typmods between backends)

2017-08-13 Thread Tom Lane
Andres Freund writes: > You both are obviously right. Another point of potential concern could > be that we'd pretyt much fully rely on dsm/dht's being available, for > the server to function correctly. Are we ok with that? Right now > postgres still works perfectly well,

[HACKERS] Re: ICU collation variant keywords and pg_collation entries (Was: [BUGS] Crash report for some ICU-52 (debian8) COLLATE and work_mem values)

2017-08-13 Thread Noah Misch
On Thu, Aug 10, 2017 at 04:51:16AM +, Noah Misch wrote: > On Mon, Aug 07, 2017 at 06:23:56PM -0400, Tom Lane wrote: > > Peter Eisentraut writes: > > > On 8/6/17 20:07, Peter Geoghegan wrote: > > >> I've looked into this. I'll give an example of what keyword

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Tom Lane
Peter Geoghegan writes: > I think that it's useful for these things to be handled in an > adversarial manner, in the same way that litigation is adversarial in > a common law court. I doubt that Noah actually set out to demoralize > anyone. He is just doing the job he was assigned.

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Tom Lane
"Augustine, Jobin" writes: > Appears that patch is not helping. > Errors like below are still appearing in the log > === > 2017-08-11 12:22:35 UTC [2840]: [1-1] user=,db=,app=,client= FATAL: could > not connect to the primary server: could

Re: [HACKERS] Comment typo in plannodes.h

2017-08-13 Thread Etsuro Fujita
On 2017/08/11 2:18, Robert Haas wrote: On Thu, Aug 10, 2017 at 8:25 AM, Etsuro Fujita wrote: Here is a small patch for fixing a comment typo in plannodes.h: s/all the partitioned table/all the partitioned tables/. Committed. Thank you. Best regards, Etsuro

Re: [HACKERS] Thoughts on unit testing?

2017-08-13 Thread Craig Ringer
On 14 August 2017 at 03:19, Peter Geoghegan wrote: > On Thu, Aug 10, 2017 at 2:53 PM, Thomas Munro > wrote: > > The current regression tests, isolation tests and TAP tests are very > > good (though I admit my experience with TAP is limited), but IMHO

[HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-08-13 Thread Michael Paquier
Hi all, Recent commit 8d98819 has added a missing permissoin check to lo_put() to make sure that the write permissions of the object are properly set before writing to a large object. When studying the problem, we bumped into the fact that it is possible to actually simplify those ACL checks and

Re: [HACKERS] [BUGS] Replication to Postgres 10 on Windows is broken

2017-08-13 Thread Tom Lane
Andres Freund writes: > On 2017-08-11 18:11:03 +0530, Augustine, Jobin wrote: >> Appears that patch is not helping. > That's too bad. Any chance you could install > https://docs.microsoft.com/en-us/sysinternals/downloads/procmon and > activate monitoring just for that phase?

Re: [HACKERS] Allow INSTEAD OF DELETE triggers to modify the tuple for RETURNING

2017-08-13 Thread Marko Tiikkaja
On Fri, Jul 1, 2016 at 2:12 AM, I wrote: > Currently the tuple returned by INSTEAD OF triggers on DELETEs is only > used to determine whether to pretend that the DELETE happened or not, which > is often not helpful enough; for example, the actual tuple might have been > concurrently UPDATEd by

[HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-08-13 Thread Tomas Vondra
Hi, Attached is a rebased version of the Generational context, originally submitted with SlabContext (which was already committed into Pg 10). The main change is that I've abandoned the pattern of defining a Data structure and then a pointer typedef, i.e. typedef struct

Re: [HACKERS] WIP: Failover Slots

2017-08-13 Thread Craig Ringer
On 12 August 2017 at 08:03, Andres Freund wrote: > On 2017-08-02 16:35:17 -0400, Robert Haas wrote: > > I actually think failover slots are quite desirable, especially now > > that we've got logical replication in core. In a review of this > > thread I don't see anyone

Re: [HACKERS] reload-through-the-top-parent switch the partition table

2017-08-13 Thread Rushabh Lathia
On Fri, Aug 11, 2017 at 10:50 PM, Robert Haas wrote: > On Fri, Aug 11, 2017 at 5:36 AM, Rushabh Lathia > wrote: > > Please find attach patch with the changes. > > I found the way that you had the logic structured in flagInhTables() > to be quite

Re: [HACKERS] scan on inheritance parent with no children in current session

2017-08-13 Thread Ashutosh Bapat
On Fri, Aug 11, 2017 at 9:08 PM, Robert Haas wrote: > > rhaas=# create table parent (a int) partition by list (a); > CREATE TABLE > rhaas=# create temp table child partition of parent for values in (1); > CREATE TABLE > rhaas=# explain verbose select * from parent; >

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2017-08-13 Thread Sokolov Yura
В Fri, 11 Aug 2017 14:05:08 -0400 Robert Haas пишет: > On Thu, Aug 10, 2017 at 11:12 AM, Alexander Korotkov > wrote: > > These results look very cool! > > I think CSN is eventually inevitable, but it's a long distance > > feature. Thus, this

Re: [HACKERS] initdb failure on Debian sid/mips64el in EventTriggerEndCompleteQuery

2017-08-13 Thread Christoph Berg
Re: To PostgreSQL Hackers 2017-08-13 <20170813130127.g3tcyzzvuvlpz...@msg.df7cb.de> > 10beta3 and 9.6.4 are both failing during initdb on mips64el on > Debian/sid (unstable): > > https://buildd.debian.org/status/fetch.php?pkg=postgresql-9.6=mips64el=9.6.4-1=1502374949=0 >

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-13 Thread Gavin Flower
On 13/08/17 16:19, Thomas Munro wrote: On Sat, Aug 12, 2017 at 3:24 AM, Tom Lane wrote: [...] I'd vote for including this in v10. There doesn't seem to be any downside to this: it's a no brainer to avoid our exploding hash table case when we can see it coming. But