Re: [HACKERS] Hash Indexes

2016-09-15 Thread Mark Kirkwood
On 16/09/16 18:35, Amit Kapila wrote: On Thu, Sep 15, 2016 at 7:53 PM, Andres Freund wrote: Hi, On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: For making hash indexes usable in production systems, we need to improve its concurrency and make them crash-safe by WAL logging them. One earlier

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Amit Kapila
On Thu, Sep 15, 2016 at 7:53 PM, Andres Freund wrote: > Hi, > > On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: >> For making hash indexes usable in production systems, we need to improve >> its concurrency and make them crash-safe by WAL logging them. > > One earlier question about this is wheth

Re: [HACKERS] Tackling JsonPath support

2016-09-15 Thread Pavel Stehule
Hi 2016-09-15 18:05 GMT+02:00 Christian Convey : > On Mon, Sep 5, 2016 at 1:44 PM, Pavel Stehule > wrote: > ... > > > I wrote XMLTABLE function, and I am thinking about JSON_TABLE function. > But > > there is one blocker - missing JsonPath support in our JSON > implementation. > > > > So one ide

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Amit Kapila
On Thu, Sep 15, 2016 at 7:25 PM, Robert Haas wrote: > On Thu, Sep 15, 2016 at 2:13 AM, Amit Kapila wrote: >> One other point, I would like to discuss is that currently, we have a >> concept for tracking active hash scans (hashscan.c) which I think is >> mainly to protect splits when the backend w

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Pavan Deolasee
On Fri, Sep 16, 2016 at 9:09 AM, Pavan Deolasee wrote: > > I also realised that we can compact the TID array in step (b) above > because we only need to store 17 bits for block numbers (we already know > which 1GB segment they belong to). Given that usable offsets are also just > 13 bits, TID arr

Re: [HACKERS] Parallel sec scan in plpgsql

2016-09-15 Thread Ashutosh Bapat
On Thu, Sep 15, 2016 at 9:15 PM, Alex Ignatov wrote: > Hello! > Does parallel secscan works in plpgsql? > Parallel seq scan is a query optimization that will work independent of the source of the query - i.e whether it comes directly from a client or a procedural language like plpgsql. So, I gues

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Pavan Deolasee
On Fri, Sep 16, 2016 at 12:24 AM, Claudio Freire wrote: > On Thu, Sep 15, 2016 at 3:48 PM, Tomas Vondra > wrote: > > For example, we always allocate the TID array as large as we can fit into > > m_w_m, but maybe we don't need to wait with switching to the bitmap until > > filling the whole array

Re: [HACKERS] kqueue

2016-09-15 Thread Thomas Munro
On Thu, Sep 15, 2016 at 11:04 AM, Thomas Munro wrote: > On Thu, Sep 15, 2016 at 10:48 AM, Keith Fiske wrote: >> Thomas Munro brought up in #postgresql on freenode needing someone to test a >> patch on a larger FreeBSD server. I've got a pretty decent machine (3.1Ghz >> Quad Core Xeon E3-1220V3, 1

Re: [HACKERS] Stopping logical replication protocol

2016-09-15 Thread Craig Ringer
On 9 September 2016 at 12:03, Craig Ringer wrote: > Setting "waiting on author" in CF per discussion of the need for tests. Have you had a chance to look at adding the tests we discussed? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Trai

Re: [HACKERS] WAL consistency check facility

2016-09-15 Thread Michael Paquier
On Thu, Sep 15, 2016 at 7:30 PM, Kuntal Ghosh wrote: > Thoughts? There are still a couple of things that this patch makes me unhappy, particularly the handling of the GUC with the xlogreader flags. I am not sure if I'll be able to look at that again within the next couple of weeks, but please be

Re: [HACKERS] patch: function xmltable

2016-09-15 Thread Peter Eisentraut
On 9/12/16 1:14 AM, Craig Ringer wrote: > I would've expected once per query. There's no way the expressions can > reference the row data, so there's no reason to evaluate them each > time. > > The only use case I see for evaluating them each time is - maybe - > DEFAULT. Where maybe there's a use

Re: [HACKERS] WIP: About CMake v2

2016-09-15 Thread Michael Paquier
On Fri, Sep 16, 2016 at 4:38 AM, Yury Zhuravlev wrote: > Michael Paquier wrote: >> >> Could it be possible to get a refreshed patch on this thread for at >> least the sake of the archives? I'd really like to see somehting >> happening here and do some progress for this CF. > > > Sure, I will do it

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-15 Thread Craig Ringer
On 2 September 2016 at 23:29, Petr Jelinek wrote: > You could put it to txid.c where all the other txid stuff is in? Yeah, even though it's in adt/ I think it'll do. I thought I'd need get_xid_in_recent_past() for catalog_xmin hot standby feedback, but upon closer examination the needed logic i

Re: [HACKERS] patch: function xmltable

2016-09-15 Thread Craig Ringer
On 15 September 2016 at 19:31, Pavel Stehule wrote: > b_expr enforces shift/reduce conflict :( No problem then. I just thought it'd be worth allowing more if it worked to do so. > I found other opened question - how we can translate empty tag to SQL value? > The Oracle should not to solve this

Re: [HACKERS] condition variables

2016-09-15 Thread Peter Geoghegan
Maybe I can leave it up to you to determine if that applies in the context of my parallel create index patch. You are the obvious candidate to review that patch anyway, of course. -- Peter Geoghegan

Re: [HACKERS] shm_mq_set_sender() crash

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 5:22 PM, Tom Lane wrote: > Robert Haas writes: >> Of course, it's also possible that the ParallelWorkerNumber code is >> entirely correct and something overwrote the null bytes that were >> supposed to be found at that location. It would be very useful to see >> (a) the v

Re: [HACKERS] Fix comment in ATExecValidateConstraint

2016-09-15 Thread Robert Haas
On Thu, Aug 18, 2016 at 9:01 PM, Amit Langote wrote: > Reads much less ambiguous, so +1. > > Except in the doc patch: > > s/change the type of a column constraint/change the type of a column/g > > I fixed that part in the attached. Thanks. Committed; sorry for the delay. (As some of those of yo

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Tom Lane
Andreas Karlsson writes: > On 09/15/2016 05:38 PM, Alvaro Herrera wrote: >> I suppose some interested party could grab the patch that Heikki >> committed to the new branches and produce a back-patch that can be >> applied to the older branches. > Here is the result of backporting the sum of the t

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Andreas Karlsson
On 09/15/2016 05:38 PM, Alvaro Herrera wrote: I suppose some interested party could grab the patch that Heikki committed to the new branches and produce a back-patch that can be applied to the older branches. Here is the result of backporting the sum of the two patches on top of REL9_4_STABLE.

Re: [HACKERS] RLS related docs

2016-09-15 Thread Joe Conway
On 09/15/2016 01:33 PM, Robert Haas wrote: > On Sun, Aug 28, 2016 at 4:23 PM, Joe Conway wrote: For COPY, I think perhaps it would be more logical to put the new note immediately after the third note which describes the privileges required, since it's kind of related, and then we ca

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Christoph Berg
Re: Heikki Linnakangas 2016-09-15 <7e4991a9-410f-5e1f-2a3a-e918e4a4b...@iki.fi> > > I'm afraid it's not that easy - Debian 9 (stretch) will release at the > > beginning of next year, and apt.postgresql.org will want to build > > 9.2/9.3/9.4 for that distribution. I guess yum.postgresql.org will > >

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-15 Thread Peter Geoghegan
On Thu, Sep 15, 2016 at 1:51 PM, Heikki Linnakangas wrote: > BTW, does a 1-way merge make any sense? I was surprised to see this in the > log, even without this patch: > > LOG: finished 1-way merge step: CPU 0.62s/7.22u sec elapsed 8.43 sec > STATEMENT: SELECT COUNT(*) FROM (SELECT * FROM medium

Re: [HACKERS] shm_mq_set_sender() crash

2016-09-15 Thread Tom Lane
Robert Haas writes: > Of course, it's also possible that the ParallelWorkerNumber code is > entirely correct and something overwrote the null bytes that were > supposed to be found at that location. It would be very useful to see > (a) the value of ParallelWorkerNumber and (b) the contents of > v

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-15 Thread Robert Haas
On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane wrote: > Christoph Berg writes: >> I've always been wondering why we don't set a log_line_prefix by >> default. > > I think the odds of getting to something that everyone would agree on > are nil, so I'm not excited about getting into that particular > bi

Re: [HACKERS] shm_mq_set_sender() crash

2016-09-15 Thread Robert Haas
On Sat, Aug 27, 2016 at 3:43 AM, Amit Kapila wrote: > On Fri, Aug 26, 2016 at 6:20 PM, Tom Lane wrote: >> Latest from lorikeet: >> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2016-08-26%2008%3A37%3A27 >> >> TRAP: FailedAssertion("!(vmq->mq_sender == ((void *)0))", File: >>

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Tom Lane
Andres Freund writes: > On 2016-09-15 16:48:59 -0400, Tom Lane wrote: >> The patch that I posted would run both the generate_series(1, 2) and >> generate_series(2,4) calls in the same SRF node, forcing them to run in >> lockstep, after which their results would be fed to the SRF node doing >> the

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Andres Freund
On 2016-09-15 16:48:59 -0400, Tom Lane wrote: > Andres Freund writes: > > Hm. One thing I wonder about with this approach, is how we're going to > > handle something absurd like: > > SELECT generate_series(1, generate_series(1, 2)), generate_series(1, > > generate_series(2,4)); > > The patch tha

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-15 Thread Peter Geoghegan
On Thu, Sep 15, 2016 at 1:51 PM, Heikki Linnakangas wrote: >> I still don't get why you're doing all of this within mergeruns() (the >> beginning of when we start merging -- we merge all quicksorted runs), >> rather than within beginmerge() (the beginning of one particular merge >> pass, of which

Re: [HACKERS] Renaming some binaries

2016-09-15 Thread Robert Haas
On Fri, Aug 26, 2016 at 5:42 PM, Peter Eisentraut wrote: > On 8/26/16 12:26 PM, Euler Taveira wrote: >> initdb: we already have 'pg_ctl init' (since 9.0) and could remove initdb. > > I have a concern specifically about pg_ctl. Depending on how your > PostgreSQL is packaged, not all of the pg_ctl

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-15 Thread Heikki Linnakangas
On 09/15/2016 10:12 PM, Peter Geoghegan wrote: On Wed, Sep 14, 2016 at 10:43 AM, Heikki Linnakangas wrote: Spotted another issue with this code just now. Shouldn't it be based on state->tapeRange? You don't want the destTape to get memory, since you don't use batch memory for tapes that are wri

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Tom Lane
Andres Freund writes: > Hm. One thing I wonder about with this approach, is how we're going to > handle something absurd like: > SELECT generate_series(1, generate_series(1, 2)), generate_series(1, > generate_series(2,4)); The patch that I posted would run both the generate_series(1, 2) and gene

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Tom Lane
Thomas Munro writes: > Very interesting. Perhaps that is why NetBSD shows a speedup with the > kqueue patch[1] but FreeBSD doesn't. I guess that if I could get the > kqueue patch to perform better on large FreeBSD systems, it would also > be a solution to this problem. I just noticed that kqueu

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Andres Freund
Hi, On 2016-09-14 19:28:25 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-09-12 19:35:22 -0400, Tom Lane wrote: > >> Anyway I'll draft a prototype and then we can compare. > > > Ok, cool. > > Here's a draft patch that is just meant to investigate what the planner > changes might loo

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Tom Lane
Marco Pfatschbacher writes: > the current implementation of PostmasterIsAlive() uses a pipe to > monitor the existence of the postmaster process. > One end of the pipe is held open in the postmaster, while the other end is > inherited to all the auxiliary and background processes when they fork. >

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Heikki Linnakangas
On 09/15/2016 05:33 PM, Christoph Berg wrote: Re: Michael Paquier 2016-09-15 On Thu, Sep 15, 2016 at 8:57 PM, Heikki Linnakangas wrote: I backpatched this to 9.5, but not further than that. The functions this modified were moved around in 9.5, so the patch wouldn't apply as is. It wouldn't b

Re: [HACKERS] Postgres abort found in 9.3.11

2016-09-15 Thread Tom Lane
"K S, Sandhya (Nokia - IN/Bangalore)" writes: > We tried to replicate the scenario without our patch(exiting postmaster) and > still we were able to see the issue. > Same error was seen this time as well. > node-0 postgres[8243]: [1-2] HINT: Is another postmaster already running on > port 5433

Re: [HACKERS] WIP: About CMake v2

2016-09-15 Thread Yury Zhuravlev
Michael Paquier wrote: Could it be possible to get a refreshed patch on this thread for at least the sake of the archives? I'd really like to see somehting happening here and do some progress for this CF. Sure, I will do it on Friday. Today I finished mingw+msys support. (mingw without msys has

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Andres Freund
On 2016-09-15 15:23:58 -0400, Tom Lane wrote: > Andres Freund writes: > > In my present patch I've *ripped out* the support for materialization > > in nodeFunctionscan.c entirely. That means that rescans referencing > > volatile functions can change their behaviour (if a function is > > re

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Thomas Munro
On Fri, Sep 16, 2016 at 1:57 AM, Marco Pfatschbacher wrote: > the current implementation of PostmasterIsAlive() uses a pipe to > monitor the existence of the postmaster process. > One end of the pipe is held open in the postmaster, while the other end is > inherited to all the auxiliary and backgr

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-15 Thread Tom Lane
Andres Freund writes: > 0003-Avoid-materializing-SRFs-in-the-FROM-list.patch > To avoid performance regressions from moving SRFM_ValuePerCall SRFs to > ROWS FROM, nodeFunctionscan.c needs to support not materializing > output. I looked over this patch a bit. > In my present patch I've *r

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Andres Freund
Hi, On 2016-09-15 15:57:55 +0200, Marco Pfatschbacher wrote: > the current implementation of PostmasterIsAlive() uses a pipe to > monitor the existence of the postmaster process. > One end of the pipe is held open in the postmaster, while the other end is > inherited to all the auxiliary and backg

[HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Marco Pfatschbacher
Hi, the current implementation of PostmasterIsAlive() uses a pipe to monitor the existence of the postmaster process. One end of the pipe is held open in the postmaster, while the other end is inherited to all the auxiliary and background processes when they fork. This leads to multiple processes

Re: [HACKERS] Postgres abort found in 9.3.11

2016-09-15 Thread K S, Sandhya (Nokia - IN/Bangalore)
Hello, We tried to replicate the scenario without our patch(exiting postmaster) and still we were able to see the issue. Same error was seen this time as well. node-0 postgres[8243]: [1-2] HINT: Is another postmaster already running on port 5433? If not, wait a few seconds and retry. node-1

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-15 Thread Peter Geoghegan
On Wed, Sep 14, 2016 at 10:43 AM, Heikki Linnakangas wrote: > Addressed all your comments one way or another, new patch attached. Comments > on some specific points below: Cool. My response here is written under time pressure, which is not ideal. I think it's still useful, though. >> * You shoul

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Claudio Freire
On Thu, Sep 15, 2016 at 3:48 PM, Tomas Vondra wrote: > For example, we always allocate the TID array as large as we can fit into > m_w_m, but maybe we don't need to wait with switching to the bitmap until > filling the whole array - we could wait as long as the bitmap fits into the > remaining par

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Tomas Vondra
On 09/15/2016 06:40 PM, Robert Haas wrote: On Thu, Sep 15, 2016 at 12:22 PM, Tom Lane wrote: Tomas Vondra writes: On 09/14/2016 07:57 PM, Tom Lane wrote: People who are vacuuming because they are out of disk space will be very very unhappy with that solution. The people are usually runn

Re: [HACKERS] RLS related docs

2016-09-15 Thread Robert Haas
On Sun, Aug 28, 2016 at 4:23 PM, Joe Conway wrote: >>> For COPY, I think perhaps it would be more logical to put the new note >>> immediately after the third note which describes the privileges >>> required, since it's kind of related, and then we can talk about the >>> RLS policies required, e.g.

Re: [HACKERS] RLS related docs

2016-09-15 Thread Robert Haas
On Tue, Aug 30, 2016 at 3:05 AM, Dean Rasheed wrote: > On 28 August 2016 at 21:23, Joe Conway wrote: >> Apologies for the delay, but new patch attached. Assuming no more >> comments, will commit this, backpatched to 9.5, in a day or two. > > Looking at this again, I think there is something fishy

Re: [HACKERS] GiST: interpretation of NaN from penalty function

2016-09-15 Thread Andrew Borodin
> Certainly, "NaN means infinity", as your patch proposes, has no more basis to > it than "NaN means zero". You are absolutley right. Now I see that best interpretation is "NaN means NaN". Seems like we need only drop a check. Nodes with NaN penalties will be considered even worser than those with

Re: [HACKERS] Use pread and pwrite instead of lseek + write and read

2016-09-15 Thread Tom Lane
Oskari Saarenmaa writes: > 17.08.2016, 22:11, Tom Lane kirjoitti: >> I'd be more excited about this if the claimed improvement were more than >> 1.5%, but you know as well as I do that that's barely above the noise >> floor for most performance measurements. I'm left wondering why bother, >> and

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-09-15 Thread Robert Haas
On Tue, Sep 6, 2016 at 6:04 AM, Simon Riggs wrote: > On 1 September 2016 at 21:28, Simon Riggs wrote: >> So the only way to handle multiple locks is to do this roughly the way >> Rod suggests. > > I've just been reading the VACUUM code and it turns out that we > already use Rod's mechanism intern

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Tom Lane
"Daniel Verite" writes: > It looks easy to make the hooks return a bool, and when returning false, > cancel the assignment of the variable. Yeah, that's about how I was envisioning it too. > I volunteer to write that patch. Thanks. > It would close the hazard that exists today of an internal p

Re: [HACKERS] [COMMITTERS] pgsql: Support OpenSSL 1.1.0.

2016-09-15 Thread Heikki Linnakangas
On 09/15/2016 07:51 PM, Heikki Linnakangas wrote: Wild guess: curculio is building with LibreSSL, which claims to be OpenSSL >= 1.1.0, but it doesn't actually implement all the functions that OpenSSL 1.1.0 does. Looks like we need some more autoconf scripting to detect LibreSSL. Or switch to det

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Daniel Verite
Tom Lane wrote: > I think changing the hook API is a pretty reasonable thing to do here > (though I'd make it its own patch and then add the autocommit change > on top). When was the last time you actually wanted to set VERBOSITY > to "fubar"? It looks easy to make the hooks return a boo

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Jesper Pedersen
On 09/15/2016 02:03 AM, Amit Kapila wrote: Same thing here - where the fields involving the hash index aren't updated. Do you mean that for such cases also you see 40-60% gain? No, UPDATEs are around 10-20% for our cases. I have done a run to look at the concurrency / TPS aspect of the i

Re: [HACKERS] [COMMITTERS] pgsql: Support OpenSSL 1.1.0.

2016-09-15 Thread Heikki Linnakangas
On 09/15/2016 07:41 PM, Tom Lane wrote: Heikki Linnakangas writes: Support OpenSSL 1.1.0. Buildfarm member curculio doesn't like this patch. I suspect the reason is it's got some slightly-too-old version of OpenSSL, but if so, we ought to try to fix configure's probe so the problem gets repo

Re: [HACKERS] Declarative partitioning - another take

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 4:53 AM, Amit Langote wrote: > Wow, this is bad. What is needed in this case is "canonicalization" of > the range partition bounds specified in the command. I think we shouldn't worry about this. It seems like unnecessary scope creep. All human beings capable of using P

Re: [HACKERS] [COMMITTERS] pgsql: Support OpenSSL 1.1.0.

2016-09-15 Thread Tom Lane
Heikki Linnakangas writes: > Support OpenSSL 1.1.0. Buildfarm member curculio doesn't like this patch. I suspect the reason is it's got some slightly-too-old version of OpenSSL, but if so, we ought to try to fix configure's probe so the problem gets reported at configure time, not somewhere down

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 12:22 PM, Tom Lane wrote: > Tomas Vondra writes: >> On 09/14/2016 07:57 PM, Tom Lane wrote: >>> People who are vacuuming because they are out of disk space will be very >>> very unhappy with that solution. > >> The people are usually running out of space for data, while th

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Tom Lane
Tomas Vondra writes: > On 09/14/2016 07:57 PM, Tom Lane wrote: >> People who are vacuuming because they are out of disk space will be very >> very unhappy with that solution. > The people are usually running out of space for data, while these files > would be temporary files placed wherever temp

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Claudio Freire
On Thu, Sep 15, 2016 at 12:50 PM, Tomas Vondra wrote: > On 09/14/2016 07:57 PM, Tom Lane wrote: >> >> Pavan Deolasee writes: >>> >>> On Wed, Sep 14, 2016 at 10:53 PM, Alvaro Herrera >>> >>> wrote: One thing not quite clear to me is how do we create the bitmap representation starti

Re: [HACKERS] File system operations.

2016-09-15 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova > wrote: >> What do you think about moving stuff from pg_upgrade/file.c to storage/file/ >> to allow reuse of this code? I think it'll be really helpful for developers >> of contrib modules >> like backup managers. > W

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 11:10 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Thu, Sep 15, 2016 at 7:29 AM, Rahila Syed wrote: In keeping with current design of hooks instead of rejecting autocommit 'ON' setting inside a transaction,the value can be set to

[HACKERS] Tackling JsonPath support

2016-09-15 Thread Christian Convey
On Mon, Sep 5, 2016 at 1:44 PM, Pavel Stehule wrote: ... > I wrote XMLTABLE function, and I am thinking about JSON_TABLE function. But > there is one blocker - missing JsonPath support in our JSON implementation. > > So one idea - implement JsonPath support and related JSON query functions. > Thi

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Tomas Vondra
On 09/14/2016 05:17 PM, Robert Haas wrote: I am kind of doubtful about this whole line of investigation because we're basically trying pretty hard to fix something that I'm not sure is broken.I do agree that, all other things being equal, the TID lookups will probably be faster with a bitma

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Tomas Vondra
On 09/14/2016 07:57 PM, Tom Lane wrote: Pavan Deolasee writes: On Wed, Sep 14, 2016 at 10:53 PM, Alvaro Herrera wrote: One thing not quite clear to me is how do we create the bitmap representation starting from the array representation in midflight without using twice as much memory transie

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Tom Lane
I wrote: > OK, I'll take care of it (since I now realize that the inconsistency > is my own fault --- I committed that GUC not you). It's unclear what > this will do for Peter's complaint though. On closer inspection, the answer is "nothing", because the select_parallel test overrides the default

[HACKERS] Parallel sec scan in plpgsql

2016-09-15 Thread Alex Ignatov
Hello! Does parallel secscan works in plpgsql? -- Alex Ignatov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hac

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-15 Thread Robert Haas
On Tue, Sep 13, 2016 at 9:07 PM, Kouhei Kaigai wrote: > In the current implementation calls recompute_limits() on the first > invocation of ExecLimit and ExecReScanLimit. Do we expect the > ps->numTuples will be also passed down to the child nodes on the same > timing? Sure, unless we find some r

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Alvaro Herrera
Christoph Berg wrote: > Re: Michael Paquier 2016-09-15 > > > On Thu, Sep 15, 2016 at 8:57 PM, Heikki Linnakangas wrote: > > > I backpatched this to 9.5, but not further than that. The functions this > > > modified were moved around in 9.5, so the patch wouldn't apply as is. It > > > wouldn't be

Re: [HACKERS] Comment typo in execIndexing.c

2016-09-15 Thread Robert Haas
On Tue, Sep 13, 2016 at 10:48 PM, Amit Langote wrote: > Spotted a typo in the header comment of execIndexing.c. Attached fixes it. > > s/exclusive constraints/exclusion constraints/g Committed, thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 11:10 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 15, 2016 at 7:29 AM, Rahila Syed wrote: >>> In keeping with current design of hooks instead of rejecting autocommit 'ON' >>> setting inside >>> a transaction,the value can be set to 'ON' with a psql_error disp

Re: [HACKERS] File system operations.

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova wrote: > What do you think about moving stuff from pg_upgrade/file.c to storage/file/ > to allow reuse of this code? I think it'll be really helpful for developers > of contrib modules > like backup managers. Well, storage/file is backend an

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 7:29 AM, Rahila Syed wrote: >> In keeping with current design of hooks instead of rejecting autocommit 'ON' >> setting inside >> a transaction,the value can be set to 'ON' with a psql_error displaying that >> the value >> will be effective when the cu

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-15 Thread Masahiko Sawada
On Thu, Sep 15, 2016 at 2:40 AM, Simon Riggs wrote: > On 14 September 2016 at 11:19, Pavan Deolasee > wrote: > >>> In >>> theory we could even start with the list of TIDs and switch to the >>> bitmap if the TID list becomes larger than the bitmap would have been, >>> but I don't know if it's wo

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-15 Thread Tom Lane
Robert Haas writes: > This seems like a very complicated mechanism of substituting for a > very simple patch. Well, if we're gonna do it, then let's just do it, but please let's have a patch that doesn't look like somebody's temporary debugging kluge. I'd suggest that this is parallel to nodeToS

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 10:44 AM, Tom Lane wrote: >> Well, sure, but at any reasonable value of min_parallel_relation_size >> that won't be a factor. The question here is whether we want the default >> value to be platform-independent. I notice that both config.sgml and >>

[HACKERS] File system operations.

2016-09-15 Thread Anastasia Lubennikova
Hi, hackers! Do we have any standard set of functions to operate with files? I mean link(), copy(), rename(), unlink(), ftruncate() and so on. Mostly I worry about the first two since they have different implementation on Windows. I found a couple of functions in src/backend/storage/file/copydir

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 7:29 AM, Rahila Syed wrote: > In keeping with current design of hooks instead of rejecting autocommit 'ON' > setting inside > a transaction,the value can be set to 'ON' with a psql_error displaying that > the value > will be effective when the current transaction has ended.

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 10:44 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 15, 2016 at 9:59 AM, Tom Lane wrote: >>> Possibly we ought to change things so that the default value of >>> min_parallel_relation_size is a fixed number of bytes rather >>> than a fixed number of blocks. Not

Re: [HACKERS] Block level parallel vacuum WIP

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 7:21 AM, Masahiko Sawada wrote: > I'm implementing this patch but I need to resolve the problem > regarding lock for extension by multiple parallel workers. > In parallel vacuum, multiple workers could try to acquire the > exclusive lock for extension on same relation. > Si

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 9:59 AM, Tom Lane wrote: >> Possibly we ought to change things so that the default value of >> min_parallel_relation_size is a fixed number of bytes rather >> than a fixed number of blocks. Not sure though. > The reason why this was originally recko

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-15 Thread Pavan Deolasee
On Thu, Sep 15, 2016 at 7:55 PM, Pavan Deolasee wrote: > > (lldb) script print print_bms_members(lldb.frame.FindVariable ("a")) > nwords = 1 bitmap: 0x200 > > Or even this if lldb.frame.FindVariable() is pushed inside the function: (lldb) script print print_bms_members('a') nwords = 1 bitmap: 0x

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-09-15 Thread Christoph Berg
Re: Michael Paquier 2016-09-15 > On Thu, Sep 15, 2016 at 8:57 PM, Heikki Linnakangas wrote: > > I backpatched this to 9.5, but not further than that. The functions this > > modified were moved around in 9.5, so the patch wouldn't apply as is. It > > wouldn't be difficult to back-patch further if

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-15 Thread Pavan Deolasee
On Thu, Sep 15, 2016 at 7:38 PM, Robert Haas wrote: > > > This seems like a very complicated mechanism of substituting for a > very simple patch. I don't have objection to the patch per se. The point of posting this was just to share other mechanisms that exists. BTW advantage of using debugger

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Andres Freund
Hi, On 2016-05-10 17:39:22 +0530, Amit Kapila wrote: > For making hash indexes usable in production systems, we need to improve > its concurrency and make them crash-safe by WAL logging them. One earlier question about this is whether that is actually a worthwhile goal. Are the speed and space b

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Sep 15, 2016 at 9:59 AM, Tom Lane wrote: > > Possibly we ought to change things so that the default value of > > min_parallel_relation_size is a fixed number of bytes rather > > than a fixed number of blocks. Not sure though. > > The reason why this was originally re

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 1:41 AM, Amit Kapila wrote: > I think it is possible without breaking pg_upgrade, if we match all > items of a page at once (and save them as local copy), rather than > matching item-by-item as we do now. We are already doing similar for > btree, refer explanation of BTSca

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 9:59 AM, Tom Lane wrote: > Possibly we ought to change things so that the default value of > min_parallel_relation_size is a fixed number of bytes rather > than a fixed number of blocks. Not sure though. The reason why this was originally reckoned in blocks is because the

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-15 Thread Robert Haas
On Wed, Sep 14, 2016 at 8:01 AM, Pavan Deolasee wrote: > On Wed, Sep 14, 2016 at 3:46 PM, Pavan Deolasee > wrote: >> lately I'm using LVM debugger (which probably does not have something >> equivalent), > > > And I was so clueless about lldb's powerful scripting interface. For > example, you can

Re: [HACKERS] select_parallel test fails with nonstandard block size

2016-09-15 Thread Tom Lane
Peter Eisentraut writes: > When building with --with-blocksize=16, the select_parallel test fails > with this difference: > explain (costs off) > select sum(parallel_restricted(unique1)) from tenk1 > group by(parallel_restricted(unique1)); > - QUERY PLAN > --

Re: [HACKERS] Hash Indexes

2016-09-15 Thread Robert Haas
On Thu, Sep 15, 2016 at 2:13 AM, Amit Kapila wrote: > One other point, I would like to discuss is that currently, we have a > concept for tracking active hash scans (hashscan.c) which I think is > mainly to protect splits when the backend which is trying to split has > some scan open. You can read

Re: [HACKERS] palloc() too large on pg_buffercache with large shared_buffers

2016-09-15 Thread Robert Haas
On Wed, Sep 14, 2016 at 7:59 PM, Kouhei Kaigai wrote: >> On Wed, Sep 14, 2016 at 12:13 AM, Kouhei Kaigai wrote: >> > It looks to me pg_buffercache tries to allocate more than 1GB using >> > palloc(), when shared_buffers is more than 256GB. >> > >> > # show shared_buffers ; >> > shared_buffers >>

Re: [HACKERS] sequences and pg_upgrade

2016-09-15 Thread Anastasia Lubennikova
15.09.2016 15:29, Peter Eisentraut: On 9/14/16 8:52 AM, Anastasia Lubennikova wrote: Could you clarify, please, why do we dump sequence in schemaOnly mode? + if (dopt.schemaOnly && dopt.sequence_data) + getSequenceData(&dopt, tblinfo, numTables, dopt.oids); The point of this

Re: [HACKERS] Declarative partitioning - another take

2016-09-15 Thread Ashutosh Bapat
Hi Amit, It looks like there is some problem while creating paramterized paths for multi-level partitioned tables. Here's a longish testcase CREATE TABLE prt1_l (a int, b int, c varchar) PARTITION BY RANGE(a); CREATE TABLE prt1_l_p1 PARTITION OF prt1_l FOR VALUES START (0) END (250) PARTITION BY

Re: [HACKERS] autonomous transactions

2016-09-15 Thread Peter Eisentraut
On 9/3/16 9:08 AM, Serge Rielau wrote: > Interestingly, despite being supported in PL/SQL on nested BEGIN END blocks, > we nearly exclusively see AT’s covering the entire function or trigger. Oracle only supports an entire PL/SQL function to be declared autonomous. But it was pretty easy in my i

Re: [HACKERS] autonomous transactions

2016-09-15 Thread Peter Eisentraut
On 8/31/16 9:11 AM, Craig Ringer wrote: > Peter, you mention "Oracle-style autonomous transaction blocks". > > What are the semantics to be expected of those with regards to: > > - Accessing objects exclusively locked by the outer xact or where the > requested lockmode conflicts with a lock held

Re: [HACKERS] autonomous transactions

2016-09-15 Thread Peter Eisentraut
On 8/31/16 8:46 AM, Greg Stark wrote: > I'm surprised by the background worker. I had envisioned autonomous > transactions being implemented by allowing a process to reserve a > second entry in PGPROC with the same pid. Or perhaps save its existing > information in a separate pgproc slot (or stack

Re: [HACKERS] WIP: About CMake v2

2016-09-15 Thread Michael Paquier
Yury, On Mon, Aug 22, 2016 at 7:48 AM, Christian Convey wrote: >>> I glad to hear it. I suppose you can just try build postgres and send all >>> problems to github tracker. >>> https://github.com/stalkerg/postgres_cmake/issues > > FYI, I had success using your "postgres_cmake" repo. I tested it

Re: [HACKERS] autonomous transactions

2016-09-15 Thread Peter Eisentraut
On 8/31/16 12:38 AM, Jaime Casanova wrote: > On 30 August 2016 at 20:50, Peter Eisentraut > wrote: >> >> - Patches to PL/pgSQL to implement Oracle-style autonomous transaction >> blocks: >> >> AS $$ >> DECLARE >> PRAGMA AUTONOMOUS_TRANSACTION; >> BEGIN >> FOR i IN 0..9 LOOP >> START TRANSA

  1   2   >