Re: [HACKERS] Reading timeline from pg_control on replication slave

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 2:48 AM, Craig Ringer wrote: > (I'd quite like ThisTimeLineID to go away as a global. It's messy and > confusing, and I'd much rather it be fetched when needed). Yeah, I agree. My take on the matter is that we could just use the status data of the

Re: [HACKERS] Parallel worker error

2017-10-30 Thread Amit Kapila
On Mon, Oct 30, 2017 at 10:04 AM, Robert Haas wrote: > On Mon, Oct 30, 2017 at 8:25 AM, Amit Kapila wrote: >> Thanks. I have closed this entry in CF app, however, I am not sure >> what is the best way to deal with the entry present in PostgreSQL

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-10-30 Thread Masahiko Sawada
On Fri, Oct 27, 2017 at 12:03 AM, Robert Haas wrote: > On Thu, Oct 26, 2017 at 12:36 PM, Masahiko Sawada > wrote: >> Since the previous patch conflicts with current HEAD, I attached the >> updated patch for next CF. > > I think we should back up

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2017-10-30 Thread Ashutosh Bapat
On Fri, Oct 27, 2017 at 1:36 AM, Tom Lane wrote: > David Rowley writes: >> On 27 October 2017 at 01:44, Ashutosh Bapat >> wrote: >>> I think Antonin has a point. The join processing may deem some base >>>

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Robert Haas
On Sun, Oct 29, 2017 at 1:19 AM, Simon Riggs wrote: > Nothing I am proposing blocks later work. That's not really true. Nobody's going to be happy if MERGE has one behavior in one set of cases and an astonishingly different behavior in another set of cases. If you adopt

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 9:43 AM, Chris Travers wrote: > Are there any cases right now where you have features added by extensions > that write to directories which are required for a rewind? In some of the stuff I maintain, I actually have one case now of a

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Chris Travers
On Mon, Oct 30, 2017 at 10:57 AM, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 9:43 AM, Chris Travers > wrote: > > Are there any cases right now where you have features added by > extensions that write to directories which are required

Re: [HACKERS] proposal: schema variables

2017-10-30 Thread Hannu Krosing
but you can always do with a (id, value) as ( values (1, 'foo'), (2, 'bar'), (3, 'baz') ) select set_config('custom.value',(select value from a where id = 2),true); if you are worried about the evaluation order On 29 October 2017 at 09:51, Chris Travers wrote: > >

Re: [HACKERS] path toward faster partition pruning

2017-10-30 Thread Rajkumar Raghuwanshi
On Mon, Oct 30, 2017 at 12:20 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > In the previous versions, RT index of the table needed to be passed to > partition.c, which I realized is no longer needed, so I removed that > requirement from the interface. As a result, patches 0002 and

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread David Rowley
On 30 October 2017 at 22:34, Andres Freund wrote: > Hi, > > On 2015-12-20 02:49:13 +1300, David Rowley wrote: >> Alternatively, if there was some way to mark the path as cold from within >> the path, rather than from the if() condition before the path, then we >> could perhaps

[HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Alvaro Herrera
Tomas Vondra wrote: > FWIW I can reproduce this on REL_10_STABLE, but not on REL9_6_STABLE. So > it seems to be due to something that changed in the last release. I've been trying to reproduce it for half an hour with no success (I think my laptop is just too slow). I bet this is related to the

[HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Connor Wolf
Hi there! I'm looking at implementing a custom indexing scheme, and I've been having trouble understanding the proper approach. Basically, I need a BK tree, which is a tree-structure useful for indexing arbitrary discrete metric-spaces (in my case, I'm interested in indexing across the

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread David Rowley
On 30 October 2017 at 22:44, Andres Freund wrote: > On 2017-10-30 22:39:01 +1300, David Rowley wrote: >> Today I was thinking, to get around that issue, we might be able to >> generate another thin wrapper around elog_start() and mark that as >> __attribute__((cold)) and fudge

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Oleg Bartunov
On Sun, Oct 29, 2017 at 10:07 AM, Connor Wolf wrote: > Hi there! > > I'm looking at implementing a custom indexing scheme, and I've been having > trouble understanding the proper approach. > > Basically, I need a BK tree, which is a tree-structure useful for indexing

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-30 Thread Andres Freund
Hi, On 2017-10-24 03:39:54 -0700, Andres Freund wrote: > Largely that's due to the overflow checks. > > For integers we currently do: > > #define SAMESIGN(a,b) (((a) < 0) == ((b) < 0)) > > /* >* Overflow check. If the inputs are of different signs then their sum >* cannot

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Alexander Korotkov
On Mon, Oct 30, 2017 at 2:07 PM, Oleg Bartunov wrote: > On Mon, Oct 30, 2017 at 12:05 PM, Oleg Bartunov > wrote: > > On Sun, Oct 29, 2017 at 10:07 AM, Connor Wolf > > wrote: > >> Hi there! > >> > >> I'm looking at

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 11:56 AM, Raúl Marín Rodríguez wrote: > both patches seem complementary. I've rebased my changes on top of that > patch > (v14) in https://git.io/vFtnT and everything seems to be working fine. Attaching patches directly to a thread is a better

Re: [HACKERS] More stats about skipped vacuums

2017-10-30 Thread Kyotaro HORIGUCHI
At Thu, 26 Oct 2017 15:06:30 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20171026.150630.115694437.horiguchi.kyot...@lab.ntt.co.jp> > At Fri, 20 Oct 2017 19:15:16 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Michael Paquier
On Fri, Oct 27, 2017 at 4:51 PM, Raúl Marín Rodríguez wrote: > I've written a small patch to add support for pow() in pgbench. Cool. > The main reason behind it is that I'm currently using a shell call to do it > which takes between 2-10 ms that can be a big percentage of

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 11:07 AM, Alvaro Herrera wrote: > Michael Paquier wrote: > >> Please add this patch to the upcoming commit fest if you would like to >> get some feedback: >> https://commitfest.postgresql.org/15/ >> >> I am adding as well Fabien in CC who worked in

Re: [HACKERS] PoC: full merge join on comparison clause

2017-10-30 Thread Alexander Kuzmenkov
Hi, I am attaching the updated patch, rebased to 820c03. On 09.10.2017 13:47, Ashutosh Bapat wrote: Hi Alexander, Commit c7a9fa399 has added another test on mergeopfamilies. I think your patch will need to take care of that test. On Wed, Oct 4, 2017 at 6:38 PM, Alexander Kuzmenkov

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Alvaro Herrera
Michael Paquier wrote: > Please add this patch to the upcoming commit fest if you would like to > get some feedback: > https://commitfest.postgresql.org/15/ > > I am adding as well Fabien in CC who worked in getting the internal > function infrastructure in the shape it is now (waaay better)

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Oleg Bartunov
On Mon, Oct 30, 2017 at 12:05 PM, Oleg Bartunov wrote: > On Sun, Oct 29, 2017 at 10:07 AM, Connor Wolf > wrote: >> Hi there! >> >> I'm looking at implementing a custom indexing scheme, and I've been having >> trouble understanding the proper

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Chris Travers
On Mon, Oct 30, 2017 at 11:36 AM, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 10:15 AM, Chris Travers > wrote: > > This also brings up a fairly major concern more generally about control > by > > the way. A lot of cases where pg_rewind

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Alexander Korotkov
Hi! On Sun, Oct 29, 2017 at 12:07 PM, Connor Wolf wrote: > I'm looking at implementing a custom indexing scheme, and I've been having > trouble understanding the proper approach. > > Basically, I need a BK tree, which is a tree-structure useful for indexing >

Re: [HACKERS] Flexible configuration for full-text search

2017-10-30 Thread Aleksandr Parfenov
I'm mostly happy with mentioned modifications, but I have few questions to clarify some points. I will send new patch in week or two. On Thu, 26 Oct 2017 20:01:14 +0200 Emre Hasegeli wrote: > To put it formally: > > ALTER TEXT SEARCH CONFIGURATION name > ADD MAPPING FOR

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 7:04 PM, Alvaro Herrera wrote: >> A sort of middle way would be to keep the secondary checkpoint around >> but never try to replay from that point, or only if a specific flag is >> provided. > > Why do you want to keep the secondary checkpoint? If

Re: [HACKERS] CUBE seems a bit confused about ORDER BY

2017-10-30 Thread Alexander Korotkov
On Sun, Oct 29, 2017 at 1:30 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Fri, Oct 20, 2017 at 1:01 AM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > I've reviewed code of ~> operator and its KNN-GiST support. > Unfortunately, it appears that it's broken in

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-10-30 Thread Robert Haas
On Tue, Oct 24, 2017 at 1:26 PM, Ivan Kartyshov wrote: > Hello. I made some bugfixes and rewrite the patch. I don't think it's a good idea to deliberately leave the state of the standby different from the state of the master on the theory that it won't matter. I

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: > I think it does the contrary. The current mechanism is, in my opinion, > downright dangerous: > https://www.postgresql.org/message-id/20160201235854.go8...@awork2.anarazel.de A sort of middle way would be to keep the

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: > > I think it does the contrary. The current mechanism is, in my opinion, > > downright dangerous: > > https://www.postgresql.org/message-id/20160201235854.go8...@awork2.anarazel.de > > A sort of

Re: [HACKERS] Comment typo

2017-10-30 Thread Magnus Hagander
On Mon, Oct 30, 2017 at 3:49 AM, Etsuro Fujita wrote: > Here is a patch to fix a typo in a comment in partition.c: > s/specificiation/specification/. > Applied, thanks. -- Magnus Hagander Me: https://www.hagander.net/ Work:

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 7:04 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Oct 24, 2017 at 7:25 PM, Andres Freund wrote: >> > I think it does the contrary. The current mechanism is, in my opinion, >> > downright dangerous: >> >

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Tom Lane
Robert Haas writes: > I was mostly just thinking out loud, listing another option rather > than advocating for it. FWIW, I just wanted the question to be debated and resolved properly. After rereading the thread Andres pointed to, I thought of a hazard that I think Andres

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 25 October 2017 at 00:17, Michael Paquier wrote: > On Mon, Oct 23, 2017 at 11:20 PM, Simon Riggs wrote: >> Remove the code that maintained two checkpoint's WAL files and all >> associated stuff. >> >> Try to avoid breaking anything else >> >>

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Alvaro Herrera
Michael Paquier wrote: > Attaching patches directly to a thread is a better practice as if > github goes away, any Postgres developers can still have an access to > any code you publish using the public archives on postgresql.org. Also, by posting to pgsql-hackers indicating intention to

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 09:44, Robert Haas wrote: > On Sun, Oct 29, 2017 at 1:19 AM, Simon Riggs wrote: >> Nothing I am proposing blocks later work. > > That's not really true. Nobody's going to be happy if MERGE has one > behavior in one set of cases

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 2:22 PM, Simon Riggs wrote: > On 25 October 2017 at 00:17, Michael Paquier > wrote: >> -* Delete old log files (those no longer needed even for previous >> -* checkpoint or the standbys in XLOG streaming). >> +

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Simon Riggs
On 29 October 2017 at 21:25, Peter Geoghegan wrote: > The semantics that I suggest (the SQL standard's semantics) will > require less code, and will be far simpler. Right now, I simply don't > understand why you're insisting on using ON CONFLICT without even > saying why. I can

[HACKERS] Patch: restrict pg_rewind to whitelisted directories

2017-10-30 Thread Chris Travers
The attached patch is cleaned up and filed for the commit fest this next month: Here's the full commit message via Mercurial. I will likely have a new branch per version the patch since that's the closest thing to a rebase in this version control system. changeset: 60492:47f87a2d2fa1 tag:

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Raúl Marín Rodríguez
Sorry about the patch. Attaching it now so it can be considered as submitted. -- *Raúl Marín Rodríguez*carto.com From a6eecfe6637bdb0cbb02a9eda7b88d9c4325bb51 Mon Sep 17 00:00:00 2001 From: Raul Marin Date: Fri, 13 Oct 2017 17:42:23 +0200 Subject: [PATCH] Add pow()

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Raúl Marín Rodríguez
Hi, both patches seem complementary. I've rebased my changes on top of that patch (v14) in https://git.io/vFtnT and everything seems to be working fine. On Mon, Oct 30, 2017 at 12:36 PM, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 11:07 AM, Alvaro Herrera >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-30 Thread Ashutosh Bapat
On Thu, Oct 26, 2017 at 7:41 PM, Masahiko Sawada wrote: > > Because I don't want to break the current user semantics. that is, > currently it's guaranteed that the subsequent reads can see the > committed result of previous writes even if the previous transactions > were

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Michael Paquier
On Sat, Oct 28, 2017 at 4:22 AM, Chris Travers wrote: > The Solution: > The solution is a whitelist of directories specified which are the only ones > which are synchronised. The relevant part of this patch is: > > +/* List of directories to synchronize: > + * base data

Re: [HACKERS] Jsonb transform for pl/python

2017-10-30 Thread Anthony Bykov
On Sun, 29 Oct 2017 19:11:02 +0100 David Fetter wrote: > Thanks for your hard work! > > Should there also be one for PL/Python3U? > > Best, > David. Hi. Actually, there is one for PL/Python3U. This patch contains following extensions: jsonb_plpythonu jsonb_plpython2u

Re: [HACKERS] git down

2017-10-30 Thread Magnus Hagander
On Mon, Oct 30, 2017 at 2:35 AM, Andreas Karlsson wrote: > On 10/27/2017 10:51 PM, Erik Rijkers wrote: > >> git.postgresql.org is down/unreachable >> >> ( git://git.postgresql.org/git/postgresql.git ) >> > > Yes, I noticed this too, but

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread Andres Freund
Hi, On 2015-12-20 02:49:13 +1300, David Rowley wrote: > Alternatively, if there was some way to mark the path as cold from within > the path, rather than from the if() condition before the path, then we > could perhaps do something in the elog() macro instead. I just couldn't > figure out a way

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Chris Travers
First, thanks for your thoughts on this, and I am interested in probing them more. On Mon, Oct 30, 2017 at 9:04 AM, Michael Paquier wrote: > On Sat, Oct 28, 2017 at 4:22 AM, Chris Travers > wrote: > > The Solution: > > The solution is a

[HACKERS] pow support for pgbench

2017-10-30 Thread Raúl Marín Rodríguez
Hi, I've written a small patch to add support for pow() in pgbench. The main reason behind it is that I'm currently using a shell call to do it which takes between 2-10 ms that can be a big percentage of the time taken by the whole transaction. For example (shortened): latency average = 11.718

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 10:15 AM, Chris Travers wrote: > This also brings up a fairly major concern more generally about control by > the way. A lot of cases where pg_rewind is called, the user doesn't > necessarily have much control on how it is called. Moreover in

Re: [HACKERS] parallelize queries containing initplans

2017-10-30 Thread tushar
On 10/30/2017 09:02 AM, Amit Kapila wrote: Thanks a lot Tushar for testing this patch. In the latest patch, I have just rebased some comments, there is no code change, so I don't expect any change in behavior, but feel free to test it once again. Thanks Amit. Sure. -- regards,tushar

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-10-30 Thread Masahiko Sawada
On Mon, Oct 30, 2017 at 5:48 PM, Ashutosh Bapat wrote: > On Thu, Oct 26, 2017 at 7:41 PM, Masahiko Sawada > wrote: >> >> Because I don't want to break the current user semantics. that is, >> currently it's guaranteed that the subsequent

Re: [HACKERS] An unlikely() experiment

2017-10-30 Thread Andres Freund
On 2017-10-30 22:39:01 +1300, David Rowley wrote: > On 30 October 2017 at 22:34, Andres Freund wrote: > > Hi, > > > > On 2015-12-20 02:49:13 +1300, David Rowley wrote: > >> Alternatively, if there was some way to mark the path as cold from within > >> the path, rather than

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Tom Lane
Alexander Korotkov writes: > I think Connor struggles to implement just an operator class. Advising him > to implement an index access method is a good way to get him away of > PostgreSQL hacking for a long time :) Yeah. To answer the question a bit more directly:

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 15:31, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 2:22 PM, Simon Riggs wrote: >> On 25 October 2017 at 00:17, Michael Paquier >> wrote: >>> -* Delete old log files (those no longer

Re: [HACKERS] Jsonb transform for pl/python

2017-10-30 Thread David Fetter
On Mon, Oct 30, 2017 at 11:15:00AM +0300, Anthony Bykov wrote: > On Sun, 29 Oct 2017 19:11:02 +0100 > David Fetter wrote: > > > Thanks for your hard work! > > > > Should there also be one for PL/Python3U? > > > > Best, > > David. > Hi. > Actually, there is one for

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-30 Thread Alvaro Herrera
Alvaro Herrera wrote: > Alvaro Herrera wrote: > > > Before pushing, I'll give a look to the regular autovacuum path to see > > if it needs a similar fix. > > Reading that one, my conclusion is that it doesn't have the same problem > because the strings are allocated in AutovacuumMemCxt which is

Re: [HACKERS] pow support for pgbench

2017-10-30 Thread Fabien COELHO
Hello Michaël, I'm fine with having pow in pgbench. I am adding as well Fabien in CC who worked in getting the internal function infrastructure in the shape it is now (waaay better) with commit 86c43f4. It might be even better if https://commitfest.postgresql.org/15/985/, which has been

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Andres Freund
On 2017-10-30 10:10:19 -0400, Tom Lane wrote: > Robert Haas writes: > > I was mostly just thinking out loud, listing another option rather > > than advocating for it. > > FWIW, I just wanted the question to be debated and resolved properly. > > After rereading the thread

[HACKERS] Fix duplicated "the" occurrences in codebase

2017-10-30 Thread Christoph Dreis
Hey, please find a patch attached that fixes duplicated "the" occurrences in the codebase. As this is my first patch, please let me know in case I did something wrong. Cheers, Christoph duplicate-the-typos.patch Description: Binary data -- Sent via pgsql-hackers mailing list

Re: [HACKERS] path toward faster partition pruning

2017-10-30 Thread Dilip Kumar
On Mon, Oct 30, 2017 at 12:20 PM, Amit Langote wrote: > On 2017/10/30 14:55, Amit Langote wrote: >> Fixed in the attached updated patch, along with a new test in 0001 to >> cover this case. Also, made a few tweaks to 0003 and 0005 (moved some >> code from the

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tomas Vondra
On 10/30/2017 09:03 PM, Tom Lane wrote: > Michael Paquier writes: >> b1328d78f is close enough, but per what I see that's actually not >> true. I have been able to reproduce the problem, which shows up within >> a window of 2-4 minutes. Still sometimes it can take way

[HACKERS] Rewriting PL/Python's typeio code

2017-10-30 Thread Tom Lane
I started to work on teaching PL/Python about domains over composite, and soon found that it was a can of worms. Aside from the difficulty of shoehorning that in with a minimal patch, there were pre-existing problems. I found that it didn't do arrays of domains right either (ok, that's an

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-10-30 Thread Nikita Glukhov
Hi, hackers! I have a question about transformation of JSON constructors into executor nodes. In first letter in this thread we wrote: JSON_OBJECT(), JSON_ARRAY() constructors and IS JSON predicate are transformed into raw function calls. Here is an example explaining what it means: =#

[HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2017-10-30 Thread Lætitia Avrot
Hello, As Amit Langot pointed out, the column_constraint definition is missing whereas it is used in ALTER TABLE synopsis. It can be easily found in the CREATE TABLE synopsis, but it's not very user friendly. I simply copied/paste the column_constraint definition from the CREATE TABLE synopsis

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tom Lane
Tomas Vondra writes: > On 10/30/2017 09:03 PM, Tom Lane wrote: >> [ scratches head ... ] Not sure how that could've introduced any >> problems? Will look. > Not sure, but I can confirm Michael's findings - I've been unable to > reproduce the issue on 1a4be103a5

Re: [HACKERS] proposal: schema variables

2017-10-30 Thread srielau
Pavel, I wouldn't put in the DROP option. Or at least not in that form of syntax. By convention CREATE persists DDL and makes object definitions visible across sessions. DECLARE defines session private objects which cannot collide with other sessions. If you want variables with a short lifetime

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tom Lane
Michael Paquier writes: > b1328d78f is close enough, but per what I see that's actually not > true. I have been able to reproduce the problem, which shows up within > a window of 2-4 minutes. Still sometimes it can take way longer, and I > spotted one test where it took

Re: [HACKERS] WIP: BRIN multi-range indexes

2017-10-30 Thread Tomas Vondra
Hi, attached is a patch series that includes both the BRIN multi-range minmax indexes discussed in this thread, and the BRIN bloom indexes initially posted in [1]. It seems easier to just deal with a single patch series, although we may end up adding just one of those proposed opclasses. There

Re: [HACKERS] [POC] hash partitioning

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 5:52 PM, amul sul wrote: > Actually, int4[] is also inappropriate type as we have started using a 64bit > hash function. We need something int8[] which is not available, so that I > have used ANYARRAYOID in the attached patch(0004). I don't know why

Re: [HACKERS] [POC] hash partitioning

2017-10-30 Thread amul sul
On Tue, Oct 31, 2017 at 9:54 AM, Robert Haas wrote: > On Mon, Oct 30, 2017 at 5:52 PM, amul sul wrote: >> Actually, int4[] is also inappropriate type as we have started using a 64bit >> hash function. We need something int8[] which is not available, so

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Alvaro Herrera
Thanks everyone for the analysis downthread. Here's a test case that provokes the crash faster. Initialize with create table brin_test (a text); create index on brin_test using brin (a) with (pages_per_range = 1); Then in one psql, run this: select brin_summarize_new_values('brin_test_a_idx')

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Alvaro Herrera
Tom Lane wrote: > So: I put the blame on the fact that summarize_range() thinks that > the tuple offset it has for the placeholder tuple is guaranteed to > hold good, even across possibly-long intervals where it's holding > no lock on the containing buffer. Yeah, I think this is a pretty

Re: [HACKERS] Parallel Hash take II

2017-10-30 Thread Thomas Munro
On Tue, Aug 1, 2017 at 9:28 AM, Andres Freund wrote: > On 2017-07-26 20:12:56 +1200, Thomas Munro wrote: >> I'll report on performance separately. > > Looking forward to that ;) Here are some experimental results from a Xeon E5-2695 v3 with a ton of RAM and spinning disks

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> So: I put the blame on the fact that summarize_range() thinks that >> the tuple offset it has for the placeholder tuple is guaranteed to >> hold good, even across possibly-long intervals where it's holding >> no lock on the

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 11:24 PM, Andres Freund wrote: >> > Perhaps it should rather be pg_add_s32_overflow, or a similar >> > naming scheme? >> >> Not sure what the s is supposed to be? Signed? > > Yes, signed. So we could add a u32 or something complementing the

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-10-30 Thread Masahiko Sawada
On Mon, Oct 30, 2017 at 10:16 PM, Robert Haas wrote: > On Tue, Oct 24, 2017 at 1:26 PM, Ivan Kartyshov > wrote: >> Hello. I made some bugfixes and rewrite the patch. > > I don't think it's a good idea to deliberately leave the state of the >

Re: [HACKERS] Re: protocol version negotiation (Re: Libpq PGRES_COPY_BOTH - version compatibility)

2017-10-30 Thread Badrul Chowdhury
Hi Robert, Thank you for the comprehensive review! We are very much in the early stages of contributing to the PG community and we clearly have lots to learn, but we look forward to becoming proficient and active members of the pg community. Regarding the patch, I have tested it extensively by

Re: [HACKERS] How to implement a SP-GiST index as a extension module?

2017-10-30 Thread Connor Wolf
I was mostly unclear on how I'd go about attaching the extension functions to the relevant indexing mechanism. From the looks of the vptree.tar.gz file (which is really, *really* helpful, incidentally!), a it's done via a custom operator class, which then gets passed to the actual index creation

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Tom Lane
I wrote: > Hmm. The index offnum being complained of is one past the end of the > lp array. I think I see what about that commit changed the behavior: > the old code for PageIndexDeleteNoCompact never changed the length > of the lp array, except in the corner case where the page is becoming >

[HACKERS] Remove inbound links to sql-createuser

2017-10-30 Thread David G. Johnston
Since CREATE USER is officially an alias for CREATE ROLE other parts of the documentation should point to CREATE ROLE, not CREATE USER. Most do but I noticed when looking at CREATE DATABASE that it did not. Further searching turned up the usage in client-auth.sgml. That one is questionable

[HACKERS] [PATCH] Comment typo in get_collation_name() comment

2017-10-30 Thread Dagfinn Ilmari Mannsåker
Hi Hackers, The comment for get_collation_name() seems to have been copy-pasted from get_constraint_name(), but missed one s/constraint/collation/. Patch attached. -- "I use RMS as a guide in the same way that a boat captain would use a lighthouse. It's good to know where it is, but you

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Peter Geoghegan
On Mon, Oct 30, 2017 at 6:21 AM, Simon Riggs wrote: >> That's not really true. Nobody's going to be happy if MERGE has one >> behavior in one set of cases and an astonishingly different behavior >> in another set of cases. If you adopt a behavior for certain cases >> that

Re: [HACKERS] Linking libpq statically to libssl

2017-10-30 Thread Stephen Frost
Daniele, * Daniele Varrazzo (daniele.varra...@gmail.com) wrote: > On Fri, Oct 27, 2017 at 2:37 PM, Tom Lane wrote: > > Daniele Varrazzo writes: > >> I have a problem building binary packages for psycopg2. Binary > >> packages ship with their own

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 18:59, Peter Geoghegan wrote: > It is most emphatically *not* just a "small matter of programming". Please explain in detail the MERGE SQL statements that you think will be problematic and why. -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 4:57 PM, Andres Freund wrote: > 0001) Introduces pg_{add,sub,mul}{16,32,64}_overflow(a, b, *result) > These use compiler intrinsics on gcc/clang. If that's not > available, they cast to a wider type and to overflow checks. For > 64bit

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-30 Thread Dagfinn Ilmari Mannsåker
Robert Haas writes: >> 0003) Removes -fwrapv. I'm *NOT* suggesting we apply this right now, but >> it seems like an important test for the new facilities. Without >> 0002, tests would fail after this, after it all tests run >> successfully. > > I suggest

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Peter Geoghegan
On Mon, Oct 30, 2017 at 11:07 AM, Simon Riggs wrote: > Please explain in detail the MERGE SQL statements that you think will > be problematic and why. Your proposal is totally incomplete, so I can only surmise its behavior in certain cases, to make a guess at what the

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Nico Williams
On Mon, Oct 30, 2017 at 10:59:43AM -0700, Peter Geoghegan wrote: > On Mon, Oct 30, 2017 at 6:21 AM, Simon Riggs wrote: > > If a general purpose solution exists, please explain what it is. > > For the umpteenth time, a general purpose solution is one that more or > less

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Oct 29, 2017 at 1:19 AM, Simon Riggs wrote: > > Nothing I am proposing blocks later work. > > That's not really true. Nobody's going to be happy if MERGE has one > behavior in one set of cases and an astonishingly

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 9:42 AM, Alvaro Herrera wrote: > Tomas Vondra wrote: >> FWIW I can reproduce this on REL_10_STABLE, but not on REL9_6_STABLE. So >> it seems to be due to something that changed in the last release. > > I've been trying to reproduce it for half an

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-10-30 Thread Ivan Kartyshov
Ants Aasma писал 2017-10-26 17:29: On Mon, Oct 23, 2017 at 12:29 PM, Ivan Kartyshov wrote: Ants Aasma писал 2017-09-26 13:00: Exposing this interface as WAITLSN will encode that visibility order matches LSN order. This removes any chance of fixing for example

Re: [HACKERS] Current int & float overflow checking is slow.

2017-10-30 Thread Andres Freund
Hi, On 2017-10-30 22:29:42 +0530, Robert Haas wrote: > On Mon, Oct 30, 2017 at 4:57 PM, Andres Freund wrote: > > 0001) Introduces pg_{add,sub,mul}{16,32,64}_overflow(a, b, *result) > > These use compiler intrinsics on gcc/clang. If that's not > > available, they

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 15:22, Simon Riggs wrote: >> You forgot to update this formula in xlog.c: >> distance = (2.0 + CheckPointCompletionTarget) * >> CheckPointDistanceEstimate; >> /* add 10% for good measure. */ >> distance *= 1.10; >> You can guess easily