Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2017-10-31 Thread Stephen Frost
Greetings, * Lætitia Avrot (laetitia.av...@gmail.com) wrote: > As Amit Langot pointed out, the column_constraint definition is missing > whereas it is used in ALTER TABLE synopsis. It can be easily found in the > CREATE TABLE synopsis, but it's not very user friendly. Agreed. > You will find

Re: [HACKERS] Patch: restrict pg_rewind to whitelisted directories

2017-10-31 Thread Robert Haas
On Mon, Oct 30, 2017 at 6:44 PM, Chris Travers wrote: > The attached patch is cleaned up and filed for the commit fest this next > month: It's generally better to post the patch on the same message as the discussion thread, or at least link back to the discussion thread

[HACKERS] Add some const decorations to prototypes

2017-10-31 Thread Peter Eisentraut
Here is a patch that adds const decorations to many char * arguments in functions. It should have no impact otherwise; there are very few code changes caused by it. Some functions have a strtol()-like behavior where they take in a const char * and return a pointer into that as another argument.

Re: [HACKERS] Remove inbound links to sql-createuser

2017-10-31 Thread Stephen Frost
David, * David G. Johnston (david.g.johns...@gmail.com) wrote: > Since CREATE USER is officially an alias for CREATE ROLE other parts of the > documentation should point to CREATE ROLE, not CREATE USER. Most do but I > noticed when looking at CREATE DATABASE that it did not. Further searching >

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-10-31 Thread Ants Aasma
On Mon, Oct 30, 2017 at 7:25 PM, Ivan Kartyshov wrote: > It sounds reasonable. I can offer the following version. > > WAIT LSN lsn_number; > WAIT LSN lsn_number TIMEOUT delay; > WAIT LSN lsn_number INFINITE; > WAIT LSN lsn_number NOWAIT; > > > WAIT [token]

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tom Lane
Michael Paquier writes: > On Tue, Oct 31, 2017 at 4:56 AM, Tom Lane wrote: >> Yeah, we're still missing an understanding of why we didn't see it >> before; the inadequate locking was surely there before. > Because 24992c6d has added a check on the

Re: [HACKERS] Query regarding permission on table_column%type access

2017-10-31 Thread Stephen Frost
Greetings, * Neha Sharma (neha.sha...@enterprisedb.com) wrote: > I have observed that even if the user does not have permission on a > table(created in by some other user),the function parameter still can have > a parameter of that table_column%type. This is because the creation of the table

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Simon Riggs
On 31 October 2017 at 12:56, Stephen Frost wrote: > Simon, > > * Simon Riggs (si...@2ndquadrant.com) wrote: >> On 30 October 2017 at 19:55, Stephen Frost wrote: >> > I don't think MERGE should be radically different from other database >> > systems and

Re: [HACKERS] Remove secondary checkpoint

2017-10-31 Thread Simon Riggs
On 30 October 2017 at 18:58, Simon Riggs wrote: > On 30 October 2017 at 15:22, Simon Riggs wrote: > >>> You forgot to update this formula in xlog.c: >>> distance = (2.0 + CheckPointCompletionTarget) * >>> CheckPointDistanceEstimate; >>> /*

Re: [HACKERS] Query regarding permission on table_column%type access

2017-10-31 Thread Tom Lane
Stephen Frost writes: > * Neha Sharma (neha.sha...@enterprisedb.com) wrote: >> I have observed that even if the user does not have permission on a >> table(created in by some other user),the function parameter still can have >> a parameter of that table_column%type. > This is

Re: [HACKERS] SQL procedures

2017-10-31 Thread Simon Riggs
On 31 October 2017 at 18:23, Peter Eisentraut wrote: > I've been working on SQL procedures. (Some might call them "stored > procedures", but I'm not aware of any procedures that are not stored, so > that's not a term that I'm using here.) I guess that the DO

Re: [HACKERS] Statement-level rollback

2017-10-31 Thread MauMau
From: Simon Riggs On 14 August 2017 at 23:58, Peter Eisentraut wrote: > On 2/28/17 02:39, Tsunakawa, Takayuki wrote: >> The code for stored functions is not written yet, but I'd like your feedback for the specification and design based on the current patch. I'll

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Peter Geoghegan
On Tue, Oct 31, 2017 at 2:25 AM, Simon Riggs wrote: > If there are challenges ahead, its reasonable to ask for test cases > for that now especially if you think you know what they already are. > Imagine we go forwards 2 months - if you dislike my patch when it > exists you

Re: [HACKERS] Fix dumping pre-10 DBs by pg_dump10 if table "name" exists

2017-10-31 Thread Vitaly Burovoy
On 10/31/17, Tom Lane wrote: > Yeah, there are quite a few unqualified casts in pg_dump, but AFAICS > all the rest are OK because the search_path is just pg_catalog. > > But I did find psql's describe.c making a similar mistake :-(. > Pushed that along with your fix. > >

Re: [HACKERS] Remove inbound links to sql-createuser

2017-10-31 Thread Stephen Frost
David, * Stephen Frost (sfr...@snowman.net) wrote: > * David G. Johnston (david.g.johns...@gmail.com) wrote: > > Since CREATE USER is officially an alias for CREATE ROLE other parts of the > > documentation should point to CREATE ROLE, not CREATE USER. Most do but I > > noticed when looking at

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tom Lane
So in a few more runs this morning using Alvaro's simplified test case, I have seen the following behaviors not previously reported: 1. Crashes in PageIndexTupleOverwrite, which has the same "invalid index offnum: %u" error report as PageIndexTupleDeleteNoCompact. I note the same message appears

Re: [HACKERS] Remove secondary checkpoint

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 2:00 PM, Simon Riggs wrote: > On 30 October 2017 at 18:58, Simon Riggs wrote: >> On 30 October 2017 at 15:22, Simon Riggs wrote: >> You forgot to update this formula in xlog.c: distance =

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Alvaro Herrera
Tom Lane wrote: > So in a few more runs this morning using Alvaro's simplified test case, > I have seen the following behaviors not previously reported: > 1. Crashes in PageIndexTupleOverwrite, which has the same "invalid index > offnum: %u" error report as PageIndexTupleDeleteNoCompact. I note

Re: [HACKERS] Fix dumping pre-10 DBs by pg_dump10 if table "name" exists

2017-10-31 Thread Tom Lane
Vitaly Burovoy writes: > I left an other "NULL::name AS rolname" at > src/bin/pg_dump/pg_dump.c:2978 because can't check (remoteVersion < > 9) it and it is under strict "selectSourceSchema(fout, > "pg_catalog");" schema set. Yeah, there are quite a few unqualified

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Tomas Vondra
Hi, On 10/31/2017 04:48 PM, Greg Stark wrote: > On 31 October 2017 at 07:05, Chris Travers wrote: >> Hi; >> >> After Andres's excellent talk at PGConf we tried benchmarking >> effective_io_concurrency on some of our servers and found that those which >> have a number of

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Alvaro Herrera
Tom Lane wrote: > I really don't understand how any of this "let's release the buffer > lock and then take it back later" logic is supposed to work reliably. So summarize_range first inserts the placeholder tuple, which is there purposefully for other processes to update concurrently; then,

Re: [HACKERS] SQL procedures

2017-10-31 Thread Tom Lane
Peter Eisentraut writes: > I've been working on SQL procedures. No comment yet on the big picture here, but ... > The provided procedural languages (an ever more > confusing term) each needed a small touch-up to handle pg_proc entries > with prorettype == 0.

Re: [HACKERS] Add some const decorations to prototypes

2017-10-31 Thread Tom Lane
Peter Eisentraut writes: > Here is a patch that adds const decorations to many char * arguments in > functions. It should have no impact otherwise; there are very few code > changes caused by it. +1 in general ... > Some functions have a strtol()-like behavior

[HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Greg Stark
On 31 October 2017 at 07:05, Chris Travers wrote: > Hi; > > After Andres's excellent talk at PGConf we tried benchmarking > effective_io_concurrency on some of our servers and found that those which > have a number of NVME storage volumes could not fill the I/O queue

Re: [HACKERS] Fix dumping pre-10 DBs by pg_dump10 if table "name" exists

2017-10-31 Thread Vitaly Burovoy
On 10/31/17, Tom Lane wrote: > Vitaly Burovoy writes: >> Recently my colleagues found a bug. > >> - "SELECT 'bigint'::name AS >> sequence_type, " >> + "SELECT

Re: [HACKERS] Fix dumping pre-10 DBs by pg_dump10 if table "name" exists

2017-10-31 Thread Tom Lane
Vitaly Burovoy writes: > Recently my colleagues found a bug. > - "SELECT 'bigint'::name AS > sequence_type, " > + "SELECT > 'bigint'::pg_catalog.name AS sequence_type, Good

Re: [HACKERS] [PATCH] Generic type subscripting

2017-10-31 Thread Arthur Zakirov
On Sun, Oct 29, 2017 at 10:56:19PM +0100, Dmitry Dolgov wrote: > > So, here is the new version of patch that contains modifications we've > discussed, namely: > > * store oids of `parse`, `fetch` and `assign` functions > > * introduce dependencies from a data type > > * as a side effect of

[HACKERS] Consistently catch errors from Python _New() functions

2017-10-31 Thread Peter Eisentraut
While reviewing some unrelated code, I noticed that we are handling error conditions from Python API functions such as PyList_New() and PyDict_New() in pretty random ways or not at all. Here is a patch to fix that. Arguably, this is a bug fix, but I'm not sure whether it's worth meddling with

[HACKERS] postgres_fdw: Add support for INSERT OVERRIDING clause

2017-10-31 Thread Peter Eisentraut
It has been pointed out to me that the command deparsing in postgres_fdw does not support the INSERT OVERRIDING clause that was added in PG10. Here is a patch that seems to fix that. I don't know much about this, whether anything else needs to be added or whether there should be tests. Perhaps

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> I really don't understand how any of this "let's release the buffer >> lock and then take it back later" logic is supposed to work reliably. > So summarize_range first inserts the placeholder tuple, which is there >

Re: [HACKERS] SQL procedures

2017-10-31 Thread Pavel Stehule
2017-10-31 18:23 GMT+01:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > I've been working on SQL procedures. (Some might call them "stored > procedures", but I'm not aware of any procedures that are not stored, so > that's not a term that I'm using here.) > > Everything that follows

[HACKERS] Transaction control in procedures

2017-10-31 Thread Peter Eisentraut
Here is a patch that implements transaction control in PL/Python procedures. (This patch goes on top of "SQL procedures" patch v1.) So you can do this: CREATE PROCEDURE transaction_test1() LANGUAGE plpythonu AS $$ for i in range(0, 10): plpy.execute("INSERT INTO test1 (a) VALUES (%d)" % i)

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tom Lane
I wrote: > maybe > we just have some run-of-the-mill bugs to find, like the off-the-end > bug I spotted in brin_doupdate. There's apparently at least one > more, but given the error message it must be something like not > checking for a page to have turned into a revmap page. Shouldn't > be too

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-10-31 Thread Shubham Barai
On 9 October 2017 at 18:57, Alexander Korotkov wrote: > On Thu, Oct 5, 2017 at 9:48 PM, Shubham Barai > wrote: > >> On 3 October 2017 at 00:32, Alexander Korotkov > > wrote: >> >>> On Mon, Oct 2, 2017 at 9:11 PM,

Re: [HACKERS] PostgreSQL 10 parenthesized single-column updates can produce errors

2017-10-31 Thread Rob McColl
Attaching patch... :-/ On Tue, Oct 31, 2017 at 4:27 PM, Rob McColl wrote: > Between 9.6.5 and 10, the handling of parenthesized single-column UPDATE > statements changed. In 9.6.5, they were treated identically to > unparenthesized single-column UPDATES. In 10, they are

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Tomas Vondra
Hi, On 10/20/2017 03:23 PM, Robert Haas wrote: > > ... > > The main points I want to make clearly understood is the current > design relies on (1) functions being labeled correctly and (2) other > dangerous code paths being unreachable because there's nothing that > runs between

Re: [HACKERS] WIP: Restricting pg_rewind to data/wal dirs

2017-10-31 Thread David Steele
On 10/30/17 6:36 AM, Michael Paquier wrote: > On Mon, Oct 30, 2017 at 10:15 AM, Chris Travers >> >> How does rep mgr or other programs using pg_rewind know what to exclude? > > Good question. Answers could come from folks such as David Steele > (pgBackRest) or Marco (barman) whom I am attaching

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-10-31 Thread Peter Geoghegan
On Thu, Oct 26, 2017 at 4:22 AM, Rushabh Lathia wrote: > Attaching the re based patch according to the v22 parallel-hash patch sets I took a quick look at this today, and noticed a few issues: * make_name() is used to name files in sharedtuplestore.c, which is what is

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Gilles Darold
Le 31/10/2017 à 22:28, srielau a écrit : > Pavel, > > There is no > DECLARE TEMP CURSOR > or > DECLARE TEMP variable in PLpgSQL > and > CREATE TEMP TABLE has a different meaning from what I understand you > envision for variables. > > But maybe I'm mistaken. Your original post did not describe the

[HACKERS] Dynamic result sets from procedures

2017-10-31 Thread Peter Eisentraut
This patch is more of a demo of what could be done, not my primary focus, but if there is interest and some assistance, maybe we can make something out of it. This patch also goes on top of "SQL procedures" version 1. The purpose is to return multiple result sets from a procedure. This is, I

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Serge Rielau
Pavel, I can imagine, so DECLARE command will be introduced as short cut for CREATE TEMP VARIABLE, but in this moment I would not to open this topic. I afraid of bikeshedding and I hope so CREATE TEMP VAR is anough. Language is important because language stays. You choice of syntax will

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread srielau
Pavel, There is no DECLARE TEMP CURSOR or DECLARE TEMP variable in PLpgSQL and CREATE TEMP TABLE has a different meaning from what I understand you envision for variables. But maybe I'm mistaken. Your original post did not describe the entire syntax: CREATE [TEMP] VARIABLE [IF NOT EXISTS] name

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Pavel Stehule
Hi 2017-10-30 22:42 GMT+01:00 srielau : > Pavel, > > I wouldn't put in the DROP option. > Or at least not in that form of syntax. > > By convention CREATE persists DDL and makes object definitions visible > across sessions. > DECLARE defines session private objects which cannot

Re: [HACKERS] SQL procedures

2017-10-31 Thread Pavel Stehule
2017-10-31 18:23 GMT+01:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > I've been working on SQL procedures. (Some might call them "stored > procedures", but I'm not aware of any procedures that are not stored, so > that's not a term that I'm using here.) > > Everything that follows

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Pavel Stehule
2017-10-31 22:08 GMT+01:00 Serge Rielau : > Pavel, > > I can imagine, so DECLARE command will be introduced as short cut for > CREATE TEMP VARIABLE, but in this moment I would not to open this topic. I > afraid of bikeshedding and I hope so CREATE TEMP VAR is anough. > >

[HACKERS] Account for cost and selectivity of HAVING quals

2017-10-31 Thread Tom Lane
Pursuant to the discussion at https://www.postgresql.org/message-id/20171029112420.8920b5f...@mx.zeyos.com here's a patch to fix the planner so that eval costs and selectivity of HAVING quals are factored into the appropriate plan node numbers. Perhaps unsurprisingly, this doesn't change the

Re: [HACKERS] PostgreSQL 10 parenthesized single-column updates can produce errors

2017-10-31 Thread Tom Lane
Rob McColl writes: > Attaching patch... :-/ The reason why hacking your way to a backwards-compatible solution is a bad idea is that it breaks the SQL standard compliance we're trying to achieve here. According to the spec, the elements of a parenthesized SET list should be

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
Hi, On 10/31/2017 08:46 PM, Tom Lane wrote: > I wrote: >> maybe >> we just have some run-of-the-mill bugs to find, like the off-the-end >> bug I spotted in brin_doupdate. There's apparently at least one >> more, but given the error message it must be something like not >> checking for a page to

Re: [HACKERS] PostgreSQL 10 parenthesized single-column updates can produce errors

2017-10-31 Thread David G. Johnston
On Tue, Oct 31, 2017 at 3:14 PM, Rob McColl wrote: > >> I believe that this is not an intended change or behavior, but is instead >> an unintentional side effect of 906bfcad7ba7cb3863fe0e2a7810be8e3cd84fbd >> Improve handling of "UPDATE ... SET (column_list) =

Re: [HACKERS] PostgreSQL 10 parenthesized single-column updates can produce errors

2017-10-31 Thread David G. Johnston
On Tue, Oct 31, 2017 at 3:43 PM, Tom Lane wrote: > According to the spec, the elements of a parenthesized > SET list should be assigned from the fields of a composite RHS. If > there's just one element of the SET list, the RHS should be a single-field > composite value, and

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Gilles Darold
Le 31/10/2017 à 23:36, Gilles Darold a écrit : > Le 31/10/2017 à 22:28, srielau a écrit : >> Pavel, >> >> There is no >> DECLARE TEMP CURSOR >> or >> DECLARE TEMP variable in PLpgSQL >> and >> CREATE TEMP TABLE has a different meaning from what I understand you >> envision for variables. >> >> But

Re: [HACKERS] PostgreSQL 10 parenthesized single-column updates can produce errors

2017-10-31 Thread Tom Lane
"David G. Johnston" writes: > ​Definitely moderates my opinion in my concurrent emai​l...though > postponement is not strictly bad given the seeming frequency of the > existing problematic syntax in the wild already. Yeah, I'd hoped to get some capability extensions

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Tomas Vondra
On 10/31/2017 11:44 PM, Tomas Vondra wrote: > ... > Unfortunately, I think we still have a problem ... I've been wondering > if we end up producing correct indexes, so I've done a simple test. > > 1) create the table as before > > 2) let the insert + vacuum run for some time, to see if there

Re: [HACKERS] Account for cost and selectivity of HAVING quals

2017-10-31 Thread Tels
Moin, On Tue, October 31, 2017 5:45 pm, Tom Lane wrote: > Pursuant to the discussion at > https://www.postgresql.org/message-id/20171029112420.8920b5f...@mx.zeyos.com > here's a patch to fix the planner so that eval costs and selectivity of > HAVING quals are factored into the appropriate plan

[HACKERS] Proposal: generic WAL compression

2017-10-31 Thread Oleg Ivanov
Hackers, a few years ago generic WAL was proposed by Alexander Korotkov (https://www.postgresql.org/message-id/flat/CAPpHfdsXwZmojm6Dx%2BTJnpYk27kT4o7Ri6X_4OSWcByu1Rm%2BVA%40mail.gmail.com#capphfdsxwzmojm6dx+tjnpyk27kt4o7ri6x_4oswcbyu1rm...@mail.gmail.com). and was committed into PostgreSQL

Re: [HACKERS] Account for cost and selectivity of HAVING quals

2017-10-31 Thread David G. Johnston
On Tue, Oct 31, 2017 at 4:31 PM, Tels wrote: > > ​​ > That looks odd to me, it first uses output_tuples in a formula, then > overwrites the value with a new value. Should these lines be swapped? > ​IIUC it is correct: the additional total_cost comes from

Re: [HACKERS] Account for cost and selectivity of HAVING quals

2017-10-31 Thread Tom Lane
"David G. Johnston" writes: > On Tue, Oct 31, 2017 at 4:31 PM, Tels wrote: >> That looks odd to me, it first uses output_tuples in a formula, then >> overwrites the value with a new value. Should these lines be swapped? > ​IIUC it is

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-10-31 Thread Alexander Korotkov
On Sun, Oct 29, 2017 at 12:47 AM, Pavel Stehule wrote: > 2017-10-28 23:35 GMT+02:00 Alexander Korotkov : > >> On Sat, Oct 28, 2017 at 3:46 PM, Pavel Stehule >> wrote: >> >>> 2017-09-22 21:31 GMT+02:00 Pavel Stehule

Re: [HACKERS] Flexible configuration for full-text search

2017-10-31 Thread Emre Hasegeli
> I'm mostly happy with mentioned modifications, but I have few questions > to clarify some points. I will send new patch in week or two. I am glad you liked it. Though, I think we should get approval from more senior community members or committers about the syntax, before we put more effort to

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-10-31 Thread Alexander Korotkov
On Tue, Oct 31, 2017 at 5:16 AM, Masahiko Sawada wrote: > On Mon, Oct 30, 2017 at 10:16 PM, Robert Haas > wrote: > > On Tue, Oct 24, 2017 at 1:26 PM, Ivan Kartyshov > > wrote: > >> Hello. I made some bugfixes and rewrite

Re: [HACKERS] ERROR: MultiXactId 3268957 has not been created yet -- apparent wraparound after missused pg_resetxlogs

2017-10-31 Thread Robert Haas
On Mon, Oct 16, 2017 at 5:41 PM, alain radix wrote: > I’m facing a problem with a PostgreSQL 9.6.2 reporting this error when > selecting a table > > ERROR: MultiXactId 3268957 has not been created yet -- apparent wraparound > > The root cause is not a Postgres bug but a

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-10-31 Thread Robert Haas
On Mon, Oct 16, 2017 at 3:02 PM, Masahiko Sawada wrote: > I guess that is the patch I proposed. However I think that there still > is room for discussion because the patch cannot skip to cleanup vacuum > when aggressive vacuum, which is one of the situation that I really >

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-31 Thread Robert Haas
On Mon, Oct 16, 2017 at 5:03 PM, Ashutosh Bapat wrote: > set_append_rel_size() crashes when it encounters a partitioned table > with a dropped column. Dropped columns do not have any translations > saved in AppendInfo::translated_vars; the corresponding entry is

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Simon Riggs
On 30 October 2017 at 19:17, Peter Geoghegan wrote: > On Mon, Oct 30, 2017 at 11:07 AM, Simon Riggs wrote: >> Please explain in detail the MERGE SQL statements that you think will >> be problematic and why. > > Your proposal is totally incomplete, so I can

Re: [HACKERS] EXPLAIN (ANALYZE, BUFFERS) reports bogus temporary buffer reads

2017-10-31 Thread Robert Haas
On Tue, Oct 17, 2017 at 2:29 AM, Thomas Munro wrote: > Vik Fearing asked off-list why hash joins appear to read slightly more > temporary data than they write. The reason is that we notch up a > phantom block read when we hit the end of each file. Harmless but it

Re: [HACKERS] [bug fix] postgres.exe crashes with access violation on Windows while starting up

2017-10-31 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier > So you are basically ready to lose any message that could be pushed > here if there is no memory context? That does not sound like a good > trade-off to me. A static buffer does

Re: [HACKERS] path toward faster partition pruning

2017-10-31 Thread Amit Langote
Thanks for the test case. On 2017/10/30 17:09, Rajkumar Raghuwanshi wrote: > I am getting wrong output when default is sub-partitioned further, below is > a test case. > > CREATE TABLE lpd(a int, b varchar, c float) PARTITION BY LIST (a); > CREATE TABLE lpd_p1 PARTITION OF lpd FOR VALUES IN

[HACKERS] Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Chris Travers
Hi; After Andres's excellent talk at PGConf we tried benchmarking effective_io_concurrency on some of our servers and found that those which have a number of NVME storage volumes could not fill the I/O queue even at the maximum setting (1000). Before we start benchmarking patched versions of

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 4:56 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Tom Lane wrote: >>> So: I put the blame on the fact that summarize_range() thinks that >>> the tuple offset it has for the placeholder tuple is guaranteed to >>> hold good,

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Geoff Winkless
Can I add my 2c worth, as someone without a horse in the race, as it were, in the hope that telling me how I've got this wrong might clarify the argument a bit (or at least you can all start shouting at me rather than each other :) ) The point of merge is to allow you to choose to either INSERT

Re: [HACKERS] Restricting maximum keep segments by repslots

2017-10-31 Thread Kyotaro HORIGUCHI
Hello, this is a rebased version. It gets a change of the meaning of monitoring value along with rebasing. In previous version, the "live" column mysteriously predicts the necessary segments will be kept or lost by the next checkpoint and the "distance" offered a still more mysterious value. In

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-10-31 Thread Kyotaro HORIGUCHI
This is a rebased version of the patch. At Fri, 17 Mar 2017 14:23:13 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170317.142313.232290068.horiguchi.kyot...@lab.ntt.co.jp> > At Tue, 7 Mar 2017 19:23:14 -0800, David Steele

Re: [HACKERS] show "aggressive" or not in autovacuum logs

2017-10-31 Thread Kyotaro HORIGUCHI
At Thu, 26 Oct 2017 12:42:23 +0200, Robert Haas wrote in > On Thu, Oct 26, 2017 at 10:18 AM, Kyotaro HORIGUCHI > wrote: > > Thank you. I forgot that point. Changed them

Re: [HACKERS] More stats about skipped vacuums

2017-10-31 Thread Kyotaro HORIGUCHI
This is just a repost as a (true) new thread. At Mon, 30 Oct 2017 20:57:50 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20171030.205750.246076862.horiguchi.kyot...@lab.ntt.co.jp> > At Fri, 20 Oct 2017 19:15:16 +0900, Masahiko Sawada

Re: [HACKERS] [bug fix] postgres.exe crashes with access violation on Windows while starting up

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 6:59 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier >> So you are basically ready to lose any message that could be pushed >> here if

Re: [HACKERS] parallelize queries containing initplans

2017-10-31 Thread tushar
On 10/30/2017 01:36 PM, tushar wrote: On 10/30/2017 09:02 AM, Amit Kapila wrote: Thanks a lot Tushar for testing this patch.  In the latest patch, I have just rebased some comments, there is no code change, so I don't expect any change in behavior, but feel free to test it once again. Thanks

[HACKERS] Query regarding permission on table_column%type access

2017-10-31 Thread Neha Sharma
Hi, I have observed that even if the user does not have permission on a table(created in by some other user),the function parameter still can have a parameter of that table_column%type. Scenario: postgres=# create user u1 with login ; CREATE ROLE postgres=# create user u2 with login ; CREATE

Re: [HACKERS] Comment typo

2017-10-31 Thread Etsuro Fujita
On 2017/10/30 22:39, Magnus Hagander wrote: On Mon, Oct 30, 2017 at 3:49 AM, Etsuro Fujita > wrote: Here is a patch to fix a typo in a comment in partition.c: s/specificiation/specification/. Applied, thanks. Thank

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Simon Riggs
On 30 October 2017 at 19:55, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Sun, Oct 29, 2017 at 1:19 AM, Simon Riggs wrote: >> > Nothing I am proposing blocks later work. >> >> That's not really true. Nobody's going to be

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-31 Thread Stephen Frost
Simon, * Simon Riggs (si...@2ndquadrant.com) wrote: > On 30 October 2017 at 19:55, Stephen Frost wrote: > > I don't think MERGE should be radically different from other database > > systems and just syntax sugar over a capability we have. > > I've proposed a SQL Standard

[HACKERS] Fix dumping pre-10 DBs by pg_dump10 if table "name" exists

2017-10-31 Thread Vitaly Burovoy
Hello, hackers! Recently my colleagues found a bug. They could not migrate from PG9.5 to PG10 due to error during pg_upgrage (the same as in the "reproduce" part below). An investigation showed there is a table "name" in the same schema where the dumped sequence is located and the PG tries to

Re: [HACKERS] Deadlock in ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-10-31 Thread Masahiko Sawada
On Tue, Oct 24, 2017 at 7:13 PM, Konstantin Knizhnik wrote: > Parallel execution of ALTER SUBSCRIPTION REFRESH PUBLICATION at several > nodes may cause deadlock: > > knizhnik 1480 0.0 0.1 1417532 16496 ? Ss 20:01 0:00 postgres: > bgworker: logical

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-10-31 Thread Thomas Munro
On Wed, Nov 1, 2017 at 11:29 AM, Peter Geoghegan wrote: > On Thu, Oct 26, 2017 at 4:22 AM, Rushabh Lathia > wrote: >> Attaching the re based patch according to the v22 parallel-hash patch sets > > I took a quick look at this today, and noticed a few

Re: [HACKERS] Re: PANIC: invalid index offnum: 186 when processing BRIN indexes in VACUUM

2017-10-31 Thread Alvaro Herrera
Tomas Vondra wrote: > FWIW I can reproduce this on 9.5, and I don't even need to run the > UPDATE part. That is, INSERT + VACUUM running concurrently is enough to > produce broken BRIN indexes :-( Hmm, I'm pretty sure we stress-tested brin in pretty much the same way. But I see this misbehavior

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-10-31 Thread Peter Geoghegan
On Tue, Oct 31, 2017 at 5:07 PM, Thomas Munro wrote: > So that's this bit: > > + pg_itoa(worker, filename); > + lts->pfile = BufFileCreateShared(fileset, filename); > > ... and: > > + pg_itoa(i, filename); > + file = BufFileOpenShared(fileset, filename); Right. >

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-10-31 Thread Masahiko Sawada
On Tue, Oct 31, 2017 at 6:17 PM, Alexander Korotkov wrote: > On Tue, Oct 31, 2017 at 5:16 AM, Masahiko Sawada > wrote: >> >> On Mon, Oct 30, 2017 at 10:16 PM, Robert Haas >> wrote: >> > On Tue, Oct 24, 2017 at 1:26 PM,

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-10-31 Thread Andreas Karlsson
Here is a rebased version of the patch. Andreas diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml index a0ca2851e5..f8c59ea127 100644 --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -926,6 +926,7 @@ ERROR: could not serialize access due to read/write dependencies among

[HACKERS] Removing LEFT JOINs in more cases

2017-10-31 Thread David Rowley
Hackers, Normally we'll only ever remove a LEFT JOIN relation if it's unused and there's no possibility that the join would cause row duplication. To check that the join wouldn't cause row duplicate we make use of proofs, such as unique indexes, or for sub-queries, we make use of DISTINCT and

Re: [HACKERS] [bug fix] postgres.exe crashes with access violation on Windows while starting up

2017-10-31 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:michael.paqu...@gmail.com] > On Tue, Oct 31, 2017 at 6:59 AM, Tsunakawa, Takayuki > wrote: > > When CurrentMemoryContext is NULL, the message is logged with > ReportEventA(). This is similar to write_console(). > > My point is that

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Serge Rielau
" Although the syntax of CREATE TEMPORARY TABLE resembles that of the SQL standard, the effect is not the same. In the standard, temporary tables are defined just once and automatically exist (starting with empty contents) in every session that needs them. PostgreSQL instead requires each

Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2017-10-31 Thread Amit Langote
On 2017/10/31 21:31, Stephen Frost wrote: > * Lætitia Avrot (laetitia.av...@gmail.com) wrote: >> As Amit Langot pointed out, the column_constraint definition is missing >> whereas it is used in ALTER TABLE synopsis. It can be easily found in the >> CREATE TABLE synopsis, but it's not very user

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Craig Ringer
> Now, I agree this is somewhat more limited than I hoped for, but OTOH it > still solves the issue I initially aimed for (processing large query > results in efficient way). I don't quite understand this part. We already send results to the client in a stream unless it's something we have to

Re: [HACKERS] Dynamic result sets from procedures

2017-10-31 Thread Craig Ringer
On 1 November 2017 at 05:08, Peter Eisentraut wrote: > CREATE PROCEDURE pdrstest1() > LANGUAGE SQL > AS $$ > DECLARE c1 CURSOR WITH RETURN FOR SELECT * FROM cp_test2; > DECLARE c2 CURSOR WITH RETURN FOR SELECT * FROM cp_test3; > $$; > > CALL pdrstest1(); FWIW,

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Andres Freund
Hi, On 2017-10-31 18:47:06 +0100, Tomas Vondra wrote: > On 10/31/2017 04:48 PM, Greg Stark wrote: > > On 31 October 2017 at 07:05, Chris Travers > wrote: > >> Hi; > >> > >> After Andres's excellent talk at PGConf we tried benchmarking > >> effective_io_concurrency on

Re: [HACKERS] Re: Anyone have experience benchmarking very high effective_io_concurrency on NVME's?

2017-10-31 Thread Craig Ringer
On 1 November 2017 at 11:49, Andres Freund wrote: > Right. It'd probably be good to be a bit more adaptive here. But it's > hard to do with posix_fadvise - we'd need an operation that actually > notifies us of IO completion. If we were using, say, asynchronous > direct IO,

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Pavel Stehule
2017-10-31 22:28 GMT+01:00 srielau : > Pavel, > > There is no > DECLARE TEMP CURSOR > or > DECLARE TEMP variable in PLpgSQL > and > sure .. DECLARE TEMP has no sense, I talked about similarity DECLARE and CREATE TEMP CREATE TEMP TABLE has a different meaning from what I

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2017-10-31 Thread Robert Haas
On Wed, Oct 4, 2017 at 5:58 PM, Ashutosh Bapat wrote: > The view with WCO is local but the modification which violates WCO is > being made on remote server by a trigger on remote table. Trying to > control that doesn't seem to be a good idea, just like we can't >

Re: [HACKERS] proposal: schema variables

2017-10-31 Thread Pavel Stehule
2017-11-01 6:07 GMT+01:00 Serge Rielau : > "Although the syntax of CREATE TEMPORARY TABLE resembles that of the SQL > standard, the effect is not the same. In the standard, temporary tables are > defined just once and automatically exist (starting with empty contents) in > every