Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 07:11:10 -0700, Andres Freund wrote: > On 2016-09-02 09:05:35 -0500, Kevin Grittner wrote: > > On Fri, Sep 2, 2016 at 3:31 AM, Robert Haas wrote: > > > On Tue, Aug 23, 2016 at 3:10 AM, Andres Freund wrote: > > > > >> =# SELECT * FROM few,

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Andres Freund writes: > Oh, and we've previously re-added that based on > complaints. C.f. d543170f2fdd6d9845aaf91dc0f6be7a2bf0d9e7 (and others > IIRC). That one wasn't about row order per se, but I agree that people *will* bitch if we change the behavior, especially if we

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Andres Freund writes: > On 2016-09-02 10:20:42 -0400, Tom Lane wrote: >> ... ISTM all we >> need is that the SRF be on the inside of the join, which is automatic >> if it's LATERAL. > Right. But there's nothing to force a lateral reference to be there > intrinsically. I've

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 3:31 AM, Robert Haas wrote: > On Tue, Aug 23, 2016 at 3:10 AM, Andres Freund wrote: >> =# SELECT * FROM few, ROWS FROM(generate_series(1,3)); >> ┌┬─┐ >> │ id │ generate_series │ >> ├┼─┤ >>

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-09-02 Thread Tom Lane
Michael Paquier writes: > Before I send a new series of patches... There is one thing that I am > still troubled with: the compilation of pgcrypto. First from > contrib/pgcrypto/Makefile I am noticing the following issue with this > block: > CF_SRCS = $(if $(subst

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Kevin Grittner writes: > On Fri, Sep 2, 2016 at 9:51 AM, Tom Lane wrote: >> regression=# select *, generate_series(1,3) from int8_tbl; > I'm sure that you realize that running a query of that form twice > against a table with more than one heap page could

Re: [HACKERS] Tracking timezone abbreviation removals in the IANA tz database

2016-09-02 Thread Tom Lane
I wrote: > So I'm leaning to the just-remove-it answer for any deleted abbreviation > that relies on a dynamic definition. Names that never had more than one > UTC offset can remain in the tznames list. After a bit more thought and consumption of caffeine, I am thinking that that won't really be

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-02 Thread Jeevan Chalke
Hi, On Mon, Aug 29, 2016 at 7:25 AM, Kouhei Kaigai wrote: > Hello, > > The attached patch adds an optional callback to support special > optimization > if ForeignScan/CustomScan are located under the Limit node in plan-tree. > > Our sort node wisely switches the behavior

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Craig Ringer
On 2 September 2016 at 21:01, Craig Ringer wrote: > On 2 September 2016 at 20:38, Craig Ringer wrote: >> On 2 Sep. 2016 8:30 pm, "Simon Riggs" wrote: >>> >>> On 2 September 2016 at 13:16, Craig Ringer

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Kevin Grittner writes: > On Fri, Sep 2, 2016 at 9:25 AM, Tom Lane wrote: >> ORDER BY is not a useful suggestion when there is nothing >> you could order by to get the old behavior. > I'm apparently missing something, because I see a column with the >

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
Hi, Thanks for looking. On 2016-09-02 14:01:32 +0530, Robert Haas wrote: > > 6) SETOF record type functions cannot directly be used in ROWS FROM() - > >as ROWS FROM "expands" records returned by functions. When converting > >something like > > CREATE OR REPLACE FUNCTION

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Andres Freund writes: > On 2016-09-02 09:05:35 -0500, Kevin Grittner wrote: >>> In general, we've been skeptical about giving any guarantees about >>> result ordering. > Well, it's historically how we behaved for SRFs. I'm pretty sure that > people will be confused if >

Re: [HACKERS] Cache Hash Index meta page.

2016-09-02 Thread Jesper Pedersen
On 07/22/2016 06:02 AM, Mithun Cy wrote: I have created a patch to cache the meta page of Hash index in backend-private memory. This is to save reading the meta page buffer every time when we want to find the bucket page. In “_hash_first” call, we try to read meta page buffer twice just to make

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 10:20:42 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-09-02 09:05:35 -0500, Kevin Grittner wrote: > >>> In general, we've been skeptical about giving any guarantees about > >>> result ordering. > > > Well, it's historically how we behaved for SRFs.

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 10:10 AM, Tom Lane wrote: > Kevin Grittner writes: >> On Fri, Sep 2, 2016 at 9:51 AM, Tom Lane wrote: >>> regression=# select *, generate_series(1,3) from int8_tbl; > >> I'm sure that you realize that running a

Re: [HACKERS] [WIP] Patches to enable extraction state of query execution from external session

2016-09-02 Thread Maksim Milyutin
01.09.2016 18:58, Tom Lane пишет: Maksim Milyutin writes: On Tue, Aug 30, 2016 at 9:34 AM, Maksim Milyutin > wrote: Yes, but the problem is that nothing gives you the guarantee that at the moment you decide

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Petr Jelinek
On 02/09/16 15:46, Craig Ringer wrote: On 2 September 2016 at 21:01, Craig Ringer wrote: On 2 September 2016 at 20:38, Craig Ringer wrote: On 2 Sep. 2016 8:30 pm, "Simon Riggs" wrote: On 2 September 2016 at 13:16, Craig

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-09-02 Thread Aleksander Alekseev
> I'll mark this as "returned with feedback". I'd be happy to take a patch > that helps to reduce sanitizer complaints, but this seems to need some work. > > Aleksander, how did you run the sanitizer? I tried to build with clang > 4.0, with the -fsanitize=memory option, and ran "make >

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 10:31 AM, Andres Freund wrote: > On 2016-09-02 09:41:28 -0500, Kevin Grittner wrote: >> On Fri, Sep 2, 2016 at 9:25 AM, Tom Lane wrote: >> > Andres Freund writes: >> >> Oh, and we've previously re-added that

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 10:10 AM, Tom Lane wrote: > Also, before getting too high and mighty with users who expect > "select * from table" to produce rows in a predictable order, > you should reflect on the number of places in our regression > tests that assume exactly that

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Tom Lane
Andres Freund writes: > ... What we could do is to add efficient > ROWS FROM (..) WITH ORDINALITY ORDER bY ordinality; > support. Hm? regression=# explain select * from rows from (generate_series(1,10)) with ordinality order by ordinality;

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-09-02 Thread Peter Geoghegan
On Thu, Aug 18, 2016 at 5:14 PM, Peter Geoghegan wrote: > I'd certainly welcome that. There are Debian packages available from > the Github version of amcheck, which is otherwise practically > identical to the most recent version of the patch posted here: > >

Re: [HACKERS] What is the posix_memalign() equivalent for the PostgreSQL?

2016-09-02 Thread Tom Lane
Anderson Carniel writes: > I would like to know if the PostgreSQL provides a memory function that > allocates aligned memory. Not beyond MAXALIGN (although the shared-memory alloc functions do align to cacheline boundaries IIRC). > If not, according to your experience, is

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-09-02 Thread Michael Paquier
On Fri, Sep 2, 2016 at 7:57 PM, Heikki Linnakangas wrote: > I decided to split ip.c anyway. I'd like to keep the files in > src/common/ip.c as small as possible, so I think it makes sense to be quite > surgical when moving things there. I kept the pg_foreach_ifaddr() function >

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 09:05:35 -0500, Kevin Grittner wrote: > On Fri, Sep 2, 2016 at 3:31 AM, Robert Haas wrote: > > On Tue, Aug 23, 2016 at 3:10 AM, Andres Freund wrote: > > >> =# SELECT * FROM few, ROWS FROM(generate_series(1,3)); > >>

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 9:11 AM, Andres Freund wrote: > On 2016-09-02 09:05:35 -0500, Kevin Grittner wrote: >> On Fri, Sep 2, 2016 at 3:31 AM, Robert Haas wrote: >>> On Tue, Aug 23, 2016 at 3:10 AM, Andres Freund wrote: >> =#

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 9:25 AM, Tom Lane wrote: > Andres Freund writes: >> Oh, and we've previously re-added that based on >> complaints. C.f. d543170f2fdd6d9845aaf91dc0f6be7a2bf0d9e7 (and others >> IIRC). > > That one wasn't about row order per se, but I

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 9:51 AM, Tom Lane wrote: > regression=# select *, generate_series(1,3) from int8_tbl; I'm sure that you realize that running a query of that form twice against a table with more than one heap page could result in rows in a different order, even if no

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 09:41:28 -0500, Kevin Grittner wrote: > On Fri, Sep 2, 2016 at 9:25 AM, Tom Lane wrote: > > Andres Freund writes: > >> Oh, and we've previously re-added that based on > >> complaints. C.f. d543170f2fdd6d9845aaf91dc0f6be7a2bf0d9e7 (and others

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 10:34:54 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-09-02 10:20:42 -0400, Tom Lane wrote: > >> ... ISTM all we > >> need is that the SRF be on the inside of the join, which is automatic > >> if it's LATERAL. > > > Right. But there's nothing to

[HACKERS] PATCH: Avoid use of __attribute__ when building with old Sun compiler versions

2016-09-02 Thread Andy Grundman
Hi, The use of __attribute__ is currently enabled by checking for __SUNPRO_C. However, this compiler only added support for __attribute__ keywords in version 5.10 [1]. This patch adds a version check to only enable this for supported versions [2]. I have tested this with Sun C 5.8 on a Solaris 9

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 10:58:59 -0500, Kevin Grittner wrote: > If it has no significant performance impact to maintain the > historical order, then I have no problem with doing so. It's not really a runtime issue, it's just a question of how to nicely constraint the join order. There's no additional

Re: [HACKERS] Tracking timezone abbreviation removals in the IANA tz database

2016-09-02 Thread Tom Lane
I wrote: > So the idea I'm toying with (again, haven't tried to code this) is to say > that *if* we can match the abbreviation to something in the referenced > zone then we'll use that, but otherwise we fall back to treating the > abbreviation as a macro for the zone name. This turned out to

Re: [HACKERS] Index Onlys Scan for expressions

2016-09-02 Thread Ildar Musin
Hi Vladimir, On 23.08.2016 23:35, Vladimir Sitnikov wrote: Hi, I've tried your indexonlypatch5.patch against REL9_6_BETA3. Here are some results. TL;DR: 1) <> does not support index-only scan for index (type, upper(vc) varchar_pattern_ops). 3) <<(... where type=42 offset 0) where upper_vc

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-02 Thread Andres Freund
On 2016-09-02 10:50:17 +0200, Petr Jelinek wrote: > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as it does not really "stream" so makes sense to improve the > behavior there. As a side note, I would really like to have cursor-like SQL > interface

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 11:31 AM, Peter Geoghegan wrote: > On Thu, Aug 18, 2016 at 5:14 PM, Peter Geoghegan wrote: >> I'd certainly welcome that. There are Debian packages available from >> the Github version of amcheck, which is otherwise practically >>

Re: [HACKERS] What is the posix_memalign() equivalent for the PostgreSQL?

2016-09-02 Thread Andres Freund
On 2016-09-02 13:05:37 -0400, Tom Lane wrote: > Anderson Carniel writes: > > If not, according to your experience, is there a > > significance difference between the performance of the O_DIRECT or not? > > AFAIK, nobody's really bothered to measure whether that would be

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-09-02 Thread Peter Geoghegan
On Fri, Sep 2, 2016 at 11:16 AM, Peter Geoghegan wrote: > There are only tiny differences, which in any case you can see in the > commit log on Github. There is no reason why code review needs to > block on this V3, IMV. Also, I don't think that we need to include V2's tests now

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-09-02 Thread Kevin Grittner
On Fri, Sep 2, 2016 at 1:16 PM, Peter Geoghegan wrote: > On Fri, Sep 2, 2016 at 11:11 AM, Kevin Grittner wrote: >> I changed the CF entry to "Waiting on Author" pending that. (I was >> starting to review the patch for commit, so it's none to early to >>

Re: [HACKERS] Postgres abort found in 9.3.11

2016-09-02 Thread K S, Sandhya (Nokia - IN/Bangalore)
Hello Tom, Apologies for delayed reply. Our setup is a hot-standby architecture. This crash is occurring only on stand-by node. Postgres continues to run without any issues on active node. Postmaster is waiting for a start and is throwing this message. Aug 22 11:44:21.462555 info node-0

Re: [HACKERS] pgbench - allow to store select results into variables

2016-09-02 Thread Fabien COELHO
Hello Amit, This patch needs to be rebased because of commit 64710452 (on 2016-08-19). Here it is! -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 285608d..a879e59 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@

Re: [HACKERS] amcheck (B-Tree integrity checking tool)

2016-09-02 Thread Peter Geoghegan
On Fri, Sep 2, 2016 at 11:11 AM, Kevin Grittner wrote: > I changed the CF entry to "Waiting on Author" pending that. (I was > starting to review the patch for commit, so it's none to early to > mention that the last posted version is not what I should be > looking at.) There

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Andres Freund
On 2016-09-02 14:04:24 +0530, Robert Haas wrote: > On Sun, Aug 28, 2016 at 3:18 AM, Andres Freund wrote: > > 0003-Avoid-materializing-SRFs-in-the-FROM-list.patch > > To avoid performance regressions from moving SRFM_ValuePerCall SRFs to > > ROWS FROM, nodeFunctionscan.c

Re: [HACKERS] [PATCH] COPY vs \copy HINT

2016-09-02 Thread Tom Lane
Craig Ringer writes: > On 2 September 2016 at 04:28, Tom Lane wrote: >> 1. I don't really think the HINT is appropriate for the not-absolute-path >> case. > Why? If the user runs > # COPY sometable FROM 'localfile.csv' WITH (FORMAT CSV); > ERROR:

Re: [HACKERS] Logical Replication WIP

2016-09-02 Thread Peter Eisentraut
Review of 0001-Add-PUBLICATION-catalogs-and-DDL.patch: The new system catalog pg_publication_rel has columns pubid, relid, and does not use the customary column name prefixes. Maybe that is OK here. I can't actually think of a naming scheme that wouldn't make things worse. The hunk in patch

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-02 Thread Bruce Momjian
On Thu, Sep 1, 2016 at 01:01:35PM -0400, Peter Eisentraut wrote: > Maybe a couple of hooks could be useful to allow people to experiment > with this. But the hooks should be more general, as described above. > But I think a few GUC settings that can be adjusted at run time could be > sufficient

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-09-02 Thread Kevin Grittner
On Mon, Jul 18, 2016 at 3:54 PM, Emre Hasegeli wrote: > My progress so far is attached as 2 patches. First one introduces an > header file for adt/float.c. Second one refactors the geometric > operations. The first patch fails to apply due to bit-rot. That's easy enough to

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-09-02 Thread Michael Paquier
On Fri, Sep 2, 2016 at 10:59 PM, Tom Lane wrote: > Michael Paquier writes: >> Before I send a new series of patches... There is one thing that I am >> still troubled with: the compilation of pgcrypto. First from >> contrib/pgcrypto/Makefile I am

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-09-02 Thread Jim Nasby
On 8/29/16 7:34 PM, Andres Freund wrote: What if we left symlinks for the config files? Or perhaps even better, > provide a tool that will create them for people that actually need > them. See the thread around

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-02 Thread Tomas Vondra
Hi, I've subscribed to review this patch in the current CF, so let me start with a brief summary of the thread as it started more than a year ago. In general the thread is about hitting the 1GB allocation size limit (and 32-bit length in FE/BE protocol) in various ways: 1) attributes are

[HACKERS] (Comment)Bug in CteScanNext

2016-09-02 Thread Jim Nasby
In CteScanNext(): /* * If we are not at the end of the tuplestore, or are going backwards, try * to fetch a tuple from tuplestore. */ eof_tuplestore = tuplestore_ateof(tuplestorestate); if (!forward && eof_tuplestore) For the comment to be

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-02 Thread Jim Nasby
On 9/2/16 4:07 PM, Bruce Momjian wrote: Couldn't SQL sessions call a PL/Perl function that could query the OS and set max_parallel_workers_per_gather appropriately? I'd certainly like to see a greater ability to utilize "hooks" without resorting to C. "hooks" in quotes because while some

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-02 Thread Peter Geoghegan
On Thu, Sep 1, 2016 at 10:01 AM, Peter Eisentraut wrote: > On 8/16/16 3:39 AM, Haribabu Kommi wrote: >> Yes, we need to consider many parameters as a system load, not just only >> the CPU. Here I attached a POC patch that implements the CPU load >> calculation

Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

2016-09-02 Thread Magnus Hagander
On Fri, Sep 2, 2016 at 8:50 AM, Michael Paquier wrote: > On Fri, Sep 2, 2016 at 2:20 AM, Peter Eisentraut > wrote: > > On 5/13/16 2:39 AM, Michael Paquier wrote: > >> So, attached are two patches that apply on HEAD to address the

Re: [HACKERS] What is the posix_memalign() equivalent for the PostgreSQL?

2016-09-02 Thread Anderson Carniel
The unexpected errors are problems to free the memory that was previously allocated with posix_memalign. After a number of memory allocation and free calls, the next free call crashes the system. On the other hand, if I replace the posix_memalign and the free calls for the following malloc/free

Re: [HACKERS] autonomous transactions

2016-09-02 Thread Jim Nasby
On 9/2/16 3:45 AM, Robert Haas wrote: So my suggestion is that you pursue the work but change the name. That might make the plpgsql issues significantly easier to deal with as well, by making it very explicit that you're doing something with a completely separate connection. That would make

Re: [HACKERS] Issue with bgworker, SPI and pgstat_report_stat

2016-09-02 Thread Tomas Vondra
On 07/11/2016 06:53 AM, Craig Ringer wrote: On 11 July 2016 at 11:49, Michael Paquier > wrote: On Mon, Jul 11, 2016 at 3:36 AM, Julien Rouhaud > wrote: > I'm not

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-02 Thread Tomas Vondra
Hi Ivan, This patch needs a rebase, as 06d7fd6e bumped the version to 1.2. On 09/02/2016 01:38 PM, Ivan Kartyshov wrote: On 09/02/2016 06:01 AM, Robert Haas wrote: I wonder whether we ought to just switch from the consistent method to the semiconsistent method and call it good. I agree with

[HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-02 Thread Claudio Freire
The attached patch allows setting maintainance_work_mem or autovacuum_work_mem higher than 1GB (and be effective), by turning the allocation of the dead_tuples into a huge allocation. This results in fewer index scans for heavily bloated tables, and could be a lifesaver in many situations (in

Re: [HACKERS] Issue with bgworker, SPI and pgstat_report_stat

2016-09-02 Thread Michael Paquier
On Sat, Sep 3, 2016 at 10:02 AM, Tomas Vondra wrote: > In any case, I think adding the pgstat_report_stat() into worker_spi seems > like a reasonable (and backpatchable) fix. Doing just that sounds reasonable seen from here. I am wondering also if it would not be

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-02 Thread Tomas Vondra
On 09/03/2016 02:21 AM, Tomas Vondra wrote: A few minor comments regarding the patch: 1) CopyStartSend seems pretty pointless - It only has one function call in it, and is called on exactly one place (and all other places simply call allowLongStringInfo directly). I'd get rid of this function

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Simon Riggs
On 2 September 2016 at 13:16, Craig Ringer wrote: > So I've moved it to xlog.c... I'm pretty sure it shouldn't live in xlog.c, but there may be some good reason I can't see yet. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Craig Ringer
On 2 Sep. 2016 8:30 pm, "Simon Riggs" wrote: > > On 2 September 2016 at 13:16, Craig Ringer wrote: > > > So I've moved it to xlog.c... > > I'm pretty sure it shouldn't live in xlog.c, but there may be some > good reason I can't see yet. Ugh. Yes.

[HACKERS] Tracking timezone abbreviation removals in the IANA tz database

2016-09-02 Thread Tom Lane
It turns out that some of the zone abbreviations shown in the IANA timezone database don't correspond to any real-world usage, but were more or less made up by the timezone database crew. And of late, those folk have started to remove these invented abbreviations in favor of just using numeric

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Craig Ringer
On 2 September 2016 at 20:38, Craig Ringer wrote: > On 2 Sep. 2016 8:30 pm, "Simon Riggs" wrote: >> >> On 2 September 2016 at 13:16, Craig Ringer wrote: >> >> > So I've moved it to xlog.c... >> >> I'm pretty sure it shouldn't

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-02 Thread Craig Ringer
On 1 September 2016 at 21:19, Simon Riggs wrote: > I agree the doc patch should go in, though I suggest reword it > slightly, like attached patch. Thanks. The rewording looks good to me and I think that Doc-correction-each-change-once.v2.patch is ready for commit.

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Robert Haas
On Sun, Aug 28, 2016 at 3:18 AM, Andres Freund wrote: > 0003-Avoid-materializing-SRFs-in-the-FROM-list.patch > To avoid performance regressions from moving SRFM_ValuePerCall SRFs to > ROWS FROM, nodeFunctionscan.c needs to support not materializing > output. > > In my

Re: [HACKERS] autonomous transactions

2016-09-02 Thread Robert Haas
On Wed, Aug 31, 2016 at 7:20 AM, Peter Eisentraut wrote: > I would like to propose the attached patch implementing autonomous > transactions for discussion and review. I'm pretty skeptical of this approach. Like Greg Stark, Serge Rielau, and Constantin Pan, I

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-02 Thread Robert Haas
On Fri, Sep 2, 2016 at 8:49 AM, Andres Freund wrote: > On 2016-09-02 08:31:42 +0530, Robert Haas wrote: >> I wonder whether we ought to just switch from the consistent method to >> the semiconsistent method and call it good. > > +1. I think, before long, we're going to have to

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

2016-09-02 Thread Amit Langote
Hi Corey, Here are some comments and a review of the patch. On 2016/06/03 5:48, Corey Huinker wrote: > A while back, there was a push to make COPY gzip-aware. That didn't happen, > but COPY FROM PROGRAM did, and it scratches the same itch. > > I have a similar need, but with file_fdw foreign

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2016-09-02 Thread Robert Haas
On Tue, Aug 23, 2016 at 3:10 AM, Andres Freund wrote: > as noted in [1] I started hacking on removing the current implementation > of SRFs in the targetlist (tSRFs henceforth). IM discussion brought the > need for a description of the problem, need and approach to light.

Re: [HACKERS] Optimizing numeric SUM() aggregate

2016-09-02 Thread Heikki Linnakangas
On 08/03/2016 08:47 PM, Andrey Borodin wrote: 1. Currenlty overflow is carried every addition. I suggest that it is possibe to do it only every (INT32_MAX - INT32_MAX / NBASE) / (NBASE - 1) addition. Please note that with this overflow count it will be neccesary to check whether two

Re: [HACKERS] [PATCH] COPY vs \copy HINT

2016-09-02 Thread Christoph Berg
Re: Craig Ringer 2016-09-02 > I thought about that but figured it didn't really matter too much, > when thinking about examples like > > # COPY batch_demo FROM '/root/secret.csv' WITH (FORMAT CSV); > ERROR: could not open file

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Amit Langote
Hi, Some minor comments. + + + Autocommit cannot be set on inside a transaction, the ongoing + transaction has to be ended by entering COMMIT or + ROLLBACK before setting autocommit on. + + I guess: "cannot be set *to* on" and likewise

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-02 Thread Petr Jelinek
On 02/09/16 09:58, Craig Ringer wrote: On 1 September 2016 at 21:19, Simon Riggs wrote: I agree the doc patch should go in, though I suggest reword it slightly, like attached patch. Thanks. The rewording looks good to me and I think that

Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

2016-09-02 Thread Michael Paquier
On Fri, Sep 2, 2016 at 2:20 AM, Peter Eisentraut wrote: > On 5/13/16 2:39 AM, Michael Paquier wrote: >> So, attached are two patches that apply on HEAD to address the problem >> of pg_basebackup that does not sync the data it writes. As >> pg_basebackup cannot

Re: [HACKERS] autonomous transactions

2016-09-02 Thread Joel Jacobson
On Thu, Sep 1, 2016 at 8:38 PM, Andres Freund wrote: > > I'm not convinced this makes much sense. All FDWs, dblink etc. already > allow you do stuff outside of a transaction. You as a DBA can prevent FDWs from being used and dblink is an extension that you don't have to

Re: [HACKERS] [PATCH] COPY vs \copy HINT

2016-09-02 Thread Craig Ringer
On 2 September 2016 at 17:05, Christoph Berg wrote: > Re: Craig Ringer 2016-09-02 > >> I thought about that but figured it didn't really matter too much, >> when thinking about examples like >> >> # COPY

Re: [HACKERS] Declarative partitioning - another take

2016-09-02 Thread Amit Langote
On 2016/09/02 14:38, Ashutosh Bapat wrote: > Here's something I observed with your set of patches posted in June. I have > not checked the latest set of patches. So, if it's something fixed, please > ignore the mail and sorry for me being lazy. > > prt1 is partitioned table and it shows following

Re: [HACKERS] Exclude schema during pg_restore

2016-09-02 Thread Michael Banck
Am Donnerstag, den 01.09.2016, 21:39 -0400 schrieb Peter Eisentraut: > On 8/31/16 4:10 AM, Michael Banck wrote: > > attached is a small patch that adds an -N option to pg_restore, in order > > to exclude a schema, in addition to -n for the restriction to a schema. > > I think this is a good idea,

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2016-09-02 Thread Haribabu Kommi
On Wed, Aug 31, 2016 at 3:19 AM, Alvaro Herrera wrote: > Haribabu Kommi wrote: > >> Apart from the above, here are the following list of command tags that >> are generated in the code, I took only the first word of the command tag >> just to see how many categories

Re: [HACKERS] Declarative partitioning - another take

2016-09-02 Thread Amit Langote
On 2016/09/02 15:57, Ashutosh Bapat wrote: > On Fri, Sep 2, 2016 at 12:23 PM, Amit Langote wrote: >> Getting rid of the parent table in the append list by other means may be a >> way to go. We know that the table is empty and safe to just drop. >> > Ok. Does a constraint (1 = 2) or something like

Re: [HACKERS] Sample configuration files

2016-09-02 Thread Robert Haas
On Mon, Aug 29, 2016 at 7:04 AM, Vik Fearing wrote: > We have sample configuration files for postgresql.conf and > recovery.conf, but we do not have them for contrib modules. This patch > attempts to add them. > > Although the patch puts the sample configuration files in the

Re: [HACKERS] Declarative partitioning - another take

2016-09-02 Thread Ashutosh Bapat
On Fri, Sep 2, 2016 at 12:23 PM, Amit Langote wrote: > On 2016/09/02 15:22, Ashutosh Bapat wrote: > >> > >> > >>> 2. A combination of constraints on the partitions should be applicable > to > >>> the parent. We aren't doing that. > >> > >> How about on seeing that

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rushabh Lathia
I don't think patch is doing correct things: 1) postgres=# \set AUTOCOMMIT off postgres=# create table foo (a int ); CREATE TABLE postgres=# \set AUTOCOMMIT on Above test not throwing psql error, as you used strcmp(newval,"ON"). You should use pg_strcasecmp. 2) postgres=# \set AUTOCOMMIT OFF

Re: [HACKERS] Declarative partitioning - another take

2016-09-02 Thread Ashutosh Bapat
> > > > 2. A combination of constraints on the partitions should be applicable to > > the parent. We aren't doing that. > > How about on seeing that a RELOPT_OTHER_MEMBER_REL is partitioned parent > table, we can have get_relation_constraints() include a constant false > clause in the list of

Re: [HACKERS] Sample configuration files

2016-09-02 Thread Vik Fearing
On 09/02/2016 08:58 AM, Robert Haas wrote: > On Mon, Aug 29, 2016 at 7:04 AM, Vik Fearing wrote: >> We have sample configuration files for postgresql.conf and >> recovery.conf, but we do not have them for contrib modules. This patch >> attempts to add them. >> >> Although

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rahila Syed
Hello, Thank you for comments. >Above test not throwing psql error, as you used strcmp(newval,"ON"). You >should use pg_strcasecmp. Corrected in the attached. >Above error coming because in below code block change, you don't have check for >command (you should check opt0 for AUTOCOMMIT command)

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2016-09-02 Thread Michael Paquier
On Fri, Sep 2, 2016 at 2:44 AM, Peter Eisentraut wrote: > On 8/11/16 9:12 PM, Michael Paquier wrote: >> Note that pg_dump[all] and pg_upgrade already have safeguards against >> those things per the same routines putting quotes for execution as >> commands into

Re: [HACKERS] Declarative partitioning - another take

2016-09-02 Thread Amit Langote
On 2016/09/02 15:22, Ashutosh Bapat wrote: >> >> >>> 2. A combination of constraints on the partitions should be applicable to >>> the parent. We aren't doing that. >> >> How about on seeing that a RELOPT_OTHER_MEMBER_REL is partitioned parent >> table, we can have get_relation_constraints()

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-02 Thread Craig Ringer
On 2 September 2016 at 16:50, Petr Jelinek wrote: >> So here's a simpler patch that just dirties the slot when it's >> replayed something from it on the SQL interface [...] > > Okay that sounds reasonable, the SQL interface is already somewhat different > than walsender as

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rushabh Lathia
On Fri, Sep 2, 2016 at 1:12 PM, Rahila Syed wrote: > Hello, > > Thank you for comments. > > >Above test not throwing psql error, as you used strcmp(newval,"ON"). You > >should use pg_strcasecmp. > Corrected in the attached. > > >Above error coming because in below code

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rahila Syed
>Document doesn't say this changes are only for implicit BEGIN.. Correcting myself here. Not just implicit BEGIN, it will throw an error on \set AUTOCOMMIT inside any any transaction provided earlier value of AUTOCOMMIT was OFF. Probably the test in which you tried it was already ON. Thank you,

Re: [HACKERS] What is the posix_memalign() equivalent for the PostgreSQL?

2016-09-02 Thread Craig Ringer
On 2 September 2016 at 01:12, Anderson Carniel wrote: > Dear all, > > I am developing an extension for the PostgreSQL that write/read some > external files from the PostgreSQL. In order to write/read, I am using the > O_DIRECT flag and using the posix_memalign to allocate

Re: [HACKERS] Confusing docs about GetForeignUpperPaths in fdwhandler.sgml

2016-09-02 Thread Etsuro Fujita
Hi Robert, Thanks for the comments! On 2016/09/02 11:55, Robert Haas wrote: On Mon, Aug 1, 2016 at 5:44 PM, Etsuro Fujita wrote: I noticed that the following note about direct modification via GetForeignUpperPaths in fdwhandler.sgml is a bit confusing. We have

Re: [HACKERS] Re: PROPOSAL: make PostgreSQL sanitizers-friendly (and prevent information disclosure)

2016-09-02 Thread Heikki Linnakangas
On 08/20/2016 10:46 PM, Tom Lane wrote: Noah Misch writes: On Fri, Aug 19, 2016 at 07:22:02PM -0400, Tom Lane wrote: So maybe we ought to work towards taking those out? Maybe. It's a policy question boiling down to our willingness to spend cycles zeroing memory in order

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-09-02 Thread Craig Ringer
Here's another update to these patches. While working on support for logical decoding on standbys I noticed that I needed something like get_xid_in_recent_past(...) there too. So I've moved it to xlog.c as TransactionIdInRecentPast too and flipped its arguments to be more convenient. No other

[HACKERS] Materialized View Incremental Refresh right join

2016-09-02 Thread hariprasath nallasamy
Hi all Using logical decoding i set up a plugin to refresh materialized views incrementally(only the changes that are did after the previous refresh). I used the approach that oracle uses for fast refresh of materialized view using log table to record delta changes. Mostly i covered

Re: [HACKERS] Sample configuration files

2016-09-02 Thread Martín Marqués
El 02/09/16 a las 04:19, Vik Fearing escribió: > >> 2. But I'm not sure that this will actually be useful to people. It >> seems like it might just be one more thing for patch authors to >> maintain. I think that if somebody wants to set a parameter defined >> for a contrib module, it's easy

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-02 Thread Etsuro Fujita
Hi Ashutosh, On 2016/08/22 15:49, Ashutosh Bapat wrote: 1. deparsePlaceHolderVar looks odd - each of the deparse* function is named as deparse + . PlaceHolderVar is not a parser node, so no string is going to be parsed as a PlaceHolderVar. May be you want to name it as

  1   2   >