Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-16 Thread Eric Grinstein
I would also like to participate as a student. 2015-02-16 13:28 GMT-02:00 Dmitry Dolgov 9erthali...@gmail.com: I would like to participate as student. On 10 February 2015 at 03:52, Thom Brown t...@linux.com wrote: Hi all, Google Summer of Code 2015 is approaching. I'm intending on

[HACKERS] Query Rewrite with Postgres' materialized views

2015-02-16 Thread Eric Grinstein
Hello, Are there any plans for implementing query rewriting (i.e. allowing the optimizer to substitute materialized views for queries) in upcoming versions? I have recently seen this answer http://www.postgresql.org/message-id/200501041006.18735.j...@agliodbs.com, saying that query rewriting

Re: [HACKERS] Perl coding error in msvc build system?

2015-02-16 Thread Peter Eisentraut
This patch been reviewed by 4 people, resulting in 2 minor suggestions for changes (adding an m modifier, and removing a bogus last). It has 2 clear upvotes and 0 downvotes. I think it should be revised along the lines suggested and committed without further ado. My patch actually only

[HACKERS] Re: NULL checks of deferenced pointers in picksplit method of intarray

2015-02-16 Thread Kevin Grittner
Michael Paquier michael.paqu...@gmail.com wrote: Coverity is pointing out that _int_split.c has unnecessary checks for deferenced pointers in 5 places. Attached is a patch to adjust those things. Pushed. Thanks! Also, as far as I understood from this code, no elements manipulated are

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-02-16 Thread Andreas Karlsson
On 02/13/2015 02:07 PM, Michael Paquier wrote: Andreas, are you planning to continue working on this patch? Peter has provided some comments that are still unanswered. Yes, but I am quite busy right now. I will try to find time some time later this week. -- Andreas Karlsson -- Sent via

[HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-16 Thread Michael Paquier
Hi, In a lot of places in the code we have many structures doing declarations of the type foo[1] to emulate variable length arrays. Attached are a set of patches aimed at replacing that with FLEXIBLE_ARRAY_MEMBER to prevent potential failures that could be caused by compiler optimizations and

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-16 Thread Ilia Ivanicki
I'm interested in participating in gsoc 2015 as student. Sincerely, Ivanitskiy Ilya.

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-16 Thread Bruce Momjian
On Sun, Feb 15, 2015 at 11:36:50AM -0500, Tom Lane wrote: I wonder if we couldn't achieve largely the same positive effects through adding a simple transaction-level timeout option. That would be far easier for users to understand and manage, it would be trivial to allow specific high-value

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-16 Thread Jim Nasby
On 2/16/15 2:41 AM, Andres Freund wrote: Since you actually don't want cancellations for the long running reporting queries it very much might be sensible to switch to a more complicated version of HeapTupleSatisfiesVacuum() if there's longrunning queries. One that can recognize if rows are

Re: [HACKERS] Issue installing doc tools on OSX

2015-02-16 Thread Peter Eisentraut
On 2/15/15 9:23 PM, David Steele wrote: That seems a bit incredible, since port should be able to resolve the dependencies by itself. I suggest that this should be reported as a bug to MacPorts. Sure, that has been my experience, but the error message was very clear. Unfortunately I did

Re: [HACKERS] Issue installing doc tools on OSX

2015-02-16 Thread Florian Pflug
On Feb16, 2015, at 23:18 , Peter Eisentraut pete...@gmx.net wrote: On 2/15/15 9:23 PM, David Steele wrote: That seems a bit incredible, since port should be able to resolve the dependencies by itself. I suggest that this should be reported as a bug to MacPorts. Sure, that has been my

Re: [HACKERS] Commit fest 2015-12 enters money time

2015-02-16 Thread Kouhei Kaigai
I couldn't find operation to add new entry on the current CF. Go to the bottom of this CF: https://commitfest.postgresql.org/4/ Then click on New patch. I couldn't find the New patch link on both of IE and Chrome, even though I logged in. Did you do something special? Thanks, -- NEC OSS

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-16 Thread Andres Freund
On 2015-02-17 05:51:22 +0900, Michael Paquier wrote: - 0002 does the same for catalog tables - 0003 changes varlena in c.h. This is done as a separate patch because this needs some other modifications as variable-length things need to be placed at the end of structures, because of: --

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-16 Thread Peter Geoghegan
On Mon, Feb 16, 2015 at 12:00 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So INSERT ON CONFLICT IGNORE on a table with an exclusion constraint might fail. I don't like that. The point of having the command in the first place is to deal with concurrency issues. If it sometimes doesn't

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Andrew Dunstan
On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to have this info many times. I would still like to see a benchmark. Average of 3

Re: [HACKERS] Commit fest 2015-12 enters money time

2015-02-16 Thread Michael Paquier
On Tue, Feb 17, 2015 at 8:50 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: I couldn't find operation to add new entry on the current CF. Go to the bottom of this CF: https://commitfest.postgresql.org/4/ Then click on New patch. I couldn't find the New patch link on both of IE and Chrome,

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Petr Jelinek
On 21/01/15 17:32, Andrew Dunstan wrote: On 01/21/2015 11:21 AM, Arne Scheffer wrote: Why is it a bad thing to call the column stddev_samp analog to the aggregate function or make a note in the documentation, that the sample stddev is used to compute the solution? ... But I will add a

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Peter Geoghegan
On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to have this info many times. I would still like to see a benchmark. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Issue installing doc tools on OSX

2015-02-16 Thread David Steele
On 2/16/15 6:10 PM, Florian Pflug wrote: On Feb16, 2015, at 23:18 , Peter Eisentraut pete...@gmx.net wrote: On 2/15/15 9:23 PM, David Steele wrote: That seems a bit incredible, since port should be able to resolve the dependencies by itself. I suggest that this should be reported as a bug to

Re: [HACKERS] Commit fest 2015-12 enters money time

2015-02-16 Thread Michael Paquier
On Tue, Feb 17, 2015 at 7:39 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2015-02-16 12:25 GMT+09:00 Michael Paquier michael.paqu...@gmail.com: On Fri, Feb 13, 2015 at 10:06 AM, Michael Paquier michael.paqu...@gmail.com wrote: In order to move on to the next CF, I am going to go through all

Re: [HACKERS] Commit fest 2015-12 enters money time

2015-02-16 Thread Kouhei Kaigai
On Tue, Feb 17, 2015 at 8:50 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: I couldn't find operation to add new entry on the current CF. Go to the bottom of this CF: https://commitfest.postgresql.org/4/ Then click on New patch. I couldn't find the New patch link on both of IE and

Re: [HACKERS] Issue installing doc tools on OSX

2015-02-16 Thread Peter Eisentraut
On 2/16/15 6:10 PM, Florian Pflug wrote: BTW, why does the list of suggested packages include docbook-xml? That is used for building the man pages. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-16 Thread Andres Freund
On 2015-02-16 16:35:46 -0500, Bruce Momjian wrote: It seems we already have a mechanism in place that allows tuning of query cancel on standbys vs. preventing standby queries from seeing old data, specifically max_standby_streaming_delay/max_standby_archive_delay. We obsessed about how users

Re: [HACKERS] NULL checks of deferenced pointers in picksplit method of intarray

2015-02-16 Thread Michael Paquier
On Tue, Feb 17, 2015 at 6:49 AM, Kevin Grittner kgri...@ymail.com wrote: Michael Paquier michael.paqu...@gmail.com wrote: Coverity is pointing out that _int_split.c has unnecessary checks for deferenced pointers in 5 places. Attached is a patch to adjust those things. Pushed. Thanks!

[HACKERS] We do not need pg_promote_v4_to_v6_addr/mask

2015-02-16 Thread Tom Lane
We have some code in the server that attempts to match IPv4 address entries in pg_hba.conf to incoming connections that are in IPv6 protocol but have addresses in the range :::: (the IPv4-in-IPv6 subrange). As revealed by today's bug report from Hugo Osvaldo Barrera, this code has

Re: [HACKERS] Join push-down support for foreign tables

2015-02-16 Thread Kouhei Kaigai
Let me put some comments in addition to where you're checking now. [design issues] * Cost estimation Estimation and evaluation of cost for remote join query is not an obvious issue. In principle, local side cannot determine the cost to run remote join without remote EXPLAIN, because local side

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Petr Jelinek
On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to have this info many times. I would still like to see a benchmark. Average of 3 runs of read-only pgbench on my system all with

Re: [HACKERS] We do not need pg_promote_v4_to_v6_addr/mask

2015-02-16 Thread Andrew Dunstan
On 02/16/2015 09:07 PM, Tom Lane wrote: I wrote: We have some code in the server that attempts to match IPv4 address entries in pg_hba.conf to incoming connections that are in IPv6 protocol but have addresses in the range :::: (the IPv4-in-IPv6 subrange). As revealed by today's

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-16 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-02-17 05:51:22 +0900, Michael Paquier wrote: diff --git a/src/include/catalog/pg_authid.h b/src/include/catalog/pg_authid.h index e01e6aa..d8789a5 100644 --- a/src/include/catalog/pg_authid.h +++ b/src/include/catalog/pg_authid.h @@

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

2015-02-16 Thread Tom Lane
Attached is an 0.3 version, rebased over today's HEAD changes (applies to commit 9e3ad1aac52454569393a947c06be0d301749362 or later), and with some better logic for transferring expanded array values into and out of plpgsql functions. Using this example: create or replace function arraysetnum(n

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Petr Jelinek
On 17/02/15 03:03, Andrew Dunstan wrote: On 02/16/2015 08:57 PM, Andrew Dunstan wrote: On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished

Re: [HACKERS] We do not need pg_promote_v4_to_v6_addr/mask

2015-02-16 Thread Tom Lane
I wrote: We have some code in the server that attempts to match IPv4 address entries in pg_hba.conf to incoming connections that are in IPv6 protocol but have addresses in the range :::: (the IPv4-in-IPv6 subrange). As revealed by today's bug report from Hugo Osvaldo Barrera,

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-16 Thread Peter Geoghegan
On Mon, Feb 16, 2015 at 4:11 PM, Peter Geoghegan p...@heroku.com wrote: Jim Nasby said something about setting the HEAP_XMIN_INVALID hint bit. Maybe he is right...if that can be made to be reliable (always WAL-logged), it could be marginally better than setting xmin to invalidTransactionId.

Re: [HACKERS] We do not need pg_promote_v4_to_v6_addr/mask

2015-02-16 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 02/16/2015 09:07 PM, Tom Lane wrote: BTW, a bit of digging in the git logs and mail archives says that the code in question was originally added in 7.4 (commit 3c9bb8886df7d56a), in response to this discussion:

Re: [HACKERS] pg_basebackup -x/X doesn't play well with archive_mode wal_keep_segments

2015-02-16 Thread Fujii Masao
On Fri, Feb 13, 2015 at 9:18 AM, Andres Freund and...@2ndquadrant.com wrote: On 2015-02-12 11:44:05 -0800, Sergey Konoplev wrote: On Thu, Feb 12, 2015 at 11:40 AM, Andres Freund and...@2ndquadrant.com wrote: This obviously should not be the case. I'll have a look in a couple of hours.

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Andrew Dunstan
On 02/16/2015 08:57 PM, Andrew Dunstan wrote: On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to have this info many times. I would

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-16 Thread Petr Jelinek
On 17/02/15 02:57, Andrew Dunstan wrote: On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to have this info many times. I would still

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-16 Thread Kyotaro HORIGUCHI
Hello, I had a look on gram.y and found other syntaxes using WITH option clause. At Wed, 11 Feb 2015 14:34:17 -0600, Jim Nasby jim.na...@bluetreble.com wrote in 54dbbcc9.1020...@bluetreble.com I suspect at least some of this stems from how command line programs tend to process options before

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-16 Thread Thom Brown
On 13 February 2015 at 15:40, Thom Brown t...@linux.com wrote: On 12 February 2015 at 14:55, Alexander Korotkov aekorot...@gmail.com wrote: Hi! On Mon, Feb 9, 2015 at 11:52 PM, Thom Brown t...@linux.com wrote: Google Summer of Code 2015 is approaching. I'm intending on registering

Re: [HACKERS] Commit fest 2015-12 enters money time

2015-02-16 Thread Kohei KaiGai
2015-02-16 12:25 GMT+09:00 Michael Paquier michael.paqu...@gmail.com: On Fri, Feb 13, 2015 at 10:06 AM, Michael Paquier michael.paqu...@gmail.com wrote: In order to move on to the next CF, I am going to go through all the remaining patches and update their status accordingly. And sorry for

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-16 Thread Magnus Hagander
On Feb 17, 2015 12:26 AM, Andres Freund and...@2ndquadrant.com wrote: On 2015-02-16 16:35:46 -0500, Bruce Momjian wrote: It seems we already have a mechanism in place that allows tuning of query cancel on standbys vs. preventing standby queries from seeing old data, specifically

Re: [HACKERS] Really bad blowups with hash outer join and nulls

2015-02-16 Thread Tomas Vondra
On 16.2.2015 03:38, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas Improving the estimates is always good, but it's not going to Tomas fix the case of non-NULL values (it shouldn't be all that Tomas difficult to create such examples with a value whose

Re: [HACKERS] GSoC 2015 - mentors, students and admins.

2015-02-16 Thread Dmitry Dolgov
I would like to participate as student. On 10 February 2015 at 03:52, Thom Brown t...@linux.com wrote: Hi all, Google Summer of Code 2015 is approaching. I'm intending on registering PostgreSQL again this year. Before I do that, I'd like to have an idea of how many people are interested

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-16 Thread Andres Freund
On 2015-02-16 05:19:11 +, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Jim Nasby jim.na...@bluetreble.com writes: On 2/15/15 10:36 AM, Tom Lane wrote: I wonder if we couldn't achieve largely the same positive effects through adding a simple transaction-level timeout

Re: [HACKERS] Replication identifiers, take 4

2015-02-16 Thread Andres Freund
On 2015-02-16 11:07:09 +0200, Heikki Linnakangas wrote: On 02/16/2015 02:21 AM, Andres Freund wrote: Furthermore the fact that the origin of records is recorded allows to avoid decoding them in logical decoding. That has both efficiency advantages (we can do so before they are stored in

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-16 Thread Andres Freund
On 2015-02-16 10:00:24 +0200, Heikki Linnakangas wrote: On 02/16/2015 02:44 AM, Peter Geoghegan wrote: Are we happy with acquiring the SpeculativeInsertLock heavy-weight lock for every insertion? That seems bad for performance reasons. Also, are we happy with adding the new fields to the proc

Re: [HACKERS] Replication identifiers, take 4

2015-02-16 Thread Heikki Linnakangas
On 02/16/2015 02:21 AM, Andres Freund wrote: Furthermore the fact that the origin of records is recorded allows to avoid decoding them in logical decoding. That has both efficiency advantages (we can do so before they are stored in memory/disk) and functionality advantages. Imagine using a

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-16 Thread Heikki Linnakangas
On 02/16/2015 02:44 AM, Peter Geoghegan wrote: On Sat, Feb 14, 2015 at 2:06 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I did not solve the potential for livelocks (discussed at http://www.postgresql.org/message-id/cam3swztftt_fehet3tu3ykcpcypynnauquz3q+naasnh-60...@mail.gmail.com).

Re: [HACKERS] Replication identifiers, take 4

2015-02-16 Thread Heikki Linnakangas
On 02/16/2015 11:18 AM, Andres Freund wrote: On 2015-02-16 11:07:09 +0200, Heikki Linnakangas wrote: How does this work if you have multiple replication systems in use in the same cluster? You might use Slony to replication one table to one system, and BDR to replication another table with

Re: [HACKERS] Replication identifiers, take 4

2015-02-16 Thread Andres Freund
On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote: Yes, thanks. Note to self and everyone else looking at this: It's important to keep in mind is that the replication IDs are completely internal to the local cluster. They are *not* sent over the wire. Well, if you *want* to, you could

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-16 Thread Syed, Rahila
Hello, Thank you for reviewing and testing the patch. + /* leave if data cannot be compressed */ + if (compressed_len == 0) + return false; This should be 0, pglz_compress returns -1 when compression fails. + if (pglz_decompress(block_image,

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-16 Thread Andres Freund
On 2015-02-16 11:30:20 +, Syed, Rahila wrote: - * As a trivial form of data compression, the XLOG code is aware that - * PG data pages usually contain an unused hole in the middle, which - * contains only zero bytes. If hole_length 0 then we have removed - * such a hole from the stored

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-16 Thread Michael Paquier
On Mon, Feb 16, 2015 at 8:30 PM, Syed, Rahila rahila.s...@nttdata.com wrote: Regarding the sanity checks that have been added recently. I think that they are useful but I am suspecting as well that only a check on the record CRC is done because that's reliable enough and not doing those

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-16 Thread Andres Freund
On 2015-02-16 20:55:20 +0900, Michael Paquier wrote: On Mon, Feb 16, 2015 at 8:30 PM, Syed, Rahila rahila.s...@nttdata.com wrote: Regarding the sanity checks that have been added recently. I think that they are useful but I am suspecting as well that only a check on the record CRC is