Re: [HACKERS] commitfest status

2014-07-30 Thread Amit Kapila
On Wed, Jul 30, 2014 at 10:57 PM, Robert Haas wrote: > > Hi, > > Is anybody working on closing out the "in progress" CommitFest? > > https://commitfest.postgresql.org/action/commitfest_view?id=22 If you or others don't have any objection, then I will do this on coming weekend. > I think anything

Re: [HACKERS] Production block comparison facility

2014-07-30 Thread Michael Paquier
On Thu, Jul 31, 2014 at 2:59 PM, Simon Riggs wrote: > On 29 July 2014 11:30, Heikki Linnakangas wrote: > >> I don't understand how this works. A full-page image contains the new page >> contents *after* the WAL-logged operation. For example, in a heap insert, >> the full-page image contains the n

Re: [HACKERS] Proposal: Incremental Backup

2014-07-30 Thread Michael Paquier
On Thu, Jul 31, 2014 at 3:00 PM, Amit Kapila wrote: > One more thing, what will happen for unlogged tables with such a > mechanism? I imagine that you can safely bypass them as they are not accessible during recovery and will start with empty relation files once recovery ends. The same applies to

Re: [HACKERS] Proposal: Incremental Backup

2014-07-30 Thread Amit Kapila
On Wed, Jul 30, 2014 at 7:00 PM, desmodemone wrote: > Hello, > I think it's very useful an incremental/differential backup method, by the way > the method has two drawbacks: > 1) In a database normally, even if the percent of modify rows is small compared to total rows, the probabilit

Re: [HACKERS] Proposal: Incremental Backup

2014-07-30 Thread Amit Kapila
On Wed, Jul 30, 2014 at 11:32 PM, Robert Haas wrote: > > IMV, the way to eventually make this efficient is to have a background > process that reads the WAL and figures out which data blocks have been > modified, and tracks that someplace. Nice idea, however I think to make this happen we need to

Re: [HACKERS] Production block comparison facility

2014-07-30 Thread Simon Riggs
On 29 July 2014 11:30, Heikki Linnakangas wrote: > I don't understand how this works. A full-page image contains the new page > contents *after* the WAL-logged operation. For example, in a heap insert, > the full-page image contains the new tuple. How can you compare that with > what's on the dis

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-07-30 Thread Amit Kapila
On Fri, Jul 18, 2014 at 10:22 AM, Dilip kumar wrote: > On 16 July 2014 12:13, Magnus Hagander Wrote, > >Yeah, those are exactly my points. I think it would be significantly simpler to do it that way, rather than forking and threading. And also easier to make portable... > > >(and as a optimizatio

Re: [HACKERS] 9.4 docs current as of

2014-07-30 Thread Bruce Momjian
On Thu, Jul 24, 2014 at 06:27:52PM +0200, Magnus Hagander wrote: > On Thu, Jul 24, 2014 at 5:17 PM, Tom Lane wrote: > > Magnus Hagander writes: > >> 9.4 beta docs are listed as "Current as of 2014-05-10". > >> I'm assuming that's just a step we missed in the version stamping? > > > > No, what tha

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-07-30 Thread Peter Geoghegan
On Wed, Jul 30, 2014 at 3:04 PM, Peter Geoghegan wrote: > I feel it should be possible for both the Datum and Heap sort cases to > use the new optimization (since they can always use sort support) > wherever they can afford to make that normalization pass (they usually > can, since copytup* is usu

Re: [HACKERS] New developer TODO suggestions

2014-07-30 Thread Andrew Dunstan
On 07/30/2014 06:11 PM, Peter Geoghegan wrote: On Wed, Jul 30, 2014 at 2:37 PM, Andrew Dunstan wrote: Both these might be possible. I am not planning on doing them, at least. My current json plans for 9.5 are limited to implementing jsonb equivalents of those json functions that didn't make it

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Mitsumasa KONDO
Hi, 2014-07-31 5:18 GMT+09:00 Fabien COELHO : > > I've committed the changes to pgbench.c and the documentation changes >> with some further wordsmithing. >> > > Ok, thanks a lot for your reviews and your help with improving the > documentation. Yeah, thanks for all relative members. > I don'

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > Please let me know if either of you are ready to experiment with the "named > pipe" idea anytime soon. > If not, I would be happy to take a crack at it, but would appreciate your > expert advice to start me down the right path (files/functions to update,

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-07-30 Thread Peter Geoghegan
On Wed, Jul 30, 2014 at 3:04 PM, Peter Geoghegan wrote: > I don't have a problem with changing the name. But the name that you > propose is all about text. This patch is intended to add an extensible > infrastructure (a new part of sort support), plus one client of that > more complete extensible

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread
Robert and Tom, Please let me know if either of you are ready to experiment with the "named pipe" idea anytime soon. If not, I would be happy to take a crack at it, but would appreciate your expert advice to start me down the right path (files/functions to update, pseudo-code, etc.). -Keith Ba

Re: [HACKERS] New developer TODO suggestions

2014-07-30 Thread Peter Geoghegan
On Wed, Jul 30, 2014 at 2:37 PM, Andrew Dunstan wrote: > Both these might be possible. I am not planning on doing them, at least. My > current json plans for 9.5 are limited to implementing jsonb equivalents of > those json functions that didn't make it into the 9.4 jsonb work due to > pressure of

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-07-30 Thread Peter Geoghegan
On Wed, Jul 30, 2014 at 10:52 AM, Robert Haas wrote: > I don't think we should commit anything that's not clearly under the > PostgreSQL license. I thought that we were comfortable with using MIT licensed code. But, come to think of it I don't know what our exact policy is. Wikipedia says "The Si

Re: [HACKERS] No, pg_size_pretty(numeric) was not such a hot idea

2014-07-30 Thread Tom Lane
Josh Loberant writes: > Was this issue ever resolved? > We are now having Nagios checks failing due to the pg_size_pretty function, > and the check runs fine on my local machine 9.1 (fails on 9.2 and 9.3, both > having two pg_size_pretty functions). Nothing was done about it so far for lack of co

Re: [HACKERS] No, pg_size_pretty(numeric) was not such a hot idea

2014-07-30 Thread Josh Loberant
Was this issue ever resolved? We are now having Nagios checks failing due to the pg_size_pretty function, and the check runs fine on my local machine 9.1 (fails on 9.2 and 9.3, both having two pg_size_pretty functions). Thanks, Josh -- View this message in context: http://postgresql.1045698.n5

Re: [HACKERS] New developer TODO suggestions

2014-07-30 Thread Andrew Dunstan
On 07/29/2014 10:43 AM, Bruce Momjian wrote: * A function that converts a json array to a PostgreSQL array of a given type if all json members are compatible with the type * Expanding the set of json/jsonb operations to introduce features that people are used to from jquery, mongo, etc. Replac

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

2014-07-30 Thread Thomas Munro
On 29 July 2014 02:35, Alvaro Herrera wrote: > David Rowley wrote: > >> I've also been looking at the isolation tests and I see that you've added a >> series of tests for NOWAIT. I was wondering why you did that as that's >> really existing code, probably if you thought the tests were a bit thin >

Re: [HACKERS] Allow GRANT TRIGGER privilege to DROP TRIGGER (Re: Bug ##7716)

2014-07-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Jul 16, 2014 at 07:45:56PM -0400, Tom Lane wrote: >> I think we should get rid of the separate TRIGGER privilege altogether, >> not make it an even bigger security hole. > Uh, how does removing a trigger cause a larger security hole? As long > as users can create

Re: [HACKERS] Making joins involving ctid work for the benefit of UPSERT

2014-07-30 Thread Bruce Momjian
On Mon, Jul 28, 2014 at 11:37:07AM -0400, Robert Haas wrote: > > Yes, but what if you don't see a conflict because it isn't visible to > > your snapshot, and then you insert, and only then (step 5), presumably > > with a dirty snapshot, you find a conflict? How does the loop > > terminate if that b

Re: [HACKERS] Making joins involving ctid work for the benefit of UPSERT

2014-07-30 Thread Peter Geoghegan
On Wed, Jul 30, 2014 at 10:36 AM, Robert Haas wrote: > You've convinced me that only indexable predicates can be sensibly > used here. I'm not sure that's the same as saying that upserts are > driven by inserts, though. I'd tend to interpret that to mean > insert-the-heap-tuple-first, but I thin

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Fabien COELHO
Hello Robert, I've committed the changes to pgbench.c and the documentation changes with some further wordsmithing. Ok, thanks a lot for your reviews and your help with improving the documentation. I don't think including the other changes in patch A is a good idea, Fine. It was mostly

Re: [HACKERS] Allow GRANT TRIGGER privilege to DROP TRIGGER (Re: Bug ##7716)

2014-07-30 Thread Bruce Momjian
On Wed, Jul 16, 2014 at 07:45:56PM -0400, Tom Lane wrote: > A look at check_object_ownership suggests that you could take the TRIGGER > case out of the generic relation path and make it a special case that > allows either ownership or TRIGGER permission. > > TBH, though, I'm not sure this is somet

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Bruce Momjian
On Wed, Jul 30, 2014 at 02:49:25PM -0400, Alvaro Herrera wrote: > Stephen Frost wrote: > > * Bruce Momjian (br...@momjian.us) wrote: > > > Actually, thinking more, Stephen Frost mentioned that the auditing > > > system has to modify database _state_, and dumping/restoring the state > > > of an exte

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Tom Lane
Stephen Frost writes: > What I wish to avoid is the situation which exists around hstore. > Perhaps I've got this wrong, but my recollection of the discussion > leads me to believe that we can't have hstore in core becasue there's > no simple migration path from an hstore-enabled installation to

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > This is really true of any extension which wants to attach information > > or track things associated with roles or other database objects. What > > I'd like to avoid is having an extension which does so through

[HACKERS] Fixed redundant i18n strings in json

2014-07-30 Thread Daniele Varrazzo
Please find attached a small tweak to a couple of strings found while updating translations. The %d version is already used elsewhere in the same file. -- Daniele diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c index 2f99908..2aa27cc 100644 --- a/src/backend/utils/adt/json

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Tom Lane
Stephen Frost writes: > * Bruce Momjian (br...@momjian.us) wrote: >> Actually, thinking more, Stephen Frost mentioned that the auditing >> system has to modify database _state_, and dumping/restoring the state >> of an extension might be tricky. > This is really true of any extension which wants

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > I think someone could write a Perl script that you run before the > upgrade to create SQL commands to restore the audit settings. Is pg_upgrade going to detect that pgaudit is installed and know to run this perl script? I don't doubt that it'd be possib

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Alvaro Herrera
Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Actually, thinking more, Stephen Frost mentioned that the auditing > > system has to modify database _state_, and dumping/restoring the state > > of an extension might be tricky. > > This is really true of any extension which wan

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Bruce Momjian
On Wed, Jul 30, 2014 at 02:29:47PM -0400, Stephen Frost wrote: > Using auditing as an example, consider this scenario: > > pgaudit grows a table which is used to say "only audit roles X, Y, Z" > (or specific tables, or connections from certain IPs, etc). > > A patch for PG 10.1 is proposed

Re: [HACKERS] pgaudit - an auditing extension for PostgreSQL

2014-07-30 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > Actually, thinking more, Stephen Frost mentioned that the auditing > system has to modify database _state_, and dumping/restoring the state > of an extension might be tricky. This is really true of any extension which wants to attach information or track

Re: [HACKERS] Proposal: Incremental Backup

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 12:35 PM, Marco Nenciarini wrote: >> I agree with much of that. However, I'd question whether we can >> really seriously expect to rely on file modification times for >> critical data-integrity operations. I wouldn't like it if somebody >> ran ntpdate to fix the time whil

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-07-30 Thread Robert Haas
On Sun, Jul 27, 2014 at 3:00 AM, Peter Geoghegan wrote: > On Thu, Jun 12, 2014 at 2:09 PM, Peter Geoghegan wrote: >> Thanks for looking into this. > > Is anyone going to look at this? I'm concerned by the licensing information in hyperloglog.c, which reads: + * Portions Copyright (c) 2014, Pos

Re: [HACKERS] Making joins involving ctid work for the benefit of UPSERT

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 1:28 PM, Peter Geoghegan wrote: > On Tue, Jul 29, 2014 at 9:56 AM, Robert Haas wrote: >> I think it would be advisable to separate the syntax from the >> implementation. Presumably you can make your implementation use some >> reasonable syntax we can all agree on, and con

[HACKERS] commitfest status

2014-07-30 Thread Robert Haas
Hi, Is anybody working on closing out the "in progress" CommitFest? https://commitfest.postgresql.org/action/commitfest_view?id=22 I think anything that is "waiting on author" should certainly be bounced at this point, and stuff that never got reviewed or is ready for committer should probably b

Re: [HACKERS] gaussian distribution pgbench -- splits v4

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 4:41 AM, Fabien COELHO wrote: >> Attached B patch does turn incorrect setrandom syntax into errors instead >> of ignoring extra parameters. >> >> First A patch is repeated to help commitfest references. > > Oops, I applied the change on the wrong part:-( > > Here is the cha

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
On Wed, Jul 30, 2014 at 7:26 PM, Fabien COELHO wrote: > > ISTM that you miss the projection on the segment if dx=0 or dy=0. >>> >> >> I don't need to find projection itself, I need only distance. When dx = 0 >> then nearest point is on horizontal line of box, so distance to it is dy. >> Same whe

Re: [HACKERS] [w32] test_shm_mq test suite permanently burns connections slots

2014-07-30 Thread Robert Haas
On Mon, Jul 28, 2014 at 9:38 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> OK, I think I see the problem. In EXEC_BACKEND mode, >> SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has >> been set. InitProcess() therefore pulls the PGPROC for the worker >> from freeProcs rath

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Fabien COELHO
ISTM that you miss the projection on the segment if dx=0 or dy=0. I don't need to find projection itself, I need only distance. When dx = 0 then nearest point is on horizontal line of box, so distance to it is dy. Same when dy = 0. When both of them are 0 then point is in the box. Indeed. I

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: >> Hm. That particular protocol is broken: two postmasters doing it at the >> same time would both pass (because neither has it open for read at the >> instant where they try to write). But we could possibly frob the idea >>

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: >> I think it would be good to spend some energy figuring out what to do >> about this. > > Well, we've been around on this multiple times before, but if we have > any new ideas, sure ... Well, I tried to compile a more comprehensive list of possib

Re: [HACKERS] Proposal: Incremental Backup

2014-07-30 Thread desmodemone
2014-07-29 18:35 GMT+02:00 Marco Nenciarini : > Il 25/07/14 20:44, Robert Haas ha scritto: > > On Fri, Jul 25, 2014 at 2:21 PM, Claudio Freire > wrote: > >> On Fri, Jul 25, 2014 at 10:14 AM, Marco Nenciarini > >> wrote: > >>> 1. Proposal > >>> = > >>> Our proposal

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
On Wed, Jul 30, 2014 at 4:06 PM, Fabien COELHO wrote: > > double dx = 0.0, dy = 0.0; >> >> if (point->x < box->low.x) >>dx = box->low.x - point->x; >> if (point->x > box->high.x) >>dx = point->x - box->high.x; >> if (point->y < box->low.y) >>dy = box->low.y - point->y; >> if (point->

Re: [HACKERS] Distance from point to box

2014-07-30 Thread Fabien COELHO
double dx = 0.0, dy = 0.0; if (point->x < box->low.x) dx = box->low.x - point->x; if (point->x > box->high.x) dx = point->x - box->high.x; if (point->y < box->low.y) dy = box->low.y - point->y; if (point->y > box->high.y) dy = point->y - box->high.y; return HYPOT(dx, dy); I feel my

[HACKERS] Distance from point to box

2014-07-30 Thread Alexander Korotkov
Hackers, while reading code written by my GSoC student for knn-spgist I faced many questions about calculation distance from point to box. 1) dist_pb calls close_pb which calls on_pb, dist_ps_internal, close_ps and so on. So, it's very complex way to calculate very simple value. I see this way ha

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-07-30 Thread Pavel Stehule
Hi 2014-07-30 10:22 GMT+02:00 Etsuro Fujita : > (2014/07/29 0:58), Robert Haas wrote: > >> On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz >> wrote: >> >>> Shigeru Hanada wrote: >>> * Naming of new behavior You named this optimization "Direct Update", but I'm not sure that this is

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2014-07-30 Thread Etsuro Fujita
(2014/07/29 0:58), Robert Haas wrote: On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz wrote: Shigeru Hanada wrote: * Naming of new behavior You named this optimization "Direct Update", but I'm not sure that this is intuitive enough to express this behavior. I would like to hear opinions of nati