Re: [HACKERS] Storing the password in .pgpass file in an encrypted format

2014-02-21 Thread Daniel Farina
On Fri, Feb 21, 2014 at 10:42 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think this thread deserves more attention: http://www.postgresql.org/message-id/caazkufajufddfp1_vghbdfyru0sj6msovvkrp87acq53ov6...@mail.gmail.com (I wrote that mail) I'm still in interested in this idea and

Re: [HACKERS] Storing the password in .pgpass file in an encrypted format

2014-02-21 Thread Daniel Farina
On Fri, Feb 21, 2014 at 6:15 PM, Greg Stark st...@mit.edu wrote: On Fri, Feb 21, 2014 at 10:18 PM, Daniel Farina dan...@heroku.com wrote: I'm still in interested in this idea and haven't found a good reason to rescind the general thinking there. It's an interesting idea. I wonder if it would

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Daniel Farina
On Tue, Dec 10, 2013 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: So my objection to what Peter is suggesting is not that it's a bad idea in isolation, but that I don't see where he's going to stop, short of reinventing every query-normalization behavior that exists in the planner. If this

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-14 Thread Daniel Farina
On Thu, Nov 14, 2013 at 7:25 PM, Peter Geoghegan p...@heroku.com wrote: On Tue, Nov 5, 2013 at 5:30 AM, Sameer Thakur samthaku...@gmail.com wrote: Hello, Please find attached pg_stat_statements-identification-v9.patch. I took a quick look. Observations: + /* Making query ID dependent

Re: [HACKERS] Save Hash Indexes

2013-11-01 Thread Daniel Farina
On Fri, Nov 1, 2013 at 6:31 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Hi, Here's an idea: when a user ask for an Hash Index transparently build a BTree index over an hash function instead. Advantages: - it works - it's crash safe - it's (much?) faster than a hash index

Re: [HACKERS] Save Hash Indexes

2013-11-01 Thread Daniel Farina
On Fri, Nov 1, 2013 at 8:52 AM, Daniel Farina dan...@heroku.com wrote: On Fri, Nov 1, 2013 at 6:31 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Also, talking with Peter Geoghegan, it's unclear that there's a use case where a hash index would be faster than a btree index over the hash

[HACKERS] What hook would you recommend for one time, post authentication?

2013-10-28 Thread Daniel Farina
What hook would you recommend that matches this criteria: * Runs post-authentication * ..Once I was putting together a little extension module[0] intended to do connection limits out-of-band with the catalog (so that hot standbys and primaries can have different imposed connection limits), but

Re: [HACKERS] What hook would you recommend for one time, post authentication?

2013-10-28 Thread Daniel Farina
On Mon, Oct 28, 2013 at 6:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: roleid = get_role_oid(port-user_name, true); Thank you for that, that appears to work very well to my purpose, as does ClientAuthentication_hook, now. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-10-22 Thread Daniel Farina
On Tue, Oct 22, 2013 at 2:56 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: Hm. It's been a long time since college statistics, but doesn't the entire concept of standard deviation depend on the assumption that the underlying distribution is

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 7:40 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 10, 2013 at 7:20 PM, Sameer Thakur samthaku...@gmail.com wrote: Please find patch attached which adds documentation for session_start and introduced fields and corrects documentation for queryid to be

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 10:12 AM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Oct 11, 2013 at 12:19 AM, Daniel Farina dan...@heroku.com wrote: Probably. The idea is that without those fields it's, to wit, impossible to explain non-monotonic movement in metrics of those queries

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 10:49 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Daniel Farina escribió: On Thu, Oct 10, 2013 at 7:40 AM, Fujii Masao masao.fu...@gmail.com wrote: In my test, I found that pg_stat_statements.total_time always indicates a zero. I guess that the patch might

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-10 Thread Daniel Farina
On Thu, Oct 10, 2013 at 1:30 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Just noticed that you changed the timer to struct Instrumentation. Not really sure about that change. Since you seem to be using only the start time and counter, wouldn't it be better to store only those?

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-05 Thread Daniel Farina
On Fri, Oct 4, 2013 at 7:22 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 3, 2013 at 5:11 PM, Sameer Thakur samthaku...@gmail.com wrote: On Wed, Oct 2, 2013 at 6:40 PM, Sameer Thakur samthaku...@gmail.com wrote: Looks pretty good. Do you want to package up the patch with your

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-10-01 Thread Daniel Farina
On Tue, Oct 1, 2013 at 5:32 AM, Sameer Thakur samthaku...@gmail.com wrote: On Tue, Oct 1, 2013 at 12:48 AM, Daniel Farina-5 [via PostgreSQL] [hidden email] wrote: On Sep 30, 2013 4:39 AM, Sameer Thakur [hidden email] wrote: Also, for onlookers, I have changed this patch around to do

Re: [HACKERS] pluggable compression support

2013-10-01 Thread Daniel Farina
On Mon, Sep 30, 2013 at 1:49 PM, Huchev hugochevr...@gmail.com wrote: How come any compressor which could put some competition to pglz is systematically pushed out of the field on the ground of unverifiable legal risks ? Because pglz has been around for a while and has not caused patent

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-09-30 Thread Daniel Farina
On Sun, Sep 29, 2013 at 10:25 AM, Sameer Thakur samthaku...@gmail.com wrote: Yes i was. Just saw a warning when pg_stat_statements is loaded that valid values for pg_stat_statements.max is between 100 and 2147483647. Not sure why though. I remember hacking that out for testing sake. I can

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-09-30 Thread Daniel Farina
On Sep 30, 2013 4:39 AM, Sameer Thakur samthaku...@gmail.com wrote: Also, for onlookers, I have changed this patch around to do the date-oriented stuff but want to look it over before stapling it up and sending it. If one cannot wait, one can look at

Re: [HACKERS] Some interesting news about Linux 3.12 OOM

2013-09-27 Thread Daniel Farina
On Wed, Sep 25, 2013 at 8:00 AM, Greg Stark st...@mit.edu wrote: On Wed, Sep 25, 2013 at 12:15 AM, Daniel Farina dan...@heroku.com wrote: Enable the memcg OOM killer only for user faults, where it's really the only option available. Is this really a big deal? I would expect most faults

Re: [HACKERS] Some interesting news about Linux 3.12 OOM

2013-09-24 Thread Daniel Farina
On Sep 24, 2013 10:12 AM, Josh Berkus j...@agliodbs.com wrote: All, I've send kernel.org a message that we're keen on seeing these changes become committed. I thought it was merged already in 3.12. There are a few related patches, but here's one: commit

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-09-20 Thread Daniel Farina
On Fri, Sep 20, 2013 at 1:11 AM, Daniel Farina dan...@fdr.io wrote: I think the n-call underestimation propagation may not be quite precise for various detailed reasons (having to do with 'sticky' queries) and to make it precise is probably more work than it's worth. And, on more reflection

[HACKERS] Some interesting news about Linux 3.12 OOM

2013-09-18 Thread Daniel Farina
I'm not sure how many of you have been tracking this but courtesy of lwn.net I have learned that it seems that the OOM killer behavior in Linux 3.12 will be significantly different. And by description, it sounds like an improvement. I thought some people reading -hackers might be interested.

[HACKERS] Unpacking scalar JSON values

2013-08-24 Thread Daniel Farina
Per report of Armin Ronacher, it's not clear how to take a scalar JSON string and unquote it into a regular Postgres text value, given what I can see here: http://www.postgresql.org/docs/9.3/static/functions-json.html Example: SELECT 'a json string'::json; (Although this some problem could play

Re: [HACKERS] Unpacking scalar JSON values

2013-08-24 Thread Daniel Farina
On Sat, Aug 24, 2013 at 3:09 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 08/24/2013 11:36 PM, Daniel Farina wrote: Per report of Armin Ronacher, it's not clear how to take a scalar JSON string and unquote it into a regular Postgres text value, given what I can see here: http

Re: [HACKERS] Unpacking scalar JSON values

2013-08-24 Thread Daniel Farina
On Sat, Aug 24, 2013 at 6:04 PM, Daniel Farina dan...@fdr.io wrote: But there's no good way I can find from the documentation to do it with a scalar: select ('va\lue'::json)::text; Triggered send by accident: select ('va\lue'::json)::text; text --- va\lue (1 row) the JSON escaping

Re: [HACKERS] [9.4 CF] Free VMs for Reviewers Testers

2013-07-09 Thread Daniel Farina
On Tue, Jul 9, 2013 at 12:24 AM, Jesper Krogh jes...@krogh.cc wrote: The really, really big ones are useful even for pushing limits, such as cr1.8xlarge, with 32 CPUs and 244GiB memory. Current spot instance price (the heavily discounted can die at any time one) is $0.343/hr. Otherwise, it's

Re: [HACKERS] [9.4 CF] Free VMs for Reviewers Testers

2013-07-08 Thread Daniel Farina
On Mon, Jul 8, 2013 at 7:25 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 07/09/2013 08:35 AM, Josh Berkus wrote: Since these are cloud servers, they won't work well for performance testing. I did some work on that a while ago, and found that I was able to get _astonishingly_ stable

Re: [HACKERS] askpass program for libpq

2013-06-15 Thread Daniel Farina
On Fri, May 17, 2013 at 2:03 PM, Daniel Farina dan...@heroku.com wrote: On Wed, Jan 9, 2013 at 5:17 AM, Peter Eisentraut pete...@gmx.net wrote: I would like to have something like ssh-askpass for libpq. The main reason is that I don't want to have passwords in plain text on disk, even

Re: [HACKERS] askpass program for libpq

2013-06-15 Thread Daniel Farina
On Sat, Jun 15, 2013 at 8:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Okay, I have a patch that does something *like* (but not the same) as this, and whose implementation is totally unreasonable, but it's enough to get a sense of how the whole thing feels

Re: [HACKERS] updated emacs configuration

2013-06-14 Thread Daniel Farina
On Thu, Jun 13, 2013 at 6:27 PM, Peter Eisentraut pete...@gmx.net wrote: First, I propose adding a .dir-locals.el file to the top-level directory with basic emacs settings. These get applied automatically. This especially covers the particular tab and indentation settings that PostgreSQL

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-10 Thread Daniel Farina
On Mon, Jun 10, 2013 at 11:59 AM, Josh Berkus j...@agliodbs.com wrote: Anyway, what I'm pointing out is that this is a business decision, and there is no way that we can make a decision for the users what to do when we run out of WAL space. And that the stop archiving option needs to be there

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-10 Thread Daniel Farina
On Mon, Jun 10, 2013 at 4:42 PM, Josh Berkus j...@agliodbs.com wrote: Daniel, Jeff, I don't doubt this, that's why I do have a no-op fallback for emergencies. The discussion was about defaults. I still think that drop-wal-from-archiving-whenever is not a good one. Yeah, we can argue

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-07 Thread Daniel Farina
On Fri, Jun 7, 2013 at 12:14 PM, Josh Berkus j...@agliodbs.com wrote: Right now, what we're telling users is You can have continuous backup with Postgres, but you'd better hire and expensive consultant to set it up for you, or use this external tool of dubious provenance which there's no

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-06 Thread Daniel Farina
On Wed, Jun 5, 2013 at 10:27 PM, Joshua D. Drake j...@commandprompt.com wrote: On 6/5/2013 10:07 PM, Daniel Farina wrote: If I told you there were some of us who would prefer to attenuate the rate that things get written rather than cancel or delay archiving for a long period of time, would

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-06 Thread Daniel Farina
On Wed, Jun 5, 2013 at 11:05 PM, Joshua D. Drake j...@commandprompt.com wrote: On 6/5/2013 10:54 PM, Peter Geoghegan wrote: On Wed, Jun 5, 2013 at 10:27 PM, Joshua D. Drake j...@commandprompt.com wrote: I just wonder if we are looking in the right place (outside of some obvious badness

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-06 Thread Daniel Farina
On Thu, Jun 6, 2013 at 9:30 PM, Jeff Janes jeff.ja...@gmail.com wrote: I would oppose that as the solution, either an unconditional one, or configurable with is it as the default. Those segments are not unneeded. I need them. That is why I set up archiving in the first place. If you need

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Daniel Farina
On Wed, Jun 5, 2013 at 6:00 PM, Joshua D. Drake j...@commandprompt.com wrote: I didn't see that proposal, link? Because the idea of slowing down wal-writing sounds insane. It's not as insane as introducing an archiving gap, PANICing and crashing, or running this hunk o junk I wrote

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Daniel Farina
On Wed, Jun 5, 2013 at 8:23 PM, Joshua D. Drake j...@commandprompt.com wrote: It's not as insane as introducing an archiving gap, PANICing and crashing, or running this hunk o junk I wrote http://github.com/fdr/ratchet Well certainly we shouldn't PANIC and crash but that is a simple fix. You

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-29 Thread Daniel Farina
On Mon, May 27, 2013 at 9:41 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 May 2013 15:36, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: On Mon, May 27, 2013 at 08:26:48AM -0400, Stephen Frost wrote: That said, many discussions and ideas do get shut down,

Re: [HACKERS] Better handling of archive_command problems

2013-05-17 Thread Daniel Farina
On Thu, May 16, 2013 at 9:13 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 16, 2013 at 10:06 PM, Daniel Farina dan...@heroku.com wrote: Do you have a sketch about mechanism to not encounter that problem? I didn't until just now, but see my email to Peter. That idea might be all

Re: [HACKERS] askpass program for libpq

2013-05-17 Thread Daniel Farina
On Wed, Jan 9, 2013 at 5:17 AM, Peter Eisentraut pete...@gmx.net wrote: I would like to have something like ssh-askpass for libpq. The main reason is that I don't want to have passwords in plain text on disk, even if .pgpass is read protected. By getting the password from an external

Re: [HACKERS] askpass program for libpq

2013-05-17 Thread Daniel Farina
On Fri, May 17, 2013 at 2:03 PM, Daniel Farina dan...@heroku.com wrote: Thanks for getting through all that text. Fin. And, thoughts? I have uploaded the resolvers, the last mail, and the patch to github: https://github.com/fdr/pq-resolvers So, if one prefers to use git to get this and track

Re: [HACKERS] Better handling of archive_command problems

2013-05-16 Thread Daniel Farina
On Thu, May 16, 2013 at 5:43 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 16, 2013 at 2:42 PM, Peter Geoghegan p...@heroku.com wrote: On Thu, May 16, 2013 at 11:16 AM, Robert Haas robertmh...@gmail.com wrote: Well, I think it IS a Postgres precept that interrupts should get a

Re: [HACKERS] on existing update construct

2013-05-15 Thread Daniel Farina
On Wed, May 15, 2013 at 11:44 AM, Dev Kumkar devdas.kum...@gmail.com wrote: Hello, Is there an alternative of Sybase on existing update construct in pgsql. ON DUPLICATE KEY UPDATE doesn't work. Thanks in advance! No, you'll have to either handle this in the application or use a stored

Re: [HACKERS] Better LWLocks with compare-and-swap (9.4)

2013-05-15 Thread Daniel Farina
On Mon, May 13, 2013 at 5:50 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: pgbench -S is such a workload. With 9.3beta1, I'm seeing this profile, when I run pgbench -S -c64 -j64 -T60 -M prepared on a 32-core Linux machine: - 64.09% postgres postgres [.] tas - tas

Re: [HACKERS] Better LWLocks with compare-and-swap (9.4)

2013-05-15 Thread Daniel Farina
On Wed, May 15, 2013 at 3:08 PM, Daniel Farina dan...@heroku.com wrote: On Mon, May 13, 2013 at 5:50 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: pgbench -S is such a workload. With 9.3beta1, I'm seeing this profile, when I run pgbench -S -c64 -j64 -T60 -M prepared on a 32-core Linux

Re: [HACKERS] Better handling of archive_command problems

2013-05-13 Thread Daniel Farina
On Mon, May 13, 2013 at 3:02 PM, Peter Geoghegan p...@heroku.com wrote: Has anyone else thought about approaches to mitigating the problems that arise when an archive_command continually fails, and the DBA must manually clean up the mess? Notably, the most common problem in this vein suffered

Re: [HACKERS] pg_controldata gobbledygook

2013-04-26 Thread Daniel Farina
On Thu, Apr 25, 2013 at 9:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: I think I've heard of scripts grepping the output of pg_controldata for this that or the other. Any rewording of the labels would break that. While I'm not

Re: [HACKERS] 9.3 release notes suggestions

2013-04-25 Thread Daniel Farina
On Wed, Apr 24, 2013 at 6:30 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 23, 2013 at 11:41 PM, Bruce Momjian br...@momjian.us wrote: Thanks for the many suggestions on improving the 9.3 release notes. There were many ideas I would have never thought of. Please keep the

Re: [HACKERS] confusing message about archive failures

2013-04-19 Thread Daniel Farina
On Wed, Apr 17, 2013 at 7:33 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wednesday, April 17, 2013, Peter Eisentraut wrote: When archive_command fails three times, it prints this message into the logs: transaction log file \%s\ could not be archived: too many failures This leaves it open

Re: [HACKERS] Enabling Checksums

2013-04-18 Thread Daniel Farina
On Wed, Apr 17, 2013 at 11:08 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-04-17 18:16:36 -0700, Daniel Farina wrote: The original paper is often shorthanded Castagnoli 93, but it exists in the IEEE's sphere of influence and is hard to find a copy of. Luckily, a pretty interesting

Re: [HACKERS] Enabling Checksums

2013-04-17 Thread Daniel Farina
On Wed, Apr 17, 2013 at 5:21 PM, Greg Smith g...@2ndquadrant.com wrote: Let me see if I can summarize where the messages flying by are at since you'd like to close this topic for now: -Original checksum feature used Fletcher checksums. Its main problems, to quote wikipedia, include that it

Re: [HACKERS] Interesting post-mortem on a near disaster with git

2013-04-03 Thread Daniel Farina
On Wed, Apr 3, 2013 at 6:18 PM, Jim Nasby j...@nasby.net wrote: What about rdiff-backup? I've set it up for personal use years ago (via the handy open source bash script backupninja) years ago and it has a pretty nice no-frills point-in-time, self-expiring, file-based automatic

Re: [HACKERS] Getting to 9.3 beta

2013-03-29 Thread Daniel Farina
On Fri, Mar 29, 2013 at 8:22 AM, Andres Freund and...@2ndquadrant.com wrote: - pg_stat_statements: query, session, and eviction identification: Seems to need at least docs = wait for author, seems to be easy enough? I would have responded by now, but recent events have unfortunately made

Re: [HACKERS] Default connection parameters for postgres_fdw and dblink

2013-03-28 Thread Daniel Farina
On Wed, Mar 27, 2013 at 8:22 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 27, 2013 at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Use a service file maybe? But you can't have it both ways: either we like the behavior of libpq absorbing defaults from the postmaster environment, or

Re: [HACKERS] GSoC project : K-medoids clustering in Madlib

2013-03-26 Thread Daniel Farina
On Tue, Mar 26, 2013 at 10:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: I suggested a couple of algorithms to be implemented in MADLib(apart from K Medoids). You could pick some(or all) of them, which would require 3 months to be completed. As for more

Re: [HACKERS] Interesting post-mortem on a near disaster with git

2013-03-25 Thread Daniel Farina
On Mon, Mar 25, 2013 at 11:07 AM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: Back when we used CVS for quite a few years I kept 7 day rolling snapshots of the CVS repo, against just such a difficulty as this. But we seem to be much better organized with infrastructure these days so I

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-22 Thread Daniel Farina
On Fri, Mar 22, 2013 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: This contains some edits to comments that referred to the obsolete and bogus TupleDesc scanning. No mechanical alterations. Applied with some substantial revisions. I didn't like

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-21 Thread Daniel Farina
This contains some edits to comments that referred to the obsolete and bogus TupleDesc scanning. No mechanical alterations. --- a/contrib/dblink/dblink.c +++ b/contrib/dblink/dblink.c @@ -2961,9 +2961,8 @@ initRemoteGucs(remoteGucs *rgs, PGconn *conn) } /* - * Scan a TupleDesc and, should it

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-20 Thread Daniel Farina
On Tue, Mar 19, 2013 at 10:37 PM, Daniel Farina dan...@heroku.com wrote: I added programming around various NULL returns reading GUCs in this revision, v4. Okay, one more of those fridge-logic bugs. Sorry for the noise. v5. A missing PG_RETHROW to get the properly finally-esque semantics

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-20 Thread Daniel Farina
On Wed, Mar 20, 2013 at 7:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Okay, one more of those fridge-logic bugs. Sorry for the noise. v5. A missing PG_RETHROW to get the properly finally-esque semantics: --- a/contrib/dblink/dblink.c +++ b/contrib

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Thu, Mar 14, 2013 at 8:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Mar 12, 2013 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, watching the remote side's datestyle and intervalstyle and matching them (for both input and output) would

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Similar in purpose to cc3f281ffb0a5d9b187e7a7b7de4a045809ff683, but taking into account that a dblink caller may choose to cause arbitrary changes to DateStyle and IntervalStyle

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'd be inclined to eat the cost of calling PQparameterStatus every time (which won't be that much) and instead try

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 3:52 PM, Greg Smith g...@2ndquadrant.com wrote: On 3/19/13 6:08 PM, Ants Aasma wrote: My main worry is that there is a reasonably large population of users out there that don't have that acceleration capability and will have to settle for performance overhead 4x worse

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 3:06 PM, Daniel Farina dan...@heroku.com wrote: On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'd be inclined to eat the cost of calling

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 6:10 PM, Daniel Farina dan...@heroku.com wrote: On Tue, Mar 19, 2013 at 3:06 PM, Daniel Farina dan...@heroku.com wrote: On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane

Re: [HACKERS] Enabling Checksums

2013-03-18 Thread Daniel Farina
On Mon, Mar 18, 2013 at 1:31 PM, Greg Smith g...@2ndquadrant.com wrote: On 3/18/13 10:52 AM, Bruce Momjian wrote: With a potential 10-20% overhead, I am unclear who would enable this at initdb time. If you survey people who are running PostgreSQL on cloud hardware, be it Amazon's EC2 or

Re: [HACKERS] Enabling Checksums

2013-03-18 Thread Daniel Farina
On Mon, Mar 18, 2013 at 7:13 PM, Greg Smith g...@2ndquadrant.com wrote: I wasn't trying to flog EBS as any more or less reliable than other types of storage. What I was trying to emphasize, similarly to your quite a stretch comment, was the uncertainty involved when such deployments fail.

Re: [HACKERS] Optimizing pglz compressor

2013-03-18 Thread Daniel Farina
On Wed, Mar 6, 2013 at 6:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With these tweaks, I was able to make pglz-based delta encoding perform roughly as well as Amit's patch. Out of curiosity, do we know how

Re: [HACKERS] Enabling Checksums

2013-03-17 Thread Daniel Farina
On Sun, Mar 17, 2013 at 5:50 PM, Greg Smith g...@2ndquadrant.com wrote: On the testing front, we've seen on-list interest in this feature from companies like Heroku and Enova, who both have some resources and practice to help testing too. Heroku can spin up test instances with workloads any

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-14 Thread Daniel Farina
On Tue, Mar 12, 2013 at 11:51 AM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Okay, I see. So inverting the thinking I wrote earlier: how about hearkening carefully to any ParameterStatus messages on the local side before entering the inner loop

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-12 Thread Daniel Farina
On Mon, Mar 11, 2013 at 7:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: I will try to make time for this, although it seems like the general approach should match pgsql_fdw if possible. Is the current thinking to forward the settings and then use the GUC

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-11 Thread Daniel Farina
On Mon, Mar 11, 2013 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: BTW, it strikes me that dblink is probably subject to at least some of these same failure modes. I'm not personally volunteering to fix any of this in dblink, but maybe someone ought to look into that. I will try to make

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-10 Thread Daniel Farina
On Sun, Mar 10, 2013 at 12:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: There's a lot left to do here of course. One thing I was wondering about was why we don't allow DEFAULTs to be attached to foreign-table columns. There was no use in it before, but it seems sensible enough now.

Re: [HACKERS] Enabling Checksums

2013-03-07 Thread Daniel Farina
On Thu, Mar 7, 2013 at 7:31 PM, Bruce Momjian br...@momjian.us wrote: On Mon, Mar 4, 2013 at 05:04:27PM -0800, Daniel Farina wrote: Putting aside the not-so-rosy predictions seen elsewhere in this thread about the availability of a high performance, reliable checksumming file system available

Re: [HACKERS] Enabling Checksums

2013-03-06 Thread Daniel Farina
On Wed, Mar 6, 2013 at 8:17 PM, Greg Smith g...@2ndquadrant.com wrote: TL;DR summary: on a system I thought was a fair middle of the road server, pgbench tests are averaging about a 2% increase in WAL writes and a 2% slowdown when I turn on checksums. There are a small number of troublesome

Re: [HACKERS] [GENERAL] Floating point error

2013-03-04 Thread Daniel Farina
On Mon, Mar 4, 2013 at 2:27 PM, Maciek Sakrejda m.sakre...@gmail.com wrote: On Sun, Mar 3, 2013 at 9:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: The real difficulty is that there may be more than one storable value that corresponds to 1.23456 to six decimal digits. To be certain that we can

Re: [HACKERS] Enabling Checksums

2013-03-04 Thread Daniel Farina
On Mon, Mar 4, 2013 at 1:22 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04.03.2013 23:00, Jeff Davis wrote: On Mon, 2013-03-04 at 22:27 +0200, Heikki Linnakangas wrote: Yeah, fragmentation will certainly hurt some workloads. But how badly, and which workloads, and how does that

Re: [HACKERS] Enabling Checksums

2013-03-01 Thread Daniel Farina
On Sun, Feb 24, 2013 at 10:30 PM, Greg Smith g...@2ndquadrant.com wrote: Attached is some bit rot updates to the checksums patches. The replace-tli one still works fine I rather badly want this feature, and if the open issues with the patch has hit zero, I'm thinking about applying it,

Re: [HACKERS] Unarchived WALs deleted after crash

2013-02-21 Thread Daniel Farina
On Thu, Feb 21, 2013 at 12:39 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 21.02.2013 02:59, Daniel Farina wrote: On Fri, Feb 15, 2013 at 9:29 AM, Simon Riggssi...@2ndquadrant.com wrote: On 15 February 2013 17:07, Heikki Linnakangashlinnakan...@vmware.com wrote: Unfortunately

Re: [HACKERS] Unarchived WALs deleted after crash

2013-02-20 Thread Daniel Farina
On Fri, Feb 15, 2013 at 9:29 AM, Simon Riggs si...@2ndquadrant.com wrote: On 15 February 2013 17:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: Unfortunately in HEAD, xxx.done file is not created when restoring archived file because of absence of the patch. We need to implement that

Re: [HACKERS] Unarchived WALs deleted after crash

2013-02-14 Thread Daniel Farina
On Thu, Feb 14, 2013 at 7:45 AM, Jehan-Guillaume de Rorthais j...@dalibo.com wrote: Hi, I am facing an unexpected behavior on a 9.2.2 cluster that I can reproduce on current HEAD. On a cluster with archive enabled but failing, after a crash of postmaster, the checkpoint occurring before

Re: [HACKERS] Fractal tree indexing

2013-02-13 Thread Daniel Farina
On Wed, Feb 13, 2013 at 8:20 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: The basic idea of a fractal tree index is to attach a buffer to every non-leaf page. On insertion, instead of descending all the way down to the correct leaf page, the new

Re: [HACKERS] Considering Gerrit for CFs

2013-02-08 Thread Daniel Farina
On Fri, Feb 8, 2013 at 2:23 AM, Magnus Hagander mag...@hagander.net wrote: On Fri, Feb 8, 2013 at 1:32 AM, Josh Berkus j...@agliodbs.com wrote: This is a few too many steps, and certainly appears completely broken to any newcomer. I agree it's way too many step. Several of those can certainly

Re: [HACKERS] Considering Gerrit for CFs

2013-02-06 Thread Daniel Farina
On Wed, Feb 6, 2013 at 3:00 PM, Joshua D. Drake j...@commandprompt.com wrote: On 02/06/2013 01:53 PM, Tom Lane wrote: ... if it's going to try to coerce us out of our email-centric habits, then I for one am very much against it. To me, the problems with the existing CF app are precisely

Re: [HACKERS] [v9.3] writable foreign tables

2013-02-05 Thread Daniel Farina
On Fri, Feb 1, 2013 at 2:22 AM, Daniel Farina dan...@heroku.com wrote: On Tue, Jan 29, 2013 at 1:19 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I noticed the v10 patch cannot be applied on the latest master branch cleanly. The attached ones were rebased. Anyway, I'm looking at the first patch

Re: [HACKERS] json api WIP patch

2013-02-04 Thread Daniel Farina
On Mon, Feb 4, 2013 at 11:38 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 4, 2013 at 11:10 AM, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2013 10:47 AM, Robert Haas wrote: The SQL standards considerations seem worth thinking about, too. We've certainly gone through a lot

Re: [HACKERS] json api WIP patch

2013-02-04 Thread Daniel Farina
On Mon, Feb 4, 2013 at 12:37 PM, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2013 03:16 PM, Daniel Farina wrote: On Mon, Feb 4, 2013 at 11:38 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 4, 2013 at 11:10 AM, Andrew Dunstan and...@dunslane.net wrote: On 02/04/2013 10:47

Re: [HACKERS] [v9.3] writable foreign tables

2013-02-01 Thread Daniel Farina
On Tue, Jan 29, 2013 at 1:19 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I noticed the v10 patch cannot be applied on the latest master branch cleanly. The attached ones were rebased. Hello, I'm just getting started looking at this, but notice that the second patch relies on

Re: [HACKERS] json api WIP patch

2013-02-01 Thread Daniel Farina
On Fri, Feb 1, 2013 at 2:08 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 31, 2013 at 7:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Thu, Jan 31, 2013 at 4:20 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/31/2013 05:06 PM, Peter

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
I have adjusted this patch a little bit to take care of the review issues, along with just doing a bit of review myself. On Thu, Oct 25, 2012 at 2:25 AM, Will Leinweber w...@heroku.com wrote: Thanks for the reviews and comments. Responses inline: . On Sat, Oct 20, 2012 at 9:19 AM, Abhijit

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
On Thu, Jan 17, 2013 at 5:07 PM, Daniel Farina dan...@heroku.com wrote: I have adjusted this patch a little bit to take care of the review issues, along with just doing a bit of review myself. I realized while making my adjustments that I pointlessly grew some input checking in the inner loop

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Daniel Farina
On Tue, Jan 15, 2013 at 11:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: There are still 34 items needing attention in CF3. I suggest that, if you have some spare time, your help would be very much appreciated there. The commitfest that started on

Re: [HACKERS] json api WIP patch

2013-01-15 Thread Daniel Farina
On Tue, Jan 15, 2013 at 12:17 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/15/2013 02:47 PM, Merlin Moncure wrote: On Tue, Jan 15, 2013 at 1:04 PM, David Fetter da...@fetter.org wrote: On Mon, Jan 14, 2013 at 07:52:56PM -0500, Andrew Dunstan wrote: On 01/14/2013 07:36 PM, Merlin

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Daniel Farina
On Tue, Jan 8, 2013 at 11:51 AM, Simon Riggs si...@2ndquadrant.com wrote: On 8 January 2013 18:46, Josh Berkus j...@agliodbs.com wrote: On 1/5/13 1:21 PM, Peter Geoghegan wrote: On 21 December 2012 14:08, Robert Haas robertmh...@gmail.com wrote: I'm sure it's possible; I don't *think* it's

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-08 Thread Daniel Farina
On Tue, Jan 8, 2013 at 5:51 PM, Josh Berkus j...@agliodbs.com wrote: Daniel, To briefly reiterate my objection, I observed that one may want to enter a case of cyclicality on a temporary basis -- to assist with some intermediate states in remastering, and it'd be nice if Postgres didn't try

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2012-12-29 Thread Daniel Farina
Attached is a cumulative patch attempting to address the below. One can see the deltas to get there at https://github.com/fdr/postgres.git error-prop-pg_stat_statements-v2. On Fri, Dec 28, 2012 at 9:58 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: However, with this approach, calls_underest

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2012-12-29 Thread Daniel Farina
On Sat, Dec 29, 2012 at 4:21 AM, Daniel Farina drfar...@acm.org wrote: These were not express goals of the patch, but so long as you are inviting features, attached is a bonus patch that exposes the queryid and also the notion of a statistics session that is re-rolled whenever the stats file

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2012-12-29 Thread Daniel Farina
On Sat, Dec 29, 2012 at 6:37 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 29 December 2012 12:21, Daniel Farina drfar...@acm.org wrote: These were not express goals of the patch, but so long as you are inviting features, attached is a bonus patch that exposes the queryid and also

  1   2   3   4   5   >