Re: [HACKERS] Schema version management

2012-05-22 Thread Daniel Farina
On Mon, May 21, 2012 at 5:25 PM, Joel Jacobson j...@trustly.com wrote: If one want to reuse the splitting to files-code of the directory format, maybe the existing option -F d could be tweaked to output in both a a machine-readable format (current way), and also a human-friendly tree of files

Re: [HACKERS] Schema version management

2012-05-21 Thread Daniel Farina
On Sun, May 20, 2012 at 9:03 PM, Joel Jacobson j...@trustly.com wrote: On Mon, May 21, 2012 at 10:06 AM, Daniel Farina dan...@heroku.com wrote: Also, now that I look more carefully, there was a lot of conversation about this patch; it seems like what you are doing now is reporting its

Re: [HACKERS] Schema version management

2012-05-20 Thread Daniel Farina
On Sun, May 20, 2012 at 12:41 PM, Joel Jacobson j...@trustly.com wrote: Hi, I just read a very interesting post about schema version management. Quote: You could set it up so that every developer gets their own test database, sets up the schema there, takes a dump, and checks that in. There

Re: [HACKERS] Schema version management

2012-05-20 Thread Daniel Farina
On Sun, May 20, 2012 at 7:36 PM, Joel Jacobson j...@trustly.com wrote: On Mon, May 21, 2012 at 8:08 AM, Daniel Farina dan...@heroku.com wrote: I think you are absolutely right, but I'm not sure if teaching pg_dump a new option is the best idea.  It's a pretty complex program as-is. I've also

Re: [HACKERS] External Open Standards

2012-05-20 Thread Daniel Farina
On Sun, May 20, 2012 at 10:34 PM, Brendan Jurd dire...@gmail.com wrote: What we don't do is *output* the 'T', but this is pretty easy to workaround, e.g., to_char(now(), '-MM-DDTHH24:MI:SS').  The scope of  actually wanting the 'T' is surely pretty minor? I'd be okay with just adding a

Re: [HACKERS] External Open Standards

2012-05-19 Thread Daniel Farina
On Sat, May 12, 2012 at 5:37 AM, Simon Riggs si...@2ndquadrant.com wrote: Do we have a full list of externally defined open standards that we follow? Are there any known incompatibilities from externally defined open standards? (I know about the SQL standard stuff). The documentation is

Re: [HACKERS] Foreign keys in pgbench

2012-05-19 Thread Daniel Farina
On Sun, May 13, 2012 at 3:03 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 13 May 2012 18:07, Jeff Janes jeff.ja...@gmail.com wrote: I think that pgbench should it make it easy to assess the impact of foreign key constraints. I agree in principle.  I favour being more inclusive about

Re: [HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 2:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, May 2, 2012 at 9:38 PM, Daniel Farina dan...@heroku.com wrote: Besides accuracy, there is a thornier problem here that has to do with hot standby (although the use case is replication more generally) when one has

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 12:51 PM, james ja...@mansionfamily.plus.com wrote: I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient

Re: [HACKERS] CLOG extension

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 1:56 PM, Robert Haas robertmh...@gmail.com wrote: Possibly.  I have some fear of ending up with too many background processes, but we may need them. I sort of care about this, but only on systems that are not very busy and could otherwise get by with fewer resources --

Re: [HACKERS] CLOG extension

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 2:26 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: I'm not sure I see the point in worrying about this at all.  I mean, a process doing nothing does not waste much resources, does it?  Other than keeping a PID that you can't use for other stuff. Not much, but we

Re: [HACKERS] CLOG extension

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 2:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Daniel Farina's message of jue may 03 17:04:03 -0400 2012: I sort of care about this, but only on systems that are not very busy and could otherwise get by with

[HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-02 Thread Daniel Farina
Hello List, I'd like to share with you some experiences we've had while investigating what we'd have to do to make very-very tiny databases. First, the formulae at http://www.postgresql.org/docs/9.1/static/kernel-resources.html#SHARED-MEMORY-PARAMETERS (17-2) seem misleading, particularly with

Re: [HACKERS] Torn page hazard in ginRedoUpdateMetapage()

2012-05-02 Thread Daniel Farina
On Wed, May 2, 2012 at 6:06 PM, Noah Misch n...@leadboat.com wrote: Can we indeed assume that all support-worthy filesystems align the start of every file to a physical sector?  I know little about modern filesystem design, but these references leave me wary of that assumption:

Re: [HACKERS] Last gasp

2012-04-11 Thread Daniel Farina
On Tue, Apr 10, 2012 at 8:12 PM, Robert Haas robertmh...@gmail.com wrote: However exactly the list turns out, there is no question that non-committers have been quite successful in getting significant feature enhancements committed in each of the last three releases, and I'm pretty confident

Re: [HACKERS] Last gasp

2012-04-05 Thread Daniel Farina
On Thu, Apr 5, 2012 at 1:40 PM, Robert Haas robertmh...@gmail.com wrote: Yep.  I think Tom and I and others were all pretty clear that there were not enough people reviewing this CommitFest, Sorry to derail the thread just a little, but I've thought a little about this and I wonder if part of

Re: [HACKERS] Faster compression, again

2012-04-04 Thread Daniel Farina
On Tue, Apr 3, 2012 at 7:29 AM, Huchev hugochevr...@gmail.com wrote: For a C implementation, it could interesting to consider LZ4 algorithm, since it is written natively in this language. In contrast, Snappy has been ported to C by Andy from the original C++ Google code, which lso translate

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-04-02 Thread Daniel Farina
On Sat, Mar 31, 2012 at 6:37 AM, Dobes Vandermeer dob...@gmail.com wrote: On Sat, Mar 31, 2012 at 1:44 AM, Daniel Farina dan...@heroku.com wrote: On Fri, Mar 30, 2012 at 10:21 AM, Daniel Farina dan...@heroku.com wrote: Any enhancement here that can't be used with libpq via, say, drop-in .so

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-30 Thread Daniel Farina
On Thu, Mar 29, 2012 at 10:55 PM, Dobes Vandermeer dob...@gmail.com wrote: Virtual hosts. Same port. In that case, the frontend would not be tied to a specific PostgreSQL server, then?  I think initially this might complicate things a bit, and you could solve it by putting an HTTP proxy in

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-30 Thread Daniel Farina
On Fri, Mar 30, 2012 at 9:11 AM, Andrew Dunstan and...@dunslane.net wrote: On 03/30/2012 11:41 AM, Robert Haas wrote: On Fri, Mar 30, 2012 at 10:55 AM, Dobes Vandermeerdob...@gmail.com  wrote: Well, in our case HTTP is a clear win (but not replacement) and SPDY a potential one (even as a

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-30 Thread Daniel Farina
On Fri, Mar 30, 2012 at 10:21 AM, Daniel Farina dan...@heroku.com wrote: Any enhancement here that can't be used with libpq via, say, drop-in .so seems unworkable to me, and that's why any solution that is basically proxying to the database is basically a non-starter outside the very earliest

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-29 Thread Daniel Farina
On Mon, Mar 26, 2012 at 12:14 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2012-03-20 at 01:38 -0700, Daniel Farina wrote: On Thu, Mar 15, 2012 at 9:39 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Mar 16, 2012 at 8:14 AM, Daniel Farina dan...@heroku.com wrote: Parallel

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-29 Thread Daniel Farina
On Thu, Mar 29, 2012 at 8:04 AM, Andrew Dunstan and...@dunslane.net wrote: 1. I've been in discussion with some people about adding simple JSON extract functions. We already have some (i.e. xpath()) for XML. 2. You might find htsql http://htsql.org/ interesting. My colleagues and myself have

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-29 Thread Daniel Farina
On Thu, Mar 29, 2012 at 12:57 PM, Daniel Farina dan...@heroku.com wrote: D'oh, I munged the order. More technical concerns: * Protocol compression -- but a bit of sand in the gears is *which* compression -- for database workloads, the performance of zlib can be a meaningful bottleneck

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-29 Thread Daniel Farina
On Thu, Mar 29, 2012 at 6:25 PM, Dobes Vandermeer dob...@gmail.com wrote: 2. You might find htsql http://htsql.org/ interesting. As a reference, or should we just bundle / integrate that with PostgreSQL somehow? It's a totally different language layer without wide-spread popularity and, as

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-29 Thread Daniel Farina
On Thu, Mar 29, 2012 at 6:37 PM, Dobes Vandermeer dob...@gmail.com wrote: On Fri, Mar 30, 2012 at 3:59 AM, Daniel Farina dan...@heroku.com wrote: On Thu, Mar 29, 2012 at 12:57 PM, Daniel Farina dan...@heroku.com wrote: More technical concerns: * Protocol compression -- but a bit of sand

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-27 Thread Daniel Farina
On Mon, Mar 26, 2012 at 4:53 PM, Robert Haas robertmh...@gmail.com wrote: I think the more important question is a policy question: do we want it to work like this?  It seems like a policy question that ought to be left to the DBA, but we have no policy management framework for DBAs to

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-27 Thread Daniel Farina
On Tue, Mar 27, 2012 at 10:46 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Isn't it the case that many web applications run under some common database user regardless of the underlying webapp user?  I wouldn't say that's an unimportant case.  Granted, the webapp user wouldn't have

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-27 Thread Daniel Farina
On Tue, Mar 27, 2012 at 10:48 AM, Daniel Farina dan...@heroku.com wrote: On Tue, Mar 27, 2012 at 10:46 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Isn't it the case that many web applications run under some common database user regardless of the underlying webapp user?  I wouldn't say

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-27 Thread Daniel Farina
On Tue, Mar 27, 2012 at 1:30 PM, Andrew Dunstan and...@dunslane.net wrote: Well, that does sort of leave an arguable vulnerability.  Should the same user only be allowed to kill the process from a connection to the same database? It might be a reasonable restriction in theory, but I doubt

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-26 Thread Daniel Farina
On Mon, Mar 26, 2012 at 1:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not.  I still wouldn't trust SIGTERMing an individual backend in a production database.  It'll probably work, but what if it doesn't? Best-case scenario is you'll need to do a panic shutdown to clear the stuck lock or

Re: [HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-23 Thread Daniel Farina
On Fri, Mar 23, 2012 at 1:52 AM, Simon Riggs si...@2ndquadrant.com wrote: So we have this? Master pg_controldata - OK txid_current_snapshot() - OK Standby pg_controldata - OK txid_current_snapshot() - lower value Are there just 2 standbys? So all standbys have acted identically? Yes, I

[HACKERS] Standbys, txid_current_snapshot, wraparound

2012-03-22 Thread Daniel Farina
Some time ago I reported bug 6291[0], which reported a Xid wraparound, both as reported in pg_controldata and by txid_current_snapshot. Unfortunately, nobody could reproduce it. Today, the same system of ours just passed the wraparound mark successfully at this time, incrementing the epoch.

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-20 Thread Daniel Farina
On Thu, Mar 15, 2012 at 9:39 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Mar 16, 2012 at 8:14 AM, Daniel Farina dan...@heroku.com wrote: Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-20 Thread Daniel Farina
On Mon, Mar 19, 2012 at 9:08 PM, Robert Haas robertmh...@gmail.com wrote: It's after midnight here so maybe I'm being slow, but I don't understand what problem the SessionId solves.  ISTM that you could solve the problem like this: 1. Acquire ProcArrayLock in exclusive mode, to keep the set

Re: [HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-20 Thread Daniel Farina
On Tue, Mar 20, 2012 at 10:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Maybe we should just not worry about this. That's been my reaction right along.  There's no evidence that PID recycling is a problem in the real world. I'm entirely willing to

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-19 Thread Daniel Farina
On Sun, Mar 18, 2012 at 7:35 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 19 March 2012 01:50, Tom Lane t...@sss.pgh.pa.us wrote: I am *not* a fan of regression tests that try to microscopically test every feature in the system. I see your point of view. I suppose I can privately hold

Re: [HACKERS] Gsoc2012 Idea --- Social Network database schema

2012-03-19 Thread Daniel Farina
On Mon, Mar 19, 2012 at 6:17 PM, Josh Berkus j...@agliodbs.com wrote: On 3/18/12 8:11 PM, HuangQi wrote: The implementation seems to be done quite fully. There is even a patch file. Why is the implementation not added into the release of Postgres? As so much has already being done, what could

Re: [HACKERS] Incorrect assumptions with low LIMITs

2012-03-19 Thread Daniel Farina
On Mon, Mar 19, 2012 at 6:19 PM, Jeff Davis pg...@j-davis.com wrote: On Sat, 2012-03-17 at 12:48 +, Simon Riggs wrote: The problems are as I described them (1) no account made for sparsity, and other factors leading to an overestimate of rows (N) (2) inappropriate assumption of the

Re: [HACKERS] Gsoc2012 Idea --- Social Network database schema

2012-03-18 Thread Daniel Farina
On Sat, Mar 17, 2012 at 8:48 PM, HuangQi huangq...@gmail.com wrote:     About the second topic, so currently TABLESAMPLE is not implemented inside Postgres? I didn't see this query before, but I googled it just now and the query seems very weird and interesting. 

[HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-17 Thread Daniel Farina
This thread evolved out of an attempt to implement pg_terminate_backend for non-superusers. I thought -- probably erroneously -- that the major objection to that was the known possibility of a PID-cycling race condition, whereby a signal could be misdirected, in the case of terminate_backend,

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-17 Thread Daniel Farina
On Sat, Mar 17, 2012 at 1:48 PM, Andrew Dunstan and...@dunslane.net wrote: Mine too. We don't want a column ordering that's different for everyone. That's a recipe for mass confusion. We want to be able to mutate the ordering for everyone, and for everyone to see the same ordering. That means

Re: [HACKERS] Gsoc2012 Idea --- Social Network database schema

2012-03-17 Thread Daniel Farina
On Sat, Mar 17, 2012 at 8:50 AM, HuangQi huangq...@gmail.com wrote:     I'm quite glad if you could offer me some advices. Thanks a lot for your help! Thank you for your interest! However, I am a little confused precisely what you are thinking about implementing. Are there particular access

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Thu, Mar 15, 2012 at 10:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Thu, Mar 15, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: But actually I don't see what you hope to gain from such a change, even if it can be made to work.  Anyone who can

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Thu, Mar 15, 2012 at 11:01 PM, Daniel Farina dan...@heroku.com wrote: Okay, well, I believe there is a race in handling common administrative signals that *might* possibly matter.  In the past, pg_cancel_backend was superuser only, which is a lot like saying only available to people who can

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Fri, Mar 16, 2012 at 3:42 PM, Noah Misch n...@leadboat.com wrote: On Thu, Mar 15, 2012 at 04:14:03PM -0700, Daniel Farina wrote: Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any transactions

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Fri, Mar 16, 2012 at 4:42 PM, Daniel Farina dan...@heroku.com wrote: Hmm. Well, here's a patch that implements exactly that, I think, That version had some screws loose due to some editor snafus. Hopefully all better. -- fdr Implement-race-free-sql-originated-backend-cancellation-v3

[HACKERS] Another review of URI for libpq, v7 submission

2012-03-15 Thread Daniel Farina
I reviewed this and so far have not found any serious problems, although as is par for the course it contains some of the fiddly bits involved in any string manipulations in C. I made a few edits -- none strictly necessary for correctness -- that the original author is free audit and/or

Re: [HACKERS] Another review of URI for libpq, v7 submission

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 7:36 AM, Alex Shulgin a...@commandprompt.com wrote: I wonder if there's any evidence as to that mangling the email addresses helps to reduce spam at all?  I mean replacing (at) back to @ and (dot) to . is piece of cake for a spam crawler. I suspect we're long past the

Re: [HACKERS] Keystone auth in PostgreSQL

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 1:14 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Mar 14, 2012 at 11:38:19AM +0530, Vivek Singh Raghuwanshi wrote: Hi All, Can i use keystone auth with PostgreSQL, it is very helpful when i am using OpenStack as a cloud service and implement DBaaS. I don't think

Re: [HACKERS] Faster compression, again

2012-03-15 Thread Daniel Farina
On Wed, Mar 14, 2012 at 11:06 AM, Daniel Farina dan...@heroku.com wrote: ...and it has been ported to C (recently, and with some quirks, like no LICENSE file...yet, although it is linked from the original Snappy project). I poked the author about the license and he fixed it in a jiffy. Now

Re: [HACKERS] Faster compression, again

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 3:14 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 6:06 PM, Daniel Farina dan...@heroku.com wrote: If we're curious how it affects replication traffic, I could probably gather statistics on LZO-compressed WAL traffic, of which we have a pretty

[HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any transactions in progress, including the idle transaction, and closing the socket. I imagine the problem is a race condition whereby a pid might be

Re: [HACKERS] Keystone auth in PostgreSQL

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 6:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Our standard answer when someone asks for $random-auth-method is to suggest that they find a PAM module for it and use PAM.  I wouldn't want to claim that PAM is a particularly great interface for this sort of thing, but it's

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 9:39 PM, Fujii Masao masao.fu...@gmail.com wrote: Shall we just do everything using the MyCancelKey (which I think could just be called SessionKey, SessionSecret, or even just Session) as to ensure we have no case of mistaken identity? Or does that end up being

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: The way MyCancelKey is checked now is backwards, in my mind.  It seems like it would be better checked by the receiving PID (one can use a check/recheck also, if so inclined

[HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
For 9.3 at a minimum. The topic of LZO became mired in doubts about: * Potential Patents * The author's intention for the implementation to be GPL Since then, Google released Snappy, also an LZ77-class implementation, and it has been ported to C (recently, and with some quirks, like no LICENSE

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
On Wed, Mar 14, 2012 at 2:03 PM, Merlin Moncure mmonc...@gmail.com wrote: er, typo: I meant to say: *non-gpl* lz based...  :-). Given that, few I would say have had the traction that LZO and Snappy have had, even though in many respects they are interchangeable in the general trade-off spectrum.

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
On Wed, Mar 14, 2012 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Given that, few I would say have had the traction that LZO and Snappy have had, even though in many respects they are interchangeable in the general trade-off spectrum. The question

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Mon, Mar 12, 2012 at 8:10 PM, Bruce Momjian br...@momjian.us wrote: To answer your specific question, I think clearing the last analyzed fields should cause autovacuum to run on analyze those tables.  What I don't know is whether not clearing the last vacuum datetime will cause the table

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Mon, Mar 12, 2012 at 9:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Copying the statistics from the old server is on the pg_upgrade TODO list.  I have avoided it because it will add an additional requirement that will make pg_upgrade more fragile in case

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 3:30 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 13, 2012 at 5:42 PM, Bruce Momjian br...@momjian.us wrote: What is the target=10 duration?  I think 10 is as low as we can acceptably recommend.  Should we recommend they run vacuumdb twice, once with

Re: [HACKERS] Chronic performance issue with Replication Failover and FSM.

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 4:53 PM, Josh Berkus j...@agliodbs.com wrote: All, I've discovered a built-in performance issue with replication failover at one site, which I couldn't find searching the archives.  I don't really see what we can do to fix it, so I'm posting it here in case others

Re: [HACKERS] Chronic performance issue with Replication Failover and FSM.

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 7:05 PM, Fujii Masao masao.fu...@gmail.com wrote: If it's really a high-UPDATE workload, wouldn't autovacuum start soon? Also, while vacuum cleanup records are applied, could not the standby also update its free space map, without having to send the actual FSM updates? I

Re: [HACKERS] query planner does not canonicalize infix operators

2012-03-12 Thread Daniel Farina
On Mon, Mar 12, 2012 at 7:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Will Leinweber w...@heroku.com writes: I created an index on an hstore function, fetchval(hstore, text), however when I use the - infix operator which resolves to the very same function, this index is not used. It should be

Re: [HACKERS] query planner does not canonicalize infix operators

2012-03-12 Thread Daniel Farina
On Mon, Mar 12, 2012 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Mon, Mar 12, 2012 at 7:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Will Leinweber w...@heroku.com writes: I created an index on an hstore function, fetchval(hstore, text), however

[HACKERS] pg_upgrade and statistics

2012-03-12 Thread Daniel Farina
As noted by the manual, pg_statistic is ported in any way when performing pg_upgrade. I have been investigating what it would take to (even via just a connected SQL superuser client running UPDATE or INSERT against pg_statistic) get at least some baseline statistics into the database as quickly

Re: [HACKERS] pg_stat_statements and planning time

2012-03-07 Thread Daniel Farina
On Wed, Mar 7, 2012 at 8:07 AM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: In the patch, I didn't change the column name total_time meaning the time spent in the executor because of the backward compatibility. But once new column plan_time is added, total_time

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Daniel Farina
On Tue, Mar 6, 2012 at 3:31 PM, Andrew Dunstan and...@dunslane.net wrote: We don't slavishly need to reproduce every piece of cron. In any case, on my Linux machine at least, batch is part of the at package, not the cron package. If you want anything at all done, then I'd suggest starting with

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Daniel Farina
On Mon, Mar 5, 2012 at 12:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2012/3/5 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: Dear Developers,    I am looking for elegant and effective way for running

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut pete...@gmx.net wrote: On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote: I'd really like to support libraries (C or otherwise) of multiple versions at the same time, when the underlying library permits. What's preventing you from doing

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote: But is it unsurmountable? -- dlsym returns a function pointer, and one would build up the operator table for the version of the extension at hand, so one might have ltree version 1.01 and ltree version 2.3 fields in the

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Daniel Farina
On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: My expectation is that this feature will make life a lot easier for a lot of Postgres users. Yes. It's hard to overstate the apparent utility of this feature in the general category of visibility and profiling. --

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-29 Thread Daniel Farina
On Mon, Feb 27, 2012 at 4:26 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: This does not appear to have any user-visible effect on caret position for all variations in coercion syntax, while giving me everything that I need. I had assumed that we were relying on things being this way, but

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-28 Thread Daniel Farina
On Sun, Feb 26, 2012 at 7:36 AM, Peter Eisentraut pete...@gmx.net wrote: On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote: Well, I'm trying to invoke the extension's make check target at extension build time. I do have a temporary installation I own somehwere in my $HOME, but that is

Re: [HACKERS] pg_stat_statements normalization: re-review

2012-02-24 Thread Daniel Farina
]: https://github.com/fdr/postgres/tree/wrench-pgss From c3ad77375062cfbeee7d4ce7e0fe274a5db76453 Mon Sep 17 00:00:00 2001 From: Daniel Farina dan...@heroku.com Date: Fri, 24 Feb 2012 01:31:54 -0800 Subject: [PATCH] Introduce NodeKey as a service to extensions in the backend Signed-off-by: Daniel

Re: [HACKERS] pg_stat_statements normalization: re-review

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 3:14 AM, Daniel Farina dan...@heroku.com wrote: At ExecutorFinish (already hookable) all NodeKeys remembered by an extension should be invalidated, as that memory is free and ready to be used again. I think this statement is false; I thought it was true because

Re: [HACKERS] ISO8601 nitpicking

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 4:45 AM, Peter Eisentraut pete...@gmx.net wrote: On tor, 2012-02-23 at 23:41 -0800, Daniel Farina wrote: As it turns out, evidence would suggests that the ISO output in Postgres isn't, unless there's an ISO standard for date and time that is referring to other than 8601

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentraut pete...@gmx.net wrote: On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote: We don't initdb with PostGIS regression testing framework but I've considered doing it for this specific case and it stroke me that even then we couldn't control

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 6:31 PM, Andrew Dunstan and...@dunslane.net wrote: Really? Here's what I just got on a severely under-resourced SL6 VM: 1.5s doesn't seem terribly slow. You are right. Come to think of it, I do seem to recall that initdb got some speed improvements; these were in 8.3

[HACKERS] pg_stat_statements normalization: re-review

2012-02-23 Thread Daniel Farina
Hello again, I'm reviewing the revised version of pg_stat_statements again. In particular, this new version has done away with the mechanical but invasive change to the lexing that preserved *both* the position and length of constant values. (See

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-23 Thread Daniel Farina
On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Sandro Santilli s...@keybit.net writes: Please see the inline extension thread where answers to your problem have been discussed. I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of no help here.

[HACKERS] ISO8601 nitpicking

2012-02-23 Thread Daniel Farina
As it turns out, evidence would suggests that the ISO output in Postgres isn't, unless there's an ISO standard for date and time that is referring to other than 8601. It does not permit use of a space between the date and the time, as seen in: SELECT now(); now

Re: [HACKERS] Should we add crc32 in libpgport?

2012-02-22 Thread Daniel Farina
On Thu, Feb 16, 2012 at 6:09 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 3, 2012 at 7:33 PM, Daniel Farina dan...@heroku.com wrote: Ah, yes, I think my optimizations were off when building, or something.  I didn't get such verbosity at first, and then I remember doing something

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-03 Thread Daniel Farina
On Thu, Feb 2, 2012 at 8:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: It's a bit disturbing that nobody has reported this from the field yet. Seems to imply that hot standby isn't being used much. I have seen this, but didn't get to dig in, as I thought it could be a problem from other things done

Re: [HACKERS] Should we add crc32 in libpgport?

2012-02-03 Thread Daniel Farina
On Tue, Jan 31, 2012 at 3:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Jan 17, 2012 at 2:14 AM, Daniel Farina dan...@heroku.com wrote: See the attached patch.  It has a detailed cover letter/comment at the top of the file. I have amended

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Sun, Jan 22, 2012 at 8:42 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 22, 2012 at 3:20 PM, Daniel Farina dan...@heroku.com wrote: A few anecdotes does not constitute evidence, but it does look like some people pay attention to any additional versioning foothold they can get

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Mon, Jan 23, 2012 at 8:17 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Ok, but then, what about .so files?  Wouldn't it make sense to be able to ship also the executable modules needed, and if not, why not? Now you can dump/restore any

Re: [HACKERS] Inline Extension

2012-01-22 Thread Daniel Farina
On Fri, Jan 20, 2012 at 3:33 PM, Daniel Farina dan...@heroku.com wrote: On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Even if you give the version number in the CREATE EXTENSION command, it's by convention that people actually maintain a sane

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Thu, Jan 19, 2012 at 8:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Frankly I don't see the point of this. If the extension is an independent piece of (SQL) code, developed separately from an application, with its own lifecycle, a

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Even if you give the version number in the CREATE EXTENSION command, it's by convention that people actually maintain a sane versioning policy. If people don't take version management seriously, you

[HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
I have been working with xlogdump and noticed that unfortunately it cannot be installed without access to a postgres build directory, which makes the exported functionality in src/include/utils/pg_crc.h useless unless one has access to pg_crc.o -- which would only happen if a build directory is

Re: [HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 3:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, short of seeing an acceptable patch for the larger thing, I don't want to accept a patch to add that field to Const, because I think it's a kluge.  I'm still feeling that there must be a better way ... Hm. Maybe it is

Re: [HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 4:56 PM, Daniel Farina dan...@heroku.com wrote: I have been working with xlogdump and noticed that unfortunately it cannot be installed without access to a postgres build directory, which makes the exported functionality in src/include/utils/pg_crc.h useless unless one

Re: [HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 6:18 PM, Robert Haas robertmh...@gmail.com wrote: I think you make a compelling case. That's enough for me to just do it. Expect a patch soon. -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

[HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-15 Thread Daniel Farina
I've *finally* gotten around to reviewing this patch. My first step was to de-bitrot it very slightly. More on that in a moment. After that, I tried using it. Installation worked nicely -- I did CREATE EXTENSION and then tried reading from pg_stat_statements. I was then given an error message

[HACKERS] pg_internal.init and an index file have the same inode

2012-01-03 Thread Daniel Farina
I'm not sure if this is an XFS problem, or Postgres. There's enough suspicious evidence that it's too hard to say. Today, I get an interesting issue raised whereby a reasonably simple query fails on a system that does take successful pg_dumps regularly. To make a short story shorter, I end up

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2012-01-01 Thread Daniel Farina
On Sun, Jan 1, 2012 at 5:18 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: That's awfully complicated. If we're going to require co-operation from the backup/archiving software, we might as well just change the procedure so that backup_label is not stored in the data

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2012-01-01 Thread Daniel Farina
On Sun, Jan 1, 2012 at 6:13 AM, Magnus Hagander mag...@hagander.net wrote: It also doesn't affect backups taken through pg_basebackup - but I guess you have good reasons for not being able to use that? Parallel archiving/de-archiving and segmentation of the backup into pieces and rate limiting

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2011-12-29 Thread Daniel Farina
On Sat, Dec 3, 2011 at 8:04 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: At the moment, if the situation is ambiguous, the system assumes that you're restoring from a backup. What your suggestion amounts to is to reverse tht assumption, and assume instead that you're doing

<    1   2   3   4   5   >