[HACKERS] Segv in pg_autovacuum in 8.0.x

2005-10-19 Thread daveg
Apologies if this is old news, but pg_autovacuum in 8.0.x has the bad habit of SEGVing and exiting when a table gets dropped out from under it. This creates problems if you rely on pg_autovacuum for the bulk of your vacuuming as it forgets it's statistics when it is restarted and so will skip

Re: [HACKERS] PG Killed by OOM Condition

2005-10-24 Thread daveg
On Mon, Oct 24, 2005 at 11:26:52PM -0500, Bruno Wolff III wrote: On Mon, Oct 24, 2005 at 23:55:07 -0400, [EMAIL PROTECTED] wrote: On Mon, Oct 24, 2005 at 10:20:39PM -0500, Bruno Wolff III wrote: On Mon, Oct 03, 2005 at 23:03:06 +1000, John Hansen [EMAIL PROTECTED] wrote: Good

Re: [HACKERS] Exclusive lock for database rename

2005-11-08 Thread daveg
On Tue, Nov 08, 2005 at 03:14:34PM -0600, Jim C. Nasby wrote: On Sat, Nov 05, 2005 at 11:48:56AM +0100, Martijn van Oosterhout wrote: On Sat, Nov 05, 2005 at 10:47:30AM +0100, Jochem van Dieten wrote: On 11/4/05, Jim C. Nasby wrote: I would argue that in cases like this (and 'this'

Re: [HACKERS] Exclusive lock for database rename

2005-11-09 Thread daveg
On Wed, Nov 09, 2005 at 09:41:49AM +0100, Martijn van Oosterhout wrote: On Tue, Nov 08, 2005 at 04:06:32PM -0800, daveg wrote: I think this wait with an exponentially rising delay hurts not helps. If the stricter lock can be granted in a short time, ie the dalay could be small

Re: [HACKERS] MERGE vs REPLACE

2005-11-16 Thread daveg
On Wed, Nov 16, 2005 at 09:49:28AM -0500, Tom Lane wrote: I think we should do REPLACE-like functionality that simply fails if the match condition isn't equality on a primary key. If we can use SQL-spec MERGE syntax for this, that's fine, but let's not think in terms of silently changing to a

Re: [PATCHES] [HACKERS] Automatically setting work_mem

2006-04-22 Thread daveg
On Sat, Apr 22, 2006 at 06:38:53PM +0100, Simon Riggs wrote: On Sat, 2006-04-22 at 13:17 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I still do, for multi-user systems. Releasing unused memory from a large CREATE INDEX will allow that memory to be swapped out, even if the

Re: [PATCHES] [HACKERS] Automatically setting work_mem

2006-04-22 Thread daveg
On Sat, Apr 22, 2006 at 01:49:25PM -0700, David Fetter wrote: On Sat, Apr 22, 2006 at 01:14:42PM -0700, David Gould wrote: To avoid running out of swap and triggering the oom killer we have had to reduce work_mem below what we prefer. Dunno about your work_mem, but you can make sure the

Re: [HACKERS] IN vs EXISTS equivalence

2008-09-03 Thread daveg
On Thu, Aug 14, 2008 at 06:50:09PM +0100, Simon Riggs wrote: On Fri, 2008-08-08 at 16:23 -0400, Tom Lane wrote: NOT IN is a lot trickier, condition: you must also assume that the comparison operator involved never yields NULL for non-null inputs. That might be okay for btree

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 03:36:19PM -0400, Tom Lane wrote: Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= [EMAIL PROTECTED] writes: AFAICS, PostgreSQL is not keeping info about when a table, database, sequence, etc was created. We cannot get that info even from OS, since CLUSTER or VACUUM FULL may

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 11:03:56PM +0300, Hannu Krosing wrote: On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: I'd be very interested in seeing a last schema modification time for pg_class objects. I don't care about it being preserved over dump and restore as my use case is more

Re: [HACKERS] pg_dump enhancement proposal

2009-12-13 Thread daveg
On Thu, Nov 12, 2009 at 04:31:37PM -0500, Tom Lane wrote: Mark Hammonds mhammo...@omniti.com writes: 2. Custom Query Exports In my use of mysqldump, I found one feature very useful: the ability to execute a custom SELECT. . .WHERE statement and then dump only the results. This

Re: [HACKERS] TCP keepalive support for libpq

2010-02-10 Thread daveg
On Tue, Feb 09, 2010 at 09:34:10AM -0500, Andrew Chernow wrote: Tollef Fog Heen wrote: (please Cc me on replies, I am not subscribed) Hi, libpq currently does not use TCP keepalives. This is a problem in our case where we have some clients waiting for notifies and then the connection is

Re: [HACKERS] a faster compression algorithm for pg_dump

2010-04-14 Thread daveg
On Tue, Apr 13, 2010 at 03:03:58PM -0400, Tom Lane wrote: Joachim Wieland j...@mcknight.de writes: If we still cannot do this, then what I am asking is: What does the project need to be able to at least link against such a compression algorithm? Well, what we *really* need is a

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-03 Thread daveg
of different job-scheduling behavior ... but there's no evidence here of a consistently measurable benefit or loss from that. IIRC daveg was volunteering to do some tests with his own data; maybe we should wait for those results. I have run extensive tests with three trials of each configuration

Re: [HACKERS] Alpha releases: How to tag

2009-08-08 Thread daveg
On Fri, Aug 07, 2009 at 06:28:34PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: I am not suggesting that this change be immediate, and it's not ivory tower. It's just how everybody else does it. You keep saying that, and it's completely meaningless. What do you know

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sat, Aug 15, 2009 at 11:34:04PM +0200, Dimitri Fontaine wrote: Nitpicking dept, I think I prefer: DO [ [LANGUAGE] language] $$ ... $$; DO plperl $$ ... $$; DO language plpython $$ ... $$; language is optional and defaults to plpgsql. +1 -dg -- David Gould da...@sonic.net

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sun, Aug 16, 2009 at 02:59:53PM +0200, Pavel Stehule wrote: 2009/8/16 Peter Eisentraut pete...@gmx.net: On sön, 2009-08-16 at 00:04 -0400, Andrew Dunstan wrote: SQL is not Lisp. Simple is  good. I didn't think Peter was really very serious. Well, I don't know if we really need to

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 09:38:15PM +0200, Dimitri Fontaine wrote: Exactly, and I think that what we're missing here is a simple tool for our users to check a new PostgreSQL release against their existing application. We already know how to either log all queries and analyze the log files

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 08:02:03PM -0700, Ron Mayer wrote: Andrew Dunstan wrote: I don't know of anyone who is likely to want to try out alphas in their normal development environments. The client I approached was specifically prepared to test beta releases that way. Perhaps end-users

Re: [HACKERS] Add YAML option to explain

2009-08-28 Thread daveg
On Fri, Aug 28, 2009 at 04:37:41PM -0700, David E. Wheeler wrote: On Aug 28, 2009, at 3:45 PM, Stephen Frost wrote: +1 from me. I've read the other comments and just plain don't agree with them. It's a small patch, adds a useful format for EXPLAIN, and would be used. One of the best

Re: [HACKERS] Add YAML option to explain

2009-08-31 Thread daveg
On Mon, Aug 31, 2009 at 02:15:08PM -, Greg Sabino Mullane wrote: Greg, can we see a few examples of the YAML output compared to both json and text? ... greg=# explain (format json, analyze on) select * from pg_class where relname ~ 'x' order by 1,2,3; QUERY PLAN

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Thu, Sep 03, 2009 at 07:57:25PM -0400, Andrew Dunstan wrote: daveg wrote: On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. I have a client who uses temp tables heavily, hundreds

Re: [HACKERS] RfD: more powerful any types

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 11:43:32AM -0400, Merlin Moncure wrote: If you are going to use printf format codes, which is good and useful being something of a standard, I'd call routine printf (not format) and actually wrap vsnprintf. The format codes in printf have a very specific meaning:

Re: [HACKERS] Ragged CSV import

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 10:27:06AM +0200, Dimitri Fontaine wrote: Maybe instead of opening FROM for COPY, having it accepted in WITH would be better, the same way (from the user point of view) that DML returning are worked on. ... WITH csv AS ( COPY t FROM stdin CSV ) INSERT INTO

Re: [HACKERS] RfD: more powerful any types

2009-09-17 Thread daveg
On Tue, Sep 15, 2009 at 07:38:18AM +0200, Pavel Stehule wrote: it isn't fair :) why you use $$ without single quote? And still this case should be vulnerable on SQL injection. Maybe you or me knows, what SQL injection means, but beginners knows nothing and this people use following bad code:

Re: [HACKERS] happy birthday Tom Lane ...

2009-09-18 Thread daveg
On Fri, Sep 18, 2009 at 01:04:23PM +0200, Hans-Juergen Schoenig -- PostgreSQL wrote: Tom, On behalf of the entire PostgreSQL team here in Austria I want to wish you a happy birthday. We hope that you fill be a vital part of PostgreSQL for many years to come. Best regards,

Re: [HACKERS] numeric_to_number() function skipping some digits

2009-09-23 Thread daveg
On Tue, Sep 22, 2009 at 10:27:19AM +0530, Jeevan Chalke wrote: It seems that Oracle reads formatting string from right-to-left. Here are few results: ('number','format') == Oracle PG ('34,50','999,99') == 3450340

Re: [HACKERS] Adding \ev view editor?

2009-09-23 Thread daveg
On Mon, Sep 21, 2009 at 02:26:05PM -0400, Andrew Dunstan wrote: andrew=# select pg_get_viewdef('foo',true); pg_get_viewdef -- SELECT 'a'::text AS b, ( SELECT 1 FROM dual) AS x, random() AS y,

Re: [HACKERS] BEGIN TRANSACTION and START TRANSACTION: different error handling

2009-09-24 Thread daveg
On Thu, Sep 24, 2009 at 12:16:43PM +0300, Hannu Krosing wrote: I expect the transaction is aborted and rollback is executed automatically. - this is not how postgreSQL behaves. PostgreSQL needs an explicit end of transaction from client, either COMMIT; or ROLLBACK; when run from psql, they

[HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. The problem this addresses is that sometimes one needs to set work_mem fairly high to get good query plans for large joins. However, some complex queries will then use huge amounts of memory

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 10:35:55AM -0400, Tom Lane wrote: daveg da...@sonic.net writes: I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. Use ulimit. That was my initial thought too. However, ulimit() is documented as superceded

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 11:47:43AM -0400, Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: Tom Lane escreveu: daveg da...@sonic.net writes: I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. Use ulimit

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-01 Thread daveg
On Tue, Sep 29, 2009 at 09:52:06PM +0530, kunal sharma wrote: Hi , We are using Postgres 8.4 and its been found going into recovery mode couple of times. The server process seems to fork another child process which is another postgres server running under same data directory and after

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-02 Thread daveg
On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: daveg escribió: I work with Kunal and have been looking into this. It appears to be the same as the bug described in: http://archives.postgresql.org/pgsql-bugs/2009-09/msg00355.php as I have localized it to a NULL

Re: [HACKERS] Deprecation

2009-10-19 Thread daveg
On Sat, Oct 17, 2009 at 03:01:27PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Sounds like a good reason to remove add_missing_from in 8.5. Seems like the general consensus is that it's okay to do that. I will go make it happen unless somebody squawks pretty soon...

Re: [HACKERS] Application name patch - v2

2009-10-19 Thread daveg
On Mon, Oct 19, 2009 at 01:00:28PM +0100, Dave Page wrote: On Mon, Oct 19, 2009 at 12:57 PM, Pavel Stehule pavel.steh...@gmail.com wrote: It is not practical. I'll log errors. Usually SQL injection generates lot of errors. Loging all statements has not sense. What is difference bad and

Re: [HACKERS] Application name patch - v2

2009-10-20 Thread daveg
On Tue, Oct 20, 2009 at 12:16:42PM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Also, how many platforms can't we do this on? If we have BSD and Windows covered already. on linux, I believe you can easily read it out of /proc/self/cmdline, no? Writing a pile of

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-20 Thread daveg
On Fri, Oct 02, 2009 at 07:57:13PM -0700, daveg wrote: On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: daveg escribió: I work with Kunal and have been looking into this. It appears to be the same as the bug described in: http://archives.postgresql.org/pgsql

Re: [HACKERS] EOL for 7.4?

2009-11-03 Thread daveg
On Tue, Nov 03, 2009 at 10:32:17AM -0800, Josh Berkus wrote: So I'm going to make a case in favor of EOL'ing 7.4. In fact, I'd be in favor of doing so in, say, February after an announcement this month. The main reason I'm in favor of this is that we have a lot of users using 7.4 out of

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:22:01AM +, Greg Stark wrote: Really I think you guys are on the wrong track trying to map Postgres releases to commercial support terms. None of the Postgres releases are supported in the sense that there's no warranty and no promises, it's all best effort. If

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:47:56AM +, Greg Stark wrote: On Fri, Nov 13, 2009 at 2:35 AM, daveg da...@sonic.net wrote: I suggest we announce now that both 7.4 and 8.0 will EOL when 8.5 is expected to ship, or to comfort those who never use .0 versions when 8.5.1 ships. What would

Re: [HACKERS] [PATCHES] TODO item: Have psql show current values for a sequence

2008-05-23 Thread daveg
On Sat, May 24, 2008 at 12:27:16AM -0300, Dickson S. Guedes wrote: Hi all, These patch implements the TODO item: Have psql show current values for a sequence. Comments are welcome. Sequence public.foo_bar_seq +---+-+-+ |Column |

Re: [PATCHES] [HACKERS] Patch for Prevent pg_dump/pg_restore from being affected by statement_timeout

2008-06-23 Thread daveg
On Mon, Jun 23, 2008 at 06:51:28PM -0400, Bruce Momjian wrote: Alex Hunsaker wrote: On Wed, Apr 16, 2008 at 4:54 PM, Alvaro Herrera [EMAIL PROTECTED] wrote: Joshua D. Drake escribi?: That is an interesting idea. Something like: pg_restore -E SET STATEMENT_TIMEOUT=0; SET

Re: [PATCHES] [HACKERS] Patch for Prevent pg_dump/pg_restore from being affected by statement_timeout

2008-06-24 Thread daveg
On Mon, Jun 23, 2008 at 07:30:53PM -0400, Bruce Momjian wrote: daveg wrote: On Mon, Jun 23, 2008 at 06:51:28PM -0400, Bruce Momjian wrote: Alex Hunsaker wrote: On Wed, Apr 16, 2008 at 4:54 PM, Alvaro Herrera [EMAIL PROTECTED] wrote: Joshua D. Drake escribi

Re: [PATCHES] [HACKERS] Patch for Prevent pg_dump/pg_restore from being affected by statement_timeout

2008-06-24 Thread daveg
On Tue, Jun 24, 2008 at 05:34:50PM -0400, Tom Lane wrote: daveg [EMAIL PROTECTED] writes: lock-timeout sets statement_timeout to a small value while locks are being taken on all the tables. Then it resets it to default. So it could reset it to whatever the new default is. reset

Re: [PATCHES] [HACKERS] Patch for Prevent pg_dump/pg_restore from being affected by statement_timeout

2008-06-24 Thread daveg
On Tue, Jun 24, 2008 at 10:41:07PM -0400, Tom Lane wrote: daveg [EMAIL PROTECTED] writes: Are we talking about the same patch? Maybe not --- I thought you were talking about a backend-side behavioral change. Because I don't know what you are refering to with timer management code

Re: [HACKERS] [PATCHES] pg_dump lock timeout

2008-07-03 Thread daveg
On Thu, Jul 03, 2008 at 11:15:10AM +0300, Marko Kreen wrote: On 5/11/08, daveg [EMAIL PROTECTED] wrote: Attached is a patch to add a commandline option to pg_dump to limit how long pg_dump will wait for locks during startup. My quick review: - It does not seem important enough

Re: [HACKERS] [PATCHES] pg_dump lock timeout

2008-07-16 Thread daveg
On Thu, Jul 03, 2008 at 05:55:01AM -0700, daveg wrote: On Thu, Jul 03, 2008 at 11:15:10AM +0300, Marko Kreen wrote: On 5/11/08, daveg [EMAIL PROTECTED] wrote: Attached is a patch to add a commandline option to pg_dump to limit how long pg_dump will wait for locks during startup

Re: [HACKERS] Additional psql requirements

2008-07-25 Thread daveg
On Fri, Jul 25, 2008 at 08:16:59AM +0100, Simon Riggs wrote: On Fri, 2008-07-25 at 10:00 +0900, ITAGAKI Takahiro wrote: Simon Riggs [EMAIL PROTECTED] wrote: * access to version number * simple mechanism for conditional execution * ability to set substitution variables from command

Re: [HACKERS] Adding WHERE clause to pg_dump

2008-07-25 Thread daveg
On Fri, Jul 25, 2008 at 08:26:35PM +0100, Simon Riggs wrote: On Fri, 2008-07-25 at 11:46 -0700, Joshua D. Drake wrote: On Fri, 2008-07-25 at 19:33 +0100, Simon Riggs wrote: On Fri, 2008-07-25 at 13:31 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Attached patch

Re: [HACKERS] Adding WHERE clause to pg_dump

2008-07-25 Thread daveg
On Fri, Jul 25, 2008 at 11:17:20PM +0100, Gregory Stark wrote: Joshua D. Drake [EMAIL PROTECTED] writes: How do we deal with this? pg_dump -w last_update_timestamp ... -t 'table*' What I see is a recipe for inconsistent, un-restorable backups without a user realizing what they have

Re: [HACKERS] [PATCHES] pg_dump additional options for performance

2008-07-27 Thread daveg
On Sun, Jul 27, 2008 at 10:37:34AM +0100, Simon Riggs wrote: On Sat, 2008-07-26 at 11:03 -0700, Joshua D. Drake wrote: 2. We have no concurrency which means, anyone with any database over 50G has unacceptable restore times. Agreed. Also the core reason for wanting -w 3. We have

Re: [HACKERS] Copy storage parameters on CREATE TABLE LIKE/INHERITS

2008-07-30 Thread daveg
On Wed, Jul 30, 2008 at 04:45:47PM +0900, ITAGAKI Takahiro wrote: Here is a patch to copy column storage parameters and reloptions on CREATE TABLE LIKE, which I proposed at: [HACKERS] Uncopied parameters on CREATE TABLE LIKE http://archives.postgresql.org/message-id/[EMAIL PROTECTED]

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-03 Thread daveg
On Sat, Aug 02, 2008 at 09:30:08PM +0200, Hans-Jürgen Schönig wrote: On Aug 2, 2008, at 8:38 PM, Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Hans-Jürgen Schönig wrote: i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: ISTR that what ended up killing the enthusiasm for this was that most people realized that this GUC was just a poor tool to take a stab at solving other problems (ie. rate limiting cpu for queries). I'm not concerned with that,

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 03:09:34PM -0400, Greg Smith wrote: On Mon, 4 Aug 2008, daveg wrote: We load the production dumps into our dev environment, which are the same hardware spec, so the costs should be identical. Not identical, just close. ANALYZE samples data from your table randomly

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 02:35:07PM -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: That's great for you, I am talking in the scope of a general solution. (Note I'd also bet that even given the same hardware, different production loads can produce different

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 11:59:03AM -0700, Josh Berkus wrote: Greg, Well that's going to depend on the application But I suppose there's nothing wrong with having options which aren't always a good idea to use. The real question I guess is whether there's ever a situation where it would

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:17:59PM -0400, Jonah H. Harris wrote: On Mon, Aug 4, 2008 at 5:08 PM, Simon Riggs [EMAIL PROTECTED] wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. This would be a

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:19:50PM -0400, Robert Treat wrote: See, this is what we ended up talking about before. Someone will say I'd like to prevent my devs from accidentally doing queries with cartesian products and they will use this to do it... but that will only work in some cases, so

Re: [HACKERS] Visibility Groups

2008-08-08 Thread daveg
On Thu, Aug 07, 2008 at 01:30:27PM +0100, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly*

Re: [HACKERS] Adjusting debug_print_plan to be more useful by default

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 06:33:33PM +0100, Simon Riggs wrote: On Tue, 2008-08-19 at 12:40 -0400, Tom Lane wrote: Back in April we changed EXPLAIN VERBOSE to not dump the internal plan tree anymore, on the grounds that non-hackers didn't want that info and hackers could get it with

Re: [HACKERS] A smaller default postgresql.conf

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 09:39:39PM +0300, Peter Eisentraut wrote: On Tuesday 19 August 2008 19:12:16 Tom Lane wrote: Well, why not just make a one-eighty and say that the default postgresql.conf is *empty* (except for whatever initdb puts into it)? Well, my original implementation of GUC

Re: [HACKERS] proposal sql: labeled function params

2008-08-23 Thread daveg
On Sat, Aug 23, 2008 at 05:08:25PM +0100, Gregory Stark wrote: Pavel Stehule [EMAIL PROTECTED] writes: Hello 2008/8/23 Peter Eisentraut [EMAIL PROTECTED]: On Friday 22 August 2008 07:41:30 Decibel! wrote: If we're really worried about it we can have a GUC for a few versions that

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: Dann Corbit wrote: The LZMA SDK is granted to the public domain: http://www.7-zip.org/sdk.html I played with this but found the SDK extremely confusing and flat out horrible. One personal dislike was the unnecessary use of

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 02:47:05PM -0500, Bruce Momjian wrote: daveg wrote: On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: Dann Corbit wrote: The LZMA SDK is granted to the public domain: http://www.7-zip.org/sdk.html I played with this but found the SDK

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 08:49:29PM -0500, Robert Haas wrote: Proprietary compression algorithms, even with Postgresql-specific license exceptions? To be fair, lzo is GPL, which is a stretch to consider proprietary. -dg -- David Gould da...@sonic.net 510 536 1443510 282

Re: [HACKERS] display previous query string of idle-in-transaction

2009-07-23 Thread daveg
On Thu, Jun 04, 2009 at 10:22:41PM -0400, Robert Haas wrote: The only thing I don't like about this is that I think it's kind of a hack to shove the IDLE in transaction designation and the query string into the same database column. I've never liked having to write: select sum(1) from

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-07-31 Thread daveg
On Thu, Jul 30, 2009 at 12:29:34PM -0500, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: I think we've pretty much established that it doesn't make things *worse*, so I'm sort of inclined to go ahead and apply it. The theoretical advantage of eliminating O(N^2) search

[HACKERS] error: could not find pg_class tuple for index 2662

2011-07-27 Thread daveg
My client has been seeing regular instances of the following sort of problem: ... 03:06:09.453 exec_simple_query, postgres.c:900 03:06:12.042 XX000: could not find pg_class tuple for index 2662 at character 13 03:06:12.042 RelationReloadIndexInfo, relcache.c:1740 03:06:12.042 INSERT INTO

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-28 Thread daveg
On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: On Wed, Jul 27, 2011 at 8:28 PM, daveg da...@sonic.net wrote: My client has been seeing regular instances of the following sort of problem: On what version of PostgreSQL? 9.0.4. I previously said: This occurs on postgresql

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-29 Thread daveg
On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: On Thu, Jul 28, 2011 at 5:46 PM, daveg da...@sonic.net wrote: On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: On Wed, Jul 27, 2011 at 8:28 PM, daveg da...@sonic.net wrote: My client has been seeing regular instances

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-29 Thread daveg
On Fri, Jul 29, 2011 at 09:55:46AM -0400, Tom Lane wrote: The thing that was bizarre about the one instance in the buildfarm was that the error was persistent, ie, once a session had failed all its subsequent attempts to access pg_class failed too. I gather from Dave's description that it's

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Thu, Jul 28, 2011 at 11:31:31PM -0700, daveg wrote: On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: REINDEX. My guess is that this is happening either right around the time the VACUUM FULL commits or right around the time the REINDEX commits. It'd be helpful to know which

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Here is the update: the problem happens with vacuum full alone, no reindex is needed to trigger it. I updated the script to avoid reindexing after vacuum. Over the past two days there are still many

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-03 Thread daveg
On Mon, Aug 01, 2011 at 01:23:49PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: I think we need to start adding some instrumentation so we can get a better handle on what's going on in your database. If I were to send you

Re: [HACKERS] Further news on Clang - spurious warnings

2011-08-03 Thread daveg
On Wed, Aug 03, 2011 at 04:03:39PM -0400, Tom Lane wrote: The C standard specifies that signed-to-unsigned conversions must work like that; and even if the standard didn't, it would surely work like that on any machine with two's-complement representation, which is to say every computer built

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Wed, Aug 03, 2011 at 11:18:20AM -0400, Tom Lane wrote: Evidently not, if it's not logging anything, but now the question is why. One possibility is that for some reason RelationGetNumberOfBlocks is persistently lying about the file size. (We've seen kernel bugs before that resulted in

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Summary: the failing process reads 0 rows from 0 blocks from the OLD relfilenode. Hmm. This seems to mean that we're somehow missing a relation mapping invalidation message, or perhaps not processing

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Summary: the failing process reads 0 rows from 0 blocks from the OLD relfilenode. Hmm. This seems to mean that we're somehow missing a relation mapping invalidation message, or perhaps not processing

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 04:16:08PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: We are seeing cannot read' and 'cannot open' errors too that would be consistant with trying to use a vanished file. Yeah, these all seem consistent with the idea that the failing backend somehow

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-05 Thread daveg
On Fri, Aug 05, 2011 at 12:10:31PM -0400, Tom Lane wrote: I wrote: Ahh ... you know what, never mind about stack traces, let's just see if the attached patch doesn't fix it. On reflection, that patch would only fix the issue for pg_class, and that's not the only catalog that gets

Re: [HACKERS] VACUUM FULL versus system catalog cache invalidation

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 09:26:02PM +0100, Simon Riggs wrote: With HOT, there is very little need to perform a VACUUM FULL on any shared catalog table. Look at the indexes... I would a suggest that VACUUM FULL perform only a normal VACUUM on shared catalog tables, then perform an actual

Re: [HACKERS] our buffer replacement strategy is kind of lame

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 01:28:49PM +0100, Simon Riggs wrote: I think there are reasonable arguments to make * prefer_cache = off (default) | on a table level storage parameter, =on will disable the use of BufferAccessStrategy * make cache_spoil_threshold a parameter, with default 0.25

[HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-12 Thread daveg
This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error: FATAL: lock AccessShareLock on object 0/1260/0 is already held This happens on postgresql 8.4.7 on a large (512GB, 32 core) system that has been up for months.

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-13 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: On Fri, Aug 12, 2011 at 7:19 PM, daveg da...@sonic.net wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error:   FATAL:  lock AccessShareLock

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-15 Thread daveg
[adding back hackers so the thread shows the resolution] On Sun, Aug 14, 2011 at 07:02:55PM -0400, Tom Lane wrote: Sounds good. Based on my own testing so far, I think that patch will probably make things measurably better for you, though it won't resolve every corner case. The most recent

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-08-22 Thread daveg
On Fri, Aug 12, 2011 at 04:19:37PM -0700, daveg wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error: FATAL: lock AccessShareLock on object 0/1260/0 is already held This happens on postgresql 8.4.7

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-23 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: On Fri, Aug 12, 2011 at 7:19 PM, daveg da...@sonic.net wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error:   FATAL:  lock AccessShareLock

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-29 Thread daveg
On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wrote: vacuumdb: vacuuming of database etsy_v2 failed: ERROR: could not access status of transaction 3429738606 DETAIL: Could not open file

Re: [HACKERS] PostgreSQL and HugePage

2010-10-19 Thread daveg
On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: On 20/10/10 16:05, Mark Kirkwood wrote: shmget and friends are hugetlbpage aware, so it seems it should 'just work'. Heh - provided you specify SHM_HUGETLB in the relevant call that is :-) I had a patch for this

Re: [HACKERS] PostgreSQL and HugePage

2010-10-20 Thread daveg
On Wed, Oct 20, 2010 at 12:28:25PM -0700, Greg Stark wrote: On Wed, Oct 20, 2010 at 12:17 PM, Greg Stark gsst...@mit.edu wrote: I don't think it's a big cost once all the processes have been forked if you're reusing them beyond perhaps slightly more efficient cache usage. Hm, this site

Re: [HACKERS] PostgreSQL and HugePage

2010-10-21 Thread daveg
On Thu, Oct 21, 2010 at 08:16:27AM -0700, Mark Wong wrote: On Tue, Oct 19, 2010 at 8:30 PM, daveg da...@sonic.net wrote: On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: On 20/10/10 16:05, Mark Kirkwood wrote: shmget and friends are hugetlbpage  aware, so it seems

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
Sorry I missed your reply, catching up now. On Wed, Aug 31, 2011 at 09:56:59PM -0400, Bruce Momjian wrote: daveg wrote: On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wrote: vacuumdb

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
On Mon, Sep 05, 2011 at 08:19:21PM -0400, Bruce Momjian wrote: daveg wrote: Can you tell me what table is showing this error? Does it happen during vacuum? Can you run a vacuum verbose to see what it is throwing the error on? Thanks. This was upgrading from 8.4.8 to 9.0.4. I

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Tue, Aug 23, 2011 at 12:15:23PM -0400, Robert Haas wrote: On Mon, Aug 22, 2011 at 3:31 AM, daveg da...@sonic.net wrote: So far I've got:  - affects system tables  - happens very soon after process startup  - in 8.4.7 and 9.0.4  - not likely to be hardware or OS related  - happens

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 10:20:24AM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: After spending some time staring at the code, I do have one idea as to what might be going on here. When a backend is terminated, ShutdownPostgres() calls AbortOutOfAnyTransaction() and then

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 04:55:24PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Tom's right to be skeptical of my theory, because it would require a CHECK_FOR_INTERRUPTS() outside of a transaction block in one of the pathways that use session-level locks, and I can't

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 06:35:08PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: It does not seem restricted to pg_authid: 2011-08-24 18:35:57.445 24987 c23 apps ERROR: lock AccessShareLock on object 16403/2615/0 And I think I've seen it on other tables too. Hmm. 2615

  1   2   >