[HACKERS] timezone buglet?

2011-10-04 Thread daveg
Postgresql 9.0.4 has the timezone: America/Blanc-Sablon However other sources seem to spell this with an underscore instead of dash: America/Blanc_Sablon It appears that beside 'America/Blanc_Sablon', other multi-word timezones are spelled with underscore. For example:

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-08 Thread daveg
On Wed, Sep 07, 2011 at 09:02:04PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: On Wed, Sep 07, 2011 at 07:39:15PM -0400, Tom Lane wrote: BTW ... what were the last versions you were running on which you had *not* seen the problem? (Just wondering about the possibility that we

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Tue, Aug 23, 2011 at 12:15:23PM -0400, Robert Haas wrote: On Mon, Aug 22, 2011 at 3:31 AM, daveg da...@sonic.net wrote: So far I've got:  - affects system tables  - happens very soon after process startup  - in 8.4.7 and 9.0.4  - not likely to be hardware or OS related  - happens

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 10:20:24AM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: After spending some time staring at the code, I do have one idea as to what might be going on here. When a backend is terminated, ShutdownPostgres() calls AbortOutOfAnyTransaction() and then

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 04:55:24PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Tom's right to be skeptical of my theory, because it would require a CHECK_FOR_INTERRUPTS() outside of a transaction block in one of the pathways that use session-level locks, and I can't

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 06:35:08PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: It does not seem restricted to pg_authid: 2011-08-24 18:35:57.445 24987 c23 apps ERROR: lock AccessShareLock on object 16403/2615/0 And I think I've seen it on other tables too. Hmm. 2615

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 06:25:23PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I thought about an error exit from client authentication, and that's a somewhat appealing explanation, but I can't quite see why we wouldn't clean up there the same as anywhere else. The

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-07 Thread daveg
On Wed, Sep 07, 2011 at 07:39:15PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Also, this is very intermittant, we have seen it only in recent months on both 8.4.7 and 9.0.4 after years of no problems. Lately we see it what feels like a few times a month. Possibly some new

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
Sorry I missed your reply, catching up now. On Wed, Aug 31, 2011 at 09:56:59PM -0400, Bruce Momjian wrote: daveg wrote: On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wrote: vacuumdb

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-09-05 Thread daveg
On Mon, Sep 05, 2011 at 08:19:21PM -0400, Bruce Momjian wrote: daveg wrote: Can you tell me what table is showing this error? Does it happen during vacuum? Can you run a vacuum verbose to see what it is throwing the error on? Thanks. This was upgrading from 8.4.8 to 9.0.4. I

Re: [HACKERS] [GENERAL] pg_upgrade problem

2011-08-29 Thread daveg
On Mon, Aug 29, 2011 at 07:49:24PM +0200, hubert depesz lubaczewski wrote: On Mon, Aug 29, 2011 at 06:54:41PM +0200, hubert depesz lubaczewski wrote: vacuumdb: vacuuming of database etsy_v2 failed: ERROR: could not access status of transaction 3429738606 DETAIL: Could not open file

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-23 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: On Fri, Aug 12, 2011 at 7:19 PM, daveg da...@sonic.net wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error:   FATAL:  lock AccessShareLock

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-08-22 Thread daveg
On Fri, Aug 12, 2011 at 04:19:37PM -0700, daveg wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error: FATAL: lock AccessShareLock on object 0/1260/0 is already held This happens on postgresql 8.4.7

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-15 Thread daveg
[adding back hackers so the thread shows the resolution] On Sun, Aug 14, 2011 at 07:02:55PM -0400, Tom Lane wrote: Sounds good. Based on my own testing so far, I think that patch will probably make things measurably better for you, though it won't resolve every corner case. The most recent

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-13 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: On Fri, Aug 12, 2011 at 7:19 PM, daveg da...@sonic.net wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error:   FATAL:  lock AccessShareLock

Re: [HACKERS] VACUUM FULL versus system catalog cache invalidation

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 09:26:02PM +0100, Simon Riggs wrote: With HOT, there is very little need to perform a VACUUM FULL on any shared catalog table. Look at the indexes... I would a suggest that VACUUM FULL perform only a normal VACUUM on shared catalog tables, then perform an actual

Re: [HACKERS] our buffer replacement strategy is kind of lame

2011-08-12 Thread daveg
On Fri, Aug 12, 2011 at 01:28:49PM +0100, Simon Riggs wrote: I think there are reasonable arguments to make * prefer_cache = off (default) | on a table level storage parameter, =on will disable the use of BufferAccessStrategy * make cache_spoil_threshold a parameter, with default 0.25

[HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-12 Thread daveg
This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error: FATAL: lock AccessShareLock on object 0/1260/0 is already held This happens on postgresql 8.4.7 on a large (512GB, 32 core) system that has been up for months.

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-05 Thread daveg
On Fri, Aug 05, 2011 at 12:10:31PM -0400, Tom Lane wrote: I wrote: Ahh ... you know what, never mind about stack traces, let's just see if the attached patch doesn't fix it. On reflection, that patch would only fix the issue for pg_class, and that's not the only catalog that gets

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Wed, Aug 03, 2011 at 11:18:20AM -0400, Tom Lane wrote: Evidently not, if it's not logging anything, but now the question is why. One possibility is that for some reason RelationGetNumberOfBlocks is persistently lying about the file size. (We've seen kernel bugs before that resulted in

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Summary: the failing process reads 0 rows from 0 blocks from the OLD relfilenode. Hmm. This seems to mean that we're somehow missing a relation mapping invalidation message, or perhaps not processing

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 12:28:31PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Summary: the failing process reads 0 rows from 0 blocks from the OLD relfilenode. Hmm. This seems to mean that we're somehow missing a relation mapping invalidation message, or perhaps not processing

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-04 Thread daveg
On Thu, Aug 04, 2011 at 04:16:08PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: We are seeing cannot read' and 'cannot open' errors too that would be consistant with trying to use a vanished file. Yeah, these all seem consistent with the idea that the failing backend somehow

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-08-03 Thread daveg
On Mon, Aug 01, 2011 at 01:23:49PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: I think we need to start adding some instrumentation so we can get a better handle on what's going on in your database. If I were to send you

Re: [HACKERS] Further news on Clang - spurious warnings

2011-08-03 Thread daveg
On Wed, Aug 03, 2011 at 04:03:39PM -0400, Tom Lane wrote: The C standard specifies that signed-to-unsigned conversions must work like that; and even if the standard didn't, it would surely work like that on any machine with two's-complement representation, which is to say every computer built

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Thu, Jul 28, 2011 at 11:31:31PM -0700, daveg wrote: On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: REINDEX. My guess is that this is happening either right around the time the VACUUM FULL commits or right around the time the REINDEX commits. It'd be helpful to know which

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-31 Thread daveg
On Sun, Jul 31, 2011 at 11:44:39AM -0400, Tom Lane wrote: daveg da...@sonic.net writes: Here is the update: the problem happens with vacuum full alone, no reindex is needed to trigger it. I updated the script to avoid reindexing after vacuum. Over the past two days there are still many

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-29 Thread daveg
On Thu, Jul 28, 2011 at 07:45:01PM -0400, Robert Haas wrote: On Thu, Jul 28, 2011 at 5:46 PM, daveg da...@sonic.net wrote: On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: On Wed, Jul 27, 2011 at 8:28 PM, daveg da...@sonic.net wrote: My client has been seeing regular instances

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-29 Thread daveg
On Fri, Jul 29, 2011 at 09:55:46AM -0400, Tom Lane wrote: The thing that was bizarre about the one instance in the buildfarm was that the error was persistent, ie, once a session had failed all its subsequent attempts to access pg_class failed too. I gather from Dave's description that it's

Re: [HACKERS] error: could not find pg_class tuple for index 2662

2011-07-28 Thread daveg
On Thu, Jul 28, 2011 at 09:46:41AM -0400, Robert Haas wrote: On Wed, Jul 27, 2011 at 8:28 PM, daveg da...@sonic.net wrote: My client has been seeing regular instances of the following sort of problem: On what version of PostgreSQL? 9.0.4. I previously said: This occurs on postgresql

[HACKERS] error: could not find pg_class tuple for index 2662

2011-07-27 Thread daveg
My client has been seeing regular instances of the following sort of problem: ... 03:06:09.453 exec_simple_query, postgres.c:900 03:06:12.042 XX000: could not find pg_class tuple for index 2662 at character 13 03:06:12.042 RelationReloadIndexInfo, relcache.c:1740 03:06:12.042 INSERT INTO

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:00:01AM +0200, Heikki Linnakangas wrote: On 08.03.2011 04:07, Greg Stark wrote: Well from that log you definitely have OldestXmin going backwards. And not by a little bit either. at 6:33 it set the all_visible flag and then at 7:01 it was almost 1.3 million

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-08 Thread daveg
On Tue, Mar 08, 2011 at 10:37:24AM +0200, Heikki Linnakangas wrote: On 08.03.2011 10:00, Heikki Linnakangas wrote: Another idea is to give up on the warning when it appears that oldestxmin has moved backwards, and assume that it's actually fine. We could still warn in other cases where the

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-07 Thread daveg
On Fri, Mar 04, 2011 at 05:52:29PM +0200, Heikki Linnakangas wrote: Hmm, if these all came from the same database, then it looks OldestXmin has moved backwards. That would explain the warnings. First one vacuum determines that all the tuples are visible to everyone and sets the flag. Then

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-04 Thread daveg
On Thu, Mar 03, 2011 at 09:04:04AM -0600, Merlin Moncure wrote: On Thu, Mar 3, 2011 at 2:16 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 03.03.2011 09:12, daveg wrote: Question: what would be the consequence of simply patching out the setting of this flag

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-03 Thread daveg
On Thu, Mar 03, 2011 at 10:16:29AM +0200, Heikki Linnakangas wrote: On 03.03.2011 09:12, daveg wrote: Question: what would be the consequence of simply patching out the setting of this flag? Assuming that the incorrect PD_ALL_VISIBLE flag is the only problem (big assumption perhaps

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Tue, Mar 01, 2011 at 01:20:43PM -0800, daveg wrote: On Tue, Mar 01, 2011 at 12:00:54AM +0200, Heikki Linnakangas wrote: On 28.02.2011 23:28, daveg wrote: On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: We'll likely need to go back and forth a few times with various

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Wed, Mar 02, 2011 at 06:45:13PM -0300, Alvaro Herrera wrote: Excerpts from daveg's message of mié mar 02 18:30:34 -0300 2011: After a restart and vacuum of all dbs with no other activity things were quiet for a couple hours and then we started seeing these PD_ALL_VISIBLE messages

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Wed, Mar 02, 2011 at 04:20:24PM -0800, bricklen wrote: On Wed, Mar 2, 2011 at 3:53 PM, daveg da...@sonic.net wrote: Postgresql version is 8.4.4. I don't see how this could be related, but since you're running on NFS, maybe it is, somehow: http://archives.postgresql.org/message-id

Re: [HACKERS] Re: PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-02 Thread daveg
On Tue, Mar 01, 2011 at 08:40:37AM -0500, Robert Haas wrote: On Mon, Feb 28, 2011 at 10:32 PM, Greg Stark gsst...@mit.edu wrote: On Tue, Mar 1, 2011 at 1:43 AM, David Christensen da...@endpoint.com wrote: Was this cluster upgraded to 8.4.4 from 8.4.0?  It sounds to me like a known bug

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-01 Thread daveg
On Mon, Feb 28, 2011 at 07:43:39PM -0600, David Christensen wrote: On Feb 28, 2011, at 3:28 PM, daveg wrote: Anything new on this? I'm seeing at on one of my clients production boxes. Also, what is the significance, ie what is the risk or damage potential if this flag is set incorrectly

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-03-01 Thread daveg
On Tue, Mar 01, 2011 at 12:00:54AM +0200, Heikki Linnakangas wrote: On 28.02.2011 23:28, daveg wrote: On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: We'll likely need to go back and forth a few times with various debugging patches until we get to the heart

Re: [HACKERS] Re: [ADMIN] PD_ALL_VISIBLE flag was incorrectly set happend during repeatable vacuum

2011-02-28 Thread daveg
On Wed, Jan 12, 2011 at 10:46:14AM +0200, Heikki Linnakangas wrote: On 12.01.2011 06:21, Fujii Masao wrote: On Sat, Dec 25, 2010 at 2:09 PM, Maxim Bogukmaxim.bo...@gmail.com wrote: While I trying create reproducible test case for BUG #5798 I encountered very strange effect on two of my

Re: [HACKERS] PostgreSQL and HugePage

2010-10-21 Thread daveg
On Thu, Oct 21, 2010 at 08:16:27AM -0700, Mark Wong wrote: On Tue, Oct 19, 2010 at 8:30 PM, daveg da...@sonic.net wrote: On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: On 20/10/10 16:05, Mark Kirkwood wrote: shmget and friends are hugetlbpage  aware, so it seems

Re: [HACKERS] PostgreSQL and HugePage

2010-10-20 Thread daveg
On Wed, Oct 20, 2010 at 12:28:25PM -0700, Greg Stark wrote: On Wed, Oct 20, 2010 at 12:17 PM, Greg Stark gsst...@mit.edu wrote: I don't think it's a big cost once all the processes have been forked if you're reusing them beyond perhaps slightly more efficient cache usage. Hm, this site

Re: [HACKERS] PostgreSQL and HugePage

2010-10-19 Thread daveg
On Wed, Oct 20, 2010 at 04:08:37PM +1300, Mark Kirkwood wrote: On 20/10/10 16:05, Mark Kirkwood wrote: shmget and friends are hugetlbpage aware, so it seems it should 'just work'. Heh - provided you specify SHM_HUGETLB in the relevant call that is :-) I had a patch for this

Re: [HACKERS] a faster compression algorithm for pg_dump

2010-04-14 Thread daveg
On Tue, Apr 13, 2010 at 03:03:58PM -0400, Tom Lane wrote: Joachim Wieland j...@mcknight.de writes: If we still cannot do this, then what I am asking is: What does the project need to be able to at least link against such a compression algorithm? Well, what we *really* need is a

Re: [HACKERS] TCP keepalive support for libpq

2010-02-10 Thread daveg
On Tue, Feb 09, 2010 at 09:34:10AM -0500, Andrew Chernow wrote: Tollef Fog Heen wrote: (please Cc me on replies, I am not subscribed) Hi, libpq currently does not use TCP keepalives. This is a problem in our case where we have some clients waiting for notifies and then the connection is

Re: [HACKERS] pg_dump enhancement proposal

2009-12-13 Thread daveg
On Thu, Nov 12, 2009 at 04:31:37PM -0500, Tom Lane wrote: Mark Hammonds mhammo...@omniti.com writes: 2. Custom Query Exports In my use of mysqldump, I found one feature very useful: the ability to execute a custom SELECT. . .WHERE statement and then dump only the results. This

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:22:01AM +, Greg Stark wrote: Really I think you guys are on the wrong track trying to map Postgres releases to commercial support terms. None of the Postgres releases are supported in the sense that there's no warranty and no promises, it's all best effort. If

Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread daveg
On Fri, Nov 13, 2009 at 02:47:56AM +, Greg Stark wrote: On Fri, Nov 13, 2009 at 2:35 AM, daveg da...@sonic.net wrote: I suggest we announce now that both 7.4 and 8.0 will EOL when 8.5 is expected to ship, or to comfort those who never use .0 versions when 8.5.1 ships. What would

Re: [HACKERS] EOL for 7.4?

2009-11-03 Thread daveg
On Tue, Nov 03, 2009 at 10:32:17AM -0800, Josh Berkus wrote: So I'm going to make a case in favor of EOL'ing 7.4. In fact, I'd be in favor of doing so in, say, February after an announcement this month. The main reason I'm in favor of this is that we have a lot of users using 7.4 out of

Re: [HACKERS] Application name patch - v2

2009-10-20 Thread daveg
On Tue, Oct 20, 2009 at 12:16:42PM -0400, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: Also, how many platforms can't we do this on? If we have BSD and Windows covered already. on linux, I believe you can easily read it out of /proc/self/cmdline, no? Writing a pile of

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-20 Thread daveg
On Fri, Oct 02, 2009 at 07:57:13PM -0700, daveg wrote: On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: daveg escribió: I work with Kunal and have been looking into this. It appears to be the same as the bug described in: http://archives.postgresql.org/pgsql

Re: [HACKERS] Deprecation

2009-10-19 Thread daveg
On Sat, Oct 17, 2009 at 03:01:27PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Sounds like a good reason to remove add_missing_from in 8.5. Seems like the general consensus is that it's okay to do that. I will go make it happen unless somebody squawks pretty soon...

Re: [HACKERS] Application name patch - v2

2009-10-19 Thread daveg
On Mon, Oct 19, 2009 at 01:00:28PM +0100, Dave Page wrote: On Mon, Oct 19, 2009 at 12:57 PM, Pavel Stehule pavel.steh...@gmail.com wrote: It is not practical. I'll log errors. Usually SQL injection generates lot of errors. Loging all statements has not sense. What is difference bad and

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-02 Thread daveg
On Fri, Oct 02, 2009 at 10:41:07AM -0400, Alvaro Herrera wrote: daveg escribió: I work with Kunal and have been looking into this. It appears to be the same as the bug described in: http://archives.postgresql.org/pgsql-bugs/2009-09/msg00355.php as I have localized it to a NULL

[HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. The problem this addresses is that sometimes one needs to set work_mem fairly high to get good query plans for large joins. However, some complex queries will then use huge amounts of memory

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 10:35:55AM -0400, Tom Lane wrote: daveg da...@sonic.net writes: I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. Use ulimit. That was my initial thought too. However, ulimit() is documented as superceded

Re: [HACKERS] Limit allocated memory per session

2009-10-01 Thread daveg
On Thu, Oct 01, 2009 at 11:47:43AM -0400, Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: Tom Lane escreveu: daveg da...@sonic.net writes: I'd like to propose adding a new GUC to limit the amount of memory a backend can allocate for its own use. Use ulimit

Re: [HACKERS] Postgres server goes in recovery mode repeteadly

2009-10-01 Thread daveg
On Tue, Sep 29, 2009 at 09:52:06PM +0530, kunal sharma wrote: Hi , We are using Postgres 8.4 and its been found going into recovery mode couple of times. The server process seems to fork another child process which is another postgres server running under same data directory and after

Re: [HACKERS] BEGIN TRANSACTION and START TRANSACTION: different error handling

2009-09-24 Thread daveg
On Thu, Sep 24, 2009 at 12:16:43PM +0300, Hannu Krosing wrote: I expect the transaction is aborted and rollback is executed automatically. - this is not how postgreSQL behaves. PostgreSQL needs an explicit end of transaction from client, either COMMIT; or ROLLBACK; when run from psql, they

Re: [HACKERS] numeric_to_number() function skipping some digits

2009-09-23 Thread daveg
On Tue, Sep 22, 2009 at 10:27:19AM +0530, Jeevan Chalke wrote: It seems that Oracle reads formatting string from right-to-left. Here are few results: ('number','format') == Oracle PG ('34,50','999,99') == 3450340

Re: [HACKERS] Adding \ev view editor?

2009-09-23 Thread daveg
On Mon, Sep 21, 2009 at 02:26:05PM -0400, Andrew Dunstan wrote: andrew=# select pg_get_viewdef('foo',true); pg_get_viewdef -- SELECT 'a'::text AS b, ( SELECT 1 FROM dual) AS x, random() AS y,

Re: [HACKERS] happy birthday Tom Lane ...

2009-09-18 Thread daveg
On Fri, Sep 18, 2009 at 01:04:23PM +0200, Hans-Juergen Schoenig -- PostgreSQL wrote: Tom, On behalf of the entire PostgreSQL team here in Austria I want to wish you a happy birthday. We hope that you fill be a vital part of PostgreSQL for many years to come. Best regards,

Re: [HACKERS] RfD: more powerful any types

2009-09-17 Thread daveg
On Tue, Sep 15, 2009 at 07:38:18AM +0200, Pavel Stehule wrote: it isn't fair :) why you use $$ without single quote? And still this case should be vulnerable on SQL injection. Maybe you or me knows, what SQL injection means, but beginners knows nothing and this people use following bad code:

Re: [HACKERS] RfD: more powerful any types

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 11:43:32AM -0400, Merlin Moncure wrote: If you are going to use printf format codes, which is good and useful being something of a standard, I'd call routine printf (not format) and actually wrap vsnprintf. The format codes in printf have a very specific meaning:

Re: [HACKERS] Ragged CSV import

2009-09-12 Thread daveg
On Fri, Sep 11, 2009 at 10:27:06AM +0200, Dimitri Fontaine wrote: Maybe instead of opening FROM for COPY, having it accepted in WITH would be better, the same way (from the user point of view) that DML returning are worked on. ... WITH csv AS ( COPY t FROM stdin CSV ) INSERT INTO

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes

Re: [HACKERS] remove flatfiles.c

2009-09-03 Thread daveg
On Thu, Sep 03, 2009 at 07:57:25PM -0400, Andrew Dunstan wrote: daveg wrote: On Tue, Sep 01, 2009 at 07:42:56PM -0400, Tom Lane wrote: I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. I have a client who uses temp tables heavily, hundreds

Re: [HACKERS] Add YAML option to explain

2009-08-31 Thread daveg
On Mon, Aug 31, 2009 at 02:15:08PM -, Greg Sabino Mullane wrote: Greg, can we see a few examples of the YAML output compared to both json and text? ... greg=# explain (format json, analyze on) select * from pg_class where relname ~ 'x' order by 1,2,3; QUERY PLAN

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 09:38:15PM +0200, Dimitri Fontaine wrote: Exactly, and I think that what we're missing here is a simple tool for our users to check a new PostgreSQL release against their existing application. We already know how to either log all queries and analyze the log files

Re: [HACKERS] 8.5 release timetable, again

2009-08-28 Thread daveg
On Thu, Aug 27, 2009 at 08:02:03PM -0700, Ron Mayer wrote: Andrew Dunstan wrote: I don't know of anyone who is likely to want to try out alphas in their normal development environments. The client I approached was specifically prepared to test beta releases that way. Perhaps end-users

Re: [HACKERS] Add YAML option to explain

2009-08-28 Thread daveg
On Fri, Aug 28, 2009 at 04:37:41PM -0700, David E. Wheeler wrote: On Aug 28, 2009, at 3:45 PM, Stephen Frost wrote: +1 from me. I've read the other comments and just plain don't agree with them. It's a small patch, adds a useful format for EXPLAIN, and would be used. One of the best

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sat, Aug 15, 2009 at 11:34:04PM +0200, Dimitri Fontaine wrote: Nitpicking dept, I think I prefer: DO [ [LANGUAGE] language] $$ ... $$; DO plperl $$ ... $$; DO language plpython $$ ... $$; language is optional and defaults to plpgsql. +1 -dg -- David Gould da...@sonic.net

Re: [HACKERS] GRANT ON ALL IN schema

2009-08-16 Thread daveg
On Sun, Aug 16, 2009 at 02:59:53PM +0200, Pavel Stehule wrote: 2009/8/16 Peter Eisentraut pete...@gmx.net: On sön, 2009-08-16 at 00:04 -0400, Andrew Dunstan wrote: SQL is not Lisp. Simple is  good. I didn't think Peter was really very serious. Well, I don't know if we really need to

Re: [HACKERS] Alpha releases: How to tag

2009-08-08 Thread daveg
On Fri, Aug 07, 2009 at 06:28:34PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: I am not suggesting that this change be immediate, and it's not ivory tower. It's just how everybody else does it. You keep saying that, and it's completely meaningless. What do you know

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-08-03 Thread daveg
of different job-scheduling behavior ... but there's no evidence here of a consistently measurable benefit or loss from that. IIRC daveg was volunteering to do some tests with his own data; maybe we should wait for those results. I have run extensive tests with three trials of each configuration

Re: [HACKERS] Review: Revise parallel pg_restore's scheduling heuristic

2009-07-31 Thread daveg
On Thu, Jul 30, 2009 at 12:29:34PM -0500, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: I think we've pretty much established that it doesn't make things *worse*, so I'm sort of inclined to go ahead and apply it. The theoretical advantage of eliminating O(N^2) search

Re: [HACKERS] display previous query string of idle-in-transaction

2009-07-23 Thread daveg
On Thu, Jun 04, 2009 at 10:22:41PM -0400, Robert Haas wrote: The only thing I don't like about this is that I think it's kind of a hack to shove the IDLE in transaction designation and the query string into the same database column. I've never liked having to write: select sum(1) from

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: Dann Corbit wrote: The LZMA SDK is granted to the public domain: http://www.7-zip.org/sdk.html I played with this but found the SDK extremely confusing and flat out horrible. One personal dislike was the unnecessary use of

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 02:47:05PM -0500, Bruce Momjian wrote: daveg wrote: On Wed, Feb 04, 2009 at 10:23:17PM -0500, Andrew Chernow wrote: Dann Corbit wrote: The LZMA SDK is granted to the public domain: http://www.7-zip.org/sdk.html I played with this but found the SDK

Re: [HACKERS] Is a plan for lmza commpression in pg_dump

2009-02-07 Thread daveg
On Sat, Feb 07, 2009 at 08:49:29PM -0500, Robert Haas wrote: Proprietary compression algorithms, even with Postgresql-specific license exceptions? To be fair, lzo is GPL, which is a stretch to consider proprietary. -dg -- David Gould da...@sonic.net 510 536 1443510 282

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 03:36:19PM -0400, Tom Lane wrote: Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= [EMAIL PROTECTED] writes: AFAICS, PostgreSQL is not keeping info about when a table, database, sequence, etc was created. We cannot get that info even from OS, since CLUSTER or VACUUM FULL may

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 11:03:56PM +0300, Hannu Krosing wrote: On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: I'd be very interested in seeing a last schema modification time for pg_class objects. I don't care about it being preserved over dump and restore as my use case is more

Re: [HACKERS] IN vs EXISTS equivalence

2008-09-03 Thread daveg
On Thu, Aug 14, 2008 at 06:50:09PM +0100, Simon Riggs wrote: On Fri, 2008-08-08 at 16:23 -0400, Tom Lane wrote: NOT IN is a lot trickier, condition: you must also assume that the comparison operator involved never yields NULL for non-null inputs. That might be okay for btree

Re: [HACKERS] proposal sql: labeled function params

2008-08-23 Thread daveg
On Sat, Aug 23, 2008 at 05:08:25PM +0100, Gregory Stark wrote: Pavel Stehule [EMAIL PROTECTED] writes: Hello 2008/8/23 Peter Eisentraut [EMAIL PROTECTED]: On Friday 22 August 2008 07:41:30 Decibel! wrote: If we're really worried about it we can have a GUC for a few versions that

Re: [HACKERS] Adjusting debug_print_plan to be more useful by default

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 06:33:33PM +0100, Simon Riggs wrote: On Tue, 2008-08-19 at 12:40 -0400, Tom Lane wrote: Back in April we changed EXPLAIN VERBOSE to not dump the internal plan tree anymore, on the grounds that non-hackers didn't want that info and hackers could get it with

Re: [HACKERS] A smaller default postgresql.conf

2008-08-19 Thread daveg
On Tue, Aug 19, 2008 at 09:39:39PM +0300, Peter Eisentraut wrote: On Tuesday 19 August 2008 19:12:16 Tom Lane wrote: Well, why not just make a one-eighty and say that the default postgresql.conf is *empty* (except for whatever initdb puts into it)? Well, my original implementation of GUC

Re: [HACKERS] Visibility Groups

2008-08-08 Thread daveg
On Thu, Aug 07, 2008 at 01:30:27PM +0100, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly*

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: ISTR that what ended up killing the enthusiasm for this was that most people realized that this GUC was just a poor tool to take a stab at solving other problems (ie. rate limiting cpu for queries). I'm not concerned with that,

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 03:09:34PM -0400, Greg Smith wrote: On Mon, 4 Aug 2008, daveg wrote: We load the production dumps into our dev environment, which are the same hardware spec, so the costs should be identical. Not identical, just close. ANALYZE samples data from your table randomly

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 02:35:07PM -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: That's great for you, I am talking in the scope of a general solution. (Note I'd also bet that even given the same hardware, different production loads can produce different

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 11:59:03AM -0700, Josh Berkus wrote: Greg, Well that's going to depend on the application But I suppose there's nothing wrong with having options which aren't always a good idea to use. The real question I guess is whether there's ever a situation where it would

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:17:59PM -0400, Jonah H. Harris wrote: On Mon, Aug 4, 2008 at 5:08 PM, Simon Riggs [EMAIL PROTECTED] wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. This would be a

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:19:50PM -0400, Robert Treat wrote: See, this is what we ended up talking about before. Someone will say I'd like to prevent my devs from accidentally doing queries with cartesian products and they will use this to do it... but that will only work in some cases, so

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-03 Thread daveg
On Sat, Aug 02, 2008 at 09:30:08PM +0200, Hans-Jürgen Schönig wrote: On Aug 2, 2008, at 8:38 PM, Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Hans-Jürgen Schönig wrote: i introduced a GUC called statement_cost_limit which can be used to error out if a statement is expected to be

Re: [HACKERS] Copy storage parameters on CREATE TABLE LIKE/INHERITS

2008-07-30 Thread daveg
On Wed, Jul 30, 2008 at 04:45:47PM +0900, ITAGAKI Takahiro wrote: Here is a patch to copy column storage parameters and reloptions on CREATE TABLE LIKE, which I proposed at: [HACKERS] Uncopied parameters on CREATE TABLE LIKE http://archives.postgresql.org/message-id/[EMAIL PROTECTED]

Re: [HACKERS] [PATCHES] pg_dump additional options for performance

2008-07-27 Thread daveg
On Sun, Jul 27, 2008 at 10:37:34AM +0100, Simon Riggs wrote: On Sat, 2008-07-26 at 11:03 -0700, Joshua D. Drake wrote: 2. We have no concurrency which means, anyone with any database over 50G has unacceptable restore times. Agreed. Also the core reason for wanting -w 3. We have

Re: [HACKERS] Additional psql requirements

2008-07-25 Thread daveg
On Fri, Jul 25, 2008 at 08:16:59AM +0100, Simon Riggs wrote: On Fri, 2008-07-25 at 10:00 +0900, ITAGAKI Takahiro wrote: Simon Riggs [EMAIL PROTECTED] wrote: * access to version number * simple mechanism for conditional execution * ability to set substitution variables from command

  1   2   >