Re: [HACKERS] Bugtraq: Having Fun With PostgreSQL

2007-06-28 Thread Jeremy Drake
On Tue, 26 Jun 2007, Andrew Dunstan wrote: Jeremy Drake wrote: 2. If you cannot tell what process is connecting on a local socket (which I suspect you cannot portably), See ident_unix() in hba.c. It might not be 100% portable but I think it's fairly close for platforms that actually

Re: [HACKERS] self defined data type with limit?

2007-06-28 Thread Michael Enke
Heikki Linnakangas wrote: Michael Enke wrote: My primary goal is to get quasi numeric ordering on text column, e.g. 1 2 10 Normal order with varchar would be 1 10 2 You don't need to custom type for that. A custom operator class with custom comparison operators is enough. Ok, I tried

Re: [HACKERS] Bgwriter LRU cleaning: we've been going at this all wrong

2007-06-28 Thread Greg Smith
On Thu, 28 Jun 2007, ITAGAKI Takahiro wrote: Do you need to increase shared_buffers in such case? If you have something going wild creating dirty buffers with a high usage count faster than they are being written to disk, increasing the size of the shared_buffers cache can just make the

Re: [HACKERS] self defined data type with limit?

2007-06-28 Thread Tom Lane
Michael Enke [EMAIL PROTECTED] writes: Heikki Linnakangas wrote: You don't need to custom type for that. A custom operator class with custom comparison operators is enough. Ok, I tried with ordinary varchar and my own operators/op class. But than: 1) the index is never used (I created it

[HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Pavan Deolasee
During one of HOT stress tests, an asserition failed at tqual.c:1178 in HeapTupleSatisfiesVacuum(). The assertion failure looked really strange because the assertion checks for HEAP_XMAX_COMMITTED which we set just couple of lines above. I inspected the core dump and found that the flag is *set*

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: So we suspected an interaction between multiple processes each holding a SHARE lock and setting/checking different bits in the infomask and we could theoritically say that such interaction can potentially lead to missing hint bit updates. Yeah. This

[HACKERS] write past chunk end in ExprContext / to_char

2007-06-28 Thread Patrick Welche
With today's CVS code (originally noticed with 8.2beta3), on a PC where INT_MAX=0x7FFF=2147483647 postgres=# select version(); version

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Heikki Linnakangas
Pavan Deolasee wrote: During one of HOT stress tests, an asserition failed at tqual.c:1178 in HeapTupleSatisfiesVacuum(). The assertion failure looked really strange because the assertion checks for HEAP_XMAX_COMMITTED which we set just couple of lines above. I inspected the core dump and found

Re: [HACKERS] write past chunk end in ExprContext / to_char

2007-06-28 Thread Tom Lane
Patrick Welche [EMAIL PROTECTED] writes: With today's CVS code (originally noticed with 8.2beta3), on a PC where INT_MAX=0x7FFF=2147483647 postgres=# select to_char(2147483648,'999,999,999'); WARNING: detected write past chunk end in ExprContext 0x845509c WARNING: detected write past

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: AFAICS, we can just simply remove the assertion. But is there any codepaths that assume that after calling HeapTupleSatisfiesSnapshot, all appropriate hint bits are set? There had better not be, since we are going to postpone setting hint bits for

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Alvaro Herrera
Tom Lane escribió: Heikki Linnakangas [EMAIL PROTECTED] writes: AFAICS, we can just simply remove the assertion. But is there any codepaths that assume that after calling HeapTupleSatisfiesSnapshot, all appropriate hint bits are set? There had better not be, since we are going to

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane escribió: A quick grep suggests that VACUUM FULL might be at risk here. That particular case seems easily fixed since VACUUM FULL must hold an exclusive lock; and we can forcibly set sync commit for VACUUM FULL. Uh, that wouldn't help. The

Re: [HACKERS] write past chunk end in ExprContext / to_char

2007-06-28 Thread imad
This is the problematic part in formatting.c, function dch_time. int siz = strlen(tmtcTzn(tmtc)); if (arg == DCH_TZ) strcpy(inout, tmtcTzn(tmtc));

[HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread Alvaro Herrera
Hi, I noticed that lazy vacuum acquires an exclusive lock at the end, to be able to truncate the table. This is not a surprise. If it cannot acquire the lock, it simply skips truncating the table and goes on with life. However, what's problematic is that if a non-zero cost delay has been set,

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Simon Riggs
On Thu, 2007-06-28 at 15:16 -0400, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: AFAICS, we can just simply remove the assertion. But is there any codepaths that assume that after calling HeapTupleSatisfiesSnapshot, all appropriate hint bits are set? There had better not

Re: [HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread Simon Riggs
On Thu, 2007-06-28 at 17:16 -0400, Alvaro Herrera wrote: I noticed that lazy vacuum acquires an exclusive lock at the end, to be able to truncate the table. This is not a surprise. If it cannot acquire the lock, it simply skips truncating the table and goes on with life. However, what's

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Simon Riggs
On Thu, 2007-06-28 at 15:29 -0400, Alvaro Herrera wrote: Tom Lane escribió: Heikki Linnakangas [EMAIL PROTECTED] writes: AFAICS, we can just simply remove the assertion. But is there any codepaths that assume that after calling HeapTupleSatisfiesSnapshot, all appropriate hint bits

Re: [HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread Alvaro Herrera
Simon Riggs wrote: On Thu, 2007-06-28 at 17:16 -0400, Alvaro Herrera wrote: I noticed that lazy vacuum acquires an exclusive lock at the end, to be able to truncate the table. This is not a surprise. If it cannot acquire the lock, it simply skips truncating the table and goes on with

Re: [HACKERS] SetBufferCommitInfoNeedsSave and race conditions

2007-06-28 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Thu, 2007-06-28 at 15:16 -0400, Tom Lane wrote: A quick grep suggests that VACUUM FULL might be at risk here. No we're clear: I caught that issue specifically for VACUUM FULL fairly early on. VF assumes all hint bits are set after the first scan, so we

Re: [HACKERS] write past chunk end in ExprContext / to_char

2007-06-28 Thread Tom Lane
imad [EMAIL PROTECTED] writes: This is the problematic part in formatting.c, function dch_time. intsiz = strlen(tmtcTzn(tmtc)); if (arg == DCH_TZ) strcpy(inout, tmtcTzn(tmtc)); else {

Re: [HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread ITAGAKI Takahiro
Alvaro Herrera [EMAIL PROTECTED] wrote: What I'm requesting here is that the sleep in count_nondeletable_pages() be removed and that change backpatched to 8.2 and 8.1. Agreed. We'd better to shorten the exclusive locking as far as possible. We don't know how many pages we can truncate until

Re: [HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: What I'm requesting here is that the sleep in count_nondeletable_pages() be removed and that change backpatched to 8.2 and 8.1. Are you sure that that is, and always will be, the only sleep in that part of the code path? Seems like it might be better to

Re: [HACKERS] AutoVacuum Behaviour Question

2007-06-28 Thread Alvaro Herrera
Bruce, please make sure to keep the list copied on replies. I think there is an important bug here and I don't want it to get lost just because I lose track of it. I'm also crossposting to pgsql-hackers. Bruce McAlister wrote: okidoki, I tried this: blueface-crm=# select relname, nspname

Re: [HACKERS] How do we create the releases?

2007-06-28 Thread Bruce Momjian
Robert Treat wrote: And while we're talking about things that suck wrt packaging, I noticed it's now been over a year since I first complained about the stable snapshots in our ftp directory being outdated (http://www.postgresql.org/ftp/stable_snapshot/), if no one is going to fix that,

Re: [HACKERS] lazy vacuum sleeps with exclusive lock on table

2007-06-28 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: What I'm requesting here is that the sleep in count_nondeletable_pages() be removed and that change backpatched to 8.2 and 8.1. Are you sure that that is, and always will be, the only sleep in that part of the code path? It is

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Well, it certainly seems like this shouldn't be happening. Maybe the table belonged to a session that crashed, but the pg_class entry has not been cleaned up -- possibly because that backend has not connected to that particular database. Hm --- a

Re: [HACKERS] write past chunk end in ExprContext / to_char

2007-06-28 Thread Bruce Momjian
Tom Lane wrote: Patrick Welche [EMAIL PROTECTED] writes: With today's CVS code (originally noticed with 8.2beta3), on a PC where INT_MAX=0x7FFF=2147483647 postgres=# select to_char(2147483648,'999,999,999'); WARNING: detected write past chunk end in ExprContext 0x845509c WARNING:

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-06-28 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Well, it certainly seems like this shouldn't be happening. Maybe the table belonged to a session that crashed, but the pg_class entry has not been cleaned up -- possibly because that backend has not connected to that particular

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Yeah, we had better investigate some way to clean them up. It was never obvious before that it mattered to get rid of orphan temp tables, but I guess it does. Would it be enough to delete the tuple from pg_class? No, you need a full

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-06-28 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Yeah, we had better investigate some way to clean them up. It was never obvious before that it mattered to get rid of orphan temp tables, but I guess it does. Would it be enough to delete the tuple from

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Oh, I was just thinking in way for Bruce to get out of his current situation. Oh, for that a manual drop table as superuser should work fine. regards, tom lane ---(end of

Re: [HACKERS] [PATCHES] Doc update for pg_start_backup

2007-06-28 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Added a note to the docs that pg_start_backup can take a long time to finish now that we spread out checkpoints: I was starting to wordsmith this, and then wondered whether it's not just a stupid idea for pg_start_backup to act that way. The reason