Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-03 Thread Kevin Grittner
"Kevin Grittner" wrote: > Any objections to my putting it on the TODO list? Hearing none, added. (Apologies for missing the box where I should have commented on the what the change did.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > Perhaps, if you didn't mind sometimes getting a wrong answer. Well, it would be a heuristic which would close the deleted file *almost* all the time. When it didn't, the next check would probably catch it. Assuming that you would never get an indication that it was differen

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> Determining whether it's still the current append target is not so >> cheap though; it would require examining shared-memory status >> which means taking a lock on that status (and it's a high-traffic >> lock already). > I haven't reviewed the inter

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > Determining whether it's still the current append target is not so > cheap though; it would require examining shared-memory status > which means taking a lock on that status (and it's a high-traffic > lock already). I haven't reviewed the internal locking techniques, so this

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Tom Lane
"Kevin Grittner" writes: > Is there a reasonably cheap way to check whether the backend has a > WAL file open and whether that one is the current append target? Detecting whether we have a WAL file open is trivial (just look at the static variable holding the file descriptor). Determining whethe

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-12-01 Thread Kevin Grittner
Tom Lane wrote: > How old were the sessions you were looking at? Days to months old. > If we think this is worth doing something about > (I'm not convinced yet) Once one knows about the issue, it's only a minor annoyance, and that infrequently, so it's not worth anything that would take si

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> There has to be something causing those sessions to touch WAL, and >> the dirty-buffer scenario doesn't seem reliable enough. > This is seeming fairly likely to be the cause, though. It may be a > combination of the nightly VACUUM FREEZE ANALYZE w

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > There has to be something causing those sessions to touch WAL, and > the dirty-buffer scenario doesn't seem reliable enough. This is seeming fairly likely to be the cause, though. It may be a combination of the nightly VACUUM FREEZE ANALYZE we typically do on every database

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > It's not about the size of a temp table, because writes to the > temp table itself aren't WAL-logged. However, the system catalog > entries for a temp table *are* WAL-logged. Definitely not issuing any CREATE TEMP statements of any kind, unless the JDBC driver is doing that

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> You sure it's not creating any temp tables? You didn't mention >> revoking TEMP privilege. > They have not been revoked, but I am sure the software publisher > doesn't explicitly create any, and I'd be very surprised if the > monitoring software d

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: > You sure it's not creating any temp tables? You didn't mention > revoking TEMP privilege. They have not been revoked, but I am sure the software publisher doesn't explicitly create any, and I'd be very surprised if the monitoring software did. The tables are small enough t

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> A backend would never open a WAL file unless it had to write a WAL >> record, so I'm having a hard time believing that these were >> totally read-only transactions. Can you give specifics? > You will note that the connections logged in as "viewer" (

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-30 Thread Kevin Grittner
Tom Lane wrote: >> It seemed strange that the only backends which were holding open >> deleted WAL files were ones where the connection was established >> with a login which has no write permissions. > > A backend would never open a WAL file unless it had to write a WAL > record, so I'm having a

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-25 Thread Tom Lane
"Kevin Grittner" writes: > I ran across the subject issue and spent some time puzzling over it. > ... > I'm not sure whether Tom's comment that "There is zero hope of > making that work." referred to the idea that we could close deleted > WAL files or to something else. Is a fix feasible? The re

Re: [HACKERS] Deleted WAL files held open by backends in Linux

2009-11-25 Thread Itagaki Takahiro
"Kevin Grittner" wrote: > I guess it is a stretch to imagine that a database would have > enough read-only connections to exhaust resources by holding open > one deleted WAL file each; unless they have, say, 200 such > connections and they're cutting things so close that a wasted 3.2GB > of disk

[HACKERS] Deleted WAL files held open by backends in Linux

2009-11-25 Thread Kevin Grittner
I ran across the subject issue and spent some time puzzling over it. It seemed strange that the only backends which were holding open deleted WAL files were ones where the connection was established with a login which has no write permissions. Eventually, I vaguely recalled there was such an issue