Re: [HACKERS] dynamic background workers, round two
[sent again, previously sent as reply, instead of reply-all, thanks
Robert]
On 2013-08-09 09:09:08 -0400, Robert Haas wrote:
> On Fri, Jul 26, 2013 at 8:50 AM, Robert Haas wrote:
> > On Fri, Jul 26, 2013 at 5:34 AM, Andres Freund
> > wrote:
> >> It doesn't need to be the postmaster, but I think we need to provide
> >> central infrastructure for that. I don't want this to end up being
> >> redone poorly in multiple places.
> >> I just wanted to mention it, it obviously doesn't need to be implemented
> >> at the same time.
> >
> > OK, makes sense. I'm still working out a design for this part of it;
> > I think I want to get the dynamic shared memory stuff working first.
Makes sense. I just wanted to mention that we should keep on the back of
our minds while working on this.
> >>> >> +#define InvalidPid (-1)
> >>> >> +
> >>> >
> >>> > That value strikes me as a rather bad idea. As a pid that represents
> >>> > init's processgroup. I.e. kill(InvalidPid) will try to kill init... I'd
> >>> > rather not spread that outside async.c.
> >>>
> >>> Well, if we ever try to kill(InvalidPid), that's a PostgreSQL bug.
> >>> And if the postgres process has permission to send signals to init,
> >>> that's an OS bug (or a PostgreSQL bug, since we refuse to run as
> >>> root). So I'm not very worried about it.
> >>
> >> I don't think somebody mistakenly kill()ing somebody with -1 is all too
> >> likely, but it's a valid value to pass. That's why I dislike using it as
> >> constant for an invalid value.
> >
> > Well, so is any negative number, but we should never be trying to kill
> > process groups.
Maybe it's just my dislike for constants that are easy to misuse - even
if it's not really relevant for the specific case.
> >>> I've got two functions that
> >>> need distinguishable return values for the not-yet-started case and
> >>> the already-dead case, neither of which can be real PIDs. If we don't
> >>> use 0 and -1, the alternative is presumably to complicate the calling
> >>> signature with another out parameter. That does not seem like a net
> >>> improvement.
> >>
> >> I actually think those cases would be better served by an error code
> >> mechanism which is extensible.
> >
> > Hmm. What signature would you suggest?
so, the two functions are:
* int GetBackgroundWorkerPid(BackgroundWorkerHandle *handle);
* int WaitForBackgroundWorkerStartup(BackgroundWorkerHandle *handle);
The return value means about the same with:
* If the return value is InvalidPid, it indicates that the worker has
not yet been started by the postmaster.
* If the return value is 0, it indicates that the worker was started by
the postmaster but is no longer running.
* Any other return value is the PID of the running worker.
So, I'd suggest something like:
typedef enum BgwHandleStatus {
BGWH_SUCCESS, /* sucessfully got status */
BGWH_NOT_YET, /* worker hasn't started yet */
BGWH_GONE, /* worker had been started, but shut down already */
BGWH_POSTMASTER_DIED /* well, there you go */
} BgwHandleStatus;
BgwHandleStatus GetBackgroundWorkerPid(BackgroundWorkerHandle *handle, pid_t
*pid);
BgwHandleStatus WaitForBackgroundWorkerStartup(BackgroundWorkerHandle *handle,
pid_t *pid);
> >> I can see the point in the argument, but it still seems to be grotty
> >> architecturally.
> >
> > Suck it up. :-)
That I am able to do ;) Holidays help :P
> > I considered the idea of using SIGUSR2, but it didn't seem worth it.
> >
> >>> I don't understand what you mean by this. Why does it ignore postmaster
> >>> death?
> >>
> >> The other callers that check for WL_POSTMASTER_DEATH actually perform
> >> drastic measures like exit()ing upon postmaster. Here you just
> >> return. With the unix latch implementation it seems to be guaranteed
> >> that the next WaitLatch() done somewhere else will also return
> >> WL_POSTMASTER_DEATH. I am not sure whether it might actually "consume"
> >> the death in the windows implementation.
> >
> > Well, IMHO, every backend should exit as quick as possible upon
> > discovering that the postmaster has died. However, for complex
> > political reasons (ah-Tom-Lane-choo!), our policy is to have the
> > auxiliary processes exit and regular backends soldier on. This seems
> > completely stupid to me, but it's not this patch's job to reverse that
> > policy decision.
While I'd phrase it more politely/complex, agreed. And agreed with not
changing policy here.
I am not trying to argue for a different behaviour, but basically
wondering whether we need something like "LatchRearmPostmasterDeath()"
for the benefit of the windows implementation.
Maybe I am just dumb or years of not really touching windows got to me,
but I find the documentation around windows specific stuff utterly
lacking. There doesn't seem to be any comment on how the windows signal
queue stuff works on a high level.
> > To put that another way, one could equally well ask why WaitLatch(),
> > or for that matter
Re: [HACKERS] killing pg_dump leaves backend process
I think this is utterly the won't way to think about this. TCP is designed to be robust against transient network outages. They are *not* supposed to cause disconnections. The purpose of keepalives is to detect connections that are still valid live connections that are stale and the remote end is not longer present for. Keepalives that trigger on the timescale of less than several times the msl are just broken and make TCP unreliable. That means they cannot trigger in less than many minutes. This case is one that should just work and should work immediately. From the users point of view when a client cleanly dies the kernel on the client is fully aware of the connection being closed and the network is working fine. The server should be aware the client has gone away *immediately*. There's no excuse for any polling or timeouts. -- greg On 10 Aug 2013 17:30, "Christopher Browne" wrote: > On Sat, Aug 10, 2013 at 12:30 AM, Tom Lane wrote: > > Tatsuo Ishii writes: > >> I noticed pg_dump does not exit gracefully when killed. > >> start pg_dump > >> kill pg_dump by ctrl-c > >> ps x > > > >> 27246 ?Ds96:02 postgres: t-ishii dbt3 [local] COPY > >> 29920 ?S 0:00 sshd: ishii@pts/5 > >> 29921 pts/5Ss 0:00 -bash > >> 30172 ?Ss 0:00 postgres: t-ishii dbt3 [local] LOCK TABLE > waiting > > > >> As you can see, after killing pg_dump, a backend process is (LOCK > >> TABLE waiting) left behind. I think this could be easily fixed by > >> adding signal handler to pg_dump so that it catches the signal and > >> issues a query cancel request. > > > > If we think that's a problem (which I'm not convinced of) then pg_dump > > is the wrong place to fix it. Any other client would behave the same > > if it were killed while waiting for some backend query. So the right > > fix would involve figuring out a way for the backend to kill itself > > if the client connection goes away while it's waiting. > > This seems to me to be quite a bit like the TCP keepalive issue. > > We noticed with Slony that if something ungraceful happens in the > networking layer (the specific thing noticed was someone shutting off > networking, e.g. "/etc/init.d/networking stop" before shutting down > Postgres+Slony), the usual timeouts are really rather excessive, on > the order of a couple hours. > > Probably it would be desirable to reduce the timeout period, so that > the server could figure out that clients are incommunicado "reasonably > quickly." It's conceivable that it would be apropos to diminish the > timeout values in postgresql.conf, or at least to recommend that users > consider doing so. > -- > When confronted by a difficult problem, solve it by reducing it to the > question, "How would the Lone Ranger handle this?" > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] killing pg_dump leaves backend process
On Sat, Aug 10, 2013 at 12:26:43PM +0100, Greg Stark wrote: > On Sat, Aug 10, 2013 at 5:30 AM, Tom Lane wrote: > > Any other client would behave the same > > if it were killed while waiting for some backend query. So the right > > fix would involve figuring out a way for the backend to kill itself > > if the client connection goes away while it's waiting. > > Well I'm not sure. Maybe every other client should also issue a query > cancel and close the connection if it gets killed. libpq could offer a > function specifically for programs to call from atexit(), signal > handlers, or exception handlers (yes, that might be a bit tricky). > > But I do see a convincing argument for doing something in the server. > Namely that if you kill -9 the client surely the server should still > detect that the connection has gone away immediately. I agree that both efforts have value. A client-side change can't replace the server-side change, and tightening the client side will be more of a neatness measure once the server-side mechanism is in place. > The problem is that I don't know of any way to detect eof on a socket > other than trying to read from it (or calling poll or select). So the > server would have to periodically poll the client even when it's not > expecting any data. The inefficiency is annoying enough and it still > won't detect the eof immediately. Yes, I think that is the way to do it. The check interval could default to something like 90s, high enough to make the cost disappear into the noise and yet a dramatic improvement over the current "no fixed time limit". I bet the utils/timeout.h infrastructure added in 9.3 will make this at least 60% easier to implement than it would have been before. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pass-through queries to foreign servers
On Mon, Aug 5, 2013 at 12:21 PM, Tom Lane wrote:
> David Gudeman writes:
>> For those who don't want to go to the link to see what I'm talking
>> about with query rewrites, I thought I'd give a brief description.
>> Foreign data wrappers currently do all of their work in the planning
>> phase but I claim that isn't the right place to optimize foreign
>> queries with aggregates and GROUP BY because optimizing those things
>> would involve collapsing multiple plan node back into a single node
>> for a foreign call.
>
> I'm not sure what the best implementation for that is, but what you
> propose here would still involve such collapsing, so this argument
> seems rather empty.
>
>> I propose to do these optimizations as query
>> rewrites instead. So for example suppose t is a foreign table on the
>> foreign server named fs. Then the query
>
>> SELECT count(*) FROM t
>
>> is rewritten to
>
>> SELECT count FROM fs('select count(*) from t') fs(count bigint)
>
>> where ts() is the pass-through query function for the server fs. To
>> implement this optimization as a query rewrite, all of the elements of
>> the result have to be real source-language constructs so the
>> pass-through query has to be available in Postgresql SQL.
>
> I don't believe in any part of that design, starting with the "pass
> through query function". For one thing, it seems narrowly targeted to the
> assumption that the FDW is a frontend for a foreign server that speaks
> SQL. If the FDW's infrastructure doesn't include some kind of textual
> query language, this isn't going to be useful for it at all. For another,
> a query rewrite system is unlikely to be able to cost out the alternatives
> and decide whether pushing the aggregation across is actually a win or
> not.
>
> The direction I think we ought to be heading is to generate explicit Paths
> representing the various ways in which aggregation can be implemented.
> The logic in grouping_planner is already overly complex, and hard to
> extend, because it's all hard-wired comparisons of alternatives. We'd be
> better off with something more like the add_path infrastructure. Once
> that's been done, maybe we can allow FDWs to add Paths representing remote
> aggregation.
>
> regards, tom lane
Well, I will have to defer to your greater knowledge of the Posgres
planner. I'm surprised that it can do that since in my experience, the
planner did a cost-based search to find the best join order but the
rest of the plan structure was generated independently of the tables.
If the Postgres planner can look at the FROM clause and generate a
completely different structure based on the types of tables it finds
there, then that doesn't sound unreasonable.
However it still seems a bit like overkill. The purpose of a
cost-based heuristic search is to deal with the fact that there are
lots and lots of possible join orders (as the number of tables grows)
and it makes a huge difference which order you chose. By contrast,
this decision is fairly easy. Either you send the computation to the
data or you bring the data to the computation. Since in the case of
SQL aggregation the computation generally reduces the amount of data
that has to be moved, the first choice is almost always the right
one.
Also, this means that you won't be able to use the code that I've
written since it's all based on query rewriting. I'm disappointed
because I wanted to contribute something back, but that's for you guys
to decide.
Regards,
David Gudeman
http://unobtainabol.blogspot.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
On Sat, Aug 10, 2013 at 10:40 PM, Tom Lane wrote: > Alvaro Herrera writes: > > Hmm, the intention is that this code path mimics what the autovacuum > > launcher does to establish its connection. It did work at some point; > > maybe I broke this before commit. I will give it a look next week. > > I'm pretty sure InitPostgres skips a bunch of stuff if > IsAutoVacuumLauncherProcess(). If you just want to make an environment > equivalent to that process's, maybe those tests should be replaced with > "if (dbname == NULL)". But the claim that you have access to shared > catalogs in this state is still wrong. > I've written up a quick proof-of-concept that splits InitPostgres(dbname, username) into two phases; InitPostgresPhase1() is equivalent to what was set up prior to the autovacuum launcher bailing out, while InitPostgresPhase2(dbname, username) contains all of the authentication and database selection logic. After calling InitPostgresPhase1(), certain accesses to shared catalogs *do* work. For example, ClientAuthentication() searches the AUTHNAME syscache. Later, a call to InitPostgresPhase2() with the dbname and username brings things to a state where normal SPI can be used. This behaviour would be quite useful for certain classes of bgworker. Documentation-wise, it would be helpful to specify precisely what is/isn't allowed in this "shared catalogs only" mode. See the attached patch -- which is just a proof-of-concept -- for the details. Regards, Andrew Tipton bgworker-shared-catalogs-v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: leave a hint when switching logging away from stderr
On Sat, Aug 10, 2013 at 10:57:56AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Fri, Aug 09, 2013 at 06:59:13PM -0400, Tom Lane wrote: > >> Does that ameliorate your concern, or do you still want it to be DEBUG1? > > > I think of the "implicit sequence" messages we moved from NOTICE to DEBUG1 > > somewhat recently. No doubt those messages had helped at times, but they > > didn't quite carry their weight at NOTICE. My gut prediction is that this > > will fall in that same utility range. But you make a valid point about > > noise > > in the startup log being easier to discount. > > Well, we can certainly back off the messages' log level if we get > complaints. But I think the argument for this patch being useful > is a great deal stronger if it's allowed to operate by default. True enough. Your plan makes sense to me now; thanks. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for XML Schema Validation
On Thu, Aug 08, 2013 at 04:42:11AM +, Kodamasimham Pridhvi (MT2012066) wrote: > With reference to "Add XML Schema validation and xmlvalidate functions > (SQL:2008)" in ToDo list, we have gone through pgsql-mailing list but we > didn't find any significant work in this area, so we are proposing our own > model for xml schema validation . please kindly go through it and let us know > how can we improve it.Please find the attached proposal document. Note that PostgreSQL 8.3 had xmlvalidate() for a time; commit 3bf822c4d722d6245a65abdd2502a9d26ab990d5 removed it due to security problems. A new implementation (or revamp of the old) must avoid reintroducing those vulnerabilities. Similar considerations apply to XML Schema. -- Noah Misch EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] killing pg_dump leaves backend process
On Sat, Aug 10, 2013 at 12:30 AM, Tom Lane wrote: > Tatsuo Ishii writes: >> I noticed pg_dump does not exit gracefully when killed. >> start pg_dump >> kill pg_dump by ctrl-c >> ps x > >> 27246 ?Ds96:02 postgres: t-ishii dbt3 [local] COPY >> 29920 ?S 0:00 sshd: ishii@pts/5 >> 29921 pts/5Ss 0:00 -bash >> 30172 ?Ss 0:00 postgres: t-ishii dbt3 [local] LOCK TABLE waiting > >> As you can see, after killing pg_dump, a backend process is (LOCK >> TABLE waiting) left behind. I think this could be easily fixed by >> adding signal handler to pg_dump so that it catches the signal and >> issues a query cancel request. > > If we think that's a problem (which I'm not convinced of) then pg_dump > is the wrong place to fix it. Any other client would behave the same > if it were killed while waiting for some backend query. So the right > fix would involve figuring out a way for the backend to kill itself > if the client connection goes away while it's waiting. This seems to me to be quite a bit like the TCP keepalive issue. We noticed with Slony that if something ungraceful happens in the networking layer (the specific thing noticed was someone shutting off networking, e.g. "/etc/init.d/networking stop" before shutting down Postgres+Slony), the usual timeouts are really rather excessive, on the order of a couple hours. Probably it would be desirable to reduce the timeout period, so that the server could figure out that clients are incommunicado "reasonably quickly." It's conceivable that it would be apropos to diminish the timeout values in postgresql.conf, or at least to recommend that users consider doing so. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?" -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal: leave a hint when switching logging away from stderr
Noah Misch writes: > On Fri, Aug 09, 2013 at 06:59:13PM -0400, Tom Lane wrote: >> Also, I'm not sure that the chattiness argument is relevant, because no >> message will be emitted at all unless you're switching to some log target >> different from the postmaster's initial stderr. So the message won't show >> up in the "official" log target files, only in an arguably vestigial >> startup-time-messages-only file. > Perhaps the chatter would most affect use, typically casual, of pg_ctl without > "-l" or similar. Well, use as casual as that would probably also involve default logging parameters, so that no switch will occur and thus this patch would print nothing new. I think that the vast majority of users who have nondefault logging parameters have them because they're using some packaged version of Postgres, and most of those are probably not using pg_ctl directly at all, but going through some initscript or equivalent to start PG. (At least, that's the set of people that this patch is trying to improve usability for.) Any initscript is going to be redirecting initial stderr to someplace besides the terminal. >> Does that ameliorate your concern, or do you still want it to be DEBUG1? > I think of the "implicit sequence" messages we moved from NOTICE to DEBUG1 > somewhat recently. No doubt those messages had helped at times, but they > didn't quite carry their weight at NOTICE. My gut prediction is that this > will fall in that same utility range. But you make a valid point about noise > in the startup log being easier to discount. Well, we can certainly back off the messages' log level if we get complaints. But I think the argument for this patch being useful is a great deal stronger if it's allowed to operate by default. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
Alvaro Herrera writes: > Andrew Tipton wrote: >> However, should one attempt to pass NULL for the dbname parameter, the >> process will die with: >> FATAL: database 0 does not exist > Hmm, the intention is that this code path mimics what the autovacuum > launcher does to establish its connection. It did work at some point; > maybe I broke this before commit. I will give it a look next week. I'm pretty sure InitPostgres skips a bunch of stuff if IsAutoVacuumLauncherProcess(). If you just want to make an environment equivalent to that process's, maybe those tests should be replaced with "if (dbname == NULL)". But the claim that you have access to shared catalogs in this state is still wrong. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
Andrew Tipton writes: > The documentation for 9.4 says, in the "Background Worker Processes" > section: > Once running, the process can connect to a database by calling > BackgroundWorkerInitializeConnection(char *dbname, char *username). This > allows the process to run transactions and queries using the SPI interface. > If dbname is NULL, the session is not connected to any particular database, > but shared catalogs can be accessed. > Based on my quick read through InitPostgres() and postinit.c, it's not even > clear that there's a way to access the shared catalogs without connecting > to a specific database. There is not, because you need e.g. pg_class and pg_attribute to be available in order to open any relation whatsoever. So this documentation is just wrong. A bg worker that chooses not to connect to a database is shutting itself out of access to all relations. There's been occasional talk of trying to refactor system catalogs into separate shared and local components. The main motivation for that was to reduce the bloat from having many copies of pg_proc etc, but it could conceivably also make it practical for processes to run with only a view of the shared catalogs. The work required seems far out of proportion to these benefits, unfortunately ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
Andrew Tipton wrote: > However, should one attempt to pass NULL for the dbname parameter, the > process will die with: > > FATAL: database 0 does not exist > > BackgroundWorkerInitializeConnection() is essentially just a wrapper around > InitPostgres(), passing it the supplied dbname and username. (And passing > InvalidOid for the dboid parameter.) When InitPostgres() finds that dbname > is null, it falls back on dboid. GetDatabaseTupleByOid() returns NULL when > given InvalidOid, resulting in the aforementioned elog(FATAL). Hmm, the intention is that this code path mimics what the autovacuum launcher does to establish its connection. It did work at some point; maybe I broke this before commit. I will give it a look next week. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] proposal: lob conversion functionality
Pavel Stehule writes: > I found so there are no simple API for working with LO from PL without > access to file system. What? See lo_open(), loread(), lowrite(), etc. > These functions can be simplified if we supports some functions like > encode, decode for LO I do not see any good reason to tie encode/decode to LOs. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
The documentation for 9.4 says, in the "Background Worker Processes" section: Once running, the process can connect to a database by calling BackgroundWorkerInitializeConnection(char *dbname, char *username). This allows the process to run transactions and queries using the SPI interface. If dbname is NULL, the session is not connected to any particular database, but shared catalogs can be accessed. If username is NULL, the process will run as the superuser created during initdb. However, should one attempt to pass NULL for the dbname parameter, the process will die with: FATAL: database 0 does not exist BackgroundWorkerInitializeConnection() is essentially just a wrapper around InitPostgres(), passing it the supplied dbname and username. (And passing InvalidOid for the dboid parameter.) When InitPostgres() finds that dbname is null, it falls back on dboid. GetDatabaseTupleByOid() returns NULL when given InvalidOid, resulting in the aforementioned elog(FATAL). Based on my quick read through InitPostgres() and postinit.c, it's not even clear that there's a way to access the shared catalogs without connecting to a specific database. Certainly the rest of the code in InitPostgres expects MyDatabaseId to be set to something useful. Regards, Andrew Tipton
Re: [HACKERS] killing pg_dump leaves backend process
> On Sat, Aug 10, 2013 at 5:30 AM, Tom Lane wrote: >> Any other client would behave the same >> if it were killed while waiting for some backend query. So the right >> fix would involve figuring out a way for the backend to kill itself >> if the client connection goes away while it's waiting. I am a little bit surprised to hear the response. I'm talking about one of the client programs that are part of PostgreSQL. IMO they should satisfy higher standard than other PostgreSQL application programs in error control and signal handling. > Well I'm not sure. Maybe every other client should also issue a query > cancel and close the connection if it gets killed. libpq could offer a > function specifically for programs to call from atexit(), signal > handlers, or exception handlers (yes, that might be a bit tricky). I'm not sure if it's a duty of libpq. Different applications need to behave differently when catching signals. I think It would be better to leave the job for applications. > But I do see a convincing argument for doing something in the server. > Namely that if you kill -9 the client surely the server should still > detect that the connection has gone away immediately. > > The problem is that I don't know of any way to detect eof on a socket > other than trying to read from it (or calling poll or select). So the > server would have to periodically poll the client even when it's not > expecting any data. The inefficiency is annoying enough and it still > won't detect the eof immediately. I think in some cases reading from socket is not reliable enough to detect a broken socket. Writing to the socket is more reliable. For this prupose Pgpool-II periodically sends "parameter status" packet to frontend while waiting for response from backend to detect the socket is broken or not. Probably PostgreSQL backend could do similar thing. > I would actually tend to think libpq should offer a way to easily send > a cancel and disconnect message immediately upon exiting or closing > the connection *and* the server should periodically poll to check for > the connection being cleanly closed to handle kill -9. > > I'm surprised this is the first time we're hearing people complain > about this. I know I've seen similar behaviour from Mysql and thought > to myself that represented pretty poor behaviour and assumed Postgres > did better. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] killing pg_dump leaves backend process
On Sat, Aug 10, 2013 at 5:30 AM, Tom Lane wrote: > Any other client would behave the same > if it were killed while waiting for some backend query. So the right > fix would involve figuring out a way for the backend to kill itself > if the client connection goes away while it's waiting. Well I'm not sure. Maybe every other client should also issue a query cancel and close the connection if it gets killed. libpq could offer a function specifically for programs to call from atexit(), signal handlers, or exception handlers (yes, that might be a bit tricky). But I do see a convincing argument for doing something in the server. Namely that if you kill -9 the client surely the server should still detect that the connection has gone away immediately. The problem is that I don't know of any way to detect eof on a socket other than trying to read from it (or calling poll or select). So the server would have to periodically poll the client even when it's not expecting any data. The inefficiency is annoying enough and it still won't detect the eof immediately. I would actually tend to think libpq should offer a way to easily send a cancel and disconnect message immediately upon exiting or closing the connection *and* the server should periodically poll to check for the connection being cleanly closed to handle kill -9. I'm surprised this is the first time we're hearing people complain about this. I know I've seen similar behaviour from Mysql and thought to myself that represented pretty poor behaviour and assumed Postgres did better. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
