Re: [HACKERS] possible memory leak with SRFs
Hi, Can someone please explain why we do not reset the expression context if an SRF is involved during execution? Consider srf(foo(col)) where foo returns a pass-by-reference datatype. Your proposed patch would cut the knees out from under argument values that the SRF could reasonably expect to still be there on subsequent calls. Yeah this is my basic confusion. But wouldn't the arguments be evaluated afresh on the subsequent call for this SRF? In that case freeing up the context of the *last* call should not be an issue I would think. And if this is indeed the case we should be using a different longer lived context and not the ecxt_per_tuple_memory context.. Regards, Nikhils -- http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQLSTATE for Hot Standby cancellation
On Thu, 2010-05-06 at 15:00 +0100, Simon Riggs wrote: On Thu, 2010-05-06 at 12:08 +0200, Yeb Havinga wrote: That's funny because when I was reading this thread, I was thinking the exact opposite: having max_standby_delay always set to 0 so I know the standby server is as up-to-date as possible. The application that accesses the hot standby has to be 'special' anyway because it might deliver not-up-to-date data. If that information about specialties regarding querying the standby server includes the warning that queries might get cancelled, they can opt for a retry themselves (is there a special return code to catch that case? like PGRES_RETRY_LATER) or a message to the user that their report is currently unavailable and they should retry in a few minutes. Very sensible. Kevin Grittner already asked for that and I alread agreed, yet it has not been implemented that way http://archives.postgresql.org/pgsql-hackers/2008-12/msg01691.php Can anyone remember a specific objection to that 'cos it still sounds very sensible to me and is a quick, low impact change. Currently the SQLSTATE is ERRCODE_ADMIN_SHUTDOWN or ERRCODE_QUERY_CANCELED if not idle. That makes it even harder to determine the retryability, since it can come back in one of two states. Proposed SQLSTATE for HS cancellation is case PROCSIG_RECOVERY_CONFLICT_BUFFERPIN: case PROCSIG_RECOVERY_CONFLICT_LOCK: case PROCSIG_RECOVERY_CONFLICT_SNAPSHOT: case PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK: recovery_conflict_errcode = ERRCODE_T_R_SERIALIZATION_FAILURE; break; case PROCSIG_RECOVERY_CONFLICT_DATABASE: case PROCSIG_RECOVERY_CONFLICT_TABLESPACE: recovery_conflict_errcode = ERRCODE_ADMIN_SHUTDOWN; break; We don't have a retryable SQLSTATE, so perhaps we should document that serialization_failure and deadlock_detected are both retryable error conditions. As the above implies HS can throw some errors that are non-retyable, so we use ERRCODE_ADMIN_SHUTDOWN. Implemented as ERRCODE_ADMIN_SHUTDOWN only in the case of a dropped database. ERRCODE_T_R_SERIALIZATION_FAILURE in other cases. -- Simon Riggs www.2ndQuadrant.com *** a/src/backend/tcop/postgres.c --- b/src/backend/tcop/postgres.c *** *** 2936,2949 ProcessInterrupts(void) DisableCatchupInterrupt(); if (DoingCommandRead) ereport(FATAL, ! (errcode(ERRCODE_ADMIN_SHUTDOWN), errmsg(terminating connection due to conflict with recovery), errdetail_recovery_conflict(), errhint(In a moment you should be able to reconnect to the database and repeat your command.))); else ereport(ERROR, ! (errcode(ERRCODE_QUERY_CANCELED), errmsg(canceling statement due to conflict with recovery), errdetail_recovery_conflict())); } --- 2936,2949 DisableCatchupInterrupt(); if (DoingCommandRead) ereport(FATAL, ! (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE), errmsg(terminating connection due to conflict with recovery), errdetail_recovery_conflict(), errhint(In a moment you should be able to reconnect to the database and repeat your command.))); else ereport(ERROR, ! (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE), errmsg(canceling statement due to conflict with recovery), errdetail_recovery_conflict())); } -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding xpath_exists function
Robert Haas wrote: Oh, I see. Well, that might be reasonable syntactic sugar, although I think you should make it wrap the path in exists() unconditionally, rather than testing for an existing wrap. I'll leave it out for now, it saves me some effort after all. Please email your patch to the list (replying to this email is fine) and add it here: https://commitfest.postgresql.org/action/commitfest_view/open Will do once I've finished. Thanks for your feedback. Regards, -- Mike Fowler Registered Linux user: 379787 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: Minor notes in CLUSTER page
On Thu, May 6, 2010 at 3:29 PM, Andy Lester a...@petdance.com wrote: I was looking for how to undo a CLUSTER call earlier today. Nothing on the CLUSTER page told me how to do it, or pointed me to the ALTER TABLE page. I figure a pointer to would help the next person in my situation. I've been annoyed by this too and so am in favor of adding a mention in the documentation, but I think we should mention both the CLUSTER and SET WITHOUT CLUSTER variants of ALTER TABLE, and since it's only one sentence I think we should add it adjacent to the existing sentence discussing remembering the index. My proposed patch attached; thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company cluster-docs-rmh.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing HS/SR - 1 vs 2 performance
Erik Rijkers wrote: Everything together: the raid is what Areca call 'raid10(1E)'. (to be honest I don't remember what that 1E exactly means - extra flexibility in the number of disks, I think). Standard RAID10 only supports an even number of disks. The 1E variants also allow putting an odd number in. If you're using an even number, like in your case, the results are the same until you start losing drives, at which point the degradation performance pattern changes a bit due to differences in how things are striped. See these for more info: http://bytepile.com/raid_class.php and http://en.wikipedia.org/wiki/Non-standard_RAID_levels#IBM_ServeRAID_1E http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp?topic=/diricinfo/fqy0_craid1e.html I don't think using RAID1E has anything to do with your results, but it is possible given your test configuration that part of the difference you're seeing relates to where on disk blocks are stored. If you take a hard drive and write two copies of something onto it, the second copy will be a little slower than the first. That's just because how drive speed drops over the surface as you move further along it. There's some examples of what that looks like at http://projects.2ndquadrant.it/sites/default/files/pg-hw-bench-2010.pdf on pages 21-23. Returning to your higher level results, one of the variables you weren't sure how to account for was caching effects on the standby--the possibility that it just didn't have the data cached the same as the master impacting results. The usual way I look for that is by graphing the pgbench TPS over time. In that situation, you can see it shoot upwards over time, very obvious pattern. Example at http://projects.2ndquadrant.it/sites/default/files/pgbench-intro.pdf on pages 36,37. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] no universally correct setting for fsync
Someone just posted to the -admin list with a database corrupted while running with fsync=off. I was all set to refer him to the documentation explaining why he should stop doing this, but to my surprise the documentation waffles on the issue way past what I think is reasonable. http://www.postgresql.org/docs/8.4/interactive/runtime-config-wal.html#GUC-FSYNC There are dire-sounding statements interspersed with: | using fsync results in a performance penalty | Due to the risks involved, there is no universally correct setting | for fsync. | If you trust your operating system, your hardware, and your | utility company (or your battery backup), you can consider | disabling fsync. Isn't this a little too rosy a picture to paint? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
On Fri, May 7, 2010 at 9:47 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Someone just posted to the -admin list with a database corrupted while running with fsync=off. I was all set to refer him to the documentation explaining why he should stop doing this, but to my surprise the documentation waffles on the issue way past what I think is reasonable. http://www.postgresql.org/docs/8.4/interactive/runtime-config-wal.html#GUC-FSYNC There are dire-sounding statements interspersed with: | using fsync results in a performance penalty | Due to the risks involved, there is no universally correct setting | for fsync. | If you trust your operating system, your hardware, and your | utility company (or your battery backup), you can consider | disabling fsync. Isn't this a little too rosy a picture to paint? I agree. I've always thought this part of the documentation made setting fsync=off much more reasonable than I feel it to be. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
On Fri, May 7, 2010 at 16:00, Robert Haas robertmh...@gmail.com wrote: On Fri, May 7, 2010 at 9:47 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Someone just posted to the -admin list with a database corrupted while running with fsync=off. I was all set to refer him to the documentation explaining why he should stop doing this, but to my surprise the documentation waffles on the issue way past what I think is reasonable. http://www.postgresql.org/docs/8.4/interactive/runtime-config-wal.html#GUC-FSYNC There are dire-sounding statements interspersed with: | using fsync results in a performance penalty | Due to the risks involved, there is no universally correct setting | for fsync. | If you trust your operating system, your hardware, and your | utility company (or your battery backup), you can consider | disabling fsync. Isn't this a little too rosy a picture to paint? I agree. I've always thought this part of the documentation made setting fsync=off much more reasonable than I feel it to be. +1, definitely. fsync=off should only be done if you *really* understand what it means, and that requires a lot more explanation than that... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
Kevin Grittner kevin.gritt...@wicourts.gov writes: | If you trust your operating system, your hardware, and your | utility company (or your battery backup), you can consider | disabling fsync. Isn't this a little too rosy a picture to paint? I think that statement is true as far as it goes, but I agree with rejiggering the surrounding text. The whole thing was written back when Postgres was by far the least reliable component of the stack. It isn't anymore. We should make it clear that fsync=off is not ever recommended for production. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] beta to release
On Mon, Jan 11, 2010 at 10:45 PM, Bruce Momjian br...@momjian.us wrote: What amazes me is how many people who closely follow our development are mystified by what we do during that pre-beta period. Hey, I'm still mystified. Maybe you and Tom could do twice-a-week status updates on what you're working on and anything you could use help with, or something. Yea, that would probably help. I know I am not very transparent in this area. I was fairly satisfied with the way that our path from the end of the last CommitFest to beta unfolded from a process standpoint this time (if not entirely from a time standpoint, but that's my own fault as much as anyone - the rest of my life got in the way of PostgreSQL). We had a list of open items on the wiki (actually several lists which were eventually merged) which we worked through and then released beta1. I felt like that was pretty transparent and I understood what the blockers were When we cleared them, we went onto the next thing. I am fuzzier on what happens now. I understand that it depends on what bug reports we get as a result of beta testing, but what I don't quite know is what the expectations are for individual developers, how we're tracking what issues still need to be resolved, or what the process is for deciding when it's time to release. Any clarification from the old hands who have been through this few times before would be much appreciated. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
Kevin Grittner wrote: There are dire-sounding statements interspersed with: | using fsync results in a performance penalty | Due to the risks involved, there is no universally correct setting | for fsync. | If you trust your operating system, your hardware, and your | utility company (or your battery backup), you can consider | disabling fsync. Isn't this a little too rosy a picture to paint? I think the critical question is really whether you are prepared to lose your database. I have a customer who rotates databases in and out of line, and processes major updates on the out of line database. If they lose the database occasionally they are prepared to wear that risk for the performance gain they get from running with fsync off. It just means that they have to recover and so the inline database will get a bit staler than usual while they do. So I think its true that there is no universally right answer. Maybe the criteria mentioned in the last para need tweaking some, though. It's not just a matter of trusting hardware etc. I have seen mishaps when idiots knock out power cords and the like. The unexpected does sometime happen, despite the best planning. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: Minor notes in CLUSTER page
Excerpts from Robert Haas's message of vie may 07 07:33:55 -0400 2010: On Thu, May 6, 2010 at 3:29 PM, Andy Lester a...@petdance.com wrote: I was looking for how to undo a CLUSTER call earlier today. Nothing on the CLUSTER page told me how to do it, or pointed me to the ALTER TABLE page. I figure a pointer to would help the next person in my situation. I've been annoyed by this too and so am in favor of adding a mention in the documentation, but I think we should mention both the CLUSTER and SET WITHOUT CLUSTER variants of ALTER TABLE, and since it's only one sentence I think we should add it adjacent to the existing sentence discussing remembering the index. My proposed patch attached; thoughts? dept. of minor nitpicks: change + use literalCLUSTER/literal or literalSET WITHOUT CLUSTER/literal to + use the literalCLUSTER ON/literal or literalSET WITHOUT CLUSTER/literal Other than that, +1 from me. (Hmm, perhaps even say what each form is for right after mentioning it, instead of mentioning both and then saying what both are for.) -- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: Minor notes in CLUSTER page
nly one sentence I think we should add it adjacent to the existing sentence discussing remembering the index. My proposed patch attached; thoughts? As long as there's a pointer to the answer I'm happy. xoa -- Andy Lester = a...@petdance.com = www.theworkinggeek.com = AIM:petdance -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
Andrew Dunstan and...@dunslane.net wrote: I think the critical question is really whether you are prepared to lose your database. Precisely; and the docs don't make that at all clear. They mention the possibility of database corruption, but downplay it: | When fsync is disabled, the operating system is allowed to do its | best in buffering, ordering, and delaying writes. This can result | in significantly improved performance. However, if the system | crashes, the results of the last few committed transactions might | be lost in part or whole. In the worst case, unrecoverable data | corruption might occur. [valid use case for fsync=off] So I think its true that there is no universally right answer. Maybe the criteria mentioned in the last para need tweaking some, though. I think it goes beyond tweaking -- I think we should have a bald statement like don't turn this off unless you're OK with losing the entire contents of the database cluster. A brief listing of some cases where that is OK might be illustrative. I never meant to suggest any statement in that section is factually wrong; it's just all too rosy, leading people to believe it's no big deal to turn it off. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] possible memory leak with SRFs
Nikhil Sontakke nikhil.sonta...@enterprisedb.com writes: Consider srf(foo(col)) where foo returns a pass-by-reference datatype. Yeah this is my basic confusion. But wouldn't the arguments be evaluated afresh on the subsequent call for this SRF? No, see ExecMakeFunctionResult(). If we did that we'd have serious problems with volatile functions, ie srf(random()). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta to release
Robert Haas robertmh...@gmail.com writes: I am fuzzier on what happens now. I understand that it depends on what bug reports we get as a result of beta testing, but what I don't quite know is what the expectations are for individual developers, how we're tracking what issues still need to be resolved, or what the process is for deciding when it's time to release. Any clarification from the old hands who have been through this few times before would be much appreciated. It's pretty fuzzy. Usually we don't even think about making a release decision until a couple of months have elapsed, and at that point we have a somewhat better handle on what sorts of problems beta has been turning up. I think trying to set release criteria for 9.0 right now would be premature. I would say the expectation for individual developers is test, and read code. It's certainly not time to be starting new feature development yet. As for tracking, historically Bruce has maintained a list of open items, but I think we ought to move that over to a wiki page. The existing PostgreSQL_9.0_Open_Items page would serve fine. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] what is good solution for support NULL inside string_to_array function?
Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: quietly removing NULL is maybe good for compatibility but is wrong for functionality. I agree. I wasn't aware of this little misfeature. Default display for NULL should be a zero-length string. That's just as broken as Pavel's suggestion. Unless you have something that is guaranteed distingishable from the output of any non-null value, you really can't make a significant improvement here. regards, tom lane Is this, perhaps, a generalized case of this long-running discussion from last year?: http://archives.postgresql.org/pgsql-hackers/2009-03/msg01350.php Cheers, Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
I never meant to suggest any statement in that section is factually wrong; it's just all too rosy, leading people to believe it's no big deal to turn it off. Yeah, that section is overdue for an update. I'll write some new text and post it to pgsql-docs. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Two C-interface issues on -Testers
All, BTW, it would be good if some other folks than me were monitoring -testers. Can someone verify that these two C interface issues are intentional? If not, they're bugs: http://archives.postgresql.org/pgsql-testers/2010-05/msg00011.php http://archives.postgresql.org/pgsql-testers/2010-05/msg00010.php -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Two C-interface issues on -Testers
Josh Berkus j...@agliodbs.com writes: Can someone verify that these two C interface issues are intentional? If not, they're bugs: http://archives.postgresql.org/pgsql-testers/2010-05/msg00011.php http://archives.postgresql.org/pgsql-testers/2010-05/msg00010.php That's ecpg, not C interface --- the latter term is unlikely to draw the attention of the right person, namely Michael. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Two C-interface issues on -Testers
On Fri, May 07, 2010 at 02:14:12PM -0400, Tom Lane wrote: That's ecpg, not C interface --- the latter term is unlikely to draw the attention of the right person, namely Michael. Right, thanks Tom. I have an email filter that filters all emails containing ecpg anf puts them into a special directory so they get attention. I will follow up with the reporters. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ 179140304, AIM/Yahoo/Skype michaelmeskes, Jabber mes...@jabber.org VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta to release
On Fri, May 7, 2010 at 11:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: I would say the expectation for individual developers is test, and read code. It's certainly not time to be starting new feature development yet. I am humbly of the opinion that the expectation you have enclosed in quotation marks is far too general to allow me to contribute anything useful. If you, or whoever is driving this release process (if indeed it's being driven and not just meandering rudderless), would care to outline something more specific that you feel it would be useful to have me test, look at, or attempt to address, then I will devote some of my time to doing that. But I won't volunteer to go reread the entire code base or test things at random in the hopes of finding a bug. I don't believe that most other developers are doing that, either. What I think they are doing is lying low and developing their patches without the benefit of feedback from the list so as to avoid getting yelled at for writing them during beta, or else doing non-PostgreSQL work until beta is over. For small patches, that probably doesn't matter very much: those can be developed just as well later on as they can now. For large patches, it's evil incarnate: people who don't start working on those patches (or don't get any useful feedback on them) for another 3 months will be at least 3 months later in finishing them (maybe more, depending on their summer vacation schedule and just when their boss will let them put time into it), and that means they'll be done right at the end of next release cycle. From a project management perspective, I think we will be FAR better off if we take the time to respond to design proposals early and often. I agree we don't have time to do detailed code review now, but telling people not to write any code or ask any questions at this stage of the process is not going to result in them spending more time on beta; it's going to result in them spending less time on PostgreSQL. In further support of the above, let's consider the three largest patches that were outstanding at the end of the 8.4 release cycle. I believe these to be (1) Hot Standby, (2) Streaming Replication, and (3) SE-PostgreSQL. How much work got done on those patches between the end of the last CommitFest for 8.4 and the beginning of the first CommitFest for 8.5? If you go back and look at the archives, I believe you will find that the answer is virtually none. SR and SEPG were submitted *virtually unchanged* from the versions that existed at the end of 8.4 and were both summarily bounced out of the 2009-07 CommitFest for exactly that reason. Hot Standby wasn't even submitted to that CommitFest. There could be many reasons for this and correlation does not imply causality, but wouldn't it have been nice at least SOME of the development that happened between July and November (by which time the bulk of the development was done) had instead happened between February and July? I sure think so. I think we'd be a whole lot happier about this release right now if we'd committed HS and SR in September rather than November and January, and while we can speculate all day about whether that would have happened under any circumstances, our process rendered it a virtual impossibility. As for tracking, historically Bruce has maintained a list of open items, but I think we ought to move that over to a wiki page. The existing PostgreSQL_9.0_Open_Items page would serve fine. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PATCH: Minor notes in CLUSTER page
On Fri, May 7, 2010 at 10:44 AM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Excerpts from Robert Haas's message of vie may 07 07:33:55 -0400 2010: On Thu, May 6, 2010 at 3:29 PM, Andy Lester a...@petdance.com wrote: I was looking for how to undo a CLUSTER call earlier today. Nothing on the CLUSTER page told me how to do it, or pointed me to the ALTER TABLE page. I figure a pointer to would help the next person in my situation. I've been annoyed by this too and so am in favor of adding a mention in the documentation, but I think we should mention both the CLUSTER and SET WITHOUT CLUSTER variants of ALTER TABLE, and since it's only one sentence I think we should add it adjacent to the existing sentence discussing remembering the index. My proposed patch attached; thoughts? dept. of minor nitpicks: change + use literalCLUSTER/literal or literalSET WITHOUT CLUSTER/literal to + use the literalCLUSTER ON/literal or literalSET WITHOUT CLUSTER/literal Other than that, +1 from me. (Hmm, perhaps even say what each form is for right after mentioning it, instead of mentioning both and then saying what both are for.) I couldn't think of a way to phrase that that was any better than the way I wrote it and still allowed me to include an xref to the ALTER TABLE page in a natural way. I am however open to suggestions. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta to release
Robert Haas robertmh...@gmail.com writes: [ argues, in effect, for starting 9.1 development right now ] I can't stop you from spending your time as you please. My development time for at least the next month or two is going to be spent on code-reading the HS/SR code and fixing bugs as they come in. I don't foresee having any time to work on my own 9.1 projects, let alone review anyone else's. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta to release
On Fri, May 7, 2010 at 5:35 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: [ argues, in effect, for starting 9.1 development right now ] I can't stop you from spending your time as you please. My development time for at least the next month or two is going to be spent on code-reading the HS/SR code and fixing bugs as they come in. I don't foresee having any time to work on my own 9.1 projects, let alone review anyone else's. I'm really making an effort to be a good community member. There are a couple of reasons I don't think that I can spend ALL of my PG time over the next few months on release prep: 1. I'm not really sure what I would spend that much time doing. 2. My employer has things they want done for 9.1. That having been said, I am perfectly happy to devote a substantial amount of time to helping with 9.0, but I think it needs a little more organization to make it productive. I am not the first person to make this observation and I'm sure I won't be the last. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
2010/5/6 Tom Lane t...@sss.pgh.pa.us: Bruce Momjian br...@momjian.us writes: OK, seems people like pg_upgrade, but do we call it pgupgrade or pg_upgrade? pg_upgrade sounds good. I just bet that some users will want it to upgrade their postgresql from 9.0.0 to 9.0.1.. The latter. The former is unreadable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Cédric Villemain -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
--On 7. Mai 2010 09:48:53 -0500 Kevin Grittner kevin.gritt...@wicourts.gov wrote: I think it goes beyond tweaking -- I think we should have a bald statement like don't turn this off unless you're OK with losing the entire contents of the database cluster. A brief listing of some cases where that is OK might be illustrative. +1 I never meant to suggest any statement in that section is factually wrong; it's just all too rosy, leading people to believe it's no big deal to turn it off. I think one mistake in this paragraph is the passing mention of performance. I've seen installations in the past with fsync=off only because the admin was pressured to get instantly more speed out of the database (think of fast_mode=on). In my opinion, phrases like performance penalty are misleading, if you need that setting in 99% of all use cases for reliable operation. I've recently even started to wonder if the performance gain with fsync=off is still that large on modern hardware. While testing large migration procedures to a new version some time ago (on an admitedly fast storage) i forgot here and then to turn it off, without a significant degradation in performance. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings
hernan gonzalez hgonza...@gmail.com writes: The issue is that psql tries (apparently) to convert to UTF8 (even when he plans to output the raw text -LATIN9 in this case) just for computing the lenght of the field, to build the table. And because for this computation he (apparently) rely on the string routines with it's own locale, instead of the DB or client encoding. I didn't believe this, since I know perfectly well that the formatting code doesn't rely on any OS-supplied width calculations. But when I tested it out, I found I could reproduce Hernan's problem on Fedora 11. Some tracing showed that the problem is here: fprintf(fout, %.*s, bytes_to_output, this_line-ptr + bytes_output[j]); As the variable name indicates, psql has carefully calculated the number of *bytes* it wants to print. However, it appears that glibc's printf code interprets the parameter as the number of *characters* to print, and to determine what's a character it assumes the string is in the environment LC_CTYPE's encoding. I haven't dug into the glibc code to check, but it's presumably barfing because the string isn't valid according to UTF8 encoding, and then failing to print anything. It appears to me that this behavior violates the Single Unix Spec, which says very clearly that the count is a count of bytes: http://www.opengroup.org/onlinepubs/007908799/xsh/fprintf.html However, I'm quite sure that our chances of persuading the glibc boys that this is a bad idea are zero. I think we're going to have to change the code to not rely on %.*s here. Even without the charset mismatch in Hernan's example, we'd be printing the wrong amount of data anytime the LC_CTYPE charset is multibyte. (IOW, the code should do the wrong thing with forced-line-wrap cases if LC_CTYPE is UTF8, even if client_encoding is too; anybody want to check?) The above coding is new in 8.4, but it's probably not the only use of %.*s --- we had better go looking for other trouble spots, too. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
Bernd Helmle maili...@oopsware.de writes: I've recently even started to wonder if the performance gain with fsync=off is still that large on modern hardware. While testing large migration procedures to a new version some time ago (on an admitedly fast storage) i forgot here and then to turn it off, without a significant degradation in performance. That says to me either that you're using a battery-backed write cache, or your fsyncs don't really work (no write barriers or something like that). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] no universally correct setting for fsync
--On 7. Mai 2010 19:49:15 -0400 Tom Lane t...@sss.pgh.pa.us wrote: Bernd Helmle maili...@oopsware.de writes: I've recently even started to wonder if the performance gain with fsync=off is still that large on modern hardware. While testing large migration procedures to a new version some time ago (on an admitedly fast storage) i forgot here and then to turn it off, without a significant degradation in performance. That says to me either that you're using a battery-backed write cache, or your fsyncs don't really work (no write barriers or something like that). Well, yes, BBU present and proven storage. Maybe i'm wrong, but it seems battery backed write caches aren't that seldom even in low end systems nowadays. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings
However, it appears that glibc's printf
code interprets the parameter as the number of *characters* to print,
and to determine what's a character it assumes the string is in the
environment LC_CTYPE's encoding.
Well, I myself have problems to believe that :-)
This would be nasty... Are you sure?
I couldn reproduce that.
I made a quick test, passing a utf-8 encoded string
(5 bytes correspoding to 4 unicode chars: niño)
And my glib (same Fedora 12) seems to count bytes,
as it should.
#includestdio.h
main () {
char s[] = ni\xc3\xb1o;
printf(|%.*s|\n,5,s);
}
This, compiled with gcc 4.4.3, run with my root locale (utf8)
did not padded a blank. ie it worked as expected.
Hernán
Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings
Sorry about a error in my previous example (mixed width and precision).
But the conclusion is the same - it works on bytes:
#includestdio.h
main () {
char s[] = ni\xc3\xb1o; /* 5 bytes , 4 utf8 chars */
printf(|%*s|\n,6,s); /* this should pad a black */
printf(|%.*s|\n,4,s); /* this should eat a char */
}
[r...@myserv tmp]# ./a.out | od -t cx1
000 | n i 303 261 o | \n | n i 303 261 | \n
7c 20 6e 69 c3 b1 6f 7c 0a 7c 6e 69 c3 b1 7c 0a
Hernán
On Fri, May 7, 2010 at 10:48 PM, hgonza...@gmail.com wrote:
However, it appears that glibc's printf
code interprets the parameter as the number of *characters* to print,
and to determine what's a character it assumes the string is in the
environment LC_CTYPE's encoding.
Well, I myself have problems to believe that :-)
This would be nasty... Are you sure?
I couldn reproduce that.
I made a quick test, passing a utf-8 encoded string
(5 bytes correspoding to 4 unicode chars: niño)
And my glib (same Fedora 12) seems to count bytes,
as it should.
#includestdio.h
main () {
char s[] = ni\xc3\xb1o;
printf(|%.*s|\n,5,s);
}
This, compiled with gcc 4.4.3, run with my root locale (utf8)
did not padded a blank. i.e. it worked as expected.
Hernán
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] possible memory leak with SRFs
Yeah this is my basic confusion. But wouldn't the arguments be evaluated afresh on the subsequent call for this SRF? No, see ExecMakeFunctionResult(). If we did that we'd have serious problems with volatile functions, ie srf(random()). Ok thanks. So if someone uses a really long-running srf with argument expression evaluations thrown in, then running into out of memory issues should be expected and then in those cases they are better off using multiple srf calls to get the same effect if they can.. Regards, Nikhils -- http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
