Re: [HACKERS] Congrats Alvaro!
Congratulations, good things to all the family! Devrim GÜNDÜZ dev...@gunduz.org writes: Alvaro, one of our hackers and committers and my colleague more than 4 years, had a new baby today. Congrats Alvaro for his second daughter ! -committers, please commit your patches for our new baby elephant! -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mailing list archiver chewing patches
Andrew Dunstan and...@dunslane.net writes: That is assuming that the MUA gives you the option of specifying the attachment MIME type. Many (including mine) do not. It would mean an extra step - I'd have to gzip each patch or something like that. That would be unfortunate,as well as imposing extra effort, because it would make the patch not display inline in many MUAs (again, like mine). Bad MUA, change MUA, or what they say… More seriously though, it's not the first time we're having some difficulties with the MHonArc setup, and I think it's also related to the poor thread following on the archives website at month boundaries. MHonArc (http://hydra.nac.uci.edu/indiv/ehood/mhonarc.html) seems to be about converting the mails into some HTML pages, and offering the web interface to get to use them, with some indexing and searches facilities. Are our indexing and searches provided by MHonArc or maintained by the community? How helpful considering alternatives, such as AOX (which runs atop PostgreSQL and would offer anonymous IMAP facility over the archives) would be? Of course it'll boil down to who's maintaining the current solution and how much time is allocated to this, the solution research and migration would have to fit in there I suppose. Same as pgfoundry. But still, should we talk about it? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Add .gitignore files to CVS?
Hi, Another occasion to show ignorance, I couldn't resist! Tom Lane t...@sss.pgh.pa.us writes: What you're talking about would require a great deal more maintenance effort, and I don't see the point compared to using a VPATH build. I've discovered VPATH builds pretty recently, in the context of packaging extensions. The concept is simple but it could be helpful if spelled in simple terms: VPATH is about finding the sources. So you build wherever you want without messing your checkout. But you build from the target directory, telling in VPATH where the sources are. HTH, regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 05:54, Josh Berkus j...@agliodbs.com wrote: Peter, Just to clarify: I am for sticking to the agreed dates. If some things are not ready by the necessary date plus/minus one, they won't make the release. If it's obvious earlier that something won't make the date, it shouldn't be committed, and maybe put on the backburner right now. But AFAICT, that's independent of when it was submitted. Some things that were submitted just the other day might be almost ready, some things that were first submitted many months ago are still not ready. In that case, Robert's comments about patches to bounce on Day 1 of the commitfest are still valid, regardless of patch size. That is, if we're getting patches which seem very unlikely to make the cut by Feburary 15 (like KNNGiST, which currently doesn't even apply), then it makes sense for the CFM to notify those authors as soon as possible that their patch is probably last in line to get reviewed. If it doesn't even build by the time the CF starts (and didn't just break in the past couple of days), can it even be considered submitted? I think it's perfectly fair to bounce something that's been sitting in waiting for author for weeks *before* the CF starts at an early stage, to focus the resources on things that are up-to-date. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 07:38, Tom Lane t...@sss.pgh.pa.us wrote: Robert Treat xzi...@users.sourceforge.net writes: ... I don't see much sense in worrying about it now; the 2 weeks between end of CF and Beta are when we need to be cut-throat. Given that this time the must-have feature is already in the tree, I think you will find people coming around quickly to the side of pushing things out rather than fighting to get things in. I think the other Robert's main point is that getting to beta in only two weeks is ridiculously optimistic (which I'm afraid I agree with). I believe that what he's proposing is tossing enough stuff overboard so that we can finish the January CF in much less than a month, and thereby have more time for alpha-level testing and stabilization of the tree. I realize it's moving the goalposts and not necessraily fair, but we could always try to cut a week off the CF for some more stabilization period if we think that'll be enough to make a difference. Now the other approach we could take is that we'll ship *something* on 7 Mar, even if it's less stable than what we've traditionally considered to be beta quality. I don't think that really helps much though; it just means we need more time in beta. That sounds mor elike a final alpha release, in that case? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 08:09, Robert Treat xzi...@users.sourceforge.net wrote: On Sunday 10 January 2010 01:38:07 Tom Lane wrote: Robert Treat xzi...@users.sourceforge.net writes: ... I don't see much sense in worrying about it now; the 2 weeks between end of CF and Beta are when we need to be cut-throat. Given that this time the must-have feature is already in the tree, I think you will find people coming around quickly to the side of pushing things out rather than fighting to get things in. I think the other Robert's main point is that getting to beta in only two weeks is ridiculously optimistic (which I'm afraid I agree with). I believe that what he's proposing is tossing enough stuff overboard so that we can finish the January CF in much less than a month, and thereby have more time for alpha-level testing and stabilization of the tree. I agree with your summary, although I'm not sure it needs to be supported at this point. While it hasn't been stated explicitly, I suspect most reviewers will be reviewing with the idea of is there any chance that this is ready for commit in the back of thier heads, and things that aren't will likely get pushed off quickly. So how about we make that explicitly stated? Now the other approach we could take is that we'll ship *something* on 7 Mar, even if it's less stable than what we've traditionally considered to be beta quality. I don't think that really helps much though; it just means we need more time in beta. There are three reasons I'd probably be comfortable with that; 1) the CF process means we've likely had more eyes on the code going in than in past releases. 2) the alpha releases mean we should have already had more review than in previous releases. 3) so far we're still looking good on pg_migrator, which should make it easier for people to test the release once we get into beta (which should help speed that cycle up). 1 is hopfully right. 2 should be, but we haven't received many bug reports for people with the alphas. Which certainly *could* mean that 1 made there not be many bugs, but it could also mean that people aren't really testing it, just tasting it. But really if beta slips because we don't like the looks of our open issues list, thats signicantly better than the last couple releases where we held everything up just to get things into CVS months after feature freeze had passed us by. It is, unless it's because we created those open items by committing things too late in the cycle, even though it wasn't quite ready, so the author won't have to wait another year for a release. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We need to rethink relation cache entry rebuild
Tom Lane wrote: I mentioned earlier that buildfarm member jaguar (that's the one that builds with CLOBBER_CACHE_ALWAYS) was showing suspicious intermittent failures. There's another one today: hmm I was just doing a CLOBBER_CACHE_ALWAYS build on one of my ARM based boxes and it seems to fall over very quickly as well however I'm not getting a useful backtrace from gdb... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
On Fri, 2010-01-08 at 23:16 +0200, Heikki Linnakangas wrote: * I removed the feature that archiver was started during recovery. The idea of that was to enable archiving from a standby server, to relieve the master server of that duty, but I found it annoying because it causes trouble if the standby and master are configured to archive to the same location; they will fight over which copies the file to the archive first. Frankly the feature doesn't seem very useful as the patch stands, because you still have to configure archiving in the master in practice; you can't take an online base backup otherwise, and you have the risk of standby falling too much behind and having to restore from base backup whenever the standby is disconnected for any reason. Let's revisit this later when it's truly useful. Agreed * We still have a related issue, though: if standby is configured to archive to the same location as master (as it always is on my laptop, where I use the postgresql.conf of the master unmodified in the server), right after failover the standby server will try to archive all the old WAL files that were streamed from the master; but they exist already in the archive, as the master archived them already. I'm not sure if this is a pilot error, or if we should do something in the server to tell apart WAL segments streamed from master and those generated in the standby server after failover. Maybe we should immediately create a .done file for every file received from master? That sounds like the right thing to do. * I don't think we should require superuser rights for replication. Although you see all WAL and potentially all data in the system through that, a standby doesn't need any write access to the master, so it would be good practice to create a dedicated account with limited privileges for replication. Agreed. I think we should have a predefined user, called replication that has only the correct rights. * A standby that connects to master, initiates streaming, and then sits idle without stalls recycling of old WAL files in the master. That will eventually lead to a full disk in master. Do we need some kind of a emergency valve on that? Can you explain how this could occur? My understanding was that the walreceiver and startup processes were capable of independent action specifically to avoid for this kind of effect. * Documentation. The patch used to move around some sections, but I think that has been partially reverted so that it now just duplicates them. It probably needs other work too, I haven't looked at the docs in any detail. I believe the docs need urgent attention. We need more people to read the docs and understand the implications so that people can then comment. It is extremely non-obvious from the patch how things work at a behaviour level. I am very concerned that there is no thought given to monitoring replication. This will make the feature difficult to use in practice. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
On Fri, 2010-01-08 at 14:20 -0800, Josh Berkus wrote: On 1/8/10 1:16 PM, Heikki Linnakangas wrote: * A standby that connects to master, initiates streaming, and then sits idle without stalls recycling of old WAL files in the master. That will eventually lead to a full disk in master. Do we need some kind of a emergency valve on that? WARNING: I haven't thought about how this would work together with HS yes. I've been reviewing things as we go along, so I'm not that tense overall. Having said that I don't understand why the problem above would occur and the sentence seems to be missing a verb between without and stalls. More explanation please. What could happen is that the standby could slowly lag behind master. We don't have any way of monitoring that, as yet. Setting ps display is not enough here. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We need to rethink relation cache entry rebuild
On Fri, 2010-01-08 at 20:01 -0500, Tom Lane wrote: I mentioned earlier that buildfarm member jaguar (that's the one that builds with CLOBBER_CACHE_ALWAYS) was showing suspicious intermittent failures. There's another one today: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jaguardt=2010-01-08%2004:00:02 Thanks for looking into that; I'm sorry that I was too busy on HS things and was forced to hand back when the issue occurred earlier. What I have in mind is to rewrite RelationClearRelation That sounds like it is an isolated fix, so that's good from where I'm sitting. I will leave off coding anything around VFI until this is done. I was realistically around 2 weeks from doing that anyway, so I don't foresee any delay for HS. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] synchronized snapshots
On Fri, 2010-01-08 at 20:36 +0100, Joachim Wieland wrote: The attached patch implements the idea of Heikki / Simon published in http://archives.postgresql.org/pgsql-hackers/2009-11/msg00271.php Since nobody objected to the idea in general, I have implemented it. As this is not currently used anywhere it doesn't give immediate benefit, it is however a prerequisite for a parallel version of pg_dump that quite some people (including myself) seem to be interested in. I'm interested in this, but realistically won't have time to review this personally in this release. Sorry about that. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication and triggering failover
On Fri, 2010-01-08 at 12:41 +0200, Heikki Linnakangas wrote: let's make the default no failover if no trigger file location is configured, and remove the notion that normal shutdown of master stops recovery. +1 -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sat, 2010-01-09 at 08:46 -0500, Robert Haas wrote: it seems much better to me to have the rule than not I think we can overplay the need for lots of rules here and the need to chase up status every 5 minutes. The first problem, in previous years, was patches spent too long on the patch queue. That is now solved, AFAICS, at least to my satisfaction. The second problem was spending 4 months on code consolidation at last commitfest before we go to beta. We are unlikely to avoid that just by saying it will be two weeks; trying to force it will cause arguments, waste developer time and possibly endanger code quality. AFAICS we are on track for a much improved situation than before and I'm personally happy with that also. Quality Completeness Timeliness, IMHO. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Congrats Alvaro!
Congratulations Alvaro! 2010/1/10 Devrim GÜNDÜZ dev...@gunduz.org: Alvaro, one of our hackers and committers and my colleague more than 4 years, had a new baby today. Congrats Alvaro for his second daughter ! -committers, please commit your patches for our new baby elephant! -- Devrim GÜNDÜZ, RHCE Command Prompt - http://www.CommandPrompt.com devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] ECPG patch causes warning
Hi! The ecpg patch at http://git.postgresql.org/gitweb?p=postgresql.git;a=commit;h=2f567552 causes a compile warning on win64 (andi think win32, but I didn't recheck that). Specifically, line 140 of typename.c has: return (-type); Where type is of type Oid, which is unsigned. This causes the warning: .\src\interfaces\ecpg\ecpglib\typename.c(140): warning C4146: unary minus operator applied to unsigned type, result still unsigned I think that line tries to fix that problem, but either it does it wrong, or just in a way that MSVC can't deal with. I think you need to cast it to a signed type as well? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Initial refactoring of plperl.c - updated
On Sun, Jan 10, 2010 at 01:16:13AM -0500, Tom Lane wrote: Tim Bunce tim.bu...@pobox.com writes: On Sat, Jan 09, 2010 at 11:16:18PM +0200, Peter Eisentraut wrote: What's the reason for the temp file here? Defensive. If the text2macro.pl program fails/dies then you'd be left with a broken output file with a newer timestamp, so the next make wouldn't rerun text2macro.pl. Doesn't make forcibly remove the target file if the command fails? [ rummages in manual... ] It does if you specify .DELETE_ON_ERROR, which we do (in Makefile.global); so this bit of complication is a waste. Okay. Andrew, perhaps you could apply the attached to fix that. (Or I could bundle it into one of the split out plperl feature patches.) Tim. diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index 43b0fd0..3733da7 100644 *** a/src/pl/plperl/GNUmakefile --- b/src/pl/plperl/GNUmakefile *** include $(top_srcdir)/src/Makefile.shlib *** 57,64 plperl.o: perlchunks.h perlchunks.h: $(PERLCHUNKS) ! $(PERL) $(srcdir)/text2macro.pl --strip='^(\#.*|\s*)$$' $^ perlchunks.htmp ! mv perlchunks.htmp perlchunks.h all: all-lib --- 57,63 plperl.o: perlchunks.h perlchunks.h: $(PERLCHUNKS) ! $(PERL) $(srcdir)/text2macro.pl --strip='^(\#.*|\s*)$$' $^ perlchunks.h all: all-lib *** submake: *** 79,85 $(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X) clean distclean maintainer-clean: clean-lib ! rm -f SPI.c $(OBJS) perlchunks.htmp perlchunks.h rm -rf results rm -f regression.diffs regression.out --- 78,84 $(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X) clean distclean maintainer-clean: clean-lib ! rm -f SPI.c $(OBJS) perlchunks.h rm -rf results rm -f regression.diffs regression.out -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] win32 socket definition
Tom Lane wrote: There's another copy of ListenSocket[] in the BackendParameters struct. I also wonder about postmaster.c's habit of using -1 for empty slots in ListenSocket ... how safe is that for Win64? On Windows, it should be INVALID_SOCKET. James -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] win32 socket definition
On Sun, Jan 10, 2010 at 13:33, James Mansion ja...@mansionfamily.plus.com wrote: Tom Lane wrote: There's another copy of ListenSocket[] in the BackendParameters struct. I also wonder about postmaster.c's habit of using -1 for empty slots in ListenSocket ... how safe is that for Win64? On Windows, it should be INVALID_SOCKET. Indeed it should, but I think we're Ok anyway. Here's the extract from the SDK headers. /* * This is used instead of -1, since the * SOCKET type is unsigned. */ #define INVALID_SOCKET (SOCKET)(~0) #define SOCKET_ERROR(-1) But it might be worthwhile going across all those places and setting them to PGINVALID_SOCKET, and typedef that as well. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 2:09 AM, Robert Treat xzi...@users.sourceforge.net wrote: But really if beta slips because we don't like the looks of our open issues list, thats signicantly better than the last couple releases where we held everything up just to get things into CVS months after feature freeze had passed us by. Yes. It seems that we're at least not going to let the final CommitFest drag on and on and on this time, which seems like a positive step. But will it help enough to make everyone feel satisfied with the results? That's less clear. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 5:31 AM, Magnus Hagander mag...@hagander.net wrote: But really if beta slips because we don't like the looks of our open issues list, thats signicantly better than the last couple releases where we held everything up just to get things into CVS months after feature freeze had passed us by. It is, unless it's because we created those open items by committing things too late in the cycle, even though it wasn't quite ready, so the author won't have to wait another year for a release. Exactly. Or even - it was ready, but even code that's ready can have bugs. The more time you give yourself before release, the more likely you are to find a few of those. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 5:27 AM, Magnus Hagander mag...@hagander.net wrote: On Sun, Jan 10, 2010 at 05:54, Josh Berkus j...@agliodbs.com wrote: Peter, Just to clarify: I am for sticking to the agreed dates. If some things are not ready by the necessary date plus/minus one, they won't make the release. If it's obvious earlier that something won't make the date, it shouldn't be committed, and maybe put on the backburner right now. But AFAICT, that's independent of when it was submitted. Some things that were submitted just the other day might be almost ready, some things that were first submitted many months ago are still not ready. In that case, Robert's comments about patches to bounce on Day 1 of the commitfest are still valid, regardless of patch size. That is, if we're getting patches which seem very unlikely to make the cut by Feburary 15 (like KNNGiST, which currently doesn't even apply), then it makes sense for the CFM to notify those authors as soon as possible that their patch is probably last in line to get reviewed. If it doesn't even build by the time the CF starts (and didn't just break in the past couple of days), can it even be considered submitted? I think it's perfectly fair to bounce something that's been sitting in waiting for author for weeks *before* the CF starts at an early stage, to focus the resources on things that are up-to-date. Well, if it's not updated before the CommitFest starts, sure. But I suspect an update will come through at the last minute. A few more large patches may show up, too, and some small ones almost certainly will. The number of patches on the CommitFest page typically increases very quickly in the last few days before we get started. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Josh Berkus wrote: I'll also say: if we can't make time-based releases work, we're probably dead as a project. MySQL and Ingres both tried feature-based releases, and look where they are now. I think you're engaging in a bit of 'post hoc ergo propter hoc' reasoning here. In any case, I think it's a false dichotomy. We always seem to treat this as an all or nothing deal. But there is no reason that we must. We could easily decide that one or two features were critical for the release and everything else would have to make the time cut. For this release those features would obviously be HS and SR. Obviously we could could at some stage decide to cut our losses and run, as we did last release (eventually). But I think that the idea that we should never at least have a stated goal to have certain features in a given release is a bit silly. If you think we could put out this coming release without having HS + SR, and not do significant damage to the project, I can only say I profoundly disagree. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] maintenance announcement for dekeni.postgresql.org and minshara.postgresql.org
Stefan Kaltenbrunner wrote: Hi all! In order to install some OS security upgrades we are going to execute planned maintainance one the following hosts affecting the mentioned services tomorrow (10th January 13:00-14:00 GMT). Actual expected downtime will be a few minutes at most so this is just a heads up. minshara.postgresql.org affecting the following services: * gothos.postgresql.org(git.postgresql.org) * fornax.postgresql.org(ftp.postgresql.org) * wysanti.postgresql.org(static web mirror) dekeni.postgresql.org affecting the following services: * coridan.postgresql.org(commitfest.postgresql.org) * mintaka.postgresql.org(media.postgresql.org) done and all services back up. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] RADIUS authentication
The attached patch implements RADIUS authentication (RFC2865-compatible). The main usecase for me in this is the ability to use (token based) one-time-password systems easily with PostgreSQL. These systems almost always support RADIUS, and the implementation is fairly simple. RADIUS can of course be used in many other scenarios as well (for example, it can be used to implement only this group-access with at least Active Directory, something our current LDAP doesn't support. We might eventually want to support that in our LDAP, but it's not there now) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ radius.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] win32 socket definition
On Sun, Jan 10, 2010 at 13:44, Magnus Hagander mag...@hagander.net wrote: On Sun, Jan 10, 2010 at 13:33, James Mansion ja...@mansionfamily.plus.com wrote: Tom Lane wrote: There's another copy of ListenSocket[] in the BackendParameters struct. I also wonder about postmaster.c's habit of using -1 for empty slots in ListenSocket ... how safe is that for Win64? On Windows, it should be INVALID_SOCKET. Indeed it should, but I think we're Ok anyway. Here's the extract from the SDK headers. /* * This is used instead of -1, since the * SOCKET type is unsigned. */ #define INVALID_SOCKET (SOCKET)(~0) #define SOCKET_ERROR (-1) But it might be worthwhile going across all those places and setting them to PGINVALID_SOCKET, and typedef that as well. That was pretty easy, provided I didn't miss too many places :-) So I did that, and applied. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Small Bug in GetConflictingVirtualXIDs
On Sun, 2009-12-27 at 20:11 +0100, Andres Freund wrote: On Tuesday 22 December 2009 11:42:30 Simon Riggs wrote: On Tue, 2009-12-22 at 03:19 +0100, Andres Freund wrote: On Monday 21 December 2009 16:48:52 Simon Riggs wrote: Giving the drop database a snapshot is not the answer. I expect Andres to be able to fix this with a simple patch that would not effect the case of normal running. Actually its less simply than I had thought at first - I don't think the code ever handled that correctly. I might be wrong there, my knowledge of the involved code is a bit sparse... The whole conflict resolution builds on the concept of waiting for an VXid, but an idle backend does not have a valid vxid. Thats correct, right? I don't see any mileage in making Startup process wait for an idle session, so no real reason to wait for others either. So here is a small patch implementing that behaviour. I've committed a slightly modified form of this patch. It was an outstanding bug, so delaying fix at this stage not worth it. I had in mind a slightly grander fix, but it's hardly a priority to polish the chrome on this one. Thanks for the bug report and fix. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG patch causes warning
Magnus Hagander mag...@hagander.net writes: The ecpg patch at http://git.postgresql.org/gitweb?p=postgresql.git;a=commit;h=2f567552 causes a compile warning on win64 (andi think win32, but I didn't recheck that). Specifically, line 140 of typename.c has: return (-type); Where type is of type Oid, which is unsigned. I think that the compiler has caught an actual mistake here. It looks to me like the patch is attempting to use a 'negative' Oid to signal a problem, and that simply is going to break as soon as the Oid counter runs past 2G. Perhaps InvalidOid is the thing to use here? I did not look at the call sites though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG patch causes warning
Tom Lane írta: Magnus Hagander mag...@hagander.net writes: The ecpg patch at http://git.postgresql.org/gitweb?p=postgresql.git;a=commit;h=2f567552 causes a compile warning on win64 (andi think win32, but I didn't recheck that). Specifically, line 140 of typename.c has: return (-type); Where type is of type Oid, which is unsigned. I think that the compiler has caught an actual mistake here. Yes, it's a mistake, but not an actual bug. The intent was to be able to catch unhandled cases in the application, just as in ecpg_dynamic_type(). The fix for sqlda_dynamic_type() is to use the same cast: return -(int) type; Should I send a patch for this? It looks to me like the patch is attempting to use a 'negative' Oid to signal a problem, and that simply is going to break as soon as the Oid counter runs past 2G. Perhaps InvalidOid is the thing to use here? I did not look at the call sites though. regards, tom lane -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
Simon Riggs wrote: On Fri, 2010-01-08 at 14:20 -0800, Josh Berkus wrote: On 1/8/10 1:16 PM, Heikki Linnakangas wrote: * A standby that connects to master, initiates streaming, and then sits idle without stalls recycling of old WAL files in the master. That will eventually lead to a full disk in master. Do we need some kind of a emergency valve on that? WARNING: I haven't thought about how this would work together with HS yes. I've been reviewing things as we go along, so I'm not that tense overall. Having said that I don't understand why the problem above would occur and the sentence seems to be missing a verb between without and stalls. More explanation please. Yeah, that sentence was broken. What could happen is that the standby could slowly lag behind master. Right, that's what I'm worried about. In the worst case it the walreceiver process in the standby might stall completely for some reason, e.g hardware problem or SIGSTOP by an administrator. We don't have any way of monitoring that, as yet. Setting ps display is not enough here. Yeah, monitoring would be nice too. But what I was wondering is whether we need some way of stopping that from filling the disk in master. (Fujii-san's suggestion of a GUC to set the max. amount of WAL to keep in the master for standbys feels good to me). -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Windows x64 [repost]
On Fri, Dec 4, 2009 at 11:42, Tsutomu Yamada tsut...@sraoss.co.jp wrote: The following patches support Windows x64. 1) use intptr_t for Datum and pointer macros. (to support Windows LLP64) almost the same as that post before. http://archives.postgresql.org/pgsql-hackers/2009-06/threads.php#01364 2) use appropriate macro and datatypes for Windows API. enables more than 32bits shared memory. 3) Build scripts for MSVC, this came from http://archives.postgresql.org/pgsql-hackers/2008-07/msg00440.php add new parameters to config.pl. You need define platform to x64 for 64bit programs. I have now applied this, with some extensive refactoring and extension. But all parts in this one should be in now, so if you are missing something, please let me know :-) I was checked where the string converted with %ld is used. An especially fatal part is not found excluding one of plperl. I have not looked at the plperl stuff yet. I'd appreciate a comment from someone who knows plperl :-) Andrew, maybe? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG patch causes warning
Boszormenyi Zoltan z...@cybertec.at writes: Tom Lane írta: I think that the compiler has caught an actual mistake here. Yes, it's a mistake, but not an actual bug. The intent was to be able to catch unhandled cases in the application, just as in ecpg_dynamic_type(). The fix for sqlda_dynamic_type() is to use the same cast: return -(int) type; Should I send a patch for this? You should send a patch to *get rid of it*. Putting in a cast only hides the fact that this logic will break on OIDs above 2G. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Windows x64 [repost]
Magnus Hagander mag...@hagander.net writes: On Fri, Dec 4, 2009 at 11:42, Tsutomu Yamada tsut...@sraoss.co.jp wrote: I was checked where the string converted with %ld is used. An especially fatal part is not found excluding one of plperl. I have not looked at the plperl stuff yet. I'd appreciate a comment from someone who knows plperl :-) Andrew, maybe? I think I dealt with that already, if Tsutomu-san is speaking of the prepared-query hash table keys in plperl and pltcl. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
On Sun, 2010-01-10 at 18:40 +0200, Heikki Linnakangas wrote: We don't have any way of monitoring that, as yet. Setting ps display is not enough here. Yeah, monitoring would be nice too. But what I was wondering is whether we need some way of stopping that from filling the disk in master. (Fujii-san's suggestion of a GUC to set the max. amount of WAL to keep in the master for standbys feels good to me). OK, now I got you. I thought that was already agreed; guess it is now. We need monitoring anywhere we have a max_* parameter. Otherwise we won't know how close we are to disaster until we hit the limit and things break down. Otherwise we will have to set parameters by trial and error, or set them so high they are meaningless. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
On sön, 2010-01-10 at 01:38 -0500, Tom Lane wrote: Now the other approach we could take is that we'll ship *something* on 7 Mar, even if it's less stable than what we've traditionally considered to be beta quality. I don't think that really helps much though; it just means we need more time in beta. Maybe we need to ponder for a minute what beta ought to mean for us. With all the new processes and guidelines being established, the transition criteria into and out of beta are in my mind pretty weakly defined. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RADIUS authentication
On sön, 2010-01-10 at 14:25 +0100, Magnus Hagander wrote: The attached patch implements RADIUS authentication (RFC2865-compatible). The main usecase for me in this is the ability to use (token based) one-time-password systems easily with PostgreSQL. These systems almost always support RADIUS, and the implementation is fairly simple. RADIUS can of course be used in many other scenarios as well (for example, it can be used to implement only this group-access with at least Active Directory, something our current LDAP doesn't support. We might eventually want to support that in our LDAP, but it's not there now) Sounds interesting; I didn't know RADIUS was still in use. There is a copy-and-paste'o in the patch, where LDAP is mentioned instead of RADIUS. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function
Dean Rasheed dean.a.rash...@googlemail.com writes: 2010/1/9 Tom Lane t...@sss.pgh.pa.us: I think that a variant of your idea could be made to work: change plpgsql_LookupIdentifiers to a three-state variable (which'd basically mean in DECLARE, in a SQL expression, anywhere else), do no lookups in DECLARE, and in SQL expressions do lookups but never throw any errors. I'll have a go at that. Sounds plausible. I've applied a patch along these lines. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RADIUS authentication
On Sun, Jan 10, 2010 at 18:55, Peter Eisentraut pete...@gmx.net wrote: On sön, 2010-01-10 at 14:25 +0100, Magnus Hagander wrote: The attached patch implements RADIUS authentication (RFC2865-compatible). The main usecase for me in this is the ability to use (token based) one-time-password systems easily with PostgreSQL. These systems almost always support RADIUS, and the implementation is fairly simple. RADIUS can of course be used in many other scenarios as well (for example, it can be used to implement only this group-access with at least Active Directory, something our current LDAP doesn't support. We might eventually want to support that in our LDAP, but it's not there now) Sounds interesting; I didn't know RADIUS was still in use. It's very much in use. It works well for that kind of scenario, and it's still very much in use by ISPs (for other things than database,s of course) There is a copy-and-paste'o in the patch, where LDAP is mentioned instead of RADIUS. Yeah, Stefan just pointed that out to me over IM. Thanks. There's also a smis-spelling of the word RADIUS :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Initial refactoring of plperl.c - updated
Tim Bunce tim.bu...@pobox.com writes: On Sun, Jan 10, 2010 at 01:16:13AM -0500, Tom Lane wrote: Doesn't make forcibly remove the target file if the command fails? Andrew, perhaps you could apply the attached to fix that. (Or I could bundle it into one of the split out plperl feature patches.) Applied. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ECPG patch causes warning
Tom Lane írta: Boszormenyi Zoltan z...@cybertec.at writes: Tom Lane írta: I think that the compiler has caught an actual mistake here. Yes, it's a mistake, but not an actual bug. The intent was to be able to catch unhandled cases in the application, just as in ecpg_dynamic_type(). The fix for sqlda_dynamic_type() is to use the same cast: return -(int) type; Should I send a patch for this? You should send a patch to *get rid of it*. Putting in a cast only hides the fact that this logic will break on OIDs above 2G. As would ecpg_dynamic_type(), then. :-( You wrote previously: Perhaps InvalidOid is the thing to use here? Perhaps InvalidOid wouldn't be the best choice to return, because this function returns int, not Oid. InvalidOid equals to ECPGt_char. Hm... it would be hiding the failure in a good way, as the type returned couldn't be mapped to any ECPGt_* type, and the value would be returned in a string anyway. We can use ECPGt_char for the unhandled case. Another way (to lose only twenty-something values from 4GB) would be to introduce ECPGt_max and return (ECPGt_max + type) to have mapping for a large amount of type Oids. There would be a drawback in this case, when a new type might get added to ECPG, and the value of ECPGt_max changes. An Oid - int mapping has the advantage of being able to catch non-standard types, but libecpg and the app may have different opinion about the value of ECPGt_max, as the app may be compiled against a different version of libecpg in a theoretical future. I think that leaves the first choice. Patch is attached. Best regards, Zoltán Böszörményi -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ *** pgsql.orig/src/interfaces/ecpg/ecpglib/typename.c 2010-01-05 18:02:03.0 +0100 --- pgsql.3/src/interfaces/ecpg/ecpglib/typename.c 2010-01-10 19:12:07.0 +0100 *** sqlda_dynamic_type(Oid type, enum COMPAT *** 136,142 #ifdef HAVE_LONG_INT_64 return ECPGt_long; #endif default: ! return (-type); } } --- 136,143 #ifdef HAVE_LONG_INT_64 return ECPGt_long; #endif + /* Unhandled types always return a string */ default: ! return ECPGt_char; } } -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Congrats Alvaro!
On Sun, Jan 10, 2010 at 05:16:48AM +0200, Devrim GUNDUZ wrote: Alvaro, one of our hackers and committers and my colleague more than 4 years, had a new baby today. Congrats Alvaro for his second daughter ! In short, +1 from Alvaro and Sra. Herrera :) -committers, please commit your patches for our new baby elephant! w00t! Cheers, David -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Tim Bunce tim.bu...@pobox.com writes: On Fri, Jan 08, 2010 at 10:36:43PM -0500, Robert Haas wrote: I kind of thought Tom said these were a bad idea, and I think I kind of agree. Tom had some concerns which I believe I've addressed. You haven't addressed them, you've simply ignored them. For the record, I think it's a bad idea to run arbitrary user-defined code in the postmaster, and I think it's a worse idea to run arbitrary user-defined code at backend shutdown (the END-blocks bit). I do not care in the least what applications you think this might enable --- the negative consequences for overall system stability seem to me to outweigh any possible arguments on that side. What happens when the supplied code has errors, takes an unreasonable amount of time to run, does something unsafe, depends on the backend not being in an error state already, etc etc? I do not have a veto over stuff like this, but if I did, it would not go in. We're not going to support multi-line values for GUCs AFAIK, so this is going to be pretty kludgy. I'm not sure what you mean by this. What he means by this is defining GUCs in a way that would make people want to use multi-line values for them. However, that doesn't have anything to do with my worries ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
On Sun, Jan 10, 2010 at 1:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Tim Bunce tim.bu...@pobox.com writes: On Fri, Jan 08, 2010 at 10:36:43PM -0500, Robert Haas wrote: I kind of thought Tom said these were a bad idea, and I think I kind of agree. Tom had some concerns which I believe I've addressed. You haven't addressed them, you've simply ignored them. That's not *completely* true. http://archives.postgresql.org/pgsql-hackers/2009-12/msg00432.php For the record, I think it's a bad idea to run arbitrary user-defined code in the postmaster, and I think it's a worse idea to run arbitrary user-defined code at backend shutdown (the END-blocks bit). I do not care in the least what applications you think this might enable --- the negative consequences for overall system stability seem to me to outweigh any possible arguments on that side. What happens when the supplied code has errors, takes an unreasonable amount of time to run, does something unsafe, depends on the backend not being in an error state already, etc etc? Same thing that happens when you put something stupid into shared_preload_libraries - you destabilize your database cluster and the world blows up. I do not have a veto over stuff like this, but if I did, it would not go in. I'm not as strongly opposed to this as you are, but I definitely think there will be some people who shoot themselves in the foot with it. I don't think it's necessarily more dangerous than shared_preload_libraries from a theoretical standpoint, but the sheer fact that this is Perl rather than C means more people will try to do it, and some of them will manage to take out the whole database cluster, which will not be awesome. I think this is a real weakness of our one-process-per-connection model. If it were possible to recycle backends for new connections, there would be no need even to consider doing things like this. Yeah, I know you don't want to do that either, just mentioning it... We're not going to support multi-line values for GUCs AFAIK, so this is going to be pretty kludgy. I'm not sure what you mean by this. What he means by this is defining GUCs in a way that would make people want to use multi-line values for them. However, that doesn't have anything to do with my worries ... Well, you did mention it previously. http://archives.postgresql.org/pgsql-hackers/2009-12/msg00384.php Anyway, I think you've put this pretty well here: the current definition will make people WANT to use multi-line values for this, and we don't support that. I think Tim's example is fairly contrived - setting a global variable here does not seem likely to be useful to very many users, and the ones who do want it will likely want also want multi-line values. What IS likely to be useful is preloading a bunch of perl modules so that backend startup doesn't take an unreasonably long time. It's nicer to write: plperl.on_perl_init='strict,warnings,LDAP,HTML::Parser,Archive::Zip' rather than: plperl.on_perl_init='use strict;use warnings;use LDAP;use HTML::Parser;use Archive::Zip;' I would strongly suggest to Tim that he rip the portions of this patch that are related to this feature out and submit them separately so that we can commit the uncontroversial portions first. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Robert Haas robertmh...@gmail.com writes: On Sun, Jan 10, 2010 at 1:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: What happens when the supplied code has errors, takes an unreasonable amount of time to run, does something unsafe, depends on the backend not being in an error state already, etc etc? Same thing that happens when you put something stupid into shared_preload_libraries - you destabilize your database cluster and the world blows up. shared_preload_libraries is under the sole control of the DBA, though. What distresses me about this is the exposure to ordinary users. In particular, that casual little atexit addition appears to mean that *unprivileged* users can break normal functioning of the database, eg by delaying or even preventing shutdown. That's a security hole of the first magnitude. Trying to execute SPI code there could make things even more fun. I also still don't care for the whole concept of on_init code from a theoretical standpoint: as I said before, loading of the plperl shared library should not be a semantically significant event from the user's viewpoint. If these GUCs were PGC_POSTMASTER it wouldn't be so bad, but Tim still seems to want them to be settable inside an individual session before the library is loaded, which makes the loading extremely visible. As an example, if people were using such functionality then the DBA couldn't start preloading plperl for performance without breaking behavior that some of his users might be depending on. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Robert Haas wrote: Anyway, I think you've put this pretty well here: the current definition will make people WANT to use multi-line values for this, and we don't support that. I think Tim's example is fairly contrived - setting a global variable here does not seem likely to be useful to very many users, and the ones who do want it will likely want also want multi-line values. What IS likely to be useful is preloading a bunch of perl modules so that backend startup doesn't take an unreasonably long time. It's nicer to write: plperl.on_perl_init='strict,warnings,LDAP,HTML::Parser,Archive::Zip' rather than: plperl.on_perl_init='use strict;use warnings;use LDAP;use HTML::Parser;use Archive::Zip;' I don't know why you would do either of these things. I at least would load one module which would in turn load others. So I'd expect to see something like this: plperl.on_perl_init = 'use lib /my/app; use MyApp::Pg;' I think the suggestion that somehow people will want to put a huge list of directives straight into postgresql.conf and that this is a reason not to provide this facility is on the wrong track completely. I would strongly suggest to Tim that he rip the portions of this patch that are related to this feature out and submit them separately so that we can commit the uncontroversial portions first. See my previous email. I suggested that Tim send three patches: one for this controversial stuff, one for the new utility functions for plperl, and one for the remainder. He and I have discussed it and I believe he is agreeable to that. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
We need monitoring anywhere we have a max_* parameter. Otherwise we won't know how close we are to disaster until we hit the limit and things break down. Otherwise we will have to set parameters by trial and error, or set them so high they are meaningless. I agree. Thing is, though, we have a de-facto max already ... when pgxlog runs out of disk space. And no monitoring *in postgresql* for that, although obviously you can use OS monitoring for it. I'm saying, even for plain PITR, it would be an improvement in manageablity if the DBA could set a maximum number of checkpoint segments before replication is abandonded or the master shuts down. It's something we've been missing. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function
2010/1/10 Tom Lane t...@sss.pgh.pa.us: I've applied a patch along these lines. Cool. Thanks, that works great. Cheers, Dean -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
On Sun, Jan 10, 2010 at 2:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Jan 10, 2010 at 1:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: What happens when the supplied code has errors, takes an unreasonable amount of time to run, does something unsafe, depends on the backend not being in an error state already, etc etc? Same thing that happens when you put something stupid into shared_preload_libraries - you destabilize your database cluster and the world blows up. shared_preload_libraries is under the sole control of the DBA, though. What distresses me about this is the exposure to ordinary users. In particular, that casual little atexit addition appears to mean that *unprivileged* users can break normal functioning of the database, eg by delaying or even preventing shutdown. That's a security hole of the first magnitude. Trying to execute SPI code there could make things even more fun. That's really a separate issue from the on_perl_init stuff, but now that you mention it it does sound like a serious problem. Preventing SPI from being executed there is probably feasible, but I don't like the idea that broken code would cause the database to fail to shut down, and that's probably not fixable (unless maybe we detach shared memory before executing this code, or something?). I also still don't care for the whole concept of on_init code from a theoretical standpoint: as I said before, loading of the plperl shared library should not be a semantically significant event from the user's viewpoint. If these GUCs were PGC_POSTMASTER it wouldn't be so bad, but Tim still seems to want them to be settable inside an individual session before the library is loaded, which makes the loading extremely visible. As an example, if people were using such functionality then the DBA couldn't start preloading plperl for performance without breaking behavior that some of his users might be depending on. Hmm. OK, I agree: that's a problem. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Jan 10, 2010 at 1:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: What happens when the supplied code has errors, takes an unreasonable amount of time to run, does something unsafe, depends on the backend not being in an error state already, etc etc? Same thing that happens when you put something stupid into shared_preload_libraries - you destabilize your database cluster and the world blows up. shared_preload_libraries is under the sole control of the DBA, though. What distresses me about this is the exposure to ordinary users. In particular, that casual little atexit addition appears to mean that *unprivileged* users can break normal functioning of the database, eg by delaying or even preventing shutdown. That's a security hole of the first magnitude. Trying to execute SPI code there could make things even more fun. I suspect that could be inhibited at least. I also still don't care for the whole concept of on_init code from a theoretical standpoint: as I said before, loading of the plperl shared library should not be a semantically significant event from the user's viewpoint. If these GUCs were PGC_POSTMASTER it wouldn't be so bad, but Tim still seems to want them to be settable inside an individual session before the library is loaded, which makes the loading extremely visible. As an example, if people were using such functionality then the DBA couldn't start preloading plperl for performance without breaking behavior that some of his users might be depending on. Well, I think the proposed plperl.on_perl_init setting could be PGC_POSTMASTER, as I previously commented. The other settings are intended to be used on interpreter start, AIUI. Maybe we need to explore the use cases more. If we made plperl.on_perl_init PGC_POSTMASTER and the other two PGC_SUSET would that alloy your concerns? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] remove redundant ownership checks
On Sun, Dec 20, 2009 at 10:53 PM, I wrote: On Thu, Dec 17, 2009 at 7:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: KaiGai Kohei kai...@ak.jp.nec.com writes: [ patch to remove EnableDisableRule's permissions check ] I don't particularly like this patch, mainly because I disagree with randomly removing permissions checks without any sort of plan about where they ought to go. So where ought they to go? I have looked this over a little bit and I guess I don't see why the lack of a grand plan for how to organize all of our permissions checks ought to keep us from removing this one on the grounds of redundancy. We have to attack this problem in small pieces if we're going to make any progress, and the pieces aren't going to get any smaller than this. Per Tom's suggestion, I did a quick grep of the Slony source code for EnableDisableRule and got no hits. I think the appropriate level of concern about the possibility that third-party code might be calling this function is to (1) add a comment to the function noting that it is the caller's responsibility to check permissions, and (2) if we're *really* concerned that someone might miss that, release-note it. It is very unclear that it's worth even that much effort, though, since we have zero evidence that there is any third-party code using this function directly. A quick Google search on EnableDisableRule hits mostly this thread. With respect to cleaning up permissions more generally, it seems to me that Stephen Frost hit the nail on the upthread when he remarked that the current coding, where we're expecting certain functions in tablecmds.c to be called with PART of the permissions checking done, is fairly counterintuitive. I think it would be interesting to see if someone could propose a refactoring that eliminates this and puts all the permissions checking in a single, appropriate location. But ISTM that this patch would be a subset of any such more comprehensive patch, so that doesn't seem like a reason to hold off on applying this either. So I am inclined to go ahead and apply this with the addition of the comment discussed above. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
On Sun, Jan 10, 2010 at 2:58 PM, Andrew Dunstan and...@dunslane.net wrote: I don't know why you would do either of these things. I at least would load one module which would in turn load others. So I'd expect to see something like this: plperl.on_perl_init = 'use lib /my/app; use MyApp::Pg;' I think the suggestion that somehow people will want to put a huge list of directives straight into postgresql.conf and that this is a reason not to provide this facility is on the wrong track completely. Hmm. I have to admit I didn't think about use lib. That does seem like a plausible thing to want to do. I would strongly suggest to Tim that he rip the portions of this patch that are related to this feature out and submit them separately so that we can commit the uncontroversial portions first. See my previous email. I suggested that Tim send three patches: one for this controversial stuff, one for the new utility functions for plperl, and one for the remainder. He and I have discussed it and I believe he is agreeable to that. OK, well then just +1 for that. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] remove redundant ownership checks
Robert Haas robertmh...@gmail.com writes: I have looked this over a little bit and I guess I don't see why the lack of a grand plan for how to organize all of our permissions checks ought to keep us from removing this one on the grounds of redundancy. We have to attack this problem in small pieces if we're going to make any progress, and the pieces aren't going to get any smaller than this. I would turn that argument around: given the lack of a grand plan, why should we remove this particular check at all? Nobody has argued that there would be a significant, or even measurable, performance gain. When and if we do have a plan, we might find ourselves putting this check back. Even if you are right in your unsubstantiated hypothesis that this change will be a subset of any future change that is made with some plan in mind, I don't see that incremental revisions of the permissions check placement are a good way to approach the problem. What I fear will result from that is gaps in permissions checking, depending on what combination of revisions of core and third-party code happen to get used in a given installation. I think we need a plan first, not random patches first. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: As an example, if people were using such functionality then the DBA couldn't start preloading plperl for performance without breaking behavior that some of his users might be depending on. If we made plperl.on_perl_init PGC_POSTMASTER and the other two PGC_SUSET would that alloy your concerns? No, they have to all be PGC_POSTMASTER to answer that concern. Only breaking things for superusers isn't really that big an improvement over breaking them for everybody. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Streaming replication status
On Sun, 2010-01-10 at 12:10 -0800, Josh Berkus wrote: We need monitoring anywhere we have a max_* parameter. Otherwise we won't know how close we are to disaster until we hit the limit and things break down. Otherwise we will have to set parameters by trial and error, or set them so high they are meaningless. I agree. Thing is, though, we have a de-facto max already ... when pgxlog runs out of disk space. What I mean is this: The purpose of monitoring is to avoid bad things happening by being able to predict that a bad thing will happen before it actually does happen. Cars have windows to allow us to see we are about to hit something. And no monitoring *in postgresql* for that, although obviously you can use OS monitoring for it. PostgreSQL doesn't need to monitor that. If the user wants to avoid out-of-space they can write a script to monitor files/space. The info is accessible, if you wish to monitor it. Currently there is no way of knowing what the average/current transit time is on replication, no way of knowing what is happening if we go idle etc.. Those things need to be included because they are not otherwise accessible. Cars need windows, not just a finely tuned engine. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Typed tables
On tor, 2009-11-05 at 19:24 +0200, Peter Eisentraut wrote:
I'm planning to work on typed tables support. The idea is that you
create a table out of a composite type (as opposed to the other way
around, which is currently done automatically).
CREATE TYPE persons_type AS (name text, bdate date);
CREATE TABLE persons OF persons_type;
Or the fancy version:
CREATE TABLE persons OF persons_type ( PRIMARY KEY (name) );
And here is the first patch for that. The feature is complete as far as
I had wanted it. I would like to add ALTER TYPE support, but that can
come as a separate patch.
diff --git a/doc/src/sgml/information_schema.sgml b/doc/src/sgml/information_schema.sgml
index 97e95ed..37c6543 100644
--- a/doc/src/sgml/information_schema.sgml
+++ b/doc/src/sgml/information_schema.sgml
@@ -4750,19 +4750,29 @@ ORDER BY c.ordinal_position;
row
entryliteraluser_defined_type_catalog/literal/entry
entrytypesql_identifier/type/entry
- entryApplies to a feature not available in productnamePostgreSQL//entry
+ entry
+ If the table is a typed table, the name of the database that
+ contains the underlying data type (always the current
+ database), else null.
+ /entry
/row
row
entryliteraluser_defined_type_schema/literal/entry
entrytypesql_identifier/type/entry
- entryApplies to a feature not available in productnamePostgreSQL//entry
+ entry
+ If the table is a typed table, the name of the schema that
+ contains the underlying data type, else null.
+ /entry
/row
row
entryliteraluser_defined_type_name/literal/entry
entrytypesql_identifier/type/entry
- entryApplies to a feature not available in productnamePostgreSQL//entry
+ entry
+ If the table is a typed table, the name of the underlying data
+ type, else null.
+ /entry
/row
row
@@ -4778,7 +4788,7 @@ ORDER BY c.ordinal_position;
row
entryliteralis_typed/literal/entry
entrytypeyes_or_no/type/entry
- entryApplies to a feature not available in productnamePostgreSQL//entry
+ entryliteralYES/literal if the table is a typed table, literalNO/literal if not/entry
/row
row
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 43764f1..28eb52e 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -32,6 +32,16 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } ] TABLE replaceable class=PAR
[ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
[ TABLESPACE replaceable class=PARAMETERtablespace/replaceable ]
+CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } ] TABLE replaceable class=PARAMETERtable_name/replaceable
+OF replaceable class=PARAMETERtype_name/replaceable [ (
+ { replaceable class=PARAMETERcolumn_name/replaceable WITH OPTIONS [ DEFAULT replaceabledefault_expr/replaceable ] [ replaceable class=PARAMETERcolumn_constraint/replaceable [ ... ] ]
+| replaceabletable_constraint/replaceable }
+[, ... ]
+) ]
+[ WITH ( replaceable class=PARAMETERstorage_parameter/replaceable [= replaceable class=PARAMETERvalue/replaceable] [, ... ] ) | WITH OIDS | WITHOUT OIDS ]
+[ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
+[ TABLESPACE replaceable class=PARAMETERtablespace/replaceable ]
+
phrasewhere replaceable class=PARAMETERcolumn_constraint/replaceable is:/phrase
[ CONSTRAINT replaceable class=PARAMETERconstraint_name/replaceable ]
@@ -154,6 +164,27 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } ] TABLE replaceable class=PAR
/varlistentry
varlistentry
+termliteralOF replaceable class=PARAMETERtype_name/replaceable/literal/term
+listitem
+ para
+ Creates a firsttermtyped table/firstterm, which takes its
+ structure from the specified composite type (name optionally
+ schema-qualified). A typed table is tied to its type; for
+ example the table will be dropped if the type is dropped
+ (with literalDROP TYPE ... CASCADE/literal).
+ /para
+
+ para
+ When a typed table is created, then the data types of the
+ columns are determined by the underlying composite type and are
+ not specified by the literalCREATE TABLE/literal command.
+ But the literalCREATE TABLE/literal command can add defaults
+ and constraints to the table and can specify storage parameters.
+ /para
+/listitem
+ /varlistentry
+
+ varlistentry
termreplaceable class=PARAMETERcolumn_name/replaceable/term
listitem
para
@@ -1182,6 +1213,17 @@ CREATE TABLE cinemas (
/programlisting
/para
+ para
+ Create a composite type and a typed table:
+programlisting
+CREATE TYPE employee_type AS (name text, salary numeric);
+
+CREATE TABLE employees OF employee_type (
+PRIMARY KEY (name),
+salary WITH OPTIONS DEFAULT 1000
+);
+/programlisting
+ /para
/refsect1
refsect1
Re: [HACKERS] Streaming replication status
Currently there is no way of knowing what the average/current transit time is on replication, no way of knowing what is happening if we go idle etc.. Those things need to be included because they are not otherwise accessible. Cars need windows, not just a finely tuned engine. Like I said, I agree. I'm just pointing out that the monitoring deficiency already exists whether or not we add a max_* parameter. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: As an example, if people were using such functionality then the DBA couldn't start preloading plperl for performance without breaking behavior that some of his users might be depending on. If we made plperl.on_perl_init PGC_POSTMASTER and the other two PGC_SUSET would that alloy your concerns? No, they have to all be PGC_POSTMASTER to answer that concern. Only breaking things for superusers isn't really that big an improvement over breaking them for everybody. Well, I don't know about Tim but I think I could live with that. And when we get some actual experience with using them we'll have a better handle on whether or not it gives us any pain. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] damage control mode
Now the other approach we could take is that we'll ship *something* on 7 Mar, even if it's less stable than what we've traditionally considered to be beta quality. I don't think that really helps much though; it just means we need more time in beta. Well, we're shipping an alpha, aren't we? My proposal for beta in 2 weeks was based on the idea of having *no* new patches in CF4, at all. Given the reality of HS+SR, I don't think it's realistic anymore either. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Typed tables
On 1/10/10 2:34 PM, Peter Eisentraut wrote: On tor, 2009-11-05 at 19:24 +0200, Peter Eisentraut wrote: I'm planning to work on typed tables support. The idea is that you create a table out of a composite type (as opposed to the other way around, which is currently done automatically). Nice. Can we come up with a better name for the feature, though? Composite Type Tables? Type-Table Inheritance? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
On Jan 10, 2010, at 11:17 AM, Robert Haas wrote: It's nicer to write: plperl.on_perl_init='strict,warnings,LDAP,HTML::Parser,Archive::Zip' rather than: plperl.on_perl_init='use strict;use warnings;use LDAP;use HTML::Parser;use Archive::Zip;' Well, no, because sometimes I just want to load something and not have functions exported (into whatever namespaces ends up calling this). So I might have something like: plplerl.on_perl_init='use HTML::Entities ();' Other times I might want those functions exported. FWIW, Bricolage has a feature like this, and you can only put stuff on one line. It's been there since 2002 or so. No one has ever complained about it; I doubt anyone would complain about this, either. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] We need to rethink relation cache entry rebuild
I wrote:
Basically I think we have to fix this by ensuring that an error escape
can't occur while a relcache entry is in a partially rebuilt state.
Attached is a draft patch for this. In addition to fixing the stated
problem, it also takes care of a thinko that I found along the way:
RelationClearRelation assumes that any rd_indexprs or rd_indpred trees
can be freed by deleting the rd_indexcxt context, as the comments in
rel.h imply. But actually the code that loads those fields was putting
the trees directly into CacheMemoryContext, meaning that a cache flush
on an index that has expressions or a predicate would result in a
session-lifespan memory leak.
I think this is not too complicated to back-patch --- if anything the
logic is simpler than before. Comments?
regards, tom lane
Index: src/backend/utils/cache/relcache.c
===
RCS file: /cvsroot/pgsql/src/backend/utils/cache/relcache.c,v
retrieving revision 1.297
diff -c -r1.297 relcache.c
*** src/backend/utils/cache/relcache.c 7 Jan 2010 20:39:45 - 1.297
--- src/backend/utils/cache/relcache.c 10 Jan 2010 23:45:58 -
***
*** 198,203
--- 198,204
/* non-export function prototypes */
+ static void RelationDestroyRelation(Relation relation);
static void RelationClearRelation(Relation relation, bool rebuild);
static void RelationReloadIndexInfo(Relation relation);
***
*** 211,220
int natts, const FormData_pg_attribute *attrs);
static HeapTuple ScanPgRelation(Oid targetRelId, bool indexOK);
! static Relation AllocateRelationDesc(Relation relation, Form_pg_class relp);
static void RelationParseRelOptions(Relation relation, HeapTuple tuple);
static void RelationBuildTupleDesc(Relation relation);
! static Relation RelationBuildDesc(Oid targetRelId, Relation oldrelation);
static void RelationInitPhysicalAddr(Relation relation);
static void load_critical_index(Oid indexoid);
static TupleDesc GetPgClassDescriptor(void);
--- 212,221
int natts, const FormData_pg_attribute *attrs);
static HeapTuple ScanPgRelation(Oid targetRelId, bool indexOK);
! static Relation AllocateRelationDesc(Form_pg_class relp);
static void RelationParseRelOptions(Relation relation, HeapTuple tuple);
static void RelationBuildTupleDesc(Relation relation);
! static Relation RelationBuildDesc(Oid targetRelId, bool insertIt);
static void RelationInitPhysicalAddr(Relation relation);
static void load_critical_index(Oid indexoid);
static TupleDesc GetPgClassDescriptor(void);
***
*** 305,319
* AllocateRelationDesc
*
* This is used to allocate memory for a new relation descriptor
! * and initialize the rd_rel field.
! *
! * If 'relation' is NULL, allocate a new RelationData object.
! * If not, reuse the given object (that path is taken only when
! * we have to rebuild a relcache entry during RelationClearRelation).
*/
static Relation
! AllocateRelationDesc(Relation relation, Form_pg_class relp)
{
MemoryContext oldcxt;
Form_pg_class relationForm;
--- 306,317
* AllocateRelationDesc
*
* This is used to allocate memory for a new relation descriptor
! * and initialize the rd_rel field from the given pg_class tuple.
*/
static Relation
! AllocateRelationDesc(Form_pg_class relp)
{
+ Relation relation;
MemoryContext oldcxt;
Form_pg_class relationForm;
***
*** 321,335
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
/*
! * allocate space for new relation descriptor, if needed
*/
! if (relation == NULL)
! relation = (Relation) palloc(sizeof(RelationData));
/*
! * clear all fields of reldesc
*/
- MemSet(relation, 0, sizeof(RelationData));
relation-rd_targblock = InvalidBlockNumber;
relation-rd_fsm_nblocks = InvalidBlockNumber;
relation-rd_vm_nblocks = InvalidBlockNumber;
--- 319,331
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
/*
! * allocate and zero space for new relation descriptor
*/
! relation = (Relation) palloc0(sizeof(RelationData));
/*
! * clear fields of reldesc that should initialize to something non-zero
*/
relation-rd_targblock = InvalidBlockNumber;
relation-rd_fsm_nblocks = InvalidBlockNumber;
relation-rd_vm_nblocks = InvalidBlockNumber;
***
*** 387,393
{
case RELKIND_RELATION:
case RELKIND_TOASTVALUE:
- case RELKIND_UNCATALOGED:
case RELKIND_INDEX:
break;
default:
--- 383,388
***
*** 415,420
--- 410,416
relation-rd_options = MemoryContextAlloc(CacheMemoryContext,
VARSIZE(options));
memcpy(relation-rd_options, options, VARSIZE(options));
+ pfree(options);
}
}
***
*** 805,828
/*
* RelationBuildDesc
*
! * Build a relation descriptor --- either a new one, or by
! * recycling the
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: No, they have to all be PGC_POSTMASTER to answer that concern. Only breaking things for superusers isn't really that big an improvement over breaking them for everybody. Well, I don't know about Tim but I think I could live with that. And when we get some actual experience with using them we'll have a better handle on whether or not it gives us any pain. Upon further review, PGC_POSTMASTER isn't going to work because of this in guc.c: /* * Only allow custom PGC_POSTMASTER variables to be created during shared * library preload; any later than that, we can't ensure that the value * doesn't change after startup. This is a fatal elog if it happens; just * erroring out isn't safe because we don't know what the calling loadable * module might already have hooked into. */ if (context == PGC_POSTMASTER !process_shared_preload_libraries_in_progress) elog(FATAL, cannot create PGC_POSTMASTER variables after startup); We certainly don't want to make it such that plperl *has* to be preloaded, so PGC_POSTMASTER is out. However, I think PGC_SIGHUP would be enough to address my basic worry, which is that people shouldn't be depending on the ability to set these things within an individual session. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] mailing list archiver chewing patches
Dimitri Fontaine wrote: Andrew Dunstan and...@dunslane.net writes: That is assuming that the MUA gives you the option of specifying the attachment MIME type. Many (including mine) do not. It would mean an extra step - I'd have to gzip each patch or something like that. That would be unfortunate,as well as imposing extra effort, because it would make the patch not display inline in many MUAs (again, like mine). Bad MUA, change MUA, or what they say… More seriously though, it's not the first time we're having some difficulties with the MHonArc setup, and I think it's also related to the poor thread following on the archives website at month boundaries. Absolutely. The month boundary problem boils down to the fact that Mhonarc does not scale very well, so we can't have mboxes that are too large. This is why most people split their archives per month, and then each month is published as an independent Mhonarc output archive. It's a horrid solution. Are our indexing and searches provided by MHonArc or maintained by the community? Searches are completely external to mhonarc. How helpful considering alternatives, such as AOX (which runs atop PostgreSQL and would offer anonymous IMAP facility over the archives) would be? Of course it'll boil down to who's maintaining the current solution and how much time is allocated to this, the solution research and migration would have to fit in there I suppose. Same as pgfoundry. But still, should we talk about it? There's some talk about writing our own archiving system, database-backed. There have been a few false starts but no concrete result so far. We need a lot more manpower invested in this problem. If there's interest, let's talk about it. My daugher was born yesterday and I'm having a bit of a calm before the storm because she's not coming home until Tuesday or so (at this time of the day, that is, because I have to take care of the other daughter). I'll be probably away for (at least) a week when she does; and I'll probably have somewhat of a shortage of spare time after that. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Feature patch 1 for plperl [PATCH]
Tim Bunce tim.bu...@pobox.com writes: I didn't get any significant feedback from the earlier draft so here's the finished 'feature patch 1' for plperl. (This builds on my earlier plperl refactoring patch, and the follow-on ppport.h patch.) Just looking over this patch, I don't think it's nearly robust enough against initialization failures. The original code wasn't very good about that either, but that was (more or less) okay because it was executing predetermined, pretested code that we really don't expect to fail. I think the standard has to be a *lot* higher if we are going to execute user-supplied perl code there. You need to make sure that things are left in a reasonably consistent, safe state if an error occurs. Along the same line, there needs to be more effort put into the errors that can be thrown when one of these failures happen. The current messages don't follow our style guidelines very well, and aren't exposed for translation. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] remove redundant ownership checks
On Sun, Jan 10, 2010 at 4:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I have looked this over a little bit and I guess I don't see why the lack of a grand plan for how to organize all of our permissions checks ought to keep us from removing this one on the grounds of redundancy. We have to attack this problem in small pieces if we're going to make any progress, and the pieces aren't going to get any smaller than this. I would turn that argument around: given the lack of a grand plan, why should we remove this particular check at all? Nobody has argued that there would be a significant, or even measurable, performance gain. When and if we do have a plan, we might find ourselves putting this check back. You're arguing against a straw man - there's clearly no argument here from performance. We generally do not choose to litter the code with redundant or irrelevant checks because it makes the code difficult to maintain and understand. Sometimes it also hurts performance, but that's not a necessary criterion for removal. Nor are we generally in the habit of keeping redundant code around because a hypothetical future refactoring might by chance end up putting exactly the same code back. Even if you are right in your unsubstantiated hypothesis that this change will be a subset of any future change that is made with some plan in mind, I don't see that incremental revisions of the permissions check placement are a good way to approach the problem. What I fear will result from that is gaps in permissions checking, depending on what combination of revisions of core and third-party code happen to get used in a given installation. I think we need a plan first, not random patches first. I think the issue at hand is completely severable from the issue of a more general plan. Again, when we find that we find that the code does the same work twice, that's typically something we would want to fix, independent of what else might or might not come later. That having been said, I'm 100% in favor of talking about what a more general plan should look like; in fact, I asked for your opinion here: http://archives.postgresql.org/pgsql-hackers/2009-12/msg01824.php ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Red-black tree for GIN
On Thu, Dec 31, 2009 at 4:19 PM, Robert Haas robertmh...@gmail.com wrote: My other question is as related to performance. Can you provide a test case that shows the performance improvement with this patch? So, we still don't have a test case for this patch. During the November CommitFest, Greg Smith griped a bit about the lack of a reproducible performance benchmark for the XLogInsert patch: http://archives.postgresql.org/pgsql-hackers/2009-12/msg00816.php ...and I would say the same logic applies to this patch, maybe even moreso. Tom has already applied a partial workaround for this problem, and I'm feeling like it won't be trivial to figure out what to measure to see the remaining issue and measure how much this new implementation helps. The coding pattern that this patch uses also merits some discussion. Basically, rbtree.c is a generic implementation of red-black trees - from a textbook - which ginbulk.c then uses for GIN. One possible advantage of this implementation is that it might make it possible for us to use the rbtree.c logic in other places, if we have other data structures that need similar treatment. But I'm not sure if that's the way we want to go. The other alternative is to drop the generalized implementation and incorporate the logic directly into ginbulk.c. I really don't know which is better, but I'd like to hear some other opinions... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Red-black tree for GIN
On Mon, Jan 11, 2010 at 2:42 AM, Robert Haas robertmh...@gmail.com wrote: The coding pattern that this patch uses also merits some discussion. Basically, rbtree.c is a generic implementation of red-black trees - from a textbook - which ginbulk.c then uses for GIN. One possible advantage of this implementation is that it might make it possible for us to use the rbtree.c logic in other places, if we have other data structures that need similar treatment. But I'm not sure if that's the way we want to go. The other alternative is to drop the generalized implementation and incorporate the logic directly into ginbulk.c. I really don't know which is better, but I'd like to hear some other opinions... So, code reuse is not the only advantage of abstraction. It's also just plain easier to understand and test code written with clear abstract interfaces. The way you describe it someone with no knowledge could look at rbtree.c and see if it's done correctly and maybe improve it. And someone reading ginbulk only has to understand the operations red-black trees support and no understand how they're implemented to follow the code. Is there any advantage of integrating the code with ginbulk.c? Does it let us get away with finer grained locks or do any tricks doing gin-specific changes while we're in the middle of rebalancing or anything like that? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RADIUS authentication
Magnus, * Magnus Hagander (mag...@hagander.net) wrote: The attached patch implements RADIUS authentication (RFC2865-compatible). Great! We have a few environments which use RADIUS auth, nice that PG might be able to use that auth method in the future. I'm not a fan of having the shared secret stored in a 'regular' config file. Could you support, or maybe just change it to, breaking that out into another file? Perhaps something simimlar to how pam_radius_auth works, where you can also list multiple servers? http://freeradius.org/pam_radius_auth/ Would also allow using the same file for multiple RADIUS-based servers.. I know pg_hba.conf can just be set to have minimal permissions (and is on Debian), but that's the kind of file that tends to end up in things like subversion repositories or puppet configs where they aren't treated as carefully since, generally, what's in them doesn't come across as super-sensetive. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] damage control mode
On Sun, Jan 10, 2010 at 6:24 PM, Josh Berkus j...@agliodbs.com wrote: Now the other approach we could take is that we'll ship *something* on 7 Mar, even if it's less stable than what we've traditionally considered to be beta quality. I don't think that really helps much though; it just means we need more time in beta. Well, we're shipping an alpha, aren't we? My proposal for beta in 2 weeks was based on the idea of having *no* new patches in CF4, at all. Given the reality of HS+SR, I don't think it's realistic anymore either. Well, the small patches are not going to hold us up much. I don't think having to deal with those is going to impact the schedule, even if they are new. It's the big ones that are going to drain time from release prep. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
On Tue, Jan 5, 2010 at 8:23 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2010/1/5 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 4, 2010 at 2:51 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't have a problem to write second and safe fmtId function (with technique used in dumputils don't need to modify libpq), although fmtId do exactly what I need. I would to understand to behave. I think you mean that you would need to understand how it should behave - in which case I agree, but I think Tom spelled that out pretty clearly upthread: close PQescapeStringConn and adapt it to be PQescapeIdentifier. The more important point here is that fmtId doesn't do exactly what you need in any case. fmtId is safe to use in pg_dump because pg_dump is only expected to work with the same or older version of the backend. It would not be safe to use it in libpq, which is expected to still work with newer backends that might have more reserved words. So I finnaly moved to libpq PQescapeIdentConn function patch is attached. No longer applies, please rebase. Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify - what to do when the queue is full
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 - do we need to limit the payload to pure ASCII ? I think yes, we need to. I also think we need to reject other payloads with elog(ERROR...). ...[snip other followups] On the one hand, I don't see the problem with ASCII here - the payload is meant as a quick shorthand convenience, not a literal payload of important information. On the other, it should at least match the current rules for the listen and notify names themselves, which means allowing more than ASCII. - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 201001102303 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAktKo40ACgkQvJuQZxSWSshg9ACg2uiDYuhBnRQqFS6Ej3O9VLcC 2TgAn035OrYcdERn4I1VI4NRQFBIcXZ/ =yJmK -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Testing with concurrent sessions
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Using DBI/DBD::Pg would raise another issue - what version of libpq would it be using? Not the one in the build being tested, that's for sure. Er...why not? That's what psql uses. Because you'd have to build DBD::Pg against the new libpq, as you do psql. That means you need DBD::Pg sources and the build environment for Perl (headers etc) not just a working Perl runtime. Big difference. Yes, but that is what I was envisioning. As you point out, that's the only sane way to make sure we have a good version of DBD::Pg with which to test. As a side effect, it put libpq through some extra paces as well. :) Is bundling a Perl module in the source tree and building it as part of the Pg build a reasonable choice? Personally, I don't think so. *shrug* It's different, but it's the best solution to the problem at hand. It wouldn't be built as part of Pg, only as part of the tests. Additionally, a dedicated testing tool like some folks have been talking about would be really handy for users who want to test their schema. I've had to write my own (in Java, or I'd be offering it) for this purpose, as psql is completely unsuitable for concurrent-run testing and I needed to show that my locking was safe and deadlock-free in some of the more complex stored procs I have. Sure, but it's the difference between waiting for someone to write something (and then dealing with the invevitable bugs, tweaks, and enhancements), or using a solid, known quantity (DBI + Test::More). - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 201001102316 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAktKplIACgkQvJuQZxSWSsj7TQCeMOXWS+uLIZE9QbeBWPxYv/rg HhEAn0QZUzE2/8uyg5Oi+K8qL/oTeDSO =R8Az -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] quoting psql varible as identifier
2010/1/11 Robert Haas robertmh...@gmail.com: On Tue, Jan 5, 2010 at 8:23 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2010/1/5 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 4, 2010 at 2:51 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I don't have a problem to write second and safe fmtId function (with technique used in dumputils don't need to modify libpq), although fmtId do exactly what I need. I would to understand to behave. I think you mean that you would need to understand how it should behave - in which case I agree, but I think Tom spelled that out pretty clearly upthread: close PQescapeStringConn and adapt it to be PQescapeIdentifier. The more important point here is that fmtId doesn't do exactly what you need in any case. fmtId is safe to use in pg_dump because pg_dump is only expected to work with the same or older version of the backend. It would not be safe to use it in libpq, which is expected to still work with newer backends that might have more reserved words. So I finnaly moved to libpq PQescapeIdentConn function patch is attached. No longer applies, please rebase. ok I'll do it today Pavel Thanks, ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Red-black tree for GIN
Robert, we have benchmark for rbtree http://www.sai.msu.su/~megera/wiki/2009-07-27 rbtree, actually, fix corner cases with ordered input, with little overhead. As you may see from knngist patch, rbtree used in gist code, so, please, leave rbtree code as is. Oleg On Sun, 10 Jan 2010, Robert Haas wrote: On Thu, Dec 31, 2009 at 4:19 PM, Robert Haas robertmh...@gmail.com wrote: My other question is as related to performance. =A0Can you provide a test case that shows the performance improvement with this patch? So, we still don't have a test case for this patch. During the November CommitFest, Greg Smith griped a bit about the lack of a reproducible performance benchmark for the XLogInsert patch: http://archives.postgresql.org/pgsql-hackers/2009-12/msg00816.php ...and I would say the same logic applies to this patch, maybe even moreso. Tom has already applied a partial workaround for this problem, and I'm feeling like it won't be trivial to figure out what to measure to see the remaining issue and measure how much this new implementation helps. The coding pattern that this patch uses also merits some discussion. Basically, rbtree.c is a generic implementation of red-black trees - from a textbook - which ginbulk.c then uses for GIN. One possible advantage of this implementation is that it might make it possible for us to use the rbtree.c logic in other places, if we have other data structures that need similar treatment. But I'm not sure if that's the way we want to go. The other alternative is to drop the generalized implementation and incorporate the logic directly into ginbulk.c. I really don't know which is better, but I'd like to hear some other opinions... ...Robert --=20 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Listen / Notify - what to do when the queue is full
On mån, 2010-01-11 at 04:05 +, Greg Sabino Mullane wrote: On the one hand, I don't see the problem with ASCII here - the payload is meant as a quick shorthand convenience, not a literal payload of important information. Is it not? The notify name itself is already a quick shorthand convenience. Who knows what the payload is actually meant for. Have use cases been presented and analyzed? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Typed tables
On sön, 2010-01-10 at 15:27 -0800, Josh Berkus wrote: On 1/10/10 2:34 PM, Peter Eisentraut wrote: On tor, 2009-11-05 at 19:24 +0200, Peter Eisentraut wrote: I'm planning to work on typed tables support. The idea is that you create a table out of a composite type (as opposed to the other way around, which is currently done automatically). Nice. Can we come up with a better name for the feature, though? Composite Type Tables? Type-Table Inheritance? Typed tables is the official SQL standard name for the feature, and it's also used in DB2 documentation. So I kind of would prefer to keep it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xpath improvement suggestion
On ons, 2010-01-06 at 23:46 +0100, Arie Bikker wrote:
Hope this is the right attachement type (I'm new at this)
BTW. here a some nice examples:
- Get the number of attributes of the first childnode:
select ( xpath('count(@*)',(xpath('*[1]','a b=cd e=f
g=j//a'))[1]))[1];
- an alternative for xpath_exist('/a/d')
select (xpath('boolean(/a/d)','a b=cd e=f g=j//a'))[1];
- fixes bug 4206
select xpath('//text()',xmlparse(document '?xml
version=1.0?elem1elem2one/elem2elem2two/elem2elem2three/elem2elem3att=2//elem1'));
- fixes bug 4294
select xpath('name(/my:a/*[last()])', 'a
xmlns=http://myns.com/ns;btext1/bctext2/c/a',
ARRAY[ARRAY['my','http://myns.com/ns']]);
Instead of converting everything to text, there have been previous
suggestions to add functionx like xpath_string, xpath_number,
xpath_boolean that return the appropriate types from xpath. This could
provide for better type safety and probably also more clarity.
In any case, please consider adding test cases like the above to the
regression tests in whatever patch comes out at the end.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
