Re: [HACKERS] Reading recovery.conf earlier
On Mon, 2009-12-07 at 13:03 +0900, Fujii Masao wrote: On Sun, Dec 6, 2009 at 2:49 AM, Simon Riggs si...@2ndquadrant.com wrote: Proposal is to split out the couple of lines in readRecoveryCommandFile() that set important state and make it read in an option block that can be used by caller. It would then be called by both postmaster (earlier in startup) and again later by startup process, as happens now. I want to do it that way so I can read file before we create shared memory, so I don't have to worry about passing details via shared memory itself. I agree with the proposal that postmaster reads the recovery.conf. Because this would enable all child processes to easily obtain the parameter values in that, like GUC parameters. OK But I'm not sure why recovery.conf should be read separately by postmaster and the startup process. How about making postmaster read all of that for the simplification? That sounds better but is more complex. We'd have to read in the file, store in memory, then after shared memory is up put the data somewhere so the startup process can read it. (Remember Windows). What postgresql.conf already does is read file separately in each process, so no data passing. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, recent changes
Simon Riggs wrote: On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: For what it's worth, this doesn't seem particularly unlikely or unusual to me. I don't know many people who shutdown both nodes of a highly available application at the same time. If they did, I wouldn't expect them to complain they couldn't run queries on the standby when an two obvious and simple workarounds exist to allow them to access their data: start the master again, or make the standby switchover, both of which are part of standard operating procedures. It might not be common or expected in a typical HA installation, but it would be a very strange limitation in my mind. It might well happen e.g in a standby used for reporting, or when you do PITR. It doesn't seem very high up the list of additional features, at least. Well, it's in the patch now. I'm just asking you to not break it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
Simon Riggs wrote: What postgresql.conf already does is read file separately in each process, so no data passing. No it doesn't. Postmaster reads the file once, and backends inherit the values at fork(). In EXEC_BACKEND case, postmaster writes all the non-default values to a separate file, which the child process reads at startup. Reading the file separately in each process would cause trouble with PGC_POSTMASTER params. All backends must agree on their values. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
2009/12/7 Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp: Tom Lane t...@sss.pgh.pa.us wrote: It was written and submitted by one person who did not bother to ask first whether anyone else thought it was worthwhile. So its presence on the CF list should not be taken as evidence that there's consensus for it. Should we have Needs Discussion phase before Needs Review ? Reviews, including me, think patches with needs-review status are worthwhile. In contrast, contributers often register their patches to CF without discussions just because of no response; they cannot find whether no response is silent approval or not. +1. Sometimes a reviewer waits for the consensus in the community when someone else waits for review (, because it is marked as Needs Review). Regards, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
On Mon, 2009-12-07 at 10:13 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: What postgresql.conf already does is read file separately in each process, so no data passing. No it doesn't. Postmaster reads the file once, and backends inherit the values at fork(). In EXEC_BACKEND case, postmaster writes all the non-default values to a separate file, which the child process reads at startup. As I mentioned, the difficulty is always the Windows case. I wasn't aware we wrote a separate file in that case, so that does potentially effect my thinking. Reading the file separately in each process would cause trouble with PGC_POSTMASTER params. All backends must agree on their values. Looking at the parameters in recovery.conf I don't believe they would cause problems if read twice. So I think reading file twice would still be simplest way forwards. If you really think that changing the file is a possibility between processes reading them, then I would just take a full temp copy of the file, read it in postmaster, read it in startup, then delete temp file. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby, recent changes
On Mon, 2009-12-07 at 10:02 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2009-12-06 at 17:26 -0500, Robert Haas wrote: For what it's worth, this doesn't seem particularly unlikely or unusual to me. I don't know many people who shutdown both nodes of a highly available application at the same time. If they did, I wouldn't expect them to complain they couldn't run queries on the standby when an two obvious and simple workarounds exist to allow them to access their data: start the master again, or make the standby switchover, both of which are part of standard operating procedures. It might not be common or expected in a typical HA installation, but it would be a very strange limitation in my mind. It might well happen e.g in a standby used for reporting, or when you do PITR. Yes, its possible that the standby can be shutdown while disconnected from a master. If that occurs, all we are saying is that we cannot use shutdown checkpoints as starting points. If the primary was set up in default mode, then wal_standby_info = on and so there will be running_xact WAL records immediately after each checkpoint to use as starting points. We don't need shutdown checkpoints as well. So adding shutdown checkpoints as a valid starting place does not help in this case, it just makes the code harder to test. It doesn't seem very high up the list of additional features, at least. Well, it's in the patch now. I'm just asking you to not break it. There's been many things in the patch that have been removed. This is by far the least important of the things removed in the name of getting this done as soon as possible, with good quality. I see no reason for your eagerness to include this feature. I have removed it; as you say, its in the repo if someone wants to add it in later. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need a mentor, and a project.
abindra wrote: Next quarter I am planning to do an Independent Study course where the main objective would be to allow me to get familiar with the internals of Postgres by working on a project(s). I would like to work on something that could possibly be accepted as a patch. This is (I think) somewhat similar to what students do during google summer and I was hoping to get some help here in terms of: 1. A good project to work on for a newbie. 2. Would someone be willing to be a mentor? It would be nice to be able to get some guidance on a one-to-one basis. I would start with the TODO list: http://wiki.postgresql.org/wiki/Todo These are things for which there is a consensus that it would be a good idea to implement them. Pick things that look interesting to you, and try to read the discussions in the archives that lead to the TODO items. Bring the topic up in the hackers list, say that you would like to work on this or that TODO item, present your ideas of how you want to do it. Ask about things where you feel insecure. If you get some support, proceed to write a patch. Ask for directions, post half-baked patches and ask for comments. That is because you will probably receive a good amount of critizism and maybe rejection, and if you invest a couple of months into working on something that nobody knows about *and* your work gets rejected, that is much worse than drawing fire right away. It's probably not easy to find a mentor (unless you have money to give away), but you may find people who are interested in what you are doing and who will help you. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
Simon Riggs wrote: On Mon, 2009-12-07 at 10:13 +0200, Heikki Linnakangas wrote: Reading the file separately in each process would cause trouble with PGC_POSTMASTER params. All backends must agree on their values. Looking at the parameters in recovery.conf I don't believe they would cause problems if read twice. So I think reading file twice would still be simplest way forwards. Yeah, I don't know if that's an issue for recovery.conf, I was just pointing out that that's one reason it's done that way for postgresql.conf. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Hi, Greg Smith g...@2ndquadrant.com writes: Robert Haas wrote: The main point here for me is that the JSON format is already parseable by YAML parsers, and can probably be turned into YAML using a very short Perl script - possibly even using a sed script. I think that it's overkill to support two formats that are that similar. It's not the case that JSON can be turned into YAML or that it just happens that it can be parsed by YAML parsers. While there was some possible divergence in earlier versions, a JSON 1.2 document *is* in YAML format already. JSON is actually a subset of YAML that uses one of the many possible YAML styles--basically, YAML accepts anything in JSON format, along with others. This means that by providing JSON output, we've *already* provided YAML output, too. Just not the nice looking output people tend to associate with YAML. Well we have JSON and agreed it was a good idea to have it. Now JSON is a subset of YAML and some would prefer another YAML style (me included). If the problem is supporting 2 formats in core rather than 3, what about replacing the current JSON support with the YAML one? At a later point we could even have JSON support back by having the YAML printer able to output different YAML styles, but I guess that's not where we are now. Vote: +1 for YAML even if that means dropping JSON. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Clearing global statistics
2009/12/7 Greg Smith g...@2ndquadrant.com: Itagaki Takahiro wrote: Greg Smith g...@2ndquadrant.com wrote: I'm thinking that I should rename this new function to pg_stat_reset_bgwriter so it's obvious how limited its target is. I don't think it is a good name because we might have another cluster-level statictics not related with bgwriter in the future. I hope you will suggest extensible method when you improve this area. I follow what you mean now. I'll take a look at allowing pg_stat_reset to act on an input as you suggested, rather than adding more of these UDFs for every time somebody adds a new area they want to target clearing. I have on my TODO to implement the ability to do stats reset on a single object (say, one table only). Please take this into consideration when you design/name this, so theres no unnecessary overlap :-) Same goes for the stats message itself. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
On Mon, Dec 7, 2009 at 5:28 PM, Simon Riggs si...@2ndquadrant.com wrote: If you really think that changing the file is a possibility between processes reading them, then I would just take a full temp copy of the file, read it in postmaster, read it in startup, then delete temp file. This seems more robust because processes which are started long after postmaster has started might use recovery.conf in the future (e.g., walreceiver in SR, read-only backends). Also, in Windows, writing only non-default values into a temp file like GUC is good for a process (like backend) which might be started many times. Reading the full of the file every startup of such process would somewhat harm the performance. Though, of course, this is overkill for your purpose. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
On mån, 2009-12-07 at 17:14 +0900, Hitoshi Harada wrote: 2009/12/7 Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp: Tom Lane t...@sss.pgh.pa.us wrote: It was written and submitted by one person who did not bother to ask first whether anyone else thought it was worthwhile. So its presence on the CF list should not be taken as evidence that there's consensus for it. Should we have Needs Discussion phase before Needs Review ? Reviews, including me, think patches with needs-review status are worthwhile. In contrast, contributers often register their patches to CF without discussions just because of no response; they cannot find whether no response is silent approval or not. +1. Sometimes a reviewer waits for the consensus in the community when someone else waits for review (, because it is marked as Needs Review). Yes, I would have had use for this myself a couple of times. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] New PostgreSQL Committers
On behalf of the core team, I'm pleased to announce that the PostgreSQL Project has expanded it's team of committers, those people who are able to make direct changes to the PostgreSQL source code respository. As the project is extremely conservative about any changes made to the source code to minimise the risk of introducing any bugs, commit access is only given to contributors who have consistently shown they work to a very high standard and have shown commitment to the project. The new committers are: Robert Haas: Robert developed the commitfest.postgresql.org website which is used to manage the process by which features are added to PostgreSQL. He has twice acted as commitfest manager, and submitted numerous patches such as join removal, auto-generation of headers bki files and the TRUNCATE privilege. Simon Riggs: Simon is well know for working on large enterprise features for PostgreSQL, including Point In Time Recovery and partitioning. Simon is currently working on allowing PITR slave servers to be used for read-only queries. Greg Stark: Greg has worked on low-level features in PostgreSQL, including asynchronous pre-fetching of data and packed variable length data types. Greg was also responsible for the CREATE INDEX CONCURRENTLY feature. ITAGAKI Takahiro: ITAGAKI-san has worked on countless patches for PostgreSQL, both fixing bugs and writing new features, recently including WHEN clauses for triggers, a buffer usage feature for EXPLAIN and a new implementation of VACUUM FULL. Congratulations! -- Dave Page PostgreSQL Core Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
In response to Dave Page : On behalf of the core team, I'm pleased to announce that the PostgreSQL Project has expanded it's team of committers, those people who are able to make direct changes to the PostgreSQL source code respository. As the project is extremely conservative about any changes made to the source code to minimise the risk of introducing any bugs, commit access is only given to contributors who have consistently shown they work to a very high standard and have shown commitment to the project. The new committers are: Congratulations! +1 Andreas -- Andreas Kretschmer Kontakt: Heynitz: 035242/47150, D1: 0160/7141639 (mehr: - Header) GnuPG: 0x31720C99, 1006 CCB4 A326 1D42 6431 2EB0 389D 1DC2 3172 0C99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
On Monday 07 December 2009 02:12:43 Greg Smith wrote: After getting off to a good start, it looks like this patch is now stuck waiting for a second review pass from Kevin right now, with no open items for Andres to correct. Since the only issues on the table seem to be that of code aesthetics and long-term planning for this style of implementation rather than specific functional bits, I'm leaning toward saying this one is ready to have a committer look at it. Any comments from Kevin or Andres about where this is at? I think it should be ready - the only know thing it needs is a s/usefull/useful/. I will take another look but I doubt I will see anything new. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default
Tom Lane t...@sss.pgh.pa.us writes: Right, just like every other thing that's pre-installed. If a particular installation wishes to let individual DB owners control this, the superuser can drop plpgsql from template1. It's not apparent to me why we need to allow non-superusers to override the project's decisions about what should be installed by default. I guess it boils down to being nice to hosting platforms, where they will want to give as much power as can be given to database owners without having those hosted people be superusers. So should the decision to remove plpgsql be on the hosting platform hands or the hosted database owner? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Tallying up the votes on this patch First, I would hope that you don't overlook the author of the patch (me) as an aye vote. :) Additionally, if you are going to tally, please consider the positive responses from people on -hackers and -general when the patch was first posted and there was a question about what the format would look like. YAML is designed to be machine-parseable (just like XML) *AND* human-readable (obstensibly like our current output). JSON tries to overcome the shortcomings of XML, but doesn't quite get it right. YAML does: it's small, intuitive, and only as verbose as needed. Human readable, yet structured enough to be read by machines as well. As far as the arguments for making an extensible system for supporting other formats, but I think this is putting the cart before the horse. While this feature is only in cvs at the moment, I've not heard a single person state another format they would like to see. I cannot think of one myself (machine parseable anyway: I'd love to see an abbreviated version that takes out the explain bits of explain analyze and changes rows= to R=, loops= to L=, etc.). This is a small patch, easy to maintain, and useful to more people than just myself, judging from the responses to the earlier thread. ObPartingShot: I hope that those in the machine readable output should never be seen by human eyes camp will support my upcoming patch to remove all extra whitespace, including newlines, from the XML format. :) - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200912070727 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAksc9QMACgkQvJuQZxSWSshJXwCfeHrspp39mqGJD5Z86121F1Ap VJEAoMIXrXe8/vS5rPnzyfbX04fJjp36 =H5BM -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
* Dimitri Fontaine: Well we have JSON and agreed it was a good idea to have it. Now JSON is a subset of YAML and some would prefer another YAML style (me included). YAML is rather difficult to parse, and most parsers assume a trusted document source because they support arbitrary class instantiations (and it's not possible to switch this off, or it's rather involved to do so). Plain JSON doesn't have this issue. -- Florian Weimerfwei...@bfk.de BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Windows x64 [repost]
2009/12/4 Tsutomu Yamada tsut...@sraoss.co.jp: Thanks to suggestion. I send pathces again by another mailer for the archive. Sorry to waste resources, below is same content that I send before. Just in case anybody was wondering, I've added myself as a reviewer of this one for next commitfest - I doubt that's very surprising :-) Others are of course more than welcome to chip in! -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] bug: fuzzystrmatch levenshtein is wrong
The current behavior of levenshtein(text,text,int,int,int) is wrong. Consider:
leki_dev=# select levenshtein('','a',2,4,5);
levenshtein
-
1
(1 row)
leki_dev=# select levenshtein('a','',2,4,5);
levenshtein
-
1
(1 row)
leki_dev=# select levenshtein('aa','a',2,4,5);
levenshtein
-
1
(1 row)
leki_dev=# select levenshtein('a','aa',2,4,5);
levenshtein
-
1
(1 row)
versus (after patch)
postgres=# select levenshtein('','a',2,4,5);
levenshtein
-
2
(1 row)
postgres=# select levenshtein('a','',2,4,5);
levenshtein
-
4
(1 row)
postgres=# select levenshtein('aa','a',2,4,5);
levenshtein
-
4
(1 row)
postgres=# select levenshtein('a','aa',2,4,5);
levenshtein
-
2
(1 row)
patch attached.
Greetings
Marcin Mańk
levenshtein.diff
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] How to cache a non-unique index?
Hi, I have added a catalog table with an index on a non-unique column. I want to cache this non-unique index like other unique indexes in syscache. Is there any way to cache a non-unique index? I just want the index to reside in the memory to speed up the index lookup. Thank you! Wang -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] EXPLAIN BUFFERS
On Mon, Dec 7, 2009 at 1:28 AM, Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp wrote: (ii) format: why does text output format have less counters than the other ones? That's because lines will be too long for text format. I think the three values in it are the most important and useful ones. I disagree. I objected to showing only part of the information when I looked at this patch last CommitFest, and I object again now. I do *NOT* want to have to use JSON or XML to get at the counters you've arbitrarily decided are not interesting to me. I think with a little creativity we can certainly get these into the text format, and I'm willing to help with that. In fact, if you want, I'll pick up this patch and make it my first commit, though since you're now a committer as well perhaps you'd prefer to do it yourself. (v) accumulative: i don't remember if we discussed it but is there a reason the number of buffers isn't accumulative? We already have cost and time that are both accumulative. I saw BufferUsageAccumDiff() function but didn't try to figure out why it isn't accumulating or passing the counters to parent nodes. It's reasonable. I'll change so if no objections. +1 to change it. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: fuzzystrmatch levenshtein is wrong
also there is integer overflow:
postgres=# select levenshtein('','',1,10,1);
levenshtein
-
-1179869184
(1 row)
should we reject arguments greater than,say, 1 ?
maximum input length is 255 currently, so the maximum numbers involved
would be about 1*255*2
This would leave some breathing room if we wanted to increase the
maximum input string length.
Greetings
Marcin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] some questions in postgresql developping
Hello, I’m a student from Nankai University in China. Now I and my team do a project which aims to integrate XML to Postgresql. What I do is to complete the function of XML Update. Now I’m researching in concurrency control. I have read the code about the concurrency control for a long time and I’m confident that I know it much. But I am puzzled that why we need to lock tuple. I think locking transaction is sufficient. I don’t think the tuple lock is good at improving executing rate or anything. I am wishing for your reply. Best Regards, --Huang Xiaocheng --Database Information System Lab, Nankai University, China
Re: [HACKERS] Writeable CTE patch
Tom Lane wrote: The only thing that I'd be comfortable with is copying the snap and modifying the copy. I don't see an easy way to do that with the current code; CopySnapshot() is static and PushUpdatedSnapshot() seems to be a bit of a pain since it messes up some of the existing code which uses the active snapshot stack. Any ideas? Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Robert Haas wrote: This is no harder than many of the other seemingly crazy things I have done, e.g. Win32 port, client library threading. ?If this is a feature we should have, I will get it done or get others to help me complete the task. Well, I have always thought that it would be sort of a feather in our cap to support this, which is why I've done a couple of reviews of it in the past. I tend to agree with Tom that only a small fraction of our users will probably want it, but then again someone's been paying KaiGai to put a pretty hefty amount of work into this over the last year-plus, so obviously someone not only wants the feature but wants it merged. Within our community, I think that there have been a lot of people who have liked the concept of this feature but very few who have liked the patch, so there's somewhat of a disconnect between our aspirations and our better technical judgment. Tom is a notable exception who I believe likes neither the concept nor the patch, which is something we may need to resolve before getting too serious about this. Agreed. SE-Linux support might expand our user base and give us additional credibility, or it might be a feature that few people use --- and I don't think anyone knows the outcome. I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? SE-Linux support would certainly put Postgres in a unique security category, and it builds on our existing good security reputation. Personally, I think AppArmor is a saner security system: http://www.novell.com/linux/security/apparmor/selinux_comparison.html (Novell-hosted URL) but I am not advocating AppArmor support. I think the whole issue is whether support for external integrated security systems is appropriate for Postgres. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
On Mon, 2009-12-07 at 19:26 +0900, Fujii Masao wrote: On Mon, Dec 7, 2009 at 5:28 PM, Simon Riggs si...@2ndquadrant.com wrote: If you really think that changing the file is a possibility between processes reading them, then I would just take a full temp copy of the file, read it in postmaster, read it in startup, then delete temp file. This seems more robust because processes which are started long after postmaster has started might use recovery.conf in the future (e.g., walreceiver in SR, read-only backends). How does this sound? At startup we will delete recovery.conf.running, if it exists. At startup recovery.conf will be copied to recovery.conf.running, which will be the file read by any additional processes that read recovery.conf during this execution. The permissions on the file will then be set to read-only. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
On Mon, Dec 7, 2009 at 6:49 AM, Dave Page dp...@pgadmin.org wrote: On behalf of the core team, I'm pleased to announce that the PostgreSQL Project has expanded it's team of committers, those people who are able to make direct changes to the PostgreSQL source code respository. As the project is extremely conservative about any changes made to the source code to minimise the risk of introducing any bugs, commit access is only given to contributors who have consistently shown they work to a very high standard and have shown commitment to the project. +1 on the congratulations, and thank you for all your work. Roberto -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default
Dimitri Fontaine dfonta...@hi-media.com writes: So should the decision to remove plpgsql be on the hosting platform hands or the hosted database owner? Why not? If they really want to prohibit use of a feature the upstream project has decided should be standard, that's their privilege. The argument against seems to be basically this should work exactly like it did before, but if that's the standard then we can never have plpgsql installed by default at all. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
On Mon, Dec 07, 2009 at 10:49:13AM +, Dave Page wrote: On behalf of the core team, I'm pleased to announce that the PostgreSQL Project has expanded it's team of committers, those people who are able to make direct changes to the PostgreSQL source code respository. As the project is extremely conservative about any changes made to the source code to minimise the risk of introducing any bugs, commit access is only given to contributors who have consistently shown they work to a very high standard and have shown commitment to the project. Congratulations to all! :) Cheers, David. The new committers are: Robert Haas: Robert developed the commitfest.postgresql.org website which is used to manage the process by which features are added to PostgreSQL. He has twice acted as commitfest manager, and submitted numerous patches such as join removal, auto-generation of headers bki files and the TRUNCATE privilege. Simon Riggs: Simon is well know for working on large enterprise features for PostgreSQL, including Point In Time Recovery and partitioning. Simon is currently working on allowing PITR slave servers to be used for read-only queries. Greg Stark: Greg has worked on low-level features in PostgreSQL, including asynchronous pre-fetching of data and packed variable length data types. Greg was also responsible for the CREATE INDEX CONCURRENTLY feature. ITAGAKI Takahiro: ITAGAKI-san has worked on countless patches for PostgreSQL, both fixing bugs and writing new features, recently including WHEN clauses for triggers, a buffer usage feature for EXPLAIN and a new implementation of VACUUM FULL. Congratulations! -- Dave Page PostgreSQL Core Team -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
2009/11/26 Tom Lane t...@sss.pgh.pa.us: Magnus Hagander mag...@hagander.net writes: I assume you are fine with the addition of some info about git, but what about the removal of those two chapters suggested? I agree that we needn't try to cover material that's in the CVS manual. As somebody mentioned upthread, a sentence or two about our branching and tagging conventions would be a lot more useful. Here's an updated patch that does what I believe the consensus of this thread was. Unless objected, I will commit this later tonight. Patch now does: * As before, update cvs documentation and add git documentation * Remove cvsup documentation * Remove cvs internal documentation * Add a link to appendix H (the source code repository) from the general getting the source chapter. It does not add any proper documentation of exactly how we deal with branches and tags at a useful level - this will come later. I would also like to propose that we actually backpatch this. At least the addition of the git documentation and the update of the CVS documentation. So we get this info out there. We don't normally backpatch things like this though, so comments on that? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ cvs_docs.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Mon, Dec 7, 2009 at 9:48 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: This is no harder than many of the other seemingly crazy things I have done, e.g. Win32 port, client library threading. ?If this is a feature we should have, I will get it done or get others to help me complete the task. Well, I have always thought that it would be sort of a feather in our cap to support this, which is why I've done a couple of reviews of it in the past. I tend to agree with Tom that only a small fraction of our users will probably want it, but then again someone's been paying KaiGai to put a pretty hefty amount of work into this over the last year-plus, so obviously someone not only wants the feature but wants it merged. Within our community, I think that there have been a lot of people who have liked the concept of this feature but very few who have liked the patch, so there's somewhat of a disconnect between our aspirations and our better technical judgment. Tom is a notable exception who I believe likes neither the concept nor the patch, which is something we may need to resolve before getting too serious about this. Agreed. SE-Linux support might expand our user base and give us additional credibility, or it might be a feature that few people use --- and I don't think anyone knows the outcome. I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? SE-Linux support would certainly put Postgres in a unique security category, and it builds on our existing good security reputation. Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, and ripping it out is likely technically fairly simple but will probably be politically impossible. Personally, I think AppArmor is a saner security system: http://www.novell.com/linux/security/apparmor/selinux_comparison.html (Novell-hosted URL) Agreed. but I am not advocating AppArmor support. I think the whole issue is whether support for external integrated security systems is appropriate for Postgres. It's not something I've run into a need for in my own work, but I think there are definitely people out there who do need it, and I'd like to see us be able to support it. One of the things that I think would be worth looking into is whether there is a way to make this pluggable, so that selinux and apparmor and trusted solaris and so on could make use of the same framework, but that requires understanding all of them well enough to design a framework that can meet all of those needs. Every framework effort we've seen from KaiGai so far has seemed extremely SE-Linux-specific and therefore pointless. But really doing this right is a big development project, and not something I can do in my free time. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
On Mon, Dec 07, 2009 at 04:08:28PM +0100, Magnus Hagander wrote: 2009/11/26 Tom Lane t...@sss.pgh.pa.us: Magnus Hagander mag...@hagander.net writes: I assume you are fine with the addition of some info about git, but what about the removal of those two chapters suggested? I agree that we needn't try to cover material that's in the CVS manual. As somebody mentioned upthread, a sentence or two about our branching and tagging conventions would be a lot more useful. Here's an updated patch that does what I believe the consensus of this thread was. Unless objected, I will commit this later tonight. Patch now does: * As before, update cvs documentation and add git documentation * Remove cvsup documentation * Remove cvs internal documentation * Add a link to appendix H (the source code repository) from the general getting the source chapter. It does not add any proper documentation of exactly how we deal with branches and tags at a useful level - this will come later. I would also like to propose that we actually backpatch this. At least the addition of the git documentation and the update of the CVS documentation. So we get this info out there. We don't normally backpatch things like this though, so comments on that? +1 for back-patching. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need a mentor, and a project.
On Mon, Dec 07, 2009 at 09:53:32AM +0100, Albe Laurenz wrote: abindra wrote: Next quarter I am planning to do an Independent Study course where the main objective would be to allow me to get familiar with the internals of Postgres by working on a project(s). I would like to work on something that could possibly be accepted as a patch. This is (I think) somewhat similar to what students do during google summer and I was hoping to get some help here in terms of: 1. A good project to work on for a newbie. 2. Would someone be willing to be a mentor? It would be nice to be able to get some guidance on a one-to-one basis. I would start with the TODO list: http://wiki.postgresql.org/wiki/Todo These are things for which there is a consensus that it would be a good idea to implement them. Pick things that look interesting to you, and try to read the discussions in the archives that lead to the TODO items. I agree the TODO list is a good place to start. Other good sources include the -hackers list and comments in the code. I was surprised when I began taking an interest in PostgreSQL how rarely interesting projects mentioned on -hackers made it into the TODO list; I've come to realize that the TODO contains, in general, very non-controversial items everyone is pretty sure we could use, whereas -hackers ranges freely over other topics which are still very interesting but often more controversial or less obviously necessary. Committed patches both large and small address TODO list items fairly rarely, so don't get too hung up on finding something from the TODO list alone. Bring the topic up in the hackers list, say that you would like to work on this or that TODO item, present your ideas of how you want to do it. Ask about things where you feel insecure. If you get some support, proceed to write a patch. Ask for directions, post half-baked patches and ask for comments. That is because you will probably receive a good amount of critizism and maybe rejection, and if you invest a couple of months into working on something that nobody knows about *and* your work gets rejected, that is much worse than drawing fire right away. +1. Especially when developing a complex patch, and especially when you're new to the community, you need to avoid working in a vacuum, for social as well as technical reasons. The more complex a patch, the more consensus you'll eventually need to achieve before getting it committed, in general, and it helps to gain that consensus early on, rather than after you've written a lot of code. The keyword proposal might be a useful search term when digging in the -hackers archives for historical examples. -- Joshua Tolley / eggyknap End Point Corporation http://www.endpoint.com signature.asc Description: Digital signature
Re: [HACKERS] YAML Was: CommitFest status/management
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: Tom Lane t...@sss.pgh.pa.us wrote: It was written and submitted by one person who did not bother to ask first whether anyone else thought it was worthwhile. So its presence on the CF list should not be taken as evidence that there's consensus for it. Should we have Needs Discussion phase before Needs Review ? Reviews, including me, think patches with needs-review status are worthwhile. In contrast, contributers often register their patches to CF without discussions just because of no response; they cannot find whether no response is silent approval or not. Hm, I guess the question would be: what is the condition for getting out of that state? It's clear who is supposed to move a patch out of 'Needs Review', 'Waiting for Author', or 'Ready for Committer' respectively. I don't know who's got the authority to decide that something has or has not achieved community consensus. Right at the moment we handle this sort of problem in a very informal way, but if it's going to become part of the commitfest state for a patch I think we need to be a bit less informal. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] tsearch parser inefficiency if text includes urls or emails - new version
Greg Smith g...@2ndquadrant.com wrote: After getting off to a good start, it looks like this patch is now stuck waiting for a second review pass from Kevin right now, with no open items for Andres to correct. Since the only issues on the table seem to be that of code aesthetics and long-term planning for this style of implementation rather than specific functional bits, I'm leaning toward saying this one is ready to have a committer look at it. Any comments from Kevin or Andres about where this is at? Yeah, really the only thing I found to complain about was one misspelled word in a comment. I am currently the hold-up, due to fighting off a bout of some virus and having other real world issues impinge. The only thing left to do, besides correcting the spelling, is to confirm the author's performance improvements and confirm that there is no degradation in a non-targeted situation. Frankly, I'd be amazed if there was a performance regression, because all it really does is pass a pointer to a new spot in an existing input buffer rather than allocating new space and copying the input from the desired spot to the end of the buffer. I can't think of any situations where calculating the new address should be slower than calculating the new address and copying from there to the end of the buffer. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to cache a non-unique index?
Xin Wang and...@gmail.com writes: I have added a catalog table with an index on a non-unique column. I want to cache this non-unique index like other unique indexes in syscache. Is there any way to cache a non-unique index? No, not with syscache. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: json format and auto_explain
Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: While testing the EXPLAIN BUFFERS patch I found a bug. I'm too tired to provide a fix right now but if someone can take it I will appreciate. It seems ExplainJSONLineEnding() doesn't expect es-grouping_stack list as a null pointer. Looks like auto_explain is under the illusion that it need not call ExplainBeginOutput/ExplainEndOutput. Fortunately the author of auto_explain can now commit the fix by himself ... Kudos, BTW :-) -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
Magnus Hagander mag...@hagander.net writes: I would also like to propose that we actually backpatch this. At least the addition of the git documentation and the update of the CVS documentation. So we get this info out there. We don't normally backpatch things like this though, so comments on that? The sort of people who would actually have a use for the information are unlikely to be looking at back branches, so I don't particularly see the point. But if you wanna do the work ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Robert Haas robertmh...@gmail.com wrote: Bruce Momjian br...@momjian.us wrote: Personally, I think AppArmor is a saner security system: http://www.novell.com/linux/security/apparmor/selinux_comparison.html Agreed. I'd like to see us be able to support it. One of the things that I think would be worth looking into is whether there is a way to make this pluggable, so that selinux and apparmor and trusted solaris and so on could make use of the same framework Given the extreme patience and diligence exhibited by KaiGai, I hesitate to say this, but it seems to me that this would be critically important for the long term success of this feature. I have no idea how much work it would be to make the interface to the external security system pluggable, but if it's at all feasible, I think it should be done. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
2009/12/7 Tom Lane t...@sss.pgh.pa.us: Magnus Hagander mag...@hagander.net writes: I would also like to propose that we actually backpatch this. At least the addition of the git documentation and the update of the CVS documentation. So we get this info out there. We don't normally backpatch things like this though, so comments on that? The sort of people who would actually have a use for the information are unlikely to be looking at back branches, so I don't particularly see the point. But if you wanna do the work ... I'd do 8.4, becuase that's what shows up under /current/ on the website. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
On Mon, Dec 7, 2009 at 5:49 AM, Dave Page dp...@pgadmin.org wrote: The new committers are: Robert Haas Simon Riggs Greg Stark ITAGAKI Takahiro Congratulations! +1 -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Robert Haas robertmh...@gmail.com writes: On Mon, Dec 7, 2009 at 9:48 AM, Bruce Momjian br...@momjian.us wrote: I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, It's not the get it in part that scares me. The problem I have with it is that I see it as a huge time sink for future maintenance problems, most of which will be classifiable as security breaches which increases the pain of dealing with them immeasurably. If I had more confidence that the basic design was right or useful I might not be so worried about the maintenance prospects, but frankly I have almost no confidence in it. This comes back to the lack of involvement of any potential user community. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Installing PL/pgSQL by default
Tom Lane t...@sss.pgh.pa.us writes: Why not? If they really want to prohibit use of a feature the upstream project has decided should be standard, that's their privilege. Well, I guess they could also automate their database creation to fix the privileges and assign the ownership of the language to the owner of the database. Then whether or not to have plpgsql there is up to the owner. For non-hosted environments, you always want to tweak some things, like installing plpgsql in the first place. So... The argument against seems to be basically this should work exactly like it did before, but if that's the standard then we can never have plpgsql installed by default at all. Don't get me wrong, I'm all for having plpgsql installed by default. I though we were talking about how to provide that and trying to decide if having to be superuser to drop plpgsql after having created the database is blocking the way forward, knowing than installing the language only requires being database owner. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] cvs chapters in our docs
Magnus Hagander escribió: 2009/12/7 Tom Lane t...@sss.pgh.pa.us: Magnus Hagander mag...@hagander.net writes: I would also like to propose that we actually backpatch this. At least the addition of the git documentation and the update of the CVS documentation. So we get this info out there. We don't normally backpatch things like this though, so comments on that? The sort of people who would actually have a use for the information are unlikely to be looking at back branches, so I don't particularly see the point. But if you wanna do the work ... I'd do 8.4, becuase that's what shows up under /current/ on the website. 8.4 makes sense to me. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] new CommitFest states (was: YAML)
On Mon, Dec 7, 2009 at 10:20 AM, Tom Lane t...@sss.pgh.pa.us wrote: Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: Tom Lane t...@sss.pgh.pa.us wrote: It was written and submitted by one person who did not bother to ask first whether anyone else thought it was worthwhile. So its presence on the CF list should not be taken as evidence that there's consensus for it. Should we have Needs Discussion phase before Needs Review ? Reviews, including me, think patches with needs-review status are worthwhile. In contrast, contributers often register their patches to CF without discussions just because of no response; they cannot find whether no response is silent approval or not. Hm, I guess the question would be: what is the condition for getting out of that state? It's clear who is supposed to move a patch out of 'Needs Review', 'Waiting for Author', or 'Ready for Committer' respectively. I don't know who's got the authority to decide that something has or has not achieved community consensus. Right at the moment we handle this sort of problem in a very informal way, but if it's going to become part of the commitfest state for a patch I think we need to be a bit less informal. This sounds suspiciously like it could lead to a situation where people want to use the CommitFest application to track feature ideas rather than actual patches. The next step would be to require that the Needs Discussion status doesn't actually require any code, and the code can be submitted when it reaches Needs Review. I think this whole line of thinking is a bad one. The CommitFest application isn't designed to manage feature requests, and I don't want to turn it into something that does. We might need a system to manage that, at some point, but let's not shoehorn it into what we have now. The right answer here is that no one should assume that a patch is wanted just because it is in the CommitFest application. This point is actually explicitly addressed in this document http://wiki.postgresql.org/wiki/Reviewing_a_Patch If people submit patches without getting consensus that the feature is something that we want, then the penalty is that their patch is more likely to be rejected. If you're worried about this, then you should get consensus before you spend time writing code. If you're not worried about it, that's your prerogative: experienced patch authors often know what will and will not fly without a drawn-out discussion, especially for very minor enhancements or bug fixes. On a related note, Greg Smith requested a state called Discussing Review, which would logically follow Needs Review and precede Waiting for Author/Ready for Committer/Returned with Feedback. I'm not altogether convinced of the value of that state, but I'm not altogether opposed to it either. If we're going to have a discussion of CommitFest states, we probably ought to talk about that one, too. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Dimitri Fontaine wrote: If the problem is supporting 2 formats in core rather than 3, what about replacing the current JSON support with the YAML one? That's a step backwards. By providing JSON format, we've also satisfied people who want YAML. Ripping out JSON would mean we *only* support YAML. There are far many more JSON parsers than YAML parsers, which is why I thought the current code committed was good enough. Anyway, the fact that I have a technical opinion suggests to me I've caught up with the discussion now, so let's talk about where we're at. I think that the ongoing discussion here is likely to consume more resources than the expected future maintenance of this small bit of code. I believe the following to be true: -The patch is small to apply -It would also be easy to remove in the future should a more modular EXPLAIN implementation replace it -Having one more legacy format to satisfy would actually improve the odds that a future modular EXPLAIN would be robustly designed -There is no way a modular explain will be written any time soon -While it's questionable whether it's strictly a majority on voting here, it's close, which suggests there is plenty of support for wanting this feature -Since nothing is removed the people who aren't in favor of this format aren't negatively impacted by it being committed All that suggests to me that we've cleared the usual do we want it? hurdles that would normally go along with deciding whether a patch should go to a committer or not. That leaves code quality then. Are there any open issues with that? There's some notes about line-breaks in the CF app. Once we have a patch with those issues resolved, this should go to a committer for final veto power on its inclusion, and then we're done here. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Need a mentor, and a project.
On Sun, Dec 6, 2009 at 9:24 PM, abin...@u.washington.edu wrote: 2. Would someone be willing to be a mentor? It would be nice to be able to get some guidance on a one-to-one basis. I might be willing to do this, but if you pick a project that is outside my area of knowledge then I might not be able to help as much. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: json format and auto_explain
Alvaro Herrera escreveu: Fortunately the author of auto_explain can now commit the fix by himself ... Kudos, BTW :-) Congratulations! -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: json format and auto_explain
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes:
Tom Lane t...@sss.pgh.pa.us wrote:
Looks like auto_explain is under the illusion that it need not call
ExplainBeginOutput/ExplainEndOutput.
Explain{Begin/End}Output are static functions, so we cannot call them
from an external contrib module. Instead, I'll suggest to call them
automatically from ExplainPrintPlan. The original codes in ExplainPrintPlan
was moved into ExplainOneResult, that name might be debatable.
This isn't an adequate solution I'm afraid --- what about the other
functions that are exported by explain.h?
I too am not totally thrilled with the idea of exporting
Explain{Begin/End}Output, but it might be the best solution.
We might also need to think about refactoring those functions:
there seem to be two different things going on there, one being
format-specific initialization which will certainly be necessary,
and one being output of a wrapper structure which might or might
not be appropriate for what auto_explain or other callers want.
In any case you need to expend more effort on the comments for the
functions. Inadequate specification of ExplainPrintPlan's call
requirements is what got us into this problem in the first place,
and the proposed patch makes that worse not better.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Kevin Grittner escribió: I'd like to see us be able to support it. One of the things that I think would be worth looking into is whether there is a way to make this pluggable, so that selinux and apparmor and trusted solaris and so on could make use of the same framework Given the extreme patience and diligence exhibited by KaiGai, I hesitate to say this, but it seems to me that this would be critically important for the long term success of this feature. I have no idea how much work it would be to make the interface to the external security system pluggable, but if it's at all feasible, I think it should be done. This is how the code was developed initially -- the patch was called PGACE and SELinux was but the first implementation on top of it. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: json format and auto_explain
On Mon, Dec 7, 2009 at 10:42 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: While testing the EXPLAIN BUFFERS patch I found a bug. I'm too tired to provide a fix right now but if someone can take it I will appreciate. It seems ExplainJSONLineEnding() doesn't expect es-grouping_stack list as a null pointer. Looks like auto_explain is under the illusion that it need not call ExplainBeginOutput/ExplainEndOutput. Fortunately the author of auto_explain can now commit the fix by himself ... Kudos, BTW :-) I just tested and this has been broken since it was committed (by Tom). However, I believe I did test it on the last version of the patch that I submitted, and I think it worked then. So we have lots of choices for who should fix this: - me, since it's my feature - Tom, since it appears that he broke it - Itagaki-san, since he's the auto_explain maintainer :-) ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] strange segfault with 8.3.8
hello, from kernel log: Dec 7 07:24:45 nsXX kernel: postgres[22481]: segfault at 7fda5e1d5000 ip 7fda604553c3 sp 7fffe41faf28 error 4 in libc-2.9.so [7fda603d1000+168000] Dec 7 07:24:45 nsXX kernel: grsec: From XX.YY.ZZ.51: Segmentation fault occurred at 7fda5e1d5000 in /usr/lib/postgresql/8.3/bin/postgres[postgres:22481] uid/euid:103/103 gid/egid:114/114, parent /usr/lib/postgresql/8.3/bin/postgres[postgres:29857] uid/euid:103/103 gid/egid:114/114 This is on 8.3.8, latest Ubuntu distribution, server under quite heavy load. afterwards we had some corrupted pages (*ERROR: compressed data is corrupt* ). any clues / directions welcome. -- Filip Rembiałkowski JID,mailto:filip.rembialkow...@gmail.com http://filip.rembialkowski.net/
Re: [HACKERS] EXPLAIN BUFFERS
Itagaki Takahiro escreveu: I think the current output is enough and useful in normal use. We can use XML or JSON format for more details. I don't think it is a good idea to have different information in different formats. I'm with Robert; *don't* do that. If you want to suppress the other ones in text format, do it in the others too. One idea is to add them only if we prefer the VERBOSE output. I think Blocks Hit: 1641 Read: 0 Temp Read: 1443 means Blocks (Hit: 1641 Read: 0 Temp Read: 1443) i.e., Blocks of hit, blocks of reads, and Blocks of temp reads. But the latter is more clear than the former. I cannot understand what you mean -- should I suppress the lines when they have all-zero values? There are nodes that don't read or write blocks. If we go this way, we need to document this behavior. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Largeobject Access Controls (r2460)
KaiGai Kohei kai...@ak.jp.nec.com wrote: Apart from SELinux, it is quite natural to apply any access controls on binary data. If we could not have any valid access controls, users will not want to store their sensitive information, such as confidential PDF files, as a large object. Absolutely. The historical security issues for large objects immediately eliminated them as a possible place to store PDF files. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Greg, thanks for your useful comments. I attached new doc patch version. I removed example changes and add link to create database cluster (I hope) everywhere. Please, look on it and let me know if there is still something what should be changed. Thanks Zdenek Greg Smith píše v ne 06. 12. 2009 v 22:29 -0500: I just noticed that there was an updated patch here that never made its way onto the CommitFest app listing. I just fixed that and took a quick look at it. I was in favor of this code change, but I have to say even I don't really like how it ended up getting documented--and I'm sure there are others would be downright hostile toward it. The biggest problem is that all of the places that used to say commandinitdb when talking about creating a cluster now just say database cluster initialization--with no link to a section covering that topic. That's not a good forward step. The part I'm more favorable toward that I expect other people to really cringe at is that the examples showing how to manually run initdb have all been switched to use pg_ctl initdb too. If we're going to have a smooth transition toward supporting both styles of working in the next release, I think what needs to happen to the documentation here is adding a very clear section saying that initdb and pg_ctl initdb are the same thing, and noting why both forms exist. Maybe a short note in both pg_ctl and initdb pointing at each other; not sure yet what's best. Then update all the current places that say initdb that have been rewritten in this doc patch to database cluster initialization to reference things appropriate still. Going as far as making all the examples exclusively use the pg_ctl form right now is probably more than the community at large wants to handle just yet I suspect. At best, maybe we could make some or all of those either use both forms, or link them to the new discussion of alternatives section. I'm glad we made some progress (and are basically at code complete now) on this patch this round. Given that this patch doesn't have a large amount of support, I think that the remaining work here is fine-tuning the documentation to cover the new option available without introducing and abrupt change people won't like. I'm going to mark this returned with feedback for now since I think that's going to take a bit more work than we really have time for right now, particularly given the limited number of people who care about this change. Zdenek, once this CommitFest clears out, I can help out with getting the documentation parts here smoothed over so this is in proper shape to commit during the next one; I don't think there's anything left you need to do. diff -r ab32ed8164e7 doc/src/sgml/config.sgml --- a/doc/src/sgml/config.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/config.sgml Mon Dec 07 17:12:32 2009 +0100 @@ -54,9 +54,10 @@ para One way to set these parameters is to edit the file filenamepostgresql.conf/indextermprimarypostgresql.conf//, -which is normally kept in the data directory. (applicationinitdb/ -installs a default copy there.) An example of what this file might look -like is: +which is normally kept in the data directory. +(link linkend=creating-clusterdatabase cluster initialization/link +process installs a default copy there.) +An example of what this file might look like is: programlisting # This is a comment log_connections = yes @@ -365,8 +366,8 @@ Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as -determined during applicationinitdb/). This parameter can -only be set at server start. +determined during link linkend=creating-clusterdatabase cluster +initialization/link). This parameter can only be set at server start. /para para @@ -747,7 +748,8 @@ Sets the amount of memory the database server uses for shared memory buffers. The default is typically 32 megabytes (literal32MB/), but might be less if your kernel settings will -not support it (as determined during applicationinitdb/). +not support it (as determined during link linkend=creating-clusterdatabase +cluster initialization)/link. This setting must be at least 128 kilobytes. (Non-default values of symbolBLCKSZ/symbol change the minimum.) However, settings significantly higher than the minimum are usually needed @@ -4267,10 +4269,10 @@ keywords literalUS/, literalNonEuro/, and literalNonEuropean/ are synonyms for literalMDY/. See xref linkend=datatype-datetime for more information. The -built-in default is literalISO, MDY/, but -applicationinitdb/application will
Re: [HACKERS] New PostgreSQL Committers
Jaime Casanova jcasa...@systemguards.com.ec wrote: Dave Page dp...@pgadmin.org wrote: The new committers are: Robert Haas Simon Riggs Greg Stark ITAGAKI Takahiro Congratulations! +1 Outstanding! Congratulations, all! -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Mon, Dec 07, 2009 at 01:09:59PM -0300, Alvaro Herrera wrote: Given the extreme patience and diligence exhibited by KaiGai, I hesitate to say this, but it seems to me that this would be critically important for the long term success of this feature. I have no idea how much work it would be to make the interface to the external security system pluggable, but if it's at all feasible, I think it should be done. This is how the code was developed initially -- the patch was called PGACE and SELinux was but the first implementation on top of it. I find it astonishing that after SE-PgSQL was implemented on top of a pluggable system (PGACE) and this system was removed at request of the community [1] that at this late phase people are suggesting it needs to be added back again. Havn't the goalposts been moved enough times? [1] http://archives.postgresql.org/pgsql-hackers/2009-01/msg02295.php (It seems we've gone from a patch that had been around for years solving actual people's problems to a patch which does barely anything and we don't know whether it solves anybodies problem). Have a nice day, -- Martijn van Oosterhout klep...@svana.org http://svana.org/kleptog/ Please line up in a tree and maintain the heap invariant while boarding. Thank you for flying nlogn airlines. signature.asc Description: Digital signature
Re: [HACKERS] new CommitFest states (was: YAML)
Robert Haas wrote: On a related note, Greg Smith requested a state called Discussing Review, which would logically follow Needs Review and precede Waiting for Author/Ready for Committer/Returned with Feedback. I'm not altogether convinced of the value of that state, but I'm not altogether opposed to it either. If we're going to have a discussion of CommitFest states, we probably ought to talk about that one, too. Don't know what it is about YAML that it encourages slipping into CF management...Discussing Review is a state patches seem to fall into for a time. I'd like to see it added mainly because it simplifies work for a lazier (than Robert) CF manager like myself, which I think is a more appropriate target for this process. Some of the process that works for him I don't think can be replicated by other personalities very well. If a patch is being actively discussed on the list, often the author is at the mercy of said discussion ending before they can make another forward step; this is why Waiting for Author isn't appropriate. Having the patch sitting in Needs Review instead is unfair to the reviewer, who would like to be able to mark it as I'm done and move on. That's also why sitting in that state takes up time for the CF manager. They need to scan all waiting for [author|review] patches to get a list of people to nudge, and in this case there is no one to nudge--we're all at the mercy of the list to reach some sort of conclusion. The obvious concern here is who has the action item them? In this case, that's the CF manager's job--to help wrap the discussion up once it's died down and figure out what state the patch should go into next. Reviewers would mark a patch Discussing Review once they're done and have sent their review to the list when it doesn't fit any of the existing next states well, and they're not sure what happens next. Basically it allows them to formally push making a hard decision about a patch upwards, which is effectively what's happening now. Then the CF manager or committer can mark it returned with feedback or flat-out rejected if the resulting discussion isn't favorable, rather than making the reviewer responsible for that, once discussion has wrapped up. Or the author/CF manager can eventually mark it waiting for author once one of them has decided that's the logical next step. I plan to turn the whole thing into a fairly easy to follow state diagram as documentation on the process, there's just this one common state I don't have a label for now: when things are trapped on the list, and nobody has an obvious next move until that settles down. There is no need or want for a Needs discussion before review or code submission. That just encourages abusing the CF app and process for something you can do quite nicely with the mailing list. If you believe in a feature but don't want to get community buy-in on the list first, write a patch to prove it works. If the reviewer torpedos your patch, don't expect you'll get a free round of discussing whether everyone wants that feature or not out of the deal. For this YAML example, had the code in the patch been junk we wouldn't even have gotten to discussion of its utility--the reviewer would have rejected it and that would have been it. And that IMHO is how it should be. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Greg Smith wrote: That's a step backwards. By providing JSON format, we've also satisfied people who want YAML. Ripping out JSON would mean we *only* support YAML. There are far many more JSON parsers than YAML parsers, which is why I thought the current code committed was good enough. XML parsers are common enough IMHO the other computer readable formats can't be that important from a computer-readability perspective, leaving their main benefit as being human friendly. I like YAML output; but I think the most compelling arguments against the patch are that if so many people want different formats it may be a good use case for external modules. And far more than yaml output, I'd like to see a flexible module system with an equivalent of cpan install yaml or gem install yaml. I suppose one could argue that instead of YAML we design a different human-oriented format for loosely structured data; but that seems even harder. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] strange segfault with 8.3.8
Filip Rembiałkowski wrote: Dec 7 07:24:45 nsXX kernel: postgres[22481]: segfault at 7fda5e1d5000 ip 7fda604553c3 sp 7fffe41faf28 error 4 in libc-2.9.so http://libc-2.9.so/[7fda603d1000+168000] Dec 7 07:24:45 nsXX kernel: grsec: From XX.YY.ZZ.51: Segmentation fault occurred at 7fda5e1d5000 in /usr/lib/postgresql/8.3/bin/postgres[postgres:22481] uid/euid:103/103 gid/egid:114/114, parent /usr/lib/postgresql/8.3/bin/postgres[postgres:29857] uid/euid:103/103 gid/egid:114/114 This is on 8.3.8, latest Ubuntu distribution, server under quite heavy load. afterwards we had some corrupted pages (//ERROR/: /compressed data/ is corrupt/). Random segfaults only under heavy load with corrupted pages afterwards normally means bad RAM. I'd suggest a round of memtest86+ on the system if it's PC hardware. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com
Re: [HACKERS] Adding support for SE-Linux security
Martijn van Oosterhout klep...@svana.org writes: I find it astonishing that after SE-PgSQL was implemented on top of a pluggable system (PGACE) and this system was removed at request of the community [1] that at this late phase people are suggesting it needs to be added back again. Havn't the goalposts been moved enough times? The reason the goalposts keep moving is that nobody has a very clear handle on what the requirements are, which stems from the lack of a clear target community with definable needs. We have had a couple of apparently-knowledgeable people pop up and say you should do this, but then they disappear again without sticking around for any detailed discussion of features (let alone code). (It seems we've gone from a patch that had been around for years solving actual people's problems to a patch which does barely anything and we don't know whether it solves anybodies problem). Do we know that any version of this patch has solved any actual people's problems? I know KaiGai-san has been putting it out as a Fedora package but there's little if any evidence that anyone's actually using that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] New PostgreSQL Committers
dp...@pgadmin.org (Dave Page) writes: Congratulations! +1 Congratulations, indeed, to this worthy set of developers! -- output = reverse(moc.liamg @ enworbbc) http://linuxfinances.info/info/multiplexor.html Power tends to corrupt and absolute power corrupts absolutely. -- First Baron Acton, 1834 - 1902 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
t...@sss.pgh.pa.us (Tom Lane) writes: Robert Haas robertmh...@gmail.com writes: On Mon, Dec 7, 2009 at 9:48 AM, Bruce Momjian br...@momjian.us wrote: I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, It's not the get it in part that scares me. The problem I have with it is that I see it as a huge time sink for future maintenance problems, most of which will be classifiable as security breaches which increases the pain of dealing with them immeasurably. Ah, yes, the importance of this is not to be underestimated... Once SE-Pg is added in, *any* bug found in it is likely to be considered a security bug, and hence a candidate for being a CERT Advisory. Some bad things are liable to happen: a) Such problems turn into a hue and cry situation requiring dropping everything else to fix the security problem. b) If everyone isn't using SE-Pg, then people won't be particularly looking for bugs, and hence bugs are likely to linger somewhat, with the consequence that a) occurs with some frequency. c) Having a series of CERT advisories issued is not going to be considered a good thing, reputation-wise! I feel about the same way about this as I did about the adding of native Windows support; I'm a bit concerned that this could be a destabilizing influence. I was wrong back then; the Windows support hasn't had the ill effects I was concerned it might have. I'd hope that my concerns about SE-Pg are just as wrong as my concerns about native Windows support. Hope doesn't make it so, alas... -- select 'cbbrowne' || '@' || 'gmail.com'; http://www3.sympatico.ca/cbbrowne/languages.html Just because it's free doesn't mean you can afford it. -- Unknown -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
Simon Riggs wrote: How does this sound? At startup we will delete recovery.conf.running, if it exists. At startup recovery.conf will be copied to recovery.conf.running, which will be the file read by any additional processes that read recovery.conf during this execution. The permissions on the file will then be set to read-only. Why not just follow the example of postresql.conf? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Chris Browne cbbro...@acm.org writes: I feel about the same way about this as I did about the adding of native Windows support; I'm a bit concerned that this could be a destabilizing influence. I was wrong back then; the Windows support hasn't had the ill effects I was concerned it might have. That's an interesting analogy. I too am not entirely convinced that it's a good comparison, but if it is, consider these points: * The goal of the Windows port was pretty well-defined and easily tested: make it work on Windows. The goalposts didn't move except perhaps when MS came out with new Windows versions. * We had a *lot* of users available to help flush out problems. * There wasn't any need to treat bugs as security sensitive, which is problematic not least because it restricts the free flow of information. Any one of those points would be good reason to think that getting SEPostgres to stability will be lots more drawn-out and painful than getting the Windows port to stability was. With all three pointing in the same direction, the tea leaves don't look good. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Florian Weimer escribió: * Dimitri Fontaine: Well we have JSON and agreed it was a good idea to have it. Now JSON is a subset of YAML and some would prefer another YAML style (me included). YAML is rather difficult to parse, and most parsers assume a trusted document source because they support arbitrary class instantiations (and it's not possible to switch this off, or it's rather involved to do so). That's irrelevant because EXPLAIN YAML is already a trusted document source. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] operator exclusion constraints
On Mon, 2009-12-07 at 00:26 -0500, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: [ exclusion constraint patch ] Applied after quite a lot of editorialization. For future reference, here is a summary of what I did: Thank you for the suggestions, and the other constructive criticism during development. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
Robert Haas escribió: Tallying up the votes on this patch, we have Tom Lane, Andrew Dunstan, Josh Drake, and myself arguing against including this in core, and Josh Berkus and Itagaki Takahiro arguing for it. Ron Mayer seemed somewhat in favor of it in his message on-list but sent a message off-list taking the opposite position. Brendan Jurd cast aspersions on the human-readability of the text format but didn't explicitly take a position on the core issue, Please add my vote for YAML, because of the human-unreadability of the default text format. In fact I'd argue we could rip out the default text format and replace it with this one. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Block-level CRC checks
On Fri, Dec 4, 2009 at 10:47 PM, Chuck McDevitt cmcdev...@greenplum.com wrote: A curiosity question regarding torn pages: How does this work on file systems that don't write in-place, but instead always do copy-on-write? My example would be Sun's ZFS file system (In Solaris BSD). Because of its snapshot rollback functionality, it never writes a page in-place, but instead always copies it to another place on disk. How does this affect the corruption caused by a torn write? Can we end up with horrible corruption on this type of filesystem where we wouldn't on normal file systems, where we are writing to a previously zeroed area on disk? Sorry if this is a stupid question... Hopefully somebody can reassure me that this isn't an issue. It's not a stupid question, we're not 100% sure but we believe ZFS doesn't need full page writes because it's immune to torn pages. I think the idea of ZFS is that the new partially written page isn't visible because it's not linked into the tree until it's been completely written. To me it appears this would depend on the drive system ordering writes very strictly which seems hard to be sure is happening. Perhaps this is tied to the tricks they do to avoid contention on the root, if they do a write barrier before every root update that seems like it should be sufficient to me, but I don't know at that level of detail. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Robert Haas wrote: Agreed. ?SE-Linux support might expand our user base and give us additional credibility, or it might be a feature that few people use --- and I don't think anyone knows the outcome. I wonder if we should rephrase this as, How hard will this feature be to add, and how hard will it be to remove in a few years if we decide we don't want it? ?SE-Linux support would certainly put Postgres in a unique security category, and it builds on our existing good security reputation. Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, and ripping it out is likely technically fairly simple but will probably be politically impossible. I figure if there is sufficient usage, we will not need to remove it, and if there isn't, we will have no objections to removing it. but I am not advocating AppArmor support. ?I think the whole issue is whether support for external integrated security systems is appropriate for Postgres. It's not something I've run into a need for in my own work, but I think there are definitely people out there who do need it, and I'd like to see us be able to support it. One of the things that I think would be worth looking into is whether there is a way to make this pluggable, so that selinux and apparmor and trusted solaris and so on could make use of the same framework, but that requires understanding all of them well enough to design a framework that can meet all of those needs. Every framework effort we've seen from KaiGai so far has seemed extremely SE-Linux-specific and therefore pointless. But really doing this right is a big development project, and not something I can do in my free time. As Alvaro mentioned, the original patch used ACE but it added too much code so the community requested its removal from the patch. It could be re-added if we have a need. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
On Mon, 2009-12-07 at 14:52 -0300, Alvaro Herrera wrote: Robert Haas escribió: Tallying up the votes on this patch, we have Tom Lane, Andrew Dunstan, Josh Drake, and myself arguing against including this in core, and Josh Berkus and Itagaki Takahiro arguing for it. Ron Mayer seemed somewhat in favor of it in his message on-list but sent a message off-list taking the opposite position. Brendan Jurd cast aspersions on the human-readability of the text format but didn't explicitly take a position on the core issue, Please add my vote for YAML, because of the human-unreadability of the default text format. In fact I'd argue we could rip out the default text format and replace it with this one. That would almost correct. My standing is we should have one format that is parsed. The current text explain output certainly does not qualify. The XML one or YAML one does. Pick one, then parse it into whatever you want from the client. Sincerely, Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering If the world pushes look it in the eye and GRR. Then push back harder. - Salamander -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Martijn van Oosterhout escribió: On Mon, Dec 07, 2009 at 01:09:59PM -0300, Alvaro Herrera wrote: This is how the code was developed initially -- the patch was called PGACE and SELinux was but the first implementation on top of it. I find it astonishing that after SE-PgSQL was implemented on top of a pluggable system (PGACE) and this system was removed at request of the community [1] that at this late phase people are suggesting it needs to be added back again. Havn't the goalposts been moved enough times? Yeah. I think the idle discussions here have created more work themselves than the hypothetical maintenance work that would be spent on this (undoubtely useful) feature. (It seems we've gone from a patch that had been around for years solving actual people's problems to a patch which does barely anything and we don't know whether it solves anybodies problem). Agreed :-( -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Bruce Momjian br...@momjian.us writes: Robert Haas wrote: Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, and ripping it out is likely technically fairly simple but will probably be politically impossible. I figure if there is sufficient usage, we will not need to remove it, and if there isn't, we will have no objections to removing it. That leaves a wide gray area where there are a few people using it but not really enough to justify the support effort. Even if there are demonstrably no users (which can never be demonstrated in practice), politically it's very hard to rip out a major feature --- it makes the project look bad. So I think the above is Pollyanna-ish nonsense. Once we ship a release with SEPostgres in it, we're committed. As Alvaro mentioned, the original patch used ACE but it added too much code so the community requested its removal from the patch. It could be re-added if we have a need. The main problem I saw with ACE was that it didn't appear to actually add any flexibility --- it was just an extra layer of function calls in an entirely SELinux-centric design. In order to have a pluggable interface layer that is worth the electrons it's written on, you need to start out with more than one target system in mind to be plugged in. So that would mean, at minimum, investigating something like AppArmor or TrustedSolaris to see what its needs are before we sit down to design the plugin layer. (Which, of course, nobody here is actually interested enough to do. But without that research there is no point in demanding a plugin layer.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Robert Haas wrote: Yes, I think that's the right way to think about it. At a guess, it's two man-months of work to get it in, and ripping it out is likely technically fairly simple but will probably be politically impossible. I figure if there is sufficient usage, we will not need to remove it, and if there isn't, we will have no objections to removing it. That leaves a wide gray area where there are a few people using it but not really enough to justify the support effort. Even if there are demonstrably no users (which can never be demonstrated in practice), politically it's very hard to rip out a major feature --- it makes the project look bad. So I think the above is Pollyanna-ish nonsense. I don't even know what Pollyanna-ish nonsense means, and it would be better if you used less flowery/inflamitory prose. Once we ship a release with SEPostgres in it, we're committed. The MS Windows port took 1-2 years to solidify and during the solidification period we accepted problems and didn't treat it as a major platform. I think if SE-Linux support is added, there would be a similar period where the features is not treated as major while we work out any problems. We might even label it that way. Labeling SE-Postgres as such might minimize the political problems of removing it in the future, if that becomes necessary. I know there has been complaints about the lack of SE-PostgreSQL developers, but given the number of developers we had for the Win32 port vs. the installed base, I think having one dedicated SE-PostgreSQL developer is much more percentage-wise than we had for MS Windows. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML Was: CommitFest status/management
On Dec 7, 2009, at 9:52 AM, Alvaro Herrera wrote: Tallying up the votes on this patch, we have Tom Lane, Andrew Dunstan, Josh Drake, and myself arguing against including this in core, and Josh Berkus and Itagaki Takahiro arguing for it. Ron Mayer seemed somewhat in favor of it in his message on-list but sent a message off-list taking the opposite position. Brendan Jurd cast aspersions on the human-readability of the text format but didn't explicitly take a position on the core issue, Please add my vote for YAML, because of the human-unreadability of the default text format. In fact I'd argue we could rip out the default text format and replace it with this one. +1 David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML
All, What it's sounding like is that we ought to have a plug-in (both for postmaster and for psql) which allows the calling of an external library to parse explain output. Then people could covert XML/JSON into whatever they wanted. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] What happened to pl/proxy and FDW?
Marko, Asko, hackers: I thought the idea was that we were going to add PL/proxy to 8.5, with support for the foriegn data wrapper syntax? What happened to that? --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML
Josh Berkus wrote: All, What it's sounding like is that we ought to have a plug-in (both for postmaster and for psql) which allows the calling of an external library to parse explain output. Then people could covert XML/JSON into whatever they wanted. Not everything is sanely convertible into some sort of plugin. A plugin mechanism for this would be FAR more trouble that it is worth, IMNSHO. We are massively over-egging this pudding (as a culinary blogger you should appreciate this analogy). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML
Josh Berkus wrote: What it's sounding like is that we ought to have a plug-in (both for postmaster and for psql) which allows the calling of an external library to parse explain output. Then people could covert XML/JSON into whatever they wanted. It would be kinder to just reject the YAML patch than to make it wait for this. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] WAL format
While looking at the streaming replication patch, I can't help but wonder why our WAL format is so complicated. WAL is divided into WAL segments, each 16 MB by default. Each WAL segment is divided into pages, 8k by default. At the beginning of each WAL page, there's a page header, but the header at the first page of each WAL segment contains a few extra fields. If a WAL record crosses a page boundary, we write as much of it as fits onto the first page, and so-called continuation records with the rest of the data on subsequent pages. In particular I wonder why we bother with the page headers. A much simpler format would be: - get rid of page headers, except for the header at the beginning of each WAL segment - get rid of continuation records - at the end of WAL segment, when there's not enough space to write the next WAL record, always write an XLOG SWITCH record to fill the rest of the segment. The page addr stored in the WAL page header gives some extra protection for detecting end of valid WAL correctly, but we rely on the prev-links and CRC within page for that anyway, so I wouldn't mind losing that. The changes to ReadRecord in the streaming replication patch feel a bit awkward, because it has to work around the fact that WAL is streamed as a stream of bytes, but ReadRecord works one page at a time. I'd like to replace ReadRecord with a simpler ring buffer approach, but handling the continuation records makes it a bit hard. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Build sizes vs docs
Came cross this when updating the cvs fix. We declare size requirements as: Also check that you have sufficient disk space. You will need about 65 MB for the source tree during compilation and about MB for the installation directory. An empty database cluster takes about 25 MB; databases take about five times the amount of space that a flat text file with the same data would take. If you are going to run the regression tests you will temporarily need up to an extra 90 MB. Use the commanddf/command command to check free disk My source *without* compile is 82 Mb, and with a build in it (linux i686) is 110Mb during compilations, rather than 65. An empty cluster takes about 33Mb. The regression database takes about 132Mb. (this is 8.4) Should we fix these numbers, or just remove them? They're clearly platform dependent, but perhaps there's still a point in including them - mainly as hints? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Install chapter broken link
Our instalation chapter (15.2) has a link to http://developer.postgresql.org/~petere/bsd-gettext/ to get gettext on other systems. this link is broken. Can somebody provide a proper one, or should we remove it? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: In particular I wonder why we bother with the page headers. Since we re-use the file for a new segment, without overwriting the old contents, it seems like we would need to do *something* to reliably determine when we've hit the end of a segment and have moved into old data from a previous use of the file. Would your proposed changes cover that adequately? (I'm not sure I understood your proposal well enough to be comfortable about that.) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
Heikki Linnakangas wrote: - at the end of WAL segment, when there's not enough space to write the next WAL record, always write an XLOG SWITCH record to fill the rest of the segment. What happens if a record is larger than a WAL segment? For example, what if I insert a 16 MB+ datum into a varlena field? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: In particular I wonder why we bother with the page headers. A much simpler format would be: - get rid of page headers, except for the header at the beginning of each WAL segment - get rid of continuation records - at the end of WAL segment, when there's not enough space to write the next WAL record, always write an XLOG SWITCH record to fill the rest of the segment. What do you do with a WAL record that doesn't fit in a segment? (They do exist.) I don't think you can eliminate continuation records. You could maybe use them only at segment boundaries but I doubt that makes things any simpler than they are now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build sizes vs docs
Magnus Hagander mag...@hagander.net writes: Came cross this when updating the cvs fix. We declare size requirements as: Also check that you have sufficient disk space. You will need about 65 MB for the source tree during compilation and about MB for the installation directory. An empty database cluster takes about 25 MB; databases take about five times the amount of space that a flat text file with the same data would take. If you are going to run the regression tests you will temporarily need up to an extra 90 MB. Use the commanddf/command command to check free disk My source *without* compile is 82 Mb, and with a build in it (linux i686) is 110Mb during compilations, rather than 65. An empty cluster takes about 33Mb. The regression database takes about 132Mb. (this is 8.4) Should we fix these numbers, or just remove them? They're clearly platform dependent, but perhaps there's still a point in including them - mainly as hints? Maybe round them off to an order of magnitude. I think it's useful to have some idea of the size requirements, even if they change over time. It wouldn't be a bad idea to say as of 8.4 or some such, too. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
Kevin Grittner kevin.gritt...@wicourts.gov writes: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: In particular I wonder why we bother with the page headers. Since we re-use the file for a new segment, without overwriting the old contents, it seems like we would need to do *something* to reliably determine when we've hit the end of a segment and have moved into old data from a previous use of the file. Would your proposed changes cover that adequately? AFAICT the proposal would make us 100% dependent on the record CRC to detect when a record has been torn (ie, only the first few sectors made it to disk). I'm a bit nervous about that from a reliability standpoint --- with a 32-bit CRC you've got a 1-in-4-billion chance of accepting bad data. Checking the page headers too gives us many more bits that have to be as-expected to consider the data good. Since the records are fed to XLogInsert as units, it seems like the actual problem might be addressable by hooking in the sync-rep data sending at that level, rather than looking at the WAL page buffers as I gather it must be doing now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
On Monday 07 December 2009 21:44:37 Tom Lane wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: In particular I wonder why we bother with the page headers. Since we re-use the file for a new segment, without overwriting the old contents, it seems like we would need to do *something* to reliably determine when we've hit the end of a segment and have moved into old data from a previous use of the file. Would your proposed changes cover that adequately? AFAICT the proposal would make us 100% dependent on the record CRC to detect when a record has been torn (ie, only the first few sectors made it to disk). I'm a bit nervous about that from a reliability standpoint --- with a 32-bit CRC you've got a 1-in-4-billion chance of accepting bad data. Checking the page headers too gives us many more bits that have to be as-expected to consider the data good. One could argue that thats a good argument to go back to 64bit CRCs. Considering that they are more seldomly computed with such a change and that CPUs got more modern... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WAL format
Alvaro Herrera alvhe...@commandprompt.com writes: Heikki Linnakangas wrote: - at the end of WAL segment, when there's not enough space to write the next WAL record, always write an XLOG SWITCH record to fill the rest of the segment. What happens if a record is larger than a WAL segment? For example, what if I insert a 16 MB+ datum into a varlena field? That case doesn't pose a problem --- the datum would be toasted into individual tuples that are certainly no larger than a page. However we do have cases where a WAL record can get arbitrarily large; in particular a commit record with many subtransactions and/or many disk files to delete. These cases do get exercised in the field too --- I can recall at least one related bug report. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Install chapter broken link
Magnus Hagander wrote: Our instalation chapter (15.2) has a link to http://developer.postgresql.org/~petere/bsd-gettext/ to get gettext on other systems. this link is broken. Can somebody provide a proper one, or should we remove it? Goals of his version described at http://archives.postgresql.org/pgsql-hackers/2001-05/msg01114.php lest anyone wonder why there was a personal port here (I know I did). The version most like that available now would be the ones listed at ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/devel/gettext/README.html The only popular platform I'm aware of that doesn't include it nowadays is OS X: http://blog.doughellmann.com/2009/06/installing-gnu-gettext-for-use-with.html which does mean the need to install your own hasn't completely gone away. Looks like all the various free BSDs are using GNU gettext now; that's what I found in the NetBSD link above, and at http://www.freebsd.org/cgi/cvsweb.cgi/ports/devel/gettext/ Even Solaris aims to be compatible with the GNU version as of 2004. I think http://www.gnu.org/software/gettext/ is the appropriate new link destination. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] YAML
Not everything is sanely convertible into some sort of plugin. A plugin mechanism for this would be FAR more trouble that it is worth, IMNSHO. We are massively over-egging this pudding (as a culinary blogger you should appreciate this analogy). OK, then let's just accept it. It's small, has a maintainer, is useful to some people, and doesn't create any wierd complications. I think, given the knowledge that YAML is now a subdialect of JSON it could potentially be made smaller, but I can't say how at the moment. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Exclusion Constraint vs. Constraint Exclusion
Folks, We have a very unfortunate naming situation with Jeff Davis's new feature, namely the shorter name, which is one permutation away from an existing and entirely unrelated feature: Constraint Exclusion, which has to do with queries over partitioned tables and like entities. Renaming it, which I believe we should do Really Soon(TM), to Operator [Exclusion] Constraints would fix this problem. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Writeable CTE patch
Marko Tiikkaja escribió: Tom Lane wrote: The only thing that I'd be comfortable with is copying the snap and modifying the copy. I don't see an easy way to do that with the current code; CopySnapshot() is static and PushUpdatedSnapshot() seems to be a bit of a pain since it messes up some of the existing code which uses the active snapshot stack. Any ideas? That API is rather new. Maybe we need a new entry point, say GetActiveSnapshotCopy or some such. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Exclusion Constraint vs. Constraint Exclusion
David Fetter wrote: Folks, We have a very unfortunate naming situation with Jeff Davis's new feature, namely the shorter name, which is one permutation away from an existing and entirely unrelated feature: Constraint Exclusion, which has to do with queries over partitioned tables and like entities. Renaming it, which I believe we should do Really Soon(TM), to Operator [Exclusion] Constraints would fix this problem. Cheers, David. +1 I thought I was just too dumb to follow the conversation. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding support for SE-Linux security
On Mon, Dec 7, 2009 at 1:00 PM, Bruce Momjian br...@momjian.us wrote: As Alvaro mentioned, the original patch used ACE but it added too much code so the community requested its removal from the patch. It could be re-added if we have a need. Well, there's no point in putting that framework back in unless we can make it sufficiently general that it could be used to serve the needs of more than one security model. And so far, the signs have not been promising. David Quigley suggests downthread that making a truly general model isn't really possible, and he may be right, or not. I was just mentioning that it's an angle I have been thinking about investigating, but it may be a dead end. The real issue is making the code committable, and then maintaining it, as Tom rightly says, forever. We've got to make sure that we're willing to take that on before we do it, and I don't think it's a small task. It isn't so much whether we want the feature as whether the level of effort is proportionate to the benefit. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] bug: json format and auto_explain
On Mon, Dec 7, 2009 at 11:07 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes:
Tom Lane t...@sss.pgh.pa.us wrote:
Looks like auto_explain is under the illusion that it need not call
ExplainBeginOutput/ExplainEndOutput.
Explain{Begin/End}Output are static functions, so we cannot call them
from an external contrib module. Instead, I'll suggest to call them
automatically from ExplainPrintPlan. The original codes in ExplainPrintPlan
was moved into ExplainOneResult, that name might be debatable.
This isn't an adequate solution I'm afraid --- what about the other
functions that are exported by explain.h?
I too am not totally thrilled with the idea of exporting
Explain{Begin/End}Output, but it might be the best solution.
We might also need to think about refactoring those functions:
there seem to be two different things going on there, one being
format-specific initialization which will certainly be necessary,
and one being output of a wrapper structure which might or might
not be appropriate for what auto_explain or other callers want.
In any case you need to expend more effort on the comments for the
functions. Inadequate specification of ExplainPrintPlan's call
requirements is what got us into this problem in the first place,
and the proposed patch makes that worse not better.
There's an awful lot of layers of function calls happening in
explain.c for no really discernable purpose that I can see.
ExplainOneQuery() calls either ExplainOneUtility() if it has a utility
command or ExplainOneQuery_hook() if that's defined or else it plans
the query and then calls ExplainOnePlan(). ExplainOneUtility() in
turn calls ExplainExecuteQuery for execute statements and handles
everything else internally. The placement of the hook seems pretty
dubious to me (does anyone actually use it?), and the whole structure
seems like it could probably be flattened a bit.
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reading recovery.conf earlier
On Mon, 2009-12-07 at 19:07 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: How does this sound? At startup we will delete recovery.conf.running, if it exists. At startup recovery.conf will be copied to recovery.conf.running, which will be the file read by any additional processes that read recovery.conf during this execution. The permissions on the file will then be set to read-only. Why not just follow the example of postresql.conf? Much better idea. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
--On 13. November 2009 23:29:41 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: t contains two DTrace probe groups. One is related to monitoring SLRU and second is about executor nodes. I merged it with the head. Original end of mail thread is here: http://archives.postgresql.org/pgsql-hackers/2009-04/msg00148.php I've started to review this. It seems to me the attached patch wasn't adjusted or discussed again to address Tom's complaints? At least the executor probes contained here hold still the same issues mentioned by Tom in the discussion linked here. -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DTrace compiler warnings
--On 14. November 2009 15:25:25 +0100 Zdenek Kotala zdenek.kot...@sun.com wrote: Hmm, const is also problem on solaris. dtrace generates probe.h file and eats const. It generates following noise on solaris build: postgres.c, line 554: warning: argument #1 is incompatible with prototype: prototype: pointer to char : ../../../src/include/utils/probes.h, line 582 argument : pointer to const char IIRC, it was discussed. I cc Robert. He should know answer why const is ignored. Have you dug into this further? -- Thanks Bernd -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
