Re: [HACKERS] Why not install pgstattuple by default?
On Wed, May 18, 2011 at 3:37 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 15:29, Marko Kreen mark...@gmail.com wrote: On Wed, May 18, 2011 at 2:57 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Some of my personal discussions of this topic have suggested that some other popular extensions like pgcrypto and hstore get converted too. I think those all fail test (3), and I'm not actually sure where pgcrypto adds any special dependency/distribution issues were it to be moved to the main database package. If this general idea catches on, a wider discussion of what else should get promoted to this extensions area would be appropriate. The ones I picked seemed the easiest to justify by this criteria set. pgcrypto would cause trouble for any builds *without* SSL. I don't think any packagers do that, but people doing manual builds would certainly get different results. What kind of trouble? It should work fine without SSL. Oh, you're right - it does. But it does provide different functionalties? Or does it actually do exactly the same stuff, just in different ways? Same stuff, assuming you use recommended algorithms (Blowfish, AES, MD5, SHA1, SHA2) OpenSSL brings in more speedy implementations (maybe), and additional algorithms (ripemd160, 3des, cast5, twofish). -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Greg Smith wrote: Attached is a second patch to move a number of extensions from contrib/ to src/test/. Extensions there are built by the default built target, making installation of the postgresql-XX-contrib package unnecessary for them to be available. That was supposed to be contrib/ to src/extension , no idea where that test bit came from. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Attached is a second patch to move a number of extensions from contrib/ to src/test/. Extensions there are built by the default built target, making installation of the postgresql-XX-contrib package unnecessary for them to be available. +1 in general on the concept :-) This request--making some of these additions available without the contrib name/package being involved--has popped up many times before, and it turys out to be really easy to resolve with the new extensions infrastructure. I think it's even a reasonable change to consider applying now, between 9.1 Beta 1 and Beta 2. The documentation adjustments are the only serious bit left here that I've been able to find, the code changes here are all internal to the build process and easy. Does this include regression tests? Or will they need some mods? I moved the following extensions: auto_explain pageinspect pg_buffercache pg_freespacemap pgrowlocks pg_stat_statements pgstattuple My criteria was picking extensions that: 1) Don't have any special dependencies 2) Are in contrib mainly because they don't need to be internal functions, not because their code quality is demo/early 3) Tend to be installed on a production server for troubleshooting problems, rather than being required by development. 4) Regularly pop up as necessary/helpful in production deployment These seem like reasonable criteria. Some of my personal discussions of this topic have suggested that some other popular extensions like pgcrypto and hstore get converted too. I think those all fail test (3), and I'm not actually sure where pgcrypto adds any special dependency/distribution issues were it to be moved to the main database package. If this general idea catches on, a wider discussion of what else should get promoted to this extensions area would be appropriate. The ones I picked seemed the easiest to justify by this criteria set. pgcrypto would cause trouble for any builds *without* SSL. I don't think any packagers do that, but people doing manual builds would certainly get different results. Any packager who grabs the shared/postgresql/extension directory in 9.1, which I expect to be all of them, shouldn't need any changes to pick up this adjustment. For example, pgstattuple installs these files: share/postgresql/extension/pgstattuple--1.0.sql share/postgresql/extension/pgstattuple--unpackaged--1.0.sql share/postgresql/extension/pgstattuple.control And these are the same locations they were already at. The location of the source and which target built it is the change here, the result isn't any different. This means that this change won't even break extensions already installed. Once the basic directory plumbing is in place, conversion of a single extension from contrib/ to src/test/ is, trivial. The diff view I did five of them in an hour once I figured out what was needed. Easiest to view the changes at https://github.com/greg2ndQuadrant/postgres/commits/move-contrib , the patch file is huge because of all the renames. https://github.com/greg2ndQuadrant/postgres/commit/d647091b18c4448c5a582d423f8839ef0c717e91 show a good example of one convert, that changes pg_freespacemap. There are more changes to the comments listing the name of the file than to any code. (Yes, I know there are some whitespace issues I introduced in the new Makefile, they should be fixed by a later commit in the series) This is where the compare view rocks: https://github.com/greg2ndQuadrant/postgres/compare/postgres:master...greg2ndQuadrant:move-contrib -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Wed, May 18, 2011 at 2:57 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Some of my personal discussions of this topic have suggested that some other popular extensions like pgcrypto and hstore get converted too. I think those all fail test (3), and I'm not actually sure where pgcrypto adds any special dependency/distribution issues were it to be moved to the main database package. If this general idea catches on, a wider discussion of what else should get promoted to this extensions area would be appropriate. The ones I picked seemed the easiest to justify by this criteria set. pgcrypto would cause trouble for any builds *without* SSL. I don't think any packagers do that, but people doing manual builds would certainly get different results. What kind of trouble? It should work fine without SSL. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Wed, May 18, 2011 at 15:29, Marko Kreen mark...@gmail.com wrote: On Wed, May 18, 2011 at 2:57 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, May 18, 2011 at 10:25, Greg Smith g...@2ndquadrant.com wrote: Some of my personal discussions of this topic have suggested that some other popular extensions like pgcrypto and hstore get converted too. I think those all fail test (3), and I'm not actually sure where pgcrypto adds any special dependency/distribution issues were it to be moved to the main database package. If this general idea catches on, a wider discussion of what else should get promoted to this extensions area would be appropriate. The ones I picked seemed the easiest to justify by this criteria set. pgcrypto would cause trouble for any builds *without* SSL. I don't think any packagers do that, but people doing manual builds would certainly get different results. What kind of trouble? It should work fine without SSL. Oh, you're right - it does. But it does provide different functionalties? Or does it actually do exactly the same stuff, just in different ways? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Greg Smith wrote: Any packager who grabs the shared/postgresql/extension directory in 9.1, which I expect to be all of them, shouldn't need any changes to pick up this adjustment. For example, pgstattuple installs these files: share/postgresql/extension/pgstattuple--1.0.sql share/postgresql/extension/pgstattuple--unpackaged--1.0.sql share/postgresql/extension/pgstattuple.control And these are the same locations they were already at. ...and the bit I missed here is that there's a fourth file here: lib/postgresql/pgstattuple.so If you look at a 9.1 spec file, such as http://svn.pgrpms.org/browser/rpm/redhat/9.1/postgresql/EL-6/postgresql-9.1.spec , you'll find: %files contrib ... %{pgbaseinstdir}/lib/pgstattuple.so Which *does* require a packager change to relocate from the postgresql-91-package to the main server one. So the theory that a change here might happen without pushing a repackaging suggestion toward packagers is busted. This does highlight that some packaging guidelines would be needed here to completely this work. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Thu, 2011-05-12 at 19:37 -0400, Tom Lane wrote: It should be okay to move, since the -devel subpackage requires the main one. Therefore there is no configuration in which pg_config would be present before and missing after the change. Thanks Tom. I can make this change in next build set. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Why not install pgstattuple by default?
On Sat, 2011-05-07 at 21:47 -0400, Greg Smith wrote: On 05/06/2011 04:06 PM, Tom Lane wrote: FWIW, I did move pg_config from -devel to the main (really client) postgresql package in Fedora, as of 9.0. That will ensure it's present in either client or server installations. Eventually that packaging will reach RHEL ... We should make sure that the PGDG packages adopt that for 9.1 then, so it starts catching on more. Unless Devrim changed to catch up since I last installed an RPM set, in that 9.0 it's still in the same place: $ rpm -qf /usr/pgsql-9.0/bin/pg_config postgresql90-devel-9.0.2-2PGDG.rhel5 I'm not sure that I can move it to main package in 9.0 package set, I need to make sure that I won't break anything. But it is pretty doable for 9.1. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Why not install pgstattuple by default?
Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= dev...@gunduz.org writes: I'm not sure that I can move it to main package in 9.0 package set, I need to make sure that I won't break anything. But it is pretty doable for 9.1. It should be okay to move, since the -devel subpackage requires the main one. Therefore there is no configuration in which pg_config would be present before and missing after the change. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Tom Lane wrote: Christopher Browne cbbro...@gmail.com writes: But people are evidently still setting packaging policies based on how things were back in 7.3, even though that perhaps isn't necessary anymore. FWIW, once you get past the client versus server distinction, I think most subpackaging decisions are based on either the idea that only a minority of people will want this, or a desire to limit how many dependencies are pulled in by the main package(s). Both of those concerns apply to various subsets of -contrib, which means it's going to be hard to persuade packagers to fold -contrib into the -server package altogether. Nor would you gain their approval by trying to pre-empt the decision. We might get somewhere by trying to identify a small set of particularly popular contrib modules that don't add any extra dependencies, and then recommending to packagers that those ones get bundled into the main server package. Certainly it's not a huge amount of code; less than 2MB these days. - % wc `dpkg -L postgresql-contrib-9.0` | tail -1 15952 67555 1770987 total Well, to add some concrete facts rather than generalities to my own post, here are the sizes of the built RPMs from my last build for Fedora: -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 postgresql-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 postgresql-contrib-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 27337677 Apr 18 10:51 postgresql-debuginfo-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 961660 Apr 18 10:50 postgresql-devel-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 7569048 Apr 18 10:50 postgresql-docs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 246506 Apr 18 10:50 postgresql-libs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl64940 Apr 18 10:50 postgresql-plperl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl65776 Apr 18 10:50 postgresql-plpython-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl45941 Apr 18 10:50 postgresql-pltcl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 5302117 Apr 18 10:50 postgresql-server-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 1370509 Apr 18 10:50 postgresql-test-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 3644113 Apr 18 10:50 postgresql-upgrade-9.0.4-1.fc13.x86_64.rpm Is that last one pg_upgrade? It seems very big. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: here are the sizes of the built RPMs from my last build for Fedora: -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 postgresql-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 postgresql-contrib-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 27337677 Apr 18 10:51 postgresql-debuginfo-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 961660 Apr 18 10:50 postgresql-devel-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 7569048 Apr 18 10:50 postgresql-docs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 246506 Apr 18 10:50 postgresql-libs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl64940 Apr 18 10:50 postgresql-plperl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl65776 Apr 18 10:50 postgresql-plpython-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl45941 Apr 18 10:50 postgresql-pltcl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 5302117 Apr 18 10:50 postgresql-server-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 1370509 Apr 18 10:50 postgresql-test-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 3644113 Apr 18 10:50 postgresql-upgrade-9.0.4-1.fc13.x86_64.rpm Is that last one pg_upgrade? It seems very big. pg_upgrade plus a supporting set of 8.4 files ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: here are the sizes of the built RPMs from my last build for Fedora: -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 postgresql-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 postgresql-contrib-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 27337677 Apr 18 10:51 postgresql-debuginfo-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 961660 Apr 18 10:50 postgresql-devel-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 7569048 Apr 18 10:50 postgresql-docs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 246506 Apr 18 10:50 postgresql-libs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl64940 Apr 18 10:50 postgresql-plperl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl65776 Apr 18 10:50 postgresql-plpython-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl45941 Apr 18 10:50 postgresql-pltcl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 5302117 Apr 18 10:50 postgresql-server-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 1370509 Apr 18 10:50 postgresql-test-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 3644113 Apr 18 10:50 postgresql-upgrade-9.0.4-1.fc13.x86_64.rpm Is that last one pg_upgrade? It seems very big. pg_upgrade plus a supporting set of 8.4 files ... OK, where do I get to dance around that pg_upgrade is packaged in Fedora thanks to you? At PGCon? LOL -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Sun, May 8, 2011 at 12:02 AM, Greg Smith g...@2ndquadrant.com wrote: Attached patch is a first cut at what moving one contrib module (in this case pg_buffercache) to a new directory structure might look like. The idea is that src/extension could become a place for first-class extensions to live. Those are ones community is committed to providing in core, but are just better implemented as extensions than in-database functions, for reasons that include security. This idea has been shared by a lot of people for a while, only problem is that it wasn't really practical to implement cleanly until the extensions code hit. I think it is now, this attempts to prove it. Since patches involving file renaming are clunky, the changes might be easier to see at https://github.com/greg2ndQuadrant/postgres/commit/507923e21e963c873a84f1b850d64e895776574f where I just pushed this change too. The install step for the modules looks like this now: gsmith@grace:~/pgwork/src/move-contrib/src/extension/pg_buffercache$ make install /bin/mkdir -p '/home/gsmith/pgwork/inst/move-contrib/lib/postgresql' /bin/mkdir -p '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension' /bin/sh ../../../config/install-sh -c -m 755 pg_buffercache.so '/home/gsmith/pgwork/inst/move-contrib/lib/postgresql/pg_buffercache.so' /bin/sh ../../../config/install-sh -c -m 644 ./pg_buffercache.control '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension/' /bin/sh ../../../config/install-sh -c -m 644 ./pg_buffercache--1.0.sql ./pg_buffercache--unpackaged--1.0.sql '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension/' $ psql -c create extension pg_buffercache CREATE EXTENSION The only clunky bit I wasn't really happy with is the amount of code duplication that comes from having a src/extension/Makefile that looks almost, but not quite, identical to contrib/Makefile. The rest of the changes don't seem too bad to me, and even that's really only 36 lines that aren't touched often. Yes, the paths are different, so backports won't happen without an extra step. But the code changes required were easier than I was expecting, due to the general good modularity of the extensions infrastructure. So long as the result ends up in share/postgresql/extension/ , whether they started in contrib/module or src/extension/module doesn't really matter to CREATE EXTENSION. But having them broke out this way makes it easy for the default Makefile to build and install them all. (I recognize I didn't do that last step yet though) I'll happily go covert pgstattuple and the rest of the internal diagnostics modules to this scheme, and do the doc cleanups, this upcoming week if it means I'll be able to use those things without installing all of contrib one day. Ditto for proposing RPM and Debian packaging changes that match them. All that work will get paid back the first time I don't have to fill out a bunch of paperwork (again) at a customer site justifying why they need to install the contrib [RPM|deb] package (which has some scary stuff in it) on all their servers, just so I can get some bloat or buffer inspection module. I would really like to see us try to group things by topic, and not just by whether or not we can all agree that the extension is important enough to be first-class (which is bound to be a bit tendentious). We probably can't completely avoid some bikeshedding on that topic, but even there it strikes me that sorting by topic might make things a bit more clear. For example, if we could somehow group together all the diagnostic tools, maybe something like the list below, I think that would be a start. Now then we might go on to argue about which are the more useful diagnostic tools, but I think it's easier to argue about that category than it is to argue in the abstract about whether you'd rather have hstore or pgstattuple, to which the answer can only be that depends. auto_explain oid2name pageinspect pg_buffercache pg_freespacemap pg_stat_statements pg_test_fsync (perhaps) pgrowlocks pgstattuple -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Mon, May 9, 2011 at 1:14 PM, Greg Smith g...@2ndquadrant.com wrote: On 05/09/2011 10:53 AM, Robert Haas wrote: I would really like to see us try to group things by topic, and not just by whether or not we can all agree that the extension is important enough to be first-class (which is bound to be a bit tendentious). Having played around with the prototype, I think it doesn't actually matter if there's a further division below the new one I introduced. The main thing I think is worth pointing out is that I only feel extensions with no external dependencies are worth the trouble of re-classifying here. If it were worth reorganizing contrib just for the sake of categorizing it, that would have been done years ago. The new thing is that extensions make it really easy to make some tools available in the server's extensions subdirectly, without actually activating them in the default install. Looking at your list: auto_explain oid2name pageinspect pg_buffercache pg_freespacemap pg_stat_statements pg_test_fsync (perhaps) pgrowlocks pgstattuple oid2name and pg_test_fsync would be out because those are real executables. I'd rather not introduce the risk/complexity of playing around with moving standalone utilities of such marginal value. Whereas I think it sets an excellent precedent if the server is shipping with some standard add-ons, built using the same extension mechanism available to external code, in the core server package. I'd certainly be happy to add auto_explain and pg_stat_statements (also extremely popular things to install for me) to that list. I'm happy enough with that set of guidelines: namely, that we'd use src/extension only for things that don't require additional dependencies, and not for things that build standalone executables. If we're going to move things around, I think we should take the trouble to categorize them along the way, and your idea of inserting one more subdirectory under src/extension for grouping seems fine to me. I don't think we should be too obstinate about trying to twist the arm of packagers who (as Tom points out) will do whatever they want in spite of us, but the current state of contrib, with all sorts of things of varying type, complexity, and value mixed together cannot possibly be a good thing. Even if the effect of all this is that some distributions end up with postgresql-server-instrumentation and postgresql-server-datatypes packages, rather than putting everything in postgresql-server, I still think that'd be better than having a monolithic lump called postgresql-contrib. Heaven only knows what could be in there (says the sys admin)... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Excerpts from Robert Haas's message of lun may 09 14:31:33 -0400 2011: I'm happy enough with that set of guidelines: namely, that we'd use src/extension only for things that don't require additional dependencies, and not for things that build standalone executables. If we're going to move things around, I think we should take the trouble to categorize them along the way, and your idea of inserting one more subdirectory under src/extension for grouping seems fine to me. For executables we already have src/bin. Do we really need a separate place for, say, pg_standby or pg_upgrade? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of lun may 09 14:31:33 -0400 2011: I'm happy enough with that set of guidelines: namely, that we'd use src/extension only for things that don't require additional dependencies, and not for things that build standalone executables. If we're going to move things around, I think we should take the trouble to categorize them along the way, and your idea of inserting one more subdirectory under src/extension for grouping seems fine to me. For executables we already have src/bin. Do we really need a separate place for, say, pg_standby or pg_upgrade? Putting them in there implies we think they are of core-code quality. I'm definitely *not* ready to grant that status to pg_upgrade, for instance. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Tom Lane t...@sss.pgh.pa.us writes: Sure, but that's a documentation issue, which again is not going to be helped by a source-tree rearrangement. So we have several problem to solve here, and I agree that source code rearrangement is fixing none of them. Maybe it would ease maintaining down the road, though, but I'll leave that choice up to you. Which contribs are ready (safe) for production? We could handle that in the version numbers, having most of contrib at version 0.9.1 (say) and some of them at version 1.0. We could also stop distributing examples in binary form, only ship them in source package. Then we need to include inspection extensions into the core packaging, but still as extensions. That's more of a packager problem, except that they need a clear and strong message about it. Maybe have a new Makefile and build those extensions as part of the server build, and install them as part as the normal install. Another mix of those ideas is to ship inspection extensions and ready for production ones all into a new package postgresql-server-extensions that the main server would depend on, and the ones that are adding more dependencies still in contrib, where not-ready for production extensions would not get built. This kind of ideas would also allow to quite easily remove things from the main server and have them as extensions, available on any install but a CREATE EXTENSION (WITH SCHEMA pg_catalog) command away. And still maintained at the same quality level in the source tree. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/09/2011 03:31 PM, Alvaro Herrera wrote: For executables we already have src/bin. Do we really need a separate place for, say, pg_standby or pg_upgrade? There's really no executables in contrib that I find myself regularly desperate for/angry at because they're not installed as an integral part of the server, the way I regularly find myself cursing some things that are now extensions. The only contrib executable I use often is pgbench, and that's normally early in server life--when it's still possible to get things installed easily most places. And it's something that can be removed when finished, in cases where people are nervous about the contrib package. Situations where pg_standby or pg_upgrade suddenly pop up as emergency needs seem unlikely too, which is also the case with oid2name, pg_test_fsync, pg_archivecleanup, and vacuumlo. I've had that happen with pg_archivecleanup exactly once since it appeared--running out of space and that was the easiest way to make the problem go away immediately and permanently--but since it was on an 8.4 server we had to download the source and build anyway. Also, my experience is that people are not that paranoid about running external binaries, even though they could potentially do harm to the database. Can always do a backup beforehand. But the idea of loading a piece of code that lives in the server all the time freaks them out. The way the word contrib implies (and sometimes is meant to mean) low quality, while stuff that ships with the main server package does not, has been beaten up here for years already. It's only a few cases where that's not fully justified, and the program can easily become an extension, that I feel are really worth changing here. There are 49 directories in contrib/ ; at best maybe 20% of them will ever fall into that category. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/09/2011 02:31 PM, Robert Haas wrote: I don't think we should be too obstinate about trying to twist the arm of packagers who (as Tom points out) will do whatever they want in spite of us, but the current state of contrib, with all sorts of things of varying type, complexity, and value mixed together cannot possibly be a good thing. I think the idea I'm running with for now means that packagers won't actually have to do anything. I'd expect typical packaging for 9.1 to include share/postgresql/extension from the build results without being more specific. You need to grab 3 files from there to get the plpgsql extension, and I can't imagine any packager listing them all by name. So if I dump the converted contrib extensions to put files under there, and remove them from the contrib build area destination, I suspect they will magically jump from the contrib to the extensions area of the server package at next package build; no packager level changes required. The more I look at this, the less obtrusive of a change it seems to be. Only people who will really notice are users who discover more in the basic server package, and of course committers with backporting to do. Since packaged builds of 9.1 current with beta1 seem to be in short supply still, this theory looks hard to prove just yet. I'm very excited that it's packaging week here however (rolls eyes), so I'll check it myself. I'll incorporate the suggestions made since I posted that test patch and do a bigger round of this next, end to end with an RPM set as the output. It sounds like everyone who has a strong opinion on what this change might look like has sketched a similar looking bikeshed. Once a reasonable implementation is hammered out, I'd rather jump to the big argument between not liking change vs. the advocacy benefits to PostgreSQL of doing this; they are considerable in my mind. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On lör, 2011-05-07 at 17:38 -0400, Andrew Dunstan wrote: On 05/07/2011 05:26 PM, Peter Eisentraut wrote: On lör, 2011-05-07 at 17:16 -0400, Andrew Dunstan wrote: pg_config is useful quite apart from its use in building things, as was discussed upthread. Link please. http://archives.postgresql.org/pgsql-hackers/2011-05/msg00275.php That thread just asserts that it might be useful, and I responded by asking for what. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/08/2011 05:24 AM, Peter Eisentraut wrote: On lör, 2011-05-07 at 17:38 -0400, Andrew Dunstan wrote: On 05/07/2011 05:26 PM, Peter Eisentraut wrote: On lör, 2011-05-07 at 17:16 -0400, Andrew Dunstan wrote: pg_config is useful quite apart from its use in building things, as was discussed upthread. Link please. http://archives.postgresql.org/pgsql-hackers/2011-05/msg00275.php That thread just asserts that it might be useful, and I responded by asking for what. As I said there: to see how the libraries are configured, for example. Just the other day I wanted to know what compilation options had been used for a particular installation. pg_config wasn't installed because the -devel package wasn't installed, and it would have saved me quite some time if pg_config had been available. Another example is to find out what the installation is using for shares, the service directory and so on. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On sön, 2011-05-08 at 07:21 -0400, Andrew Dunstan wrote: As I said there: to see how the libraries are configured, for example. Just the other day I wanted to know what compilation options had been used for a particular installation. pg_config wasn't installed because the -devel package wasn't installed, and it would have saved me quite some time if pg_config had been available. Another example is to find out what the installation is using for shares, the service directory and so on. Yeah, those are decent reasons. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
My example is of doing self-discovery to see if all needful database components seem to be properly installed. E.g. - the app needs pgcrypto, intarray, and a custom data type. The install script can consequently inform the production folk either looks good, or, alternately, seems problematic! Actually, I haven't coded a sample of the look for custom SPI types part, but it's a natural extension of what I have. Of course, it only provides a legitimate test when run on the database server, which isn't always how production folk want to do it, but that's part of a different argument...
Re: [HACKERS] Why not install pgstattuple by default?
Christopher Browne cbbro...@gmail.com writes: I don't expect the extension system to help with any of this, since if production folk try to install minimal sets of packages, they're liable to consciously exclude extension support. The improvement would come from drawing contrib a bit closer to core, and encouraging packagers (dpkg, rpm, ports) to fold contrib into base rather than separating it. I'm sure that would get some pushback, though. What I think the next step is here is to better classify contribs/ into what we consider production ready core extension (adminpack, hstore, ltree, pgstattuple, you name it — that's the trick), code example (spi, some more I guess) and extensibility examples (for hooks or whatnot). We've been talking about renaming contrib for a long time, but that will not cut it. Classifying it and agreeing to maintain some parts of it the same way we maintain the core is what's asked here. Is there a will to go there? If there's a will to maintain some contribs the way core is maintained itself, we have to pick a new name for that, and to pick a list of current contribs to move in there. Then packagers will either include that in the main package or have the main package depend on the new one. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Sat, May 7, 2011 at 8:54 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: We've been talking about renaming contrib for a long time, but that will not cut it. Classifying it and agreeing to maintain some parts of it the same way we maintain the core is what's asked here. Is there a will to go there? I'm game. I doubt it'll be a lot more maintenance; we already pretty much patch contrib when we patch everything else. It'll just be easier to sort the wheat from the chaff. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, 06 May 2011 20:06:04 -, Tom Lane t...@sss.pgh.pa.us wrote: Bundling pg_config into a -libs package is probably not going to happen, at least not on Red Hat systems, because it would create multilib issues (ie, you're supposed to be able to install 32-bit and 64-bit libraries concurrently, but there's noplace to put a /usr/bin file without causing a conflict). There is no separate directory for 64bit binaries like /usr/bin and /usr/bin/amd64 on Solaris? And even in the absence of such a convention I'd still like to have both 32bit and 64bit binaries of the server/client available. I have a feeling this topic is digressing a bit. -- Johann Oskarssonhttp://www.2ndquadrant.com/|[] PostgreSQL Development, 24x7 Support, Training and Services --+-- | Blog: http://my.opera.com/myrkraverk/blog/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Greg Stark gsst...@mit.edu writes: On Fri, May 6, 2011 at 11:32 PM, Greg Smith g...@2ndquadrant.com wrote: I use pgstattuple, pageinspect, pg_freespacemap, and pg_buffercache regularly enough that I wish they were more common. Throw in pgrowlocks and you've got the whole group Robert put into the debug set. It makes me sad every time I finish a utility using one of these and realize I'll have to include the whole make sure you have the contrib modules installed disclaimer in its documentation again. Well the lightweight way to achieve what you want is to just move these functions into core. I'm completely not in favor of that. We have spent man-years upon man-years on making Postgres an extensible system. If we can't actually *use* the extension features then that was all a waste of effort. If anything I'd rather see us looking at what parts of the current core system could be pushed out to extensions. The geometric types are a pretty obvious candidate, for example. The only argument I see as particularly frightening on that front is people playing the sekurity card. Yeah, and it's a reasonable argument. Even if it's not a reasonable argument, you won't win any friends from the other side of the fence by taking away their ability to choose. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On fre, 2011-05-06 at 14:32 -0400, Greg Smith wrote: Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. Do you need pg_config to install extensions? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Em 07-05-2011 13:42, Peter Eisentraut escreveu: Do you need pg_config to install extensions? No. But we need it to build other extensions. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On lör, 2011-05-07 at 17:35 -0300, Euler Taveira de Oliveira wrote: Em 07-05-2011 13:42, Peter Eisentraut escreveu: Do you need pg_config to install extensions? No. But we need it to build other extensions. But for that you need the -dev[el] package anyway, so there would be no point in moving pg_config out of it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/07/2011 12:42 PM, Peter Eisentraut wrote: On fre, 2011-05-06 at 14:32 -0400, Greg Smith wrote: Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. Do you need pg_config to install extensions? No, but you still need it to build them. PGXN is a source code distribution method, not a binary one. It presumes users can build modules they download using PGXS. No pg_config, no working PGXS, no working PGXN. For such a small binary to ripple out to that impact is bad. The repmgr program we distribute has the same problem, so I've been getting first-hand reports of just how many people are likely to run into this recently. You have to install postgresql-devel with RPM and on Debian, the very non-obvious postgresql-server-dev-$version Anyway, didn't want to hijack this thread beyond pointing out that if there any package reshuffling that happens for contrib changes, it should check for and resolve this problem too. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/07/2011 04:43 PM, Peter Eisentraut wrote: On lör, 2011-05-07 at 17:35 -0300, Euler Taveira de Oliveira wrote: Em 07-05-2011 13:42, Peter Eisentraut escreveu: Do you need pg_config to install extensions? No. But we need it to build other extensions. But for that you need the -dev[el] package anyway, so there would be no point in moving pg_config out of it. pg_config is useful quite apart from its use in building things, as was discussed upthread. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On lör, 2011-05-07 at 17:16 -0400, Andrew Dunstan wrote: pg_config is useful quite apart from its use in building things, as was discussed upthread. Link please. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On lör, 2011-05-07 at 17:06 -0400, Greg Smith wrote: The repmgr program we distribute has the same problem, so I've been getting first-hand reports of just how many people are likely to run into this recently. You have to install postgresql-devel with RPM and on Debian, the very non-obvious postgresql-server-dev-$version I'm pretty sure that for installing modules that need compilation from CPAN or PyPI, you will need the corresponding p*-dev* package installed. Nothing new here. I don't think you will get distributors to abandon the concept of -dev(el) packages for this. Just saying. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/07/2011 05:26 PM, Peter Eisentraut wrote: On lör, 2011-05-07 at 17:16 -0400, Andrew Dunstan wrote: pg_config is useful quite apart from its use in building things, as was discussed upthread. Link please. http://archives.postgresql.org/pgsql-hackers/2011-05/msg00275.php cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/06/2011 04:06 PM, Tom Lane wrote: FWIW, I did move pg_config from -devel to the main (really client) postgresql package in Fedora, as of 9.0. That will ensure it's present in either client or server installations. Eventually that packaging will reach RHEL ... We should make sure that the PGDG packages adopt that for 9.1 then, so it starts catching on more. Unless Devrim changed to catch up since I last installed an RPM set, in that 9.0 it's still in the same place: $ rpm -qf /usr/pgsql-9.0/bin/pg_config postgresql90-devel-9.0.2-2PGDG.rhel5 While Peter's question about whether it's really all that useful is reasonable, I'd at least like to get a better error message when you don't have everything needed to compile extensions. I think the shortest path to that is making pg_config more likely to be installed, then to check whether the file pg_config --pgxs references exists. I'll see if I can turn that idea into an actual change to propose. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Attached patch is a first cut at what moving one contrib module (in this case pg_buffercache) to a new directory structure might look like. The idea is that src/extension could become a place for first-class extensions to live. Those are ones community is committed to providing in core, but are just better implemented as extensions than in-database functions, for reasons that include security. This idea has been shared by a lot of people for a while, only problem is that it wasn't really practical to implement cleanly until the extensions code hit. I think it is now, this attempts to prove it. Since patches involving file renaming are clunky, the changes might be easier to see at https://github.com/greg2ndQuadrant/postgres/commit/507923e21e963c873a84f1b850d64e895776574f where I just pushed this change too. The install step for the modules looks like this now: gsmith@grace:~/pgwork/src/move-contrib/src/extension/pg_buffercache$ make install /bin/mkdir -p '/home/gsmith/pgwork/inst/move-contrib/lib/postgresql' /bin/mkdir -p '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension' /bin/sh ../../../config/install-sh -c -m 755 pg_buffercache.so '/home/gsmith/pgwork/inst/move-contrib/lib/postgresql/pg_buffercache.so' /bin/sh ../../../config/install-sh -c -m 644 ./pg_buffercache.control '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension/' /bin/sh ../../../config/install-sh -c -m 644 ./pg_buffercache--1.0.sql ./pg_buffercache--unpackaged--1.0.sql '/home/gsmith/pgwork/inst/move-contrib/share/postgresql/extension/' $ psql -c create extension pg_buffercache CREATE EXTENSION The only clunky bit I wasn't really happy with is the amount of code duplication that comes from having a src/extension/Makefile that looks almost, but not quite, identical to contrib/Makefile. The rest of the changes don't seem too bad to me, and even that's really only 36 lines that aren't touched often. Yes, the paths are different, so backports won't happen without an extra step. But the code changes required were easier than I was expecting, due to the general good modularity of the extensions infrastructure. So long as the result ends up in share/postgresql/extension/ , whether they started in contrib/module or src/extension/module doesn't really matter to CREATE EXTENSION. But having them broke out this way makes it easy for the default Makefile to build and install them all. (I recognize I didn't do that last step yet though) I'll happily go covert pgstattuple and the rest of the internal diagnostics modules to this scheme, and do the doc cleanups, this upcoming week if it means I'll be able to use those things without installing all of contrib one day. Ditto for proposing RPM and Debian packaging changes that match them. All that work will get paid back the first time I don't have to fill out a bunch of paperwork (again) at a customer site justifying why they need to install the contrib [RPM|deb] package (which has some scary stuff in it) on all their servers, just so I can get some bloat or buffer inspection module. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us diff --git a/contrib/Makefile b/contrib/Makefile index 6967767..04cf330 100644 --- a/contrib/Makefile +++ b/contrib/Makefile @@ -30,7 +30,6 @@ SUBDIRS = \ pageinspect \ passwordcheck \ pg_archivecleanup \ - pg_buffercache \ pg_freespacemap \ pg_standby \ pg_stat_statements \ diff --git a/contrib/pg_buffercache/Makefile b/contrib/pg_buffercache/Makefile deleted file mode 100644 index 323c0ac..000 --- a/contrib/pg_buffercache/Makefile +++ /dev/null @@ -1,18 +0,0 @@ -# contrib/pg_buffercache/Makefile - -MODULE_big = pg_buffercache -OBJS = pg_buffercache_pages.o - -EXTENSION = pg_buffercache -DATA = pg_buffercache--1.0.sql pg_buffercache--unpackaged--1.0.sql - -ifdef USE_PGXS -PG_CONFIG = pg_config -PGXS := $(shell $(PG_CONFIG) --pgxs) -include $(PGXS) -else -subdir = contrib/pg_buffercache -top_builddir = ../.. -include $(top_builddir)/src/Makefile.global -include $(top_srcdir)/contrib/contrib-global.mk -endif diff --git a/contrib/pg_buffercache/pg_buffercache--1.0.sql b/contrib/pg_buffercache/pg_buffercache--1.0.sql deleted file mode 100644 index 9407d21..000 --- a/contrib/pg_buffercache/pg_buffercache--1.0.sql +++ /dev/null @@ -1,17 +0,0 @@ -/* contrib/pg_buffercache/pg_buffercache--1.0.sql */ - --- Register the function. -CREATE FUNCTION pg_buffercache_pages() -RETURNS SETOF RECORD -AS 'MODULE_PATHNAME', 'pg_buffercache_pages' -LANGUAGE C; - --- Create a view for convenient access. -CREATE VIEW pg_buffercache AS - SELECT P.* FROM pg_buffercache_pages() AS P - (bufferid integer, relfilenode oid, reltablespace oid, reldatabase oid, - relforknumber int2, relblocknumber int8, isdirty bool, usagecount int2); - --- Don't want these to be available to public. -REVOKE ALL ON
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 00:34, Josh Berkus josh.ber...@pgexperts.com wrote: Hackers, I've run into a couple of occasions lately where I really wanted pgstattuple on a production server in order to check table/index bloat. However, in the production environment at a large site installing a contrib module can involve a process which takes days or weeks. That can be said for a lot of things in contrib. pg_standby in 8.4 for example. Or adminpack. Or dblink. Or hstore. There's a mix of example stuff and actually pretty darn useful in production stuff. I'm sure you can find a couple of hundred emails in the archives on this very topic. From 9.1, it'll be a simple CREATE EXTENSION command - so much of the problem goes away. Well. It doesn't go away, but it gets a lot more neatly swept under the rug. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Em 06-05-2011 05:06, Magnus Hagander escreveu: On Fri, May 6, 2011 at 00:34, Josh Berkusjosh.ber...@pgexperts.com wrote: Hackers, I've run into a couple of occasions lately where I really wanted pgstattuple on a production server in order to check table/index bloat. However, in the production environment at a large site installing a contrib module can involve a process which takes days or weeks. I already faced that problem too. From 9.1, it'll be a simple CREATE EXTENSION command - so much of the problem goes away. Well. It doesn't go away, but it gets a lot more neatly swept under the rug. That's half of the history. Admin needs to install postgresql-contrib package. Sometimes it takes too much time to convince clients that some additional supplied modules are useful for them. Now that we have extensions, why not build and package the contrib modules by default? 'make world' is not the answer. There is not an option for install all pieces of software. Let's install pg+contrib and leave only 'CREATE EXTENSION foo' for the admins. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 18:22, Euler Taveira de Oliveira eu...@timbira.com wrote: Em 06-05-2011 05:06, Magnus Hagander escreveu: On Fri, May 6, 2011 at 00:34, Josh Berkusjosh.ber...@pgexperts.com wrote: Hackers, I've run into a couple of occasions lately where I really wanted pgstattuple on a production server in order to check table/index bloat. However, in the production environment at a large site installing a contrib module can involve a process which takes days or weeks. I already faced that problem too. From 9.1, it'll be a simple CREATE EXTENSION command - so much of the problem goes away. Well. It doesn't go away, but it gets a lot more neatly swept under the rug. That's half of the history. Admin needs to install postgresql-contrib package. Sometimes it takes too much time to convince clients that some additional supplied modules are useful for them. Now that we have extensions, why not build and package the contrib modules by default? 'make world' is not the answer. There is not an option for install all pieces of software. Let's install pg+contrib and leave only 'CREATE EXTENSION foo' for the admins. That's mostly an issue to be solved by the packagers. Some contrib modules add dependencies, but those that don't could easily be packaged in the main server package. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 1:32 PM, Magnus Hagander mag...@hagander.net wrote: On Fri, May 6, 2011 at 18:22, Euler Taveira de Oliveira eu...@timbira.com wrote: Em 06-05-2011 05:06, Magnus Hagander escreveu: On Fri, May 6, 2011 at 00:34, Josh Berkusjosh.ber...@pgexperts.com wrote: Hackers, I've run into a couple of occasions lately where I really wanted pgstattuple on a production server in order to check table/index bloat. However, in the production environment at a large site installing a contrib module can involve a process which takes days or weeks. I already faced that problem too. From 9.1, it'll be a simple CREATE EXTENSION command - so much of the problem goes away. Well. It doesn't go away, but it gets a lot more neatly swept under the rug. That's half of the history. Admin needs to install postgresql-contrib package. Sometimes it takes too much time to convince clients that some additional supplied modules are useful for them. Now that we have extensions, why not build and package the contrib modules by default? 'make world' is not the answer. There is not an option for install all pieces of software. Let's install pg+contrib and leave only 'CREATE EXTENSION foo' for the admins. That's mostly an issue to be solved by the packagers. Some contrib modules add dependencies, but those that don't could easily be packaged in the main server package. It seems to me that there's something of a packaging policy question to this. A long time ago, on a pre-buildfarm planet, far, far away, it was pretty uncertain what contrib modules could be hoped to run on what platform. At Afilias, we used to have to be *really* picky, because the subset that ran on Solaris and AIX were not even close to all of them. pgstattuples *was* one that the DBAs always wanted, but what would compile was alway hit-and-miss. Once we got AIX running a buildfarm node, that led to getting *ALL* of contrib working there, and I'm pretty sure that similar happened with other platforms at around the same time (I'm thinking this was 7.4, but it might have been 8.0) Be that all as it may, there has been a sea change, where we have moved from sporadic usability of contrib to it being *continually* tested on *all* buildfarm platforms, which certainly adds to the confidence level. But people are evidently still setting packaging policies based on how things were back in 7.3, even though that perhaps isn't necessary anymore. Certainly it's not a huge amount of code; less than 2MB these days. - % wc `dpkg -L postgresql-contrib-9.0` | tail -1 15952 67555 1770987 total I'm getting paper cuts quite a bit these days over the differences between what different packaging systems decide to install. The one *I* get notably bit on, of late, is that I have written code that expects to have pg_config to do some degree of self-discovery, only to find production folk complaining that they only have psql available in their environment. I don't expect the extension system to help with any of this, since if production folk try to install minimal sets of packages, they're liable to consciously exclude extension support. The improvement would come from drawing contrib a bit closer to core, and encouraging packagers (dpkg, rpm, ports) to fold contrib into base rather than separating it. I'm sure that would get some pushback, though. -- When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/06/2011 01:55 PM, Christopher Browne wrote: Once we got AIX running a buildfarm node, that led to getting *ALL* of contrib working there, and I'm pretty sure that similar happened with other platforms at around the same time (I'm thinking this was 7.4, but it might have been 8.0) FYI, the buildfarm started in late 2004, near the end of the 8.0 development cycle. It quickly led to a number of contrib fixes. Time flies when you're having fun ... cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Em 06-05-2011 14:55, Christopher Browne escreveu: The improvement would come from drawing contrib a bit closer to core, and encouraging packagers (dpkg, rpm, ports) to fold contrib into base rather than separating it. I'm sure that would get some pushback, though. I'm in favor of find out what are the popular extensions and make them into base; the other ones could be moved to PGXN. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Christopher Browne wrote: I'm getting paper cuts quite a bit these days over the differences between what different packaging systems decide to install. The one *I* get notably bit on, of late, is that I have written code that expects to have pg_config to do some degree of self-discovery, only to find production folk complaining that they only have psql available in their environment. Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 2:32 PM, Greg Smith g...@2ndquadrant.com wrote: Christopher Browne wrote: I'm getting paper cuts quite a bit these days over the differences between what different packaging systems decide to install. The one *I* get notably bit on, of late, is that I have written code that expects to have pg_config to do some degree of self-discovery, only to find production folk complaining that they only have psql available in their environment. Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. I'm agreeable to that, in general. If there's a server package and a client package, it likely only fits with the server package. On a host where only the client is installed, they won't be able to install extensions, so it's pretty futile to have it there. -- When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/06/2011 03:14 PM, Christopher Browne wrote: On Fri, May 6, 2011 at 2:32 PM, Greg Smithg...@2ndquadrant.com wrote: Christopher Browne wrote: I'm getting paper cuts quite a bit these days over the differences between what different packaging systems decide to install. The one *I* get notably bit on, of late, is that I have written code that expects to have pg_config to do some degree of self-discovery, only to find production folk complaining that they only have psql available in their environment. Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. I'm agreeable to that, in general. If there's a server package and a client package, it likely only fits with the server package. On a host where only the client is installed, they won't be able to install extensions, so it's pretty futile to have it there. I don't agree. It can be useful even there, to see how the libraries are configured, for example. I'd be inclined to bundle it with postgresql-libs or the moral equivalent. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 21:19, Andrew Dunstan and...@dunslane.net wrote: On 05/06/2011 03:14 PM, Christopher Browne wrote: On Fri, May 6, 2011 at 2:32 PM, Greg Smithg...@2ndquadrant.com wrote: Christopher Browne wrote: I'm getting paper cuts quite a bit these days over the differences between what different packaging systems decide to install. The one *I* get notably bit on, of late, is that I have written code that expects to have pg_config to do some degree of self-discovery, only to find production folk complaining that they only have psql available in their environment. Given the other improvements in being able to build extensions in 9.1, we really should push packagers to move pg_config from the PostgreSQL development package into the main one starting in that version. I've gotten bit by this plenty of times. I'm agreeable to that, in general. If there's a server package and a client package, it likely only fits with the server package. On a host where only the client is installed, they won't be able to install extensions, so it's pretty futile to have it there. I don't agree. It can be useful even there, to see how the libraries are configured, for example. I'd be inclined to bundle it with postgresql-libs or the moral equivalent. +1. And it's not like it wastes huge amount of space... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Magnus Hagander mag...@hagander.net writes: On Fri, May 6, 2011 at 21:19, Andrew Dunstan and...@dunslane.net wrote: On 05/06/2011 03:14 PM, Christopher Browne wrote: If there's a server package and a client package, it likely only fits with the server package. On a host where only the client is installed, they won't be able to install extensions, so it's pretty futile to have it there. I don't agree. It can be useful even there, to see how the libraries are configured, for example. I'd be inclined to bundle it with postgresql-libs or the moral equivalent. +1. Well, actually, I think packagers have generally put it into a -devel subpackage. If it were in either a server or client package there would be much less of an issue. Bundling pg_config into a -libs package is probably not going to happen, at least not on Red Hat systems, because it would create multilib issues (ie, you're supposed to be able to install 32-bit and 64-bit libraries concurrently, but there's noplace to put a /usr/bin file without causing a conflict). FWIW, I did move pg_config from -devel to the main (really client) postgresql package in Fedora, as of 9.0. That will ensure it's present in either client or server installations. Eventually that packaging will reach RHEL ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/06/2011 04:06 PM, Tom Lane wrote: Magnus Hagandermag...@hagander.net writes: On Fri, May 6, 2011 at 21:19, Andrew Dunstanand...@dunslane.net wrote: On 05/06/2011 03:14 PM, Christopher Browne wrote: If there's a server package and a client package, it likely only fits with the server package. On a host where only the client is installed, they won't be able to install extensions, so it's pretty futile to have it there. I don't agree. It can be useful even there, to see how the libraries are configured, for example. I'd be inclined to bundle it with postgresql-libs or the moral equivalent. +1. Well, actually, I think packagers have generally put it into a -devel subpackage. If it were in either a server or client package there would be much less of an issue. Bundling pg_config into a -libs package is probably not going to happen, at least not on Red Hat systems, because it would create multilib issues (ie, you're supposed to be able to install 32-bit and 64-bit libraries concurrently, but there's noplace to put a /usr/bin file without causing a conflict). FWIW, I did move pg_config from -devel to the main (really client) postgresql package in Fedora, as of 9.0. That will ensure it's present in either client or server installations. Eventually that packaging will reach RHEL ... That's reasonable, and certainly better than having it in -devel. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Christopher Browne cbbro...@gmail.com writes: But people are evidently still setting packaging policies based on how things were back in 7.3, even though that perhaps isn't necessary anymore. FWIW, once you get past the client versus server distinction, I think most subpackaging decisions are based on either the idea that only a minority of people will want this, or a desire to limit how many dependencies are pulled in by the main package(s). Both of those concerns apply to various subsets of -contrib, which means it's going to be hard to persuade packagers to fold -contrib into the -server package altogether. Nor would you gain their approval by trying to pre-empt the decision. We might get somewhere by trying to identify a small set of particularly popular contrib modules that don't add any extra dependencies, and then recommending to packagers that those ones get bundled into the main server package. Certainly it's not a huge amount of code; less than 2MB these days. - % wc `dpkg -L postgresql-contrib-9.0` | tail -1 15952 67555 1770987 total Well, to add some concrete facts rather than generalities to my own post, here are the sizes of the built RPMs from my last build for Fedora: -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 postgresql-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 postgresql-contrib-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 27337677 Apr 18 10:51 postgresql-debuginfo-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 961660 Apr 18 10:50 postgresql-devel-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 7569048 Apr 18 10:50 postgresql-docs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 246506 Apr 18 10:50 postgresql-libs-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl64940 Apr 18 10:50 postgresql-plperl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl65776 Apr 18 10:50 postgresql-plpython-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl45941 Apr 18 10:50 postgresql-pltcl-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 5302117 Apr 18 10:50 postgresql-server-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 1370509 Apr 18 10:50 postgresql-test-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 3644113 Apr 18 10:50 postgresql-upgrade-9.0.4-1.fc13.x86_64.rpm The separate debuginfo package is distro policy enforced by toolchain; I couldn't do anything about that even if I wanted to. The separate -libs subpackage is also hard to avoid because of distro policy about multilib installations. Separating devel support files (such as headers) is also standard practice. The other subdivisions are either my fault or those of my predecessors. plperl, plpython, and pltcl are split out for dependency reasons, ie to not have the -server package require you to install those languages and their respective ecosystems. I think the separation of the -docs, -test, and -upgrade subpackages is also pretty easy to defend on the grounds that they're big and not everyone wants 'em, especially not in production. That leaves us with these three subpackages about which there's room for argument: -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 postgresql-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 postgresql-contrib-9.0.4-1.fc13.x86_64.rpm -rw-r--r--. 1 tgl tgl 5302117 Apr 18 10:50 postgresql-server-9.0.4-1.fc13.x86_64.rpm Merging -contrib into the server package would increase the size of the latter by almost 10%, which is enough to bother people. Also, a bit of dependency extraction shows that -contrib has these dependencies beyond the ones in the two main packages: libcrypt.so.1 libossp-uuid.so.16 libxslt.so.1 That's not a particularly large list, I guess, but they're still the sorts of dependencies that don't win any friends when it's time to get the distro to fit on a DVD. Bottom line is that I'd rather have a smaller postgresql-server package that gets included in the shipping DVD than a complete one that gets kicked off because it's too large and pulls in too many other non-core dependencies. So, again, some selective migration of contrib modules into the main -server package might be doable, but the key word there is selective. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
All, We might get somewhere by trying to identify a small set of particularly popular contrib modules that don't add any extra dependencies, and then recommending to packagers that those ones get bundled into the main server package. Yeah, I wasn't thinking of including all of contrib. There's a lot of reasons not to do that. I was asking about pgstattuple in particular, since it's: (a) small (b) has no external dependancies (c) adds no stability risk or performance overhead (d) is usually needed on production systems when it's needed at all It's possible that we have one or two other diagnostic utilities which meet the above profile. pageinspect, maybe? The reason why this is such an issue is that for big users with high-demand production environments, installing any software, even postgresql-devel or postgresql-contrib packages, are big major IT deals which require weeks of advance scheduling. As a result, diagnostic tools from contrib tend not to be used because the problem they need to diagnose is much more urgent than that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 5:58 PM, Josh Berkus j...@agliodbs.com wrote: Yeah, I wasn't thinking of including all of contrib. There's a lot of reasons not to do that. Slightly off-topic, but I really think we would benefit from trying to divide up contrib. Right now it's a mixture of (a) debugging and instrumentation tools (e.g. pgstattuple, pageinspect, pgrowlocks, pg_freespacemap, pg_buffercache), (b) server functionality that is generally useful but considered worth including in core (e.g. hstore, citext, pg_trgm), (c) deprecated modules that we keep around mostly for hysterical reasons (tsearch2, xml2, intagg), and (d) examples and regression test support (dummy_seclabel, spi, start-scripts). I think it would make things a lot easier for both packagers and actual users if we separated these things into different directories, e.g.: debugging and instrumentation tools - src/debug server functionality - contrib server functionality (deprecated) - contrib/deprecated examples regression test suport - src/test/examples -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 05/06/2011 05:58 PM, Josh Berkus wrote: Yeah, I wasn't thinking of including all of contrib. There's a lot of reasons not to do that. I was asking about pgstattuple in particular, since it's: (a) small (b) has no external dependancies (c) adds no stability risk or performance overhead (d) is usually needed on production systems when it's needed at all It's possible that we have one or two other diagnostic utilities which meet the above profile. pageinspect, maybe? I use pgstattuple, pageinspect, pg_freespacemap, and pg_buffercache regularly enough that I wish they were more common. Throw in pgrowlocks and you've got the whole group Robert put into the debug set. It makes me sad every time I finish a utility using one of these and realize I'll have to include the whole make sure you have the contrib modules installed disclaimer in its documentation again. These are the only ones I'd care about moving into a more likely place. The rest of the contrib modules are the sort where if you need them, you realize that early and get them installed. These are different by virtue of their need popping up most often during emergencies. The fact that I believe they all match the low impact criteria too makes it even easier to consider. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Robert Haas robertmh...@gmail.com writes: On Fri, May 6, 2011 at 5:58 PM, Josh Berkus j...@agliodbs.com wrote: Yeah, I wasn't thinking of including all of contrib. There's a lot of reasons not to do that. Slightly off-topic, but I really think we would benefit from trying to divide up contrib. [ snip ] I think it would make things a lot easier for both packagers and actual users if we separated these things into different directories, e.g.: debugging and instrumentation tools - src/debug server functionality - contrib server functionality (deprecated) - contrib/deprecated examples regression test suport - src/test/examples From a packager's standpoint, that would be entirely worthless. The source tree's just a source tree, they don't care what lives where within it. I was just thinking about what it'd take to actually repackage things for Fedora, and the main problem is here: %files contrib ... %{_datadir}/pgsql/contrib/ ... If you're not adept at reading RPM specfiles, what that is saying is that everything that make install has stuck under ${prefix}/share/pgsql/contrib/ is to be included in the contrib RPM. To selectively move some stuff to the server RPM, I'd have to replace that one line with a file-by-file list of *everything* in share/contrib, and then move some of those lines to the %files server section, and then look forward to having to maintain that list in future versions. I'm already maintaining a file-by-file list of contrib's .so's, and I can tell you it's a PITA. As a packager, what I'd really want to see from a division into recommended and not-so-recommended packages is that they get installed into different subdirectories by make install. Then I could just point RPM at those directories and I'd be done. I don't know how practical this is from our development standpoint, nor from a user's standpoint --- I doubt we want to ask people to use different CREATE EXTENSION commands depending on the preferredness of the extension. A possibly workable compromise would be to provide two separate makefile installation targets for preferred and less preferred modules. The RPM script could then do something like make install-contrib-preferred ls -R .../sharedir contrib.files.for.server-package make install-contrib-second-class-citizens ls -R .../sharedir all.contrib.files ... and then some magic with comm to separate out the contrib ... files not mentioned in contrib.files.for.server-package ... Pretty grotty but it would work. Anyway my point is that this is all driven off the *installed* file tree. A specfile writer doesn't know nor want to know where make install is getting things from in the source tree. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On 5/6/11 3:19 PM, Robert Haas wrote: Slightly off-topic, but I really think we would benefit from trying to divide up contrib. I don't agree, unless by divide up you mean move several things to extensions. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
These are the only ones I'd care about moving into a more likely place. The rest of the contrib modules are the sort where if you need them, you realize that early and get them installed. These are different by virtue of their need popping up most often during emergencies. The fact that I believe they all match the low impact criteria too makes it even easier to consider. Yes, precisely. If I need intarray, I'm going to need it for a development push, which is planned well in advance. But if I need pageinspect, it's almost certainly because an emergency has arisen. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 11:32 PM, Greg Smith g...@2ndquadrant.com wrote: I use pgstattuple, pageinspect, pg_freespacemap, and pg_buffercache regularly enough that I wish they were more common. Throw in pgrowlocks and you've got the whole group Robert put into the debug set. It makes me sad every time I finish a utility using one of these and realize I'll have to include the whole make sure you have the contrib modules installed disclaimer in its documentation again. Well the lightweight way to achieve what you want is to just move these functions into core. There's a pretty good argument to be made for debugging tools being considered an integral part of a base system. I remember making the same argument when Sun first made the radical move for a Unix vendor to stop shipping a working C compiler and debugger as part of the base Solaris packages. The only argument I see as particularly frightening on that front is people playing the sekurity card. A naive attacker who obtains access to the postgres account could do more damage than they might be able to do without these modules installed. Of course an attacker with postgres can do just about anything but it's not entirely baseless -- we don't set up the database with modules like plsh installed by default for example. The only actual security issue I can think of is that the pageinspect module would let users look at deleted records more easily. It would be pretty tricky, but not impossible, to do that without it. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 6:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: As a packager, what I'd really want to see from a division into recommended and not-so-recommended packages is that they get installed into different subdirectories by make install. Then I could just point RPM at those directories and I'd be done. Well, that might be good, too. But, right now, if someone pulls up our documentation, or our source tree, they could easily be forgiven for thinking that hstore and dummy_seclabel are comparable, and they aren't. I don't know how practical this is from our development standpoint, nor from a user's standpoint --- I doubt we want to ask people to use different CREATE EXTENSION commands depending on the preferredness of the extension. Certainly not. A possibly workable compromise would be to provide two separate makefile installation targets for preferred and less preferred modules. The RPM script could then do something like make install-contrib-preferred ls -R .../sharedir contrib.files.for.server-package make install-contrib-second-class-citizens ls -R .../sharedir all.contrib.files ... and then some magic with comm to separate out the contrib ... files not mentioned in contrib.files.for.server-package ... Pretty grotty but it would work. Anyway my point is that this is all driven off the *installed* file tree. A specfile writer doesn't know nor want to know where make install is getting things from in the source tree. This isn't any uglier than some other RPM hacks I've seen, and less ugly than some, but you'd have a better sense of that than I do. At any rate, having the various categories separated in the source tree can't possibly hurt the effort to make something like this work, and might make it somewhat easier. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
Robert Haas robertmh...@gmail.com writes: On Fri, May 6, 2011 at 6:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: As a packager, what I'd really want to see from a division into recommended and not-so-recommended packages is that they get installed into different subdirectories by make install. Well, that might be good, too. But, right now, if someone pulls up our documentation, or our source tree, they could easily be forgiven for thinking that hstore and dummy_seclabel are comparable, and they aren't. Sure, but that's a documentation issue, which again is not going to be helped by a source-tree rearrangement. As somebody who spends a lot of time on back-patching, I'm not excited in the least by suggestions to rearrange the source tree for marginal cosmetic benefits, which is all that I see here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why not install pgstattuple by default?
On Fri, May 6, 2011 at 9:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, May 6, 2011 at 6:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: As a packager, what I'd really want to see from a division into recommended and not-so-recommended packages is that they get installed into different subdirectories by make install. Well, that might be good, too. But, right now, if someone pulls up our documentation, or our source tree, they could easily be forgiven for thinking that hstore and dummy_seclabel are comparable, and they aren't. Sure, but that's a documentation issue, which again is not going to be helped by a source-tree rearrangement. I disagree - I think it would be helpful to rearrange both things. As somebody who spends a lot of time on back-patching, I'm not excited in the least by suggestions to rearrange the source tree for marginal cosmetic benefits, which is all that I see here. I understand, but we have back-patched only 32 patches that touch contrib into REL9_0_STABLE since its creation, of which 9 were done by you, and only 4 of those would have required adjustment under the separation criteria I proposed. I think, therefore, that the impact would be bearable. Source-code rearrangement is never going to be completely free, but that seems like a tolerable level of annoyance. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers