RE: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> I'm looking at some point in time in the future doing a > 'postgresql-upgrade' RPM that would include pre-built postmasters and > other binaries necessary to dump any previous version PostgreSQL (since > about 6.2.1 or so -- 6.2.1 was the first RedHat official PostgreSQL RPM, > although there were 6.1.1 RPM's before that, and there is still a > postgres95-1.09 RPM out there), linked to the current libs for that > RPM's OS release. It would be a large RPM (and the source RPM for it > would be _huge_, containing entire tarballs for at least 6.2.1, 6.3.2, > 6.4.2, 6.5.3, and 7.0.3). But, this may be the only way to make this > work barring a real migration utility. > How about instead of one huge utility that does everything for all combinations, you release specific utilities for different upgrades. Example, one RPM for 6.5.3 -> 7.0.3 and another RPM for 6.4.2 -> 7.0.2. It would result in more RPMs but they would be smaller and probably easier to maintain.
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen <[EMAIL PROTECTED]> writes: > All I'm saying is that regression should be _runnable_ in all modes > without needing anything but a shell and the PostgreSQL binary > installation. I think this'd be mostly a waste of effort. IMHO, 99% of the problems the regression tests might expose will be exposed if they are run against the RPMs by the RPM maker. (Something we have sometimes failed to do in the past ;-).) The regress tests are not that good at detecting environment-specific problems; in fact, they go out of their way to suppress environmental differences. So I don't see any strong need to support regression test running in binary distributions. Especially not if we have to kluge around a lack of essential tools. regards, tom lane
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian wrote: > > However, the dependency upon the new version of the OS being able to run > > the old executables could be a killer in the future if executable > > compatibility is removed -- after all, an upgrade might not be from the > > immediately prior version of the OS. > That is a tough one. I see your point. How would the RPM do this > anyway? It is running the same version of the OS right? Did they move > the data files from the old OS to the new OS and now they want to > upgrade? Hmm. Well, let's suppose the following: J. Random User has a database server that has been running smoothly for ages on RedHat 5.2, running PostgreSQL 6.3.2. He has had no reason to upgrade since -- while MVCC was a nice feature, he was really waiting for OUTER JOIN before upgrading, as his server is lightly loaded and won't benefit greatly from MVCC. Likewise, he's not upgraded from RedHat 5.2, because until RedHat got the 2.4 kernel into a distribution, he wasn't ready to upgrade, as he needs improved NFS performance, available in Linux kernel 2.4. And he wasn't about to go with a version of GCC that doesn't exist. So he skips the whole RedHat 6.x series -- he doesn't want to mess with kernel 2.2 in any form, thanks to its abyssmal NFS performance. So he waits on RedHat 7.2 to be released -- around October 2001 (if the typical RedHat schedule holds). At this point, PostgreSQL 7.2.1 is the standards bearer, with OUTER JOIN support that he craves, and robust WAL for excellent recoverability, amongst other Neat Features(TM). Now, by the time of RedHat 7.2, kernel 2.4 is up to .15 or so, with gcc 3.0 freshly (and officially) released, and glibc 2.2.5 finally fixing the problems that had plagued both pre-2.2 glibc's AND the earliest 2.2 glibc's -- but, the upshot is that glibc 2.0 compatibility is toast. Now, J Random slides in the new OS CD on a backup of his main server, and upgrades. RedHat 7.2's installer is very smart -- if no packages are left that use glibc 2.0, it doesn't install the compat-libs necessary for glibc 2.0 apps to run. The PostgreSQL RPMset's server subpackage preinstall script runs about two-thirds of the way through the upgrade, and backs up the old 6.3.2 executables necessary to pull a dump. The old 6.3.2 rpm database entries are removed, and, as far as the system is concerned, no dependency upon glibc 2.0 remains, so no compat-libs get installed. J Random checks out the new installation, and finds a conspicuous log message telling him to read /usr/share/doc/postgresql-7.2.1/README.rpm. He does so, and runs the (fixed by then) postgresql-dump script, which attempts to start an old backend and do a pg_dumpall -- but, horrors, the old postmaster can't start, glibc 2.0 is gone and glibc 2.2 blows core loaded under postmaster-6.3.2. ARGGHHH That's the scenario I have nightmares about. Really. > Yes, but if we added capabilities every time someone wanted something so > it worked better in their environment, this software would be a mess, > right? Yes, it would. I'll work on a patch, and we'll see what it looks like. > > been e-mails from users of other OS's -- even those that compile from > > source -- expressing a desire for a smoother upgrade cycle. The RPM's, > Yes, we all agree upgrades should be smoother. The problem is that the > cost/benefit analysis always pushed us away from improving it. I understand. I'm looking at some point in time in the future doing a 'postgresql-upgrade' RPM that would include pre-built postmasters and other binaries necessary to dump any previous version PostgreSQL (since about 6.2.1 or so -- 6.2.1 was the first RedHat official PostgreSQL RPM, although there were 6.1.1 RPM's before that, and there is still a postgres95-1.09 RPM out there), linked to the current libs for that RPM's OS release. It would be a large RPM (and the source RPM for it would be _huge_, containing entire tarballs for at least 6.2.1, 6.3.2, 6.4.2, 6.5.3, and 7.0.3). But, this may be the only way to make this work barring a real migration utility. > > And, BTW, welcome back from the summit. I heard that there was a little > > 'excitement' there :-). > Yes, it was very nice. I will post a summary to announce/general today. Good. And a welcome back to Tom as well, as he went too, IIRC. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen <[EMAIL PROTECTED]> writes: > Now, J Random slides in the new OS CD on a backup of his main server, > and upgrades. RedHat 7.2's installer is very smart -- if no packages > are left that use glibc 2.0, it doesn't install the compat-libs > necessary for glibc 2.0 apps to run. Actually, glibc is a bad example of things to break - it has versioned symbols, so postgresql is pretty likely to continue working (barring doing extremely low-level stuff, like doing weird things to the loader or depend on buggy behaviour (like Oracle did)). Postgresql doesn't use C++ either (which is a horrible mess wrt. binary compatibility - there is no such thing, FTTB). However, if it depended on kernel specific behaviour (like things in /proc, which may or may not have changed its output format) it could break. -- Trond Eivind Glomsrød Red Hat, Inc.
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> Bruce Momjian wrote: > > > At the risk of being redundant, here goes. As I've explained before, > > > the RPM upgrade environment, thanks to our standing with multiple > > > distributions as being shipped as a part of the OS, could be run as part > > > OK, maybe doing it in an RPM is the wrong way to go. If an old version > > exists, maybe the RPM is only supposed to install the software in a > > saved location, and the users must execute a command after the RPM > > install that starts the old postmaster, does the dump, puts the new > > PostgreSQL server in place, and reloads the database. > > That's more or less what's being done now. The RPM's preinstall script > (run before any files are overwritten from the new package) backs up the > required executables from the old installation. The RPM then overwrites > the necessary files, and then any old files left over are removed, along > with database removal of their records. > > A script (actually, two scripts due to a bug in the first one) is > provided to dump the database using the old executables. Which works OK > as long as the new OS release is executable compatible with the old > release. Oliver Elphick originally wrote the script for the Debian > packages, and I adapted it to the RPM environment. > > However, the dependency upon the new version of the OS being able to run > the old executables could be a killer in the future if executable > compatibility is removed -- after all, an upgrade might not be from the > immediately prior version of the OS. That is a tough one. I see your point. How would the RPM do this anyway? It is running the same version of the OS right? Did they move the data files from the old OS to the new OS and now they want to upgrade? Hmm. > > You are basically saying that because you can ship without a compiler > > sometimes, we are supposed to change the way our regression tests work. > > Let's suppose SCO says they don't ship with a compiler, and wants us to > > change our code to accomodate it. Should we? You can be certain we > > would not, and in the RPM case, you get the same answer. > > > If the patch is trivial, we will work around OS limitations, but we do > > not redesign code to work around OS limitations. We expect the OS to > > get the proper features. That is what we do with NT. Cygwin provides > > the needed features. > > No, I'm saying that someone running any OS might want to do this: > 1.) They have two machines, a development machine and a production > machine. Due to budget constraints, the dev machine is an el cheapo > version of the production machine (for the sake of argument, let's say > dev is a Sun Ultra 1 bare-bones workstation, and the production is a > high end SMP Ultra server, both running the same version of Solaris). Yes, but if we added capabilities every time someone wanted something so it worked better in their environment, this software would be a mess, right? > > This is the "Hey, other people can do it, why can't you" issue. We are > > looking for suggestions from Linux users in how this can be done. > > Perhaps running a separate command after the RPM has been installed is > > the only way to go. > > It's not really an RPM issue -- it's a PostgreSQL issue -- there have > been e-mails from users of other OS's -- even those that compile from > source -- expressing a desire for a smoother upgrade cycle. The RPM's, > Debian packages, and other binary packages just put the extant problem > in starker contrast. Until such occurs, I'll just have to continue > doing what I'm doing -- which I consider a stop-gap, not a solution. Yes, we all agree upgrades should be smoother. The problem is that the cost/benefit analysis always pushed us away from improving it. > > And, BTW, welcome back from the summit. I heard that there was a little > 'excitement' there :-). Yes, it was very nice. I will post a summary to announce/general today. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian wrote: > > At the risk of being redundant, here goes. As I've explained before, > > the RPM upgrade environment, thanks to our standing with multiple > > distributions as being shipped as a part of the OS, could be run as part > OK, maybe doing it in an RPM is the wrong way to go. If an old version > exists, maybe the RPM is only supposed to install the software in a > saved location, and the users must execute a command after the RPM > install that starts the old postmaster, does the dump, puts the new > PostgreSQL server in place, and reloads the database. That's more or less what's being done now. The RPM's preinstall script (run before any files are overwritten from the new package) backs up the required executables from the old installation. The RPM then overwrites the necessary files, and then any old files left over are removed, along with database removal of their records. A script (actually, two scripts due to a bug in the first one) is provided to dump the database using the old executables. Which works OK as long as the new OS release is executable compatible with the old release. Oliver Elphick originally wrote the script for the Debian packages, and I adapted it to the RPM environment. However, the dependency upon the new version of the OS being able to run the old executables could be a killer in the future if executable compatibility is removed -- after all, an upgrade might not be from the immediately prior version of the OS. > You are basically saying that because you can ship without a compiler > sometimes, we are supposed to change the way our regression tests work. > Let's suppose SCO says they don't ship with a compiler, and wants us to > change our code to accomodate it. Should we? You can be certain we > would not, and in the RPM case, you get the same answer. > If the patch is trivial, we will work around OS limitations, but we do > not redesign code to work around OS limitations. We expect the OS to > get the proper features. That is what we do with NT. Cygwin provides > the needed features. No, I'm saying that someone running any OS might want to do this: 1.) They have two machines, a development machine and a production machine. Due to budget constraints, the dev machine is an el cheapo version of the production machine (for the sake of argument, let's say dev is a Sun Ultra 1 bare-bones workstation, and the production is a high end SMP Ultra server, both running the same version of Solaris). 2.) For greater security, the production machine has been severely crippled WRT development tools -- if a cracker gets in, don't give him any ammunition. Good procedure to follow for publicly exposed database servers, like those that sit behind websites. Requiring such a server to have a development system installed is a misfeature, IMHO. 3.) After compiling and testing PostgreSQL on dev, the user transfers the binaries only over to production. All is well, at first. 4.) But then the load on production goes up -- and PostgreSQL starts spitting errors and FATAL's. The problem cannot be duplicated on the dev machine -- looks like a Solaris SMP issue. 5.) The user decides to run regression on production in parallel mode to help debug the problem -- but cannot figure out how to do so without installing make and other development tools on it, when he specifically did not want those tools on there for security. Serial regression, which is easily started in a no-make mode, doesn't expose the problem. All I'm saying is that regression should be _runnable_ in all modes without needing anything but a shell and the PostgreSQL binary installation. This is the problem -- it is not OS-specific. > > RPM's are expected to 'rpm -U' and you can simply _use_ the new version, > > with little to no preparation. At least that is the theory. And it > > works for most packages. > This is the "Hey, other people can do it, why can't you" issue. We are > looking for suggestions from Linux users in how this can be done. > Perhaps running a separate command after the RPM has been installed is > the only way to go. It's not really an RPM issue -- it's a PostgreSQL issue -- there have been e-mails from users of other OS's -- even those that compile from source -- expressing a desire for a smoother upgrade cycle. The RPM's, Debian packages, and other binary packages just put the extant problem in starker contrast. Until such occurs, I'll just have to continue doing what I'm doing -- which I consider a stop-gap, not a solution. And, BTW, welcome back from the summit. I heard that there was a little 'excitement' there :-). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> > Well, we certainly don't want to make changes that make things harder or > > more confusing for non-RPM installs. How are they affected here? > > They wouldn't be. Peter E has seemingly done an excellent job in this > area. I say seemingly because I haven't built an RPM from the 7.1 branch > yet, but from what he has posted, he seems to understand the issue. > Many thanks, Peter. OK, glad that is done. > > > * Upgrades that don't require an ASCII database dump for migration. This > > > can either be implemented as a program to do a pg_dump of an arbitrary > > > version of data, or as a binary migration utility. Currently, I'm > > > I really don't see the issue here. > > At the risk of being redundant, here goes. As I've explained before, > the RPM upgrade environment, thanks to our standing with multiple > distributions as being shipped as a part of the OS, could be run as part > of a general-purpose OS upgrade. In the environment of the general > purpose OS upgrade, the RPM's installation scripts cannot fire up a > backend, nor can it assume one is running or is not running, nor can the > RPM installation scripts fathom from the run-time environment whether > they are being run from a command line or from the OS upgrade (except on > Linux Mandrake, which allows such usage). OK, maybe doing it in an RPM is the wrong way to go. If an old version exists, maybe the RPM is only supposed to install the software in a saved location, and the users must execute a command after the RPM install that starts the old postmaster, does the dump, puts the new PostgreSQL server in place, and reloads the database. > > Well, no compiler? I can't see how we would do that without making > > other OS installs harder. That is really the core of the issue. We > > can't be making changes that make things harder for other OS's. Those > > have to be isolated in the RPM, or in some other middle layer. > > And I've done that in the past with the older serialized regression > tests. > > I don't see how executing a shell script instead of executing a make > command would make it any harder for other OS users. I am not trying to > make it harder for other OS users. I _am_ trying to make it easier for > users who are getting funny results from queries to be able to run > regression tests as a standardized way to see where the problem lies. > Maybe there is a hardware issue -- regression testing might be the only > way to have a standard way to pinpoint the problem. You are basically saying that because you can ship without a compiler sometimes, we are supposed to change the way our regression tests work. Let's suppose SCO says they don't ship with a compiler, and wants us to change our code to accomodate it. Should we? You can be certain we would not, and in the RPM case, you get the same answer. If the patch is trivial, we will work around OS limitations, but we do not redesign code to work around OS limitations. We expect the OS to get the proper features. That is what we do with NT. Cygwin provides the needed features. > > Please give us more information about how the current upgrade is a > > problem. We don't hear that much from other OS's. How are RPM's > > specific, and maybe we can get a plan for a solution. > > RPM's are expected to 'rpm -U' and you can simply _use_ the new version, > with little to no preparation. At least that is the theory. And it > works for most packages. This is the "Hey, other people can do it, why can't you" issue. We are looking for suggestions from Linux users in how this can be done. Perhaps running a separate command after the RPM has been installed is the only way to go. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen writes: > In the environment of the general purpose OS upgrade, the RPM's > installation scripts cannot fire up a backend, nor can it assume one > is running or is not running, nor can the RPM installation scripts > fathom from the run-time environment whether they are being run from a > command line or from the OS upgrade (except on Linux Mandrake, which > allows such usage). I don't understand why this is so. It seems perfectly possible that some %preremovebeforeupdate starts a postmaster, runs pg_dumpall, saves the file somewhere, then the %postinstallafterupdate runs the inverse operation. Disk space is not a valid objection, you'll never get away without 2x storage. Security is not a problem either. Are you not upgrading in proper dependency order or what? Everybody does dump, remove, install, undump; so can the RPMs. Okay, so it's not as great as a new KDE starting up and asking "may I update your configuration files?", but understand that the storage format is optimized for performance, not easy processing by external tools or something like that. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Peter Eisentraut wrote: > Lamar Owen writes: > > Getting the installation that 'make install' spits out massaged into > > an FHS compliant setup is the majority of the RPM's spec file. > ./configure --prefix=/usr --sysconfdir=/etc > Off you go... (I'll refrain from commenting further on the FHS.) I know alot of people don't like LSB/FHS, but, like it or not, I have to work with it. And, many many thanks for putting in the work on the configuration as you have. > > * Upgrades that don't require an ASCII database dump for migration. > Let me ask you this question: When any given RPM-based Linux distribution > will update their system from ext2 to, say, ReiserFS across the board, how > are they going to do it? Sincere question. Like the TRS-80 model III, whose TRSDOS 1.3 could not read the TRS-80 Model I's disks, written on TRSDOS 2.3 (TRSDOS's versioning was absolutely horrendous). TRSDOS 1.3 included a CONVERT utility that could read files from the old filesystem. I'm sure that the newer distributions using ReiserFS as the primary filesystem will include legacy Ext2/3 support, at least for read-only, for many versions to come. And that's my big beef -- a newer version of PostgreSQL can't even pg_dump an old database. If that single function was supported, I would have no problem with the upgrade whatsoever. > > * A less source-centric mindset. Let's see, how to explain? The > > regression tests are a good example. You need make. You need the source > > installed, configured, and built in the usual location. > This is not an excuse, but almost every package behaves this way. Test > suites are designed to be run after "make all" and before "make install". > When you ship a binary package then you're saying to users "I did the > building and installation (and presumably everything else that the authors > recommend along the way) for you." Yes, and I do just that. Regression testing is a regular part of my build process here. > RPM packages usually don't work very > well on systems that are not exactly like the one they were built on Boy, don't I know it.~;-/ > Getting the regression tests to work from anywhere is not very hard, but > it's not the most interesting project for most people. :-) I know. I'll probably do it myself, as that is something I _can_ do. > > I think I may have a solution for the library versioning problem. > > Rather than symlink libpq.so->libpq.so.2->libpq.so.2.x, I'll copy > > libpq.so.2.1 to libpq.so.2 and symlink libpq.so to that. > I'd still claim that if RPM thinks it's smarter than the dynamic loader, > then it's broken. All the shared libraries on Linux have a symlink from > more general to more specific names. PostgreSQL can't be the first to hit > this problem. RPM is getting it's .so dependency list straight from the mouth of the dynamic loader itself. RPM uses shell scripts, customizable for each system on which RPM runs, to determine the automatic dependencies -- those shell scripts run the dynamic loader to get the list of requires. So, the dynamic loader itself is providing the list. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
[Since I've rested over the weekend, I hope I don't come across this morning as an angry old snarl, like some of my previous posts on this subject unfortunately have been.] Bruce Momjian wrote: > > * Location-agnostic installation. Documentation (which I'll be happy to > > contribute) on that. Peter E is already working in this area. Getting > > the installation that 'make install' spits out massaged into an FHS > > compliant setup is the majority of the RPM's spec file. > Well, we certainly don't want to make changes that make things harder or > more confusing for non-RPM installs. How are they affected here? They wouldn't be. Peter E has seemingly done an excellent job in this area. I say seemingly because I haven't built an RPM from the 7.1 branch yet, but from what he has posted, he seems to understand the issue. Many thanks, Peter. > > * Upgrades that don't require an ASCII database dump for migration. This > > can either be implemented as a program to do a pg_dump of an arbitrary > > version of data, or as a binary migration utility. Currently, I'm > I really don't see the issue here. At the risk of being redundant, here goes. As I've explained before, the RPM upgrade environment, thanks to our standing with multiple distributions as being shipped as a part of the OS, could be run as part of a general-purpose OS upgrade. In the environment of the general purpose OS upgrade, the RPM's installation scripts cannot fire up a backend, nor can it assume one is running or is not running, nor can the RPM installation scripts fathom from the run-time environment whether they are being run from a command line or from the OS upgrade (except on Linux Mandrake, which allows such usage). Thus, if a system administrator upgrades a system, or if an end user who has a pgaccess-customized data entry system for things as mundane as an address list or recipe book, there is no opportunity to do a dump. The dump has to be performed _after_ the RPM upgrade. Now, this is far from optimal, I know. I _know_ that the user should take pains with their data. I know that there should be a backup. I also know that a user of PostgreSQL should realize that 'this is just the way it is done' and do things Our Way. I also know that few new users will do it 'Our Way'. No other package that I am aware of requires the manual intervention that PostgreSQL does, with the possible exception of upgrading to a different file system -- but that is something most new users won't do, and is something that is more difficult to automate. However, over the weekend, while resting (I did absolutely NO computer work this weekend -- too close to burnout), I had a brainstorm. A binary migration tool does not need to be written, if a concession to the needs of some users who just simply want to upgrade can be made. Suppose we can package old backends (with newer network code to connect to new clients). Suppose further that postmaster can be made intelligent enough to fire up old backends for old data, using PG_VERSION as a key. Suppose a NOTICE can be fired off warning the user that 'The Database is running in Compatibility Mode -- some features may not be available. Please perform a dump of your data, reinitialize the database, and restore your data to access new features of version x.y'. I'm highly considering doing just that from a higher level. It will not be nearly as smooth, but doable. Of course, that increases maintenance work, and I know it does. But I'm trying to find a middle ground here, since providing a true migration utility (even if it just produces a dump of the old data) seems out of reach at this time. We are currently forcing something like a popular word processing program once did -- it's proprietary file format changed. It was coded so that it could not even read the old files. But both the old and the new versions could read and write an interchange format. People who blindly upgraded their word processor were hit with a major problem. There was even a notice in the README -- which could be read after the program was installed. While the majority of us use PostgreSQL as a server behind websites and other clients, there will be a large number of new users who want to use it for much more mundane tasks. Like address books, or personal information management, or maybe even tax records. Frontends to PostgreSQL, thanks to PostgreSQL's advanced features, are likely to span the gamut -- we already have OnShore TimeSheet for time tracking and payroll, as one example. And I even see database-backed intranet-style web scripts being used on a client workstation for these sorts of things. I personally do just that with my home Linux box -- I have a number of AOLserver dynamic pages that use PostgreSQL for many mundane tasks (a multilevel sermon database is one). While I don't need handholding in the upgrade process, I have provided support to users that do -- who are astonished at the way w
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen writes: > Getting the installation that 'make install' spits out massaged into > an FHS compliant setup is the majority of the RPM's spec file. ./configure --prefix=/usr --sysconfdir=/etc Off you go... (I'll refrain from commenting further on the FHS.) > * Upgrades that don't require an ASCII database dump for migration. Let me ask you this question: When any given RPM-based Linux distribution will update their system from ext2 to, say, ReiserFS across the board, how are they going to do it? Sincere question. > * A less source-centric mindset. Let's see, how to explain? The > regression tests are a good example. You need make. You need the source > installed, configured, and built in the usual location. This is not an excuse, but almost every package behaves this way. Test suites are designed to be run after "make all" and before "make install". When you ship a binary package then you're saying to users "I did the building and installation (and presumably everything else that the authors recommend along the way) for you." RPM packages usually don't work very well on systems that are not exactly like the one they were built on, so this seems to be a fair assumption. Getting the regression tests to work from anywhere is not very hard, but it's not the most interesting project for most people. :-) > I think I may have a solution for the library versioning problem. > Rather than symlink libpq.so->libpq.so.2->libpq.so.2.x, I'll copy > libpq.so.2.1 to libpq.so.2 and symlink libpq.so to that. I'd still claim that if RPM thinks it's smarter than the dynamic loader, then it's broken. All the shared libraries on Linux have a symlink from more general to more specific names. PostgreSQL can't be the first to hit this problem. -- Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian wrote: > > > However, the upgrade gotcha has left a very bitter taste in more than > > one user's mouth. I'll not say more about that now, as I've said quite > > enough in the past. And I'm still trying to figure out enough of the > > internals of the storage manager to try to write the migration tools > > myself. But, I have other fish to fry right now, the biggest being > > cross-distribution RPM's. > > Actually, I would prefer to see how we can improve what we have before > making a binary conversion utility that will have to be updated for > every release. > > > Meet Mr. Middle Layer. :-) The PostgreSQL spec file that controls the > > RPM build is one of the most complex ones in the RedHat distribution, > > AFAIK. There's the middle layer. It does quite a bit of finagling > > already. > > Yes, I suspected the RPM was the middle layer. To the extent we can > make that easier, let's hear it. Tell us what you need to do, and what > you can't do, and see if any of us can figure out how to make things > easier. Ok, here goes: * Location-agnostic installation. Documentation (which I'll be happy to contribute) on that. Peter E is already working in this area. Getting the installation that 'make install' spits out massaged into an FHS compliant setup is the majority of the RPM's spec file. * Upgrades that don't require an ASCII database dump for migration. This can either be implemented as a program to do a pg_dump of an arbitrary version of data, or as a binary migration utility. Currently, I'm saving old executables to run under a special environment to pull a dump -- but it is far from optimal. What if the OS upgrade behind 99% of the upgrades makes it where those old executables can't run due to binary incompatibility (say I'm going from RedHat 3.0.3 to RedHat 7 -- 3.0.3, IIRC, as a.out...( and I know 3.0.3 didn't have PostgreSQL RPMs).)? What I could actually do to prevent that problem is build all of PostgreSQL's 6.1.x, 6.2.x, 6.3.x, 6.4.x, and 6.5.x and include the necessary backend executables as part of the RPM But I think you see the problem there. However, that would in my mind be better than the current situation, albeit taking up a lot of space. * A less source-centric mindset. Let's see, how to explain? The regression tests are a good example. You need make. You need the source installed, configured, and built in the usual location. You need portions of contrib. RPM's need to be installable on compiler-crippled servers for security. While the demand for regression testing on such a box may not be there, it certainly does give the user something to use to get standard output for bug reports. As a point, I run PostgreSQL in production on a compilerless machine. No compiler == more security. And Linux has enough security problems without a compiler being available :-(. Oh, and I have no make on that machine either. The documentation as well as many of the examples assume too much, IMHO, about the install location and the install methodology. I think I may have a solution for the library versioning problem. Rather than symlink libpq.so->libpq.so.2->libpq.so.2.x, I'll copy libpq.so.2.1 to libpq.so.2 and symlink libpq.so to that. A little more code for me. There is no real danger in version confusion with RPM's versioning and upgrade methodology, as long as you consistently use the RPMset. The PostgreSQL version number is readily found from an RPM database query, making the so version immaterial. The upgrade issue is the hot trigger for me at this time. It is and has been a major drain on my time and effort, as well as Trond's and others, to get the RPM upgrade working even remotely smoothly. And I am willing to code -- once I know how to go about doing it in the backend. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> Ok, here goes: Cool, a list. > * Location-agnostic installation. Documentation (which I'll be happy to > contribute) on that. Peter E is already working in this area. Getting > the installation that 'make install' spits out massaged into an FHS > compliant setup is the majority of the RPM's spec file. Well, we certainly don't want to make changes that make things harder or more confusing for non-RPM installs. How are they affected here? > * Upgrades that don't require an ASCII database dump for migration. This > can either be implemented as a program to do a pg_dump of an arbitrary > version of data, or as a binary migration utility. Currently, I'm > saving old executables to run under a special environment to pull a dump > -- but it is far from optimal. What if the OS upgrade behind 99% of the > upgrades makes it where those old executables can't run due to binary > incompatibility (say I'm going from RedHat 3.0.3 to RedHat 7 -- 3.0.3, > IIRC, as a.out...( and I know 3.0.3 didn't have PostgreSQL RPMs).)? > What I could actually do to prevent that problem is build all of > PostgreSQL's 6.1.x, 6.2.x, 6.3.x, 6.4.x, and 6.5.x and include the > necessary backend executables as part of the RPM But I think you see > the problem there. However, that would in my mind be better than the > current situation, albeit taking up a lot of space. I really don't see the issue here. We can compress ASCII dump files, so the space need should not be too bad. Can't you just check to see if there is enough space, and error out if there is not? If the 2GIG limit is a problem, can't the split utility drop the files in <2gig chunks that can be pasted together in a pipe on reload? > * A less source-centric mindset. Let's see, how to explain? The > regression tests are a good example. You need make. You need the source > installed, configured, and built in the usual location. You need > portions of contrib. RPM's need to be installable on compiler-crippled > servers for security. While the demand for regression testing on such a > box may not be there, it certainly does give the user something to use > to get standard output for bug reports. As a point, I run PostgreSQL in > production on a compilerless machine. No compiler == more security. > And Linux has enough security problems without a compiler being > available :-(. Oh, and I have no make on that machine either. Well, no compiler? I can't see how we would do that without making other OS installs harder. That is really the core of the issue. We can't be making changes that make things harder for other OS's. Those have to be isolated in the RPM, or in some other middle layer. > > The documentation as well as many of the examples assume too much, IMHO, > about the install location and the install methodology. Well, if we are not specific, things get very confusing for those other OS's. Being specific about locations makes things easier. Seems we may need to patch RPM installs to fix that. Certainly a pain, but I see no other options. > > I think I may have a solution for the library versioning problem. > Rather than symlink libpq.so->libpq.so.2->libpq.so.2.x, I'll copy > libpq.so.2.1 to libpq.so.2 and symlink libpq.so to that. A little more > code for me. There is no real danger in version confusion with RPM's > versioning and upgrade methodology, as long as you consistently use the > RPMset. The PostgreSQL version number is readily found from an RPM > database query, making the so version immaterial. Oh, that is good. > > The upgrade issue is the hot trigger for me at this time. It is and has > been a major drain on my time and effort, as well as Trond's and others, > to get the RPM upgrade working even remotely smoothly. And I am willing > to code -- once I know how to go about doing it in the backend. Please give us more information about how the current upgrade is a problem. We don't hear that much from other OS's. How are RPM's specific, and maybe we can get a plan for a solution. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> However, the upgrade gotcha has left a very bitter taste in more than > one user's mouth. I'll not say more about that now, as I've said quite > enough in the past. And I'm still trying to figure out enough of the > internals of the storage manager to try to write the migration tools > myself. But, I have other fish to fry right now, the biggest being > cross-distribution RPM's. Actually, I would prefer to see how we can improve what we have before making a binary conversion utility that will have to be updated for every release. > Meet Mr. Middle Layer. :-) The PostgreSQL spec file that controls the > RPM build is one of the most complex ones in the RedHat distribution, > AFAIK. There's the middle layer. It does quite a bit of finagling > already. Yes, I suspected the RPM was the middle layer. To the extent we can make that easier, let's hear it. Tell us what you need to do, and what you can't do, and see if any of us can figure out how to make things easier. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Trond Eivind Glomsrød wrote: > Lamar Owen <[EMAIL PROTECTED]> writes: > > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > > might fail if 2.1 were to be loaded under it (hypothetically)). > There usually are no such problems, and I'm not aware of any specific > to postgresql either. There have been reports to the pgsql-bugs list and to the PHP list about this very issue. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian wrote: > > I appreciate the fact that we are not here to make it easy for > > distributors to package our software. I also appreciate the fact that > > if you don't at least make an effort to work with major distributors > > (and RedHat, TurboLinux, Caldera, and SuSE together comprise a major > > userbase) that you run the risk of not being distributed in favor of an > > inferior product. > Let them. It is their decision. Frankly, I have seen this attitude > before, and I don't like it. Just the mention that "Gee, if you don't > cooperate, we may yank you," is really a veiled threat. I don't even see it as a veiled threat, Bruce. It simply _is_ a threat. There are other RDBMS choices. Currently PostgreSQL is the Officially Sanctioned RDBMS for multiple Linux distributions. As our capabilities increase, it will make us more and more attractive as the Choice, Top Shelf Open Source RDBMS. However, the upgrade gotcha has left a very bitter taste in more than one user's mouth. I'll not say more about that now, as I've said quite enough in the past. And I'm still trying to figure out enough of the internals of the storage manager to try to write the migration tools myself. But, I have other fish to fry right now, the biggest being cross-distribution RPM's. > > Linux is far from the only OS to be supported by > > PostgreSQL, true. But Linux is also the most popular OS for PostgreSQL > > deployment. > True, it is the most popular, but that doesn't make the others less > important. No, it doesn't. > This whole statement comes across as, "You run on Linux, and look, you > took the time to run on other OS's too. How quaint." I ran Unix before there was linux. I ran Unix years before Linus was even out of High School. Well, that is if you count Tandy Xenix V7 and System III as Unix. Or AT&T 3B1 SysVR2. Or Apollo DomainOS SR10.2. Or Ultrix on a VAX 11/750 (running in tandem with VMS). And I'm considering moving my most critical public servers from Linux over to OpenBSD. A Linux bigot I'm not. > > However, there are known problems that can bite people who are not using > > RPM's and are not running Linux. Some of those problems are such that > > it will take someone with more knowledge than I currently possess to > Again, your comments where quite helpful. We need more of them. We > need to hear more about the problems people are having with RPM's, and > how to make them better. Bruce, sometimes I fear my own lack of communications skills. If I can make my wife fighting mad at me with me having no clue as to what I said that made her mad, I fear I can make anyone mad, without knowing what I said to do so. So, I guess you could say I'm a little paranoid about my communications skills. So, I'm glad you considered my comments helpful -- I was beginning to get worried. > There must be a list of known problems. Let's hear them, so we can try > to solve them as a group. However, in general, we do not make dramatic > change to work around OS bugs, and do not plan to make major changes to > work around the limitations of RPM's. My bet is that some middle layer > can be created that will fix that for us. Meet Mr. Middle Layer. :-) The PostgreSQL spec file that controls the RPM build is one of the most complex ones in the RedHat distribution, AFAIK. There's the middle layer. It does quite a bit of finagling already. And the work that Peter E is doing is helping my cause significantly. Bruce, when I recover fully from the illness I've had the last few days, I'll try to come up with a coherent listing of what I've had to work around in the past. My current headache won't let me think straight right now, which makes it likely that I won't effectively communicate the issues. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen <[EMAIL PROTECTED]> writes: > My gut feel is that RedHat may be better off shipping 7.0.x if the > library version numbers are a contributory problem. We could provide compat-packages with just neeeded libraries. > The data upgrade problem is a bigger problem. To which RedHat might > just want to stay at 7.0.x until either a tool is written to > painlessly migrate or until the next major RedHat is released. We upgrade everything from 3.0.3 (we no longer support upgrades from 2.0 as we couldn't find a specific way to identify such a system and we didn't want accidentaly upgrade other distributions), so there is pain anyway. > Of course, that doesn't affect what I do as far as building 7.1 RPM's > for distribution from the PostgreSQL site (or by anyone who so desires > to distribute them). I have no choice for my own self but to stay on > the curve. I need TOAST and OUTER JOINS too much. Others very likely have the same need. I'll be looking into issues with these later. > So, what I feel may be the best compromise is for RedHat (and myself) to > continue building 7.0.x RPM's with bugfixes, etc, while I build 7.1 ad > subsequent RPMset's for those who know what they're doing and not > blindly upgrading their systems. > Trond, do you have any comments on that? Or is the likely migration to > kernel 2.4 in the next RedHat going to make a compatability compromise > here moot? No, the 2.4 kernel should go right in - I've been using it extensively on my system until recently (the most recent pretest has problems with flock for sendmail). Anyway, I've had a look at psql in objdump: Dynamic Section: NEEDED libpq.so.2.1 NEEDED libcrypt.so.1 NEEDED libnsl.so.1 NEEDED libdl.so.2 NEEDED libm.so.6 NEEDED libutil.so.1 NEEDED libreadline.so.4.1 NEEDED libtermcap.so.2 NEEDED libncurses.so.5 NEEDED libc.so.6 [...] It links against nice, round versions of most libraries but wants specific versions of readline ad libpq. -- Trond Eivind Glomsrød Red Hat, Inc.
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen <[EMAIL PROTECTED]> writes: > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > might fail if 2.1 were to be loaded under it (hypothetically)). > > Now, that doesn't directly effect the PostgreSQL RPM's. What it does > effect is the guy who wants to install PHP from with PostgreSQL support > enabled and cannot because of a failed dependency. Who gets blamed? > PostgreSQL. > > Trond may correct me on this, but I don't know of a workaround for > this. There usually are no such problems, and I'm not aware of any specific to postgresql either. -- Trond Eivind Glomsrød Red Hat, Inc.
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
[ Charset ISO-8859-1 unsupported, converting... ] > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > Let them. It is their decision. Frankly, I have seen this attitude > > before, and I don't like it. Just the mention that "Gee, if you don't > > cooperate, we may yank you," is really a veiled threat. Now, I know you > > aren't saying that, but the "if you don't play nice, we will drop you" > > argument sounds a lot more like MS that a Linux vendor should be acting, > > especially since they are not telling us what they want or assisting in > > the work. > > FWIW, I've never threatened to do so. If I wanted to, I would just do > it[1] - threats are bad and never cause anything but bad feelings. Sounds good. > > That being said, my favorite wishes (in addition to as much SQL > compliance and performance as possible, of course) are: OK, here is a nice list I can address constructively: > > * migration on upgrade Yes, we have problems. 7.1 will have a more robust pg_dump, and I hope that fixes most of those problems. As you see issues here, we need to hear about it. > * old libraries being able to speak to newer databases, so old > binaries can continue working after database upgrades We have always been able to do that. Old clients can talk to newer databases, though new can't necessary talk to older. To the extent that the client assumes a particular database structure, we have problems there, especially psql. I most cases, those match the server, so I think we are OK here, and most 3-rd party stuff doesn't touch the system tables in areas that are changed frequently. > * good sonames on libraries - if a library hasn't changed, bumping the > number to show it's part of a new version isn't necesarry. If it is > backwards compatible, just bump the minor version, if it isn't, bump > the major version. Or even better, use versioned symbols (I don't > know how many other OSes than Linux and Solaris supports this, > though). We only bump minor .so numbers, except for 6.5 I think where we had a major overhaul of libpq and the major was bumped. I don't see another major bump on the horizon. I don't think we have ever shipped a server that could not talk to clients at least one major revison backwards. The big question is how RPM's handle that. I have no idea. > As for assisting, at least Red Hat contributes to a lot of projects, > some of which are important to postgres on one or more platforms: gdb, > gcc, glibc and the linux kernel. There just isn't enough resources to > do everything, but I try to help out with the RPMs. We only need help to the extent RPM people are asking for major feature additions that affect only RPM/Linux, and frankly, the RPM/Linux users should be supplying patches to us for that anyway. No need for the company to get involved. > When we make patches for packages, we try to cooperate with the > author(s) to get them in - happily, we haven't had much of a need for > that with postgresql. Yes, I have never seen one. > > The "We are big. Just fix it and let us know when it is ready" attitude > > does not work here, and that is what I am hearing mostly from the RPM > > people. > > I haven't heard anyone say that. Some of the RPM users have made some demands that sound a little like that. :-) > > There must be a list of known problems. Let's hear them, so we can try > > to solve them as a group. However, in general, we do not make dramatic > > change to work around OS bugs, and do not plan to make major changes to > > work around the limitations of RPM's. > > I don't think there are any apart from the upgrade issues - if library > versioning follows the standard, that certainly won't be a problem. I would love to get a detailed list of upgrade problems so we can be sure 7.1 has them fixed. Certainly 7.1 is already a big improvement for upgrades. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian <[EMAIL PROTECTED]> writes: > Let them. It is their decision. Frankly, I have seen this attitude > before, and I don't like it. Just the mention that "Gee, if you don't > cooperate, we may yank you," is really a veiled threat. Now, I know you > aren't saying that, but the "if you don't play nice, we will drop you" > argument sounds a lot more like MS that a Linux vendor should be acting, > especially since they are not telling us what they want or assisting in > the work. FWIW, I've never threatened to do so. If I wanted to, I would just do it[1] - threats are bad and never cause anything but bad feelings. That being said, my favorite wishes (in addition to as much SQL compliance and performance as possible, of course) are: * migration on upgrade * old libraries being able to speak to newer databases, so old binaries can continue working after database upgrades * good sonames on libraries - if a library hasn't changed, bumping the number to show it's part of a new version isn't necesarry. If it is backwards compatible, just bump the minor version, if it isn't, bump the major version. Or even better, use versioned symbols (I don't know how many other OSes than Linux and Solaris supports this, though). As for assisting, at least Red Hat contributes to a lot of projects, some of which are important to postgres on one or more platforms: gdb, gcc, glibc and the linux kernel. There just isn't enough resources to do everything, but I try to help out with the RPMs. When we make patches for packages, we try to cooperate with the author(s) to get them in - happily, we haven't had much of a need for that with postgresql. > The "We are big. Just fix it and let us know when it is ready" attitude > does not work here, and that is what I am hearing mostly from the RPM > people. I haven't heard anyone say that. > There must be a list of known problems. Let's hear them, so we can try > to solve them as a group. However, in general, we do not make dramatic > change to work around OS bugs, and do not plan to make major changes to > work around the limitations of RPM's. I don't think there are any apart from the upgrade issues - if library versioning follows the standard, that certainly won't be a problem. [1] which I'm not even close to doing - I've spent a bit of time lately hunting down aliasing bugs in MySQL which causes wrong SQL query results if compiled with "-O2". Ouch. -- Trond Eivind Glomsrød Red Hat, Inc.
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Tom Lane wrote: > > Lamar Owen <[EMAIL PROTECTED]> writes: > > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > > might fail if 2.1 were to be loaded under it (hypothetically)). > > If so, I claim RPM is broken. > > The whole point of major/minor version numbering for .so's is that > a minor version bump is supposed to be binary-upward-compatible. > If the RPM stuff has arbitrarily decided that it won't honor that > definition, why do we bother with multiple numbers at all? > > > So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? > > To answer your question, there are no pending changes in libpq that > would mandate a major version bump (ie, nothing binary-incompatible, > AFAIK). We could ship it with the exact same version number, but then > how are people to tell whether they have a 7.0 or 7.1 libpq? And that is a very good point. Hey, I'm caught in the middle here :-). I want to see PostgreSQL succeed and excel (which, to me, means becoming the RDBMS of choice) on RPM-based Linux distributions, which I am sure is a goal of others too. And I'm sure no one here is against that. But, there is friction between RedHat's (to use the first example of a distributor to pop into my head) needs and the needs of the PostgreSQL group. My gut feel is that RedHat may be better off shipping 7.0.x if the library version numbers are a contributory problem. The data upgrade problem is a bigger problem. To which RedHat might just want to stay at 7.0.x until either a tool is written to painlessly migrate or until the next major RedHat is released. Of course, that doesn't affect what I do as far as building 7.1 RPM's for distribution from the PostgreSQL site (or by anyone who so desires to distribute them). I have no choice for my own self but to stay on the curve. I need TOAST and OUTER JOINS too much. So, what I feel may be the best compromise is for RedHat (and myself) to continue building 7.0.x RPM's with bugfixes, etc, while I build 7.1 ad subsequent RPMset's for those who know what they're doing and not blindly upgrading their systems. Trond, do you have any comments on that? Or is the likely migration to kernel 2.4 in the next RedHat going to make a compatability compromise here moot? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> > To answer your question, there are no pending changes in libpq that > > would mandate a major version bump (ie, nothing binary-incompatible, > > AFAIK). We could ship it with the exact same version number, but then > > how are people to tell whether they have a 7.0 or 7.1 libpq? > > If there isn't any changes, why bump it? This is huge software. There are changes to every library in every major release, major for us meaning, i.e., 7.0->7.1. That is why I bump the numbers. The interesting issue is that the version number changes for .so do _not_ mean they only talk with servers of the same release. They will talk to future servers of higher release numbers. This is done because there is a backend protocol number that is passed from client to server which determines how the server should behave with that client. We can't always have new clients talking to older servers because the old servers may not know the newer protocol. We could get fancy and trade version numbers and try to get it working, but it has not been a priority, and few have asked for it. Having old clients talking to new databases has been enough for most users. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Tom Lane <[EMAIL PROTECTED]> writes: > Lamar Owen <[EMAIL PROTECTED]> writes: > > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > > might fail if 2.1 were to be loaded under it (hypothetically)). You link against libpq.so.2 , not libpq.so.2.1. This isn't a problem. > If the RPM stuff has arbitrarily decided that it won't honor that > definition, why do we bother with multiple numbers at all? There is no such problem. > > So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? > > To answer your question, there are no pending changes in libpq that > would mandate a major version bump (ie, nothing binary-incompatible, > AFAIK). We could ship it with the exact same version number, but then > how are people to tell whether they have a 7.0 or 7.1 libpq? If there isn't any changes, why bump it? -- Trond Eivind Glomsrød Red Hat, Inc.
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> > What would be good is for someone to constructively make a posting about > > the known problems, and come up with acceptable solutions. Asking us to > > fix it really isn't going to help because we don't deal with RPM's here, > > and we don't have enough free time to make significant changes to meet > > the needs of RPM's. > > Which is why I stepped up to the plate last year to help with RPM's. > > I apologize if you took my post (which I edited greatly) as 'venting' -- > it was not my intention to 'vent', much less offend. I just want to > know what to expect from the 7.1 release. I feel that that is germane > to the Hackers list, as the knowledge necessary to answer the question > is to be found on the list. (and you answered the question above). No, I was not pointing to you when I mentioned venting. There have been other RPM threads lately. I just want information on how to make things better for RPM's, not vents. > Like it or not, in the eyes of many people having solid RPM's is a core > issue. If there are gotchas, I want to document them so people don't > get blindsided. Or work around them. Or ask why the change is > necessary in the first place. Sure. > I appreciate the fact that we are not here to make it easy for > distributors to package our software. I also appreciate the fact that > if you don't at least make an effort to work with major distributors > (and RedHat, TurboLinux, Caldera, and SuSE together comprise a major > userbase) that you run the risk of not being distributed in favor of an > inferior product. Let them. It is their decision. Frankly, I have seen this attitude before, and I don't like it. Just the mention that "Gee, if you don't cooperate, we may yank you," is really a veiled threat. Now, I know you aren't saying that, but the "if you don't play nice, we will drop you" argument sounds a lot more like MS that a Linux vendor should be acting, especially since they are not telling us what they want or assisting in the work. The "We are big. Just fix it and let us know when it is ready" attitude does not work here, and that is what I am hearing mostly from the RPM people. > I also appreciate and applaud the cross-platform mentality of the > PostgreSQL developers. Linux is far from the only OS to be supported by > PostgreSQL, true. But Linux is also the most popular OS for PostgreSQL > deployment. True, it is the most popular, but that doesn't make the others less important. This whole statement comes across as, "You run on Linux, and look, you took the time to run on other OS's too. How quaint." In the history of this project, Linux was an after-thought. None of our platforms are inferior or superior, except to the extent that the platform does not support Unix standard functions (like NT/Cygwin). > However, there are known problems that can bite people who are not using > RPM's and are not running Linux. Some of those problems are such that > it will take someone with more knowledge than I currently possess to > solve. One is the issue of upgrading/migrating tools. This is not an > RPM-specific issue. To me, that is the only big issue that I have > spoken about in a way that could even remotely be construed as > 'venting'. And it is not a Linux-specific issue. It is a core issue. Again, your comments where quite helpful. We need more of them. We need to hear more about the problems people are having with RPM's, and how to make them better. There must be a list of known problems. Let's hear them, so we can try to solve them as a group. However, in general, we do not make dramatic change to work around OS bugs, and do not plan to make major changes to work around the limitations of RPM's. My bet is that some middle layer can be created that will fix that for us. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
> Lamar Owen <[EMAIL PROTECTED]> writes: > > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > > might fail if 2.1 were to be loaded under it (hypothetically)). > > If so, I claim RPM is broken. > > The whole point of major/minor version numbering for .so's is that > a minor version bump is supposed to be binary-upward-compatible. > If the RPM stuff has arbitrarily decided that it won't honor that > definition, why do we bother with multiple numbers at all? > > > So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? > > To answer your question, there are no pending changes in libpq that > would mandate a major version bump (ie, nothing binary-incompatible, > AFAIK). We could ship it with the exact same version number, but then > how are people to tell whether they have a 7.0 or 7.1 libpq? Yes, we need to have new numbers so binaries from different releases use the proper .so files. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Bruce Momjian wrote: > Lamar Owen wrote: > > May I ask: is it necessary? Have there been version-bumping changes to > > libpq since 7.0.x? (With the rate that necessary improvement is > > happening to PostgreSQL, probably). > No, only major releases have bumps. > See pgsql/src/tools/RELEASE_CHANGES. I edit interfaces/*/Makefile and > increase the minor number for every interface by one. Thanks for the pointer. > Let me add one thing on this RPM issue. There has been a lot of talk > recently about RPM's, and what they should do, and what they don't do, > and who should be blamed. Unfortunately, much of the discussion has > been very unproductive and more like 'venting'. > I really don't appreciate people 'venting' on these lists, especially > since we have _nothing_ to do with RPM's. All we do is make the > PostgreSQL software. > What would be good is for someone to constructively make a posting about > the known problems, and come up with acceptible solutions. Asking us to > fix it really isn't going to help because we don't deal with RPM's here, > and we don't have enough free time to make significant changes to meet > the needs of RPM's. Which is why I stepped up to the plate last year to help with RPM's. I apologize if you took my post (which I edited greatly) as 'venting' -- it was not my intention to 'vent', much less offend. I just want to know what to expect from the 7.1 release. I feel that that is germane to the Hackers list, as the knowledge necessary to answer the question is to be found on the list. (and you answered the question above). Like it or not, in the eyes of many people having solid RPM's is a core issue. If there are gotchas, I want to document them so people don't get blindsided. Or work around them. Or ask why the change is necessary in the first place. I appreciate the fact that we are not here to make it easy for distributors to package our software. I also appreciate the fact that if you don't at least make an effort to work with major distributors (and RedHat, TurboLinux, Caldera, and SuSE together comprise a major userbase) that you run the risk of not being distributed in favor of an inferior product. I also appreciate and applaud the cross-platform mentality of the PostgreSQL developers. Linux is far from the only OS to be supported by PostgreSQL, true. But Linux is also the most popular OS for PostgreSQL deployment. However, there are known problems that can bite people who are not using RPM's and are not running Linux. Some of those problems are such that it will take someone with more knowledge than I currently possess to solve. One is the issue of upgrading/migrating tools. This is not an RPM-specific issue. To me, that is the only big issue that I have spoken about in a way that could even remotely be construed as 'venting'. And it is not a Linux-specific issue. It is a core issue. I'll shut up now, as I have cross-distribution RPM problems to solve. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Re: [HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
Lamar Owen <[EMAIL PROTECTED]> writes: > Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be > fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 > might fail if 2.1 were to be loaded under it (hypothetically)). If so, I claim RPM is broken. The whole point of major/minor version numbering for .so's is that a minor version bump is supposed to be binary-upward-compatible. If the RPM stuff has arbitrarily decided that it won't honor that definition, why do we bother with multiple numbers at all? > So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? To answer your question, there are no pending changes in libpq that would mandate a major version bump (ie, nothing binary-incompatible, AFAIK). We could ship it with the exact same version number, but then how are people to tell whether they have a 7.0 or 7.1 libpq? regards, tom lane
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
[Taken off GENERAL, added HACKERS to cc:] Bruce Momjian wrote: > > He's meaning the libpq version for dynamic link loading. Is the > > libpq.so lib changing versions (like the change from 6.5.x to 7.0.x > > changed from libpq.so.2.0 to libpq.so.2.1, which broke binary RPM > > compatibility for other RPM's linked against libpq.so.2.0, which failed > > when libpq.so.2.1 came on the scene). I think the answer is no, but I > > haven't checked the details yet. > I usually up the .so version numbers before entering beta. That way, > they get marked as newer than older versions. May I ask: is it necessary? Have there been version-bumping changes to libpq since 7.0.x? (With the rate that necessary improvement is happening to PostgreSQL, probably). Let me explain: RPM's contain a plethora of dependency information, some of which is added manually, but most of which is generated automatically. These dependencies are based on which 'soname' is needed to satisfy dynamic linking requirements, interpreter requirements, etc. With version numbers as part of the name, a change in version numbers changes the dependency. Unfortunately RPM deems a dependency upon libpq.so.2.0 to not be fulfilled by libpq.so.2.1 (how _can_ it know? A client linked to 2.0 might fail if 2.1 were to be loaded under it (hypothetically)). Now, that doesn't directly effect the PostgreSQL RPM's. What it does effect is the guy who wants to install PHP from with PostgreSQL support enabled and cannot because of a failed dependency. Who gets blamed? PostgreSQL. Trond may correct me on this, but I don't know of a workaround for this. And any workaround has to be applied to packages that depend upon PostgreSQL, not to the PostgreSQL RPM's (which I would gladly modify) -- although I am going to try something -- I know that a symlink to the old soname works, even though it is a kludge and, IMO, stinks like a polecat. But, enough rant. That _is_ I believe what Trond was asking about. We have been bitten before with people installing the PHP from RedHat 6.2 after installing the PostgreSQL 7.0.x RPMset -- and dependency failures wreaked havoc. So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? Actually, Bruce, it would do me and Trond a great favor if a list of what so's are getting bumped and to what version were to be posted. At least we can plan for a transition at that point. I just hate to pull a threepeat on RedHat customers. (RH 5.0 shipped PG 6.2.1. RH 5.1 shipped PG 6.3.2. BONG!) (RH 6.0 shipped 6.4.2 (bong!) RH 6.1 shipped 6.5.2 (double BONG!)). RH 7 shipped 7.0.x (small bong) -- RH 7.1 ships 7.1.x (ouch bong). Whew. Trond, you ready for this? [Note: I have been ill, so this message may be more incoherent than my normal scattered self] -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
[HACKERS] Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?)
[ Blind CC to general added for comment below.] > [Taken off GENERAL, added HACKERS to cc:] > > Bruce Momjian wrote: > > > He's meaning the libpq version for dynamic link loading. Is the > > > libpq.so lib changing versions (like the change from 6.5.x to 7.0.x > > > changed from libpq.so.2.0 to libpq.so.2.1, which broke binary RPM > > > compatibility for other RPM's linked against libpq.so.2.0, which failed > > > when libpq.so.2.1 came on the scene). I think the answer is no, but I > > > haven't checked the details yet. > > > I usually up the .so version numbers before entering beta. That way, > > they get marked as newer than older versions. > > May I ask: is it necessary? Have there been version-bumping changes to > libpq since 7.0.x? (With the rate that necessary improvement is > happening to PostgreSQL, probably). No, only major releases have bumps. > > But, enough rant. That _is_ I believe what Trond was asking about. We > have been bitten before with people installing the PHP from RedHat 6.2 > after installing the PostgreSQL 7.0.x RPMset -- and dependency failures > wreaked havoc. > > So, PostgreSQL 7.1 is slated to be libpq.so.2.2, then? > > Actually, Bruce, it would do me and Trond a great favor if a list of > what so's are getting bumped and to what version were to be posted. At > least we can plan for a transition at that point. See pgsql/src/tools/RELEASE_CHANGES. I edit interfaces/*/Makefile and increase the minor number for every interface by one. Let me add one thing on this RPM issue. There has been a lot of talk recently about RPM's, and what they should do, and what they don't do, and who should be blamed. Unfortunately, much of the discussion has been very unproductive and more like 'venting'. I really don't appreciate people 'venting' on these lists, especially since we have _nothing_ to do with RPM's. All we do is make the PostgreSQL software. If people want to discuss RPM's on the ports list, or want to create a new list just about RPM's, that's OK, but venting is bad, and venting on a list that has nothing to do with RPM's is even worse. What would be good is for someone to constructively make a posting about the known problems, and come up with acceptible solutions. Asking us to fix it really isn't going to help because we don't deal with RPM's here, and we don't have enough free time to make significant changes to meet the needs of RPM's. Also, remember we support many Unix platforms, and Linux is only one of them. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup.| Drexel Hill, Pennsylvania 19026
