Re: [HACKERS] pg_upgrade: downgradebility
Jonah, It's been two weeks, and I haven't seen anything, either here or on pg_foundry. Is this project derailed? -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] pg_upgrade: downgradebility
On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: I suggest again that you ought to be running your basic design past the list sooner rather than later. We will be posting it as soon as we can. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] pg_upgrade: downgradebility
On 9/20/06, Josh Berkus josh@agliodbs.com wrote: Supposedly someone from EnterpriseDB is working on this as well; where are you? Josh, We're currently working on the design docs and a beta. Denis, Faiz, and/or Ahsan will post here when it's posted. You should expect some activity in this area in the next two weeks or so. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_upgrade: downgradebility
Jonah, We're currently working on the design docs and a beta. Denis, Faiz, and/or Ahsan will post here when it's posted. You should expect some activity in this area in the next two weeks or so. So, are you saying that you already have code? If not, care to open up the process? We have others interested in contributing. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_upgrade: downgradebility
On 9/21/06, Josh Berkus josh@agliodbs.com wrote: So, are you saying that you already have code? Yes. If not, care to open up the process? We have others interested in contributing. We'll be opening it in the next few weeks. A large majority of it is working but we want to solidify the design and current issues before releasing it for comment and suggestion. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pg_upgrade: downgradebility
Jonah H. Harris [EMAIL PROTECTED] writes: On 9/21/06, Josh Berkus josh@agliodbs.com wrote: If not, care to open up the process? We have others interested in contributing. We'll be opening it in the next few weeks. A large majority of it is working but we want to solidify the design and current issues before releasing it for comment and suggestion. I see more and more people falling into the trap of doing a lot of work *before* showing their ideas to the community. This is an excellent plan if what you are hoping to accomplish is to waste a lot of work. Otherwise, you'll do a lot better to get the community involved sooner. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_upgrade: downgradebility
Jonah H. Harris wrote: On 9/21/06, Josh Berkus josh@agliodbs.com wrote: So, are you saying that you already have code? Yes. If not, care to open up the process? We have others interested in contributing. We'll be opening it in the next few weeks. A large majority of it is working but we want to solidify the design and current issues before releasing it for comment and suggestion. I don't understand why anybody thinks this makes any sense. You know, if I could post the Mammoth Replicator code for comment, I'd do it as soon as possible, design flaws and coding bugs included. It would help me get early feedback on dumb or bright decisions, so that I don't waste time pursuing fixes that are not invasive enough, or that are too invasive. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] pg_upgrade: downgradebility
On 9/21/06, Tom Lane [EMAIL PROTECTED] wrote: I see more and more people falling into the trap of doing a lot of work *before* showing their ideas to the community. This is an excellent plan if what you are hoping to accomplish is to waste a lot of work. Otherwise, you'll do a lot better to get the community involved sooner. Thanks for your opinion, but I disagree completely. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] pg_upgrade: downgradebility
On 9/21/06, Alvaro Herrera [EMAIL PROTECTED] wrote: You know, if I could post the Mammoth Replicator code for comment, I'd do it as soon as possible, design flaws and coding bugs included. It's easy to say something on the contrary when you know you can't back it up to begin with. Before everyone grabs their pitchforks, I would like to comment that this was going to be a strictly commercial product and as such, had already been through some design. We decided to open source it for everyone and get community involvement, so excuse us if we're going to spend just a little more time on it. I don't think I've seen anyone actually propose a design and state they were going to work on it (at least until 8.3)... so it's funny that I'm trying to follow the new, keep the community informed policy and yet everyone is still not happy enough. In short, our design is based on many of the pg_upgrade concepts but rewritten in C... if you have any blatant faults in pg_upgrade, that's the best place to start discussion. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_upgrade: downgradebility
Jonah H. Harris wrote: On 9/21/06, Tom Lane [EMAIL PROTECTED] wrote: I see more and more people falling into the trap of doing a lot of work *before* showing their ideas to the community. This is an excellent plan if what you are hoping to accomplish is to waste a lot of work. Otherwise, you'll do a lot better to get the community involved sooner. Thanks for your opinion, but I disagree completely. History in this community has basically shown that although you disagree you are completely wrong. Good luck with your project. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_upgrade: downgradebility
Jonah, I don't think I've seen anyone actually propose a design and state they were going to work on it (at least until 8.3)... so it's funny that I'm trying to follow the new, keep the community informed policy and yet everyone is still not happy enough. Ooops, didn't mean to start a which-hunt. To be fair, I e-mailed Denis about the project before it came up here, and he didn't respond (still hasn't). The current pgFoundry project is fine if this is going to be some proprietary code which EDB is going to drop on the community. However, that runs a significant risk of major design issues which will cause the project to never be used, and to be replaced with something else. Even the name of the pgFoundry project, as proposed, pretty much prohibits serious contributions from anyone else: it's EnterpriseDB Migrator, not pg_upgrade. So I think the thing to decide is whether this is EDB freeware or a real community OSS effort. I think that we'll be happy to have either, but right now it's in some grey area. --Josh Berkus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_upgrade: downgradebility
On 9/21/06, Josh Berkus josh@agliodbs.com wrote: Ooops, didn't mean to start a which-hunt. Not you're fault :) So I think the thing to decide is whether this is EDB freeware or a real community OSS effort. I think that we'll be happy to have either, but right now it's in some grey area. We certainly want it to be a community effort, we're just cleaning up the design right now. The code itself will be posted as-is for anyone to break, patch, and enhance. Even if the design is the only thing the community latches onto, improves, and drives into a new working pg_upgrade, that would be a good thing. We're just asking for a little patience. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_upgrade: downgradebility
We would help if we do it in an open pgfoundry project. Bizgres? - Luke Msg is shrt cuz m on ma treo -Original Message- From: Josh Berkus [mailto:[EMAIL PROTECTED] Sent: Thursday, September 21, 2006 11:08 PM Eastern Standard Time To: Jonah H. Harris Cc: pgsql-hackers@postgresql.org; Andrew Sullivan Subject:Re: [HACKERS] pg_upgrade: downgradebility Jonah, I don't think I've seen anyone actually propose a design and state they were going to work on it (at least until 8.3)... so it's funny that I'm trying to follow the new, keep the community informed policy and yet everyone is still not happy enough. Ooops, didn't mean to start a which-hunt. To be fair, I e-mailed Denis about the project before it came up here, and he didn't respond (still hasn't). The current pgFoundry project is fine if this is going to be some proprietary code which EDB is going to drop on the community. However, that runs a significant risk of major design issues which will cause the project to never be used, and to be replaced with something else. Even the name of the pgFoundry project, as proposed, pretty much prohibits serious contributions from anyone else: it's EnterpriseDB Migrator, not pg_upgrade. So I think the thing to decide is whether this is EDB freeware or a real community OSS effort. I think that we'll be happy to have either, but right now it's in some grey area. --Josh Berkus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_upgrade: downgradebility
Jonah, We certainly want it to be a community effort, we're just cleaning up the design right now. The code itself will be posted as-is for anyone to break, patch, and enhance. Even if the design is the only thing the community latches onto, improves, and drives into a new working pg_upgrade, that would be a good thing. We're just asking for a little patience. OK ... can we re-name the project, please, though? It's going to be hard for employees of other companies to contribute to the EnterpriseDB Migrator. Maybe PGMigrator with EnterpriseDB leading off the discription? --Josh Berkus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_upgrade: downgradebility
On 9/21/06, Josh Berkus josh@agliodbs.com wrote: OK ... can we re-name the project, please, though? It's going to be hard for employees of other companies to contribute to the EnterpriseDB Migrator. Maybe PGMigrator with EnterpriseDB leading off the discription? I can't personally make that call but I'll discuss it with everyone tomorrow and get back to you. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_upgrade: downgradebility
Jonah H. Harris [EMAIL PROTECTED] writes: In short, our design is based on many of the pg_upgrade concepts but rewritten in C... if you have any blatant faults in pg_upgrade, that's the best place to start discussion. Well, actually, there's a reason why pg_upgrade isn't in the CVS tree anymore ... it never quite worked and for many releases has been in absolutely no danger of working. So you've not exactly allayed my concerns by stating the above. I suggest again that you ought to be running your basic design past the list sooner rather than later. There are a lot of smart people on this list --- they can help you not make mistakes. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] pg_upgrade: downgradebility
I discussed with Gavin about pg_downgrade process. I think that it should be much dangerous and more complex problem than upgrade. Some operation on the new system should makes downgrade impossible ... My experience with database upgrades is that downgrade is requested only if there some show stopper. Upgrade process is performed on test machine at first time to avoid any problems. My first question is how important is downgrade for You and Your customers? And second is how to verify that downgrade is possible? Following article describes how to downgrade oracle: http://www.stanford.edu/dept/itss/docs/oracle/10g/server.101/b10763/downgrade.htm The main problem is when new functionality is used for example in the PL/SQL or new data type is used in the table definition. Oracle has compatible parameter which disable new features. I think that there is not way how to secure this in postgres now? And it should be big change in the code. Any suggestion? Zdenek ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_upgrade: downgradebility
On Wed, Sep 20, 2006 at 12:54:14PM +0200, Zdenek Kotala wrote: My first question is how important is downgrade for You and Your customers? And second is how to verify that downgrade is possible? Well, one way to do it is to set up a Slony replica using the older version of the software. So, if you've upgraded to 8.1.x, you replicate to an old 8.0.x back end as well. If 8.1 doesn't work for you, you just MOVE everything back to the 8.0.x back end, and you're golden. A -- Andrew Sullivan | [EMAIL PROTECTED] Information security isn't a technological problem. It's an economics problem. --Bruce Schneier ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_upgrade: downgradebility
On Wed, 20 Sep 2006, Andrew Sullivan wrote: On Wed, Sep 20, 2006 at 12:54:14PM +0200, Zdenek Kotala wrote: My first question is how important is downgrade for You and Your customers? And second is how to verify that downgrade is possible? Well, one way to do it is to set up a Slony replica using the older version of the software. So, if you've upgraded to 8.1.x, you replicate to an old 8.0.x back end as well. If 8.1 doesn't work for you, you just MOVE everything back to the 8.0.x back end, and you're golden. Well, I think that people who really want downgrade in such a tool are those for which slony replication is just not an option. That is, data in the range of hundreds of gigabytes. Using slony to upgrade is often not practical either. I wonder if pg_upgrade could be designed in such a way that upgrade is the same as downgrade from a development point of view. That is, the tool can change the system from one binary format to another. Thanks, Gavin ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] pg_upgrade: downgradebility
On Wed, Sep 20, 2006 at 09:28:42PM +1000, Gavin Sherry wrote: Well, I think that people who really want downgrade in such a tool are those for which slony replication is just not an option. That is, data in the range of hundreds of gigabytes. Using slony to upgrade is often not practical either. Yes, I figured as much. I was just suggesting that people who _really_ need the fault tolerance of being able to fall back do have an option, though it's probably somewhat painful to use. (You can use Slony for hundreds of gigs, although if the throughput is high enough, it might be tricky to get started.) I wonder if pg_upgrade could be designed in such a way that upgrade is the same as downgrade from a development point of view. That is, the tool can change the system from one binary format to another. I guess the question is whether there is any case where you can map an old version of some feature into some number of more-granular new versions, but going the other way is impossible. I know that sounds all hand-wavy. The analogy I'm thinking of is cases I've worked on where protocols change: the old protocol has one flag that can happily be mapped into, say, three in the new system. But in the new system, you can have just one of those flags by itself, and there's no obvious way to preserve that data when moving back to the old protocol. If we never have that sort of case with the binary formats, then what you propose ought to work. A -- Andrew Sullivan | [EMAIL PROTECTED] Unfortunately reformatting the Internet is a little more painful than reformatting your hard drive when it gets out of whack. --Scott Morris ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_upgrade: downgradebility
Gavin Sherry wrote: On Wed, 20 Sep 2006, Andrew Sullivan wrote: On Wed, Sep 20, 2006 at 12:54:14PM +0200, Zdenek Kotala wrote: My first question is how important is downgrade for You and Your customers? And second is how to verify that downgrade is possible? Well, one way to do it is to set up a Slony replica using the older version of the software. So, if you've upgraded to 8.1.x, you replicate to an old 8.0.x back end as well. If 8.1 doesn't work for you, you just MOVE everything back to the 8.0.x back end, and you're golden. Well, I think that people who really want downgrade in such a tool are those for which slony replication is just not an option. That is, data in the range of hundreds of gigabytes. Using slony to upgrade is often not practical either I agree with Gavin, Slony need a lot of extra resources. It is similarly to use pg_dump for upgrade/downgrade. I wonder if pg_upgrade could be designed in such a way that upgrade is the same as downgrade from a development point of view. That is, the tool can change the system from one binary format to another. The main problem is that newer format has more functionality that older. There is not possible perform downgrade if some create store procedure with SQL command which is not supported in the older version. Downgrade must check it but is it possible to perform 100% check now (without postgres code change)? Zdenek ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_upgrade: downgradebility
Zdenek, Andrew, Overall, I'd tend to say that downgradability should be for a v2 version of the tool. It's simply not as important. For one thing, there's always reverting-to-backup. However, we *do* need to be able to halt the upgrade and reverse it if it starts erroring out. So that might end up being the same code. Supposedly someone from EnterpriseDB is working on this as well; where are you? -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
