Re: aufs based upgrade tests
On Sat, Mar 21, 2009 at 2:28 AM, Alan Pope a...@popey.com wrote: That would fail for users who make one-time changes to their data. For example users who download their mail via pop. If they upgrade using aufs and then proceed to test the new features of their mail client, it might be configured to download new mail via pop and delete it from the server. In this case if they choose to not upgrade but throw the overlay away they lose their new mail and the ability to get it back. This could also happen if we overlay /var/spool/mail. The type of user is who stores mail in /var is likely to be more computer literate that one who stores mail in ~. However I have already suggested not overlaying /var/apt/cache, may we shouldn't overlay /var/spool/mail either. This could get a bit piecemeal though, as we probably want to overlay at least part of /var, e.g. /var/lib/apt/lists/. -- John C. McCabe-Dansted PhD Student University of Western Australia -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
to, 2009-03-19 kello 16:20 +0100, Martin Soto kirjoitti: Seconded, you start your session once with the new version installed, and there's often no way back. Applications will surely update their configuration data as soon as they're started, and there's no guarantee that they'll work when the older system is reinstated. In my humble opinion, such applications are broken. They will already break when, for example, $HOME is shared over NFS, and different hosts are not completely in sync, version-wise. As a possible solution, what about putting user accounts under aufs as well? This would have the advantage of allowing for testing how applications update their configuration data, which is also a good idea because this is a common source of bugs. This seems like a good idea, so I second it, because applications often are broken. It might be good to optionally allow /home to be excluded. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
hi, Am Freitag, den 20.03.2009, 09:49 +0200 schrieb Lars Wirzenius: to, 2009-03-19 kello 16:20 +0100, Martin Soto kirjoitti: Seconded, you start your session once with the new version installed, and there's often no way back. Applications will surely update their configuration data as soon as they're started, and there's no guarantee that they'll work when the older system is reinstated. In my humble opinion, such applications are broken. They will already break when, for example, $HOME is shared over NFS, and different hosts are not completely in sync, version-wise. As a possible solution, what about putting user accounts under aufs as well? This would have the advantage of allowing for testing how applications update their configuration data, which is also a good idea because this is a common source of bugs. This seems like a good idea, so I second it, because applications often are broken. It might be good to optionally allow /home to be excluded. how about just adding a test user in the overlay that gets removed with the aufs and have the option that application settings can be taken over from an existing user for this account ? ciao oli signature.asc Description: This is a digitally signed message part -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Wed, Mar 18, 2009 at 09:18:50AM +0100, Michael Vogt wrote: On Tue, Mar 17, 2009 at 03:53:23PM +0200, Marius Gedminas wrote: On Mon, Mar 16, 2009 at 07:00:12PM +0100, Michael Vogt wrote: Yes, after the upgrade the system will be jaunty until the next reboot, then the writable overlay is removed and the system is exactly in the same state as before the upgrade. Does this overlay cover the whole filesystem? If people start creating files in /home that are going to be lost on reboot, it would be nice to warn them about that. [..] The /home dir is excluded from the overlay. There is still a certain risk with this however, assume hypothetical e.g. the new rhythmbox is run after the test upgrade and it converts its database files to a new format that the old (intrepid) version can no longer read. This is pretty rare, but it does sometimes happens. Is there any reason why we don't overlay /home? Just so that you can test upgrade for existing configs? - Alexander -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
2009/3/18 Alexander Sack a...@ubuntu.com: Is there any reason why we don't overlay /home? Just so that you can test upgrade for existing configs? That would fail for users who make one-time changes to their data. For example users who download their mail via pop. If they upgrade using aufs and then proceed to test the new features of their mail client, it might be configured to download new mail via pop and delete it from the server. In this case if they choose to not upgrade but throw the overlay away they lose their new mail and the ability to get it back. Cheers, Al. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On 18/03/2009 Michael Vogt wrote: This is pretty rare, but it does sometimes happens. How can this be rare if the user is supposed to test things after the upgrade? I didn't think about it but it's actually risky - every application can potentially be non-forward-compatible. Users should be adviced to use the guest user account to test new applications, or perhaps it can be done in a way that only a special account sees the new system? Vincenzo -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Thu, 2009-03-19 at 15:56 +0100, Vincenzo Ciancia wrote: ... How can this be rare if the user is supposed to test things after the upgrade? I didn't think about it but it's actually risky - every application can potentially be non-forward-compatible. Users should be adviced to use the guest user account to test new applications, or perhaps it can be done in a way that only a special account sees the new system? Seconded, you start your session once with the new version installed, and there's often no way back. Applications will surely update their configuration data as soon as they're started, and there's no guarantee that they'll work when the older system is reinstated. As a possible solution, what about putting user accounts under aufs as well? This would have the advantage of allowing for testing how applications update their configuration data, which is also a good idea because this is a common source of bugs. Cheers, M. S. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Tue, Mar 17, 2009 at 03:53:23PM +0200, Marius Gedminas wrote: On Mon, Mar 16, 2009 at 07:00:12PM +0100, Michael Vogt wrote: Yes, after the upgrade the system will be jaunty until the next reboot, then the writable overlay is removed and the system is exactly in the same state as before the upgrade. Does this overlay cover the whole filesystem? If people start creating files in /home that are going to be lost on reboot, it would be nice to warn them about that. [..] The /home dir is excluded from the overlay. There is still a certain risk with this however, assume hypothetical e.g. the new rhythmbox is run after the test upgrade and it converts its database files to a new format that the old (intrepid) version can no longer read. This is pretty rare, but it does sometimes happens. Cheers, Michael -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Mon, Mar 16, 2009 at 07:00:12PM +0100, Michael Vogt wrote: Yes, after the upgrade the system will be jaunty until the next reboot, then the writable overlay is removed and the system is exactly in the same state as before the upgrade. Does this overlay cover the whole filesystem? If people start creating files in /home that are going to be lost on reboot, it would be nice to warn them about that. I created as a stub wiki page: https://wiki.ubuntu.com/AufsBasedUpgrades Marius Gedminas -- Some of the more environmentally aware dinosaurs were worried about the consequences of an accident with the new Iridium enriched fusion reactor. If it goes off only the cockroaches and mammals will survive... they said. signature.asc Description: Digital signature -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Sat, Mar 14, 2009 at 02:13:24PM +, (``-_-´´) -- BUGabundo wrote: Olá Michael e a todos. On Friday 13 March 2009 18:19:28 Michael Vogt wrote: during the last UDS we talked informally about using the aufs overlay filesystem layer for release upgrade testing. I build a prototype implementation of this now that should be ok for public testing. The idea discussed with Evan Dandrea (and others) was to create a writable overlay into /tmp on top the systemdirs in / and then run the release upgrade. This way we can test easily if the system would upgrade cleanly (if no dpkg errors/maintainer script failures happen). All writes go into /tmp so after the upgrade and on the next reboot the system is back to its pre-upgraded state again (modulo /home, that is not overlayed). It also means the next boot takes a *long* time to clean /tmp - when I did test it on one of my production machines that wait made me *really* nervous :) But its ok, it just takes long (up to ~20 minutes or so). Feedback is welcome This idea seems like a really nice idea, and one that in some other form is requested by users/testers. I would like to add to points: * if all tests go OK, and we end up with this on koala (to late for FFe on jaunty, right?), a checkbox when using update-manager -d / cli question on do-release-upgrade to use Sandbox would be much nicer then running all that code. * to save the system state prior to upgrade, so that a user can restore the system if even after successful package upgrade, some application/kernel/driver upgrade doesnt go as good. Thanks for this feedback. The longer term goal is provide the two improvements you suggested :) The current version is a first step to build up experience with the system. This is why its limited to testing currently, but in the longer run we hope to make it more capable. Cheers, Michael -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Sun, Mar 15, 2009 at 02:27:16AM -0400, John Vivirito wrote: On 03/14/2009 10:13 AM, (``-_-´´) -- BUGabundo wrote: Olá Michael e a todos. [..] This idea seems like a really nice idea, and one that in some other form is requested by users/testers. I would like to add to points: * if all tests go OK, and we end up with this on koala (to late for FFe on jaunty, right?), a checkbox when using update-manager -d / cli question on do-release-upgrade to use Sandbox would be much nicer then running all that code. * to save the system state prior to upgrade, so that a user can restore the system if even after successful package upgrade, some application/kernel/driver upgrade doesnt go as good. I am a bit on the short end of this topic due to trouble with having this set to digest mode. What exactly is this going to do. It sounds very interesting. is this similar to system restore in windows? The following quote makes it sound like after reboot it is going to restore itself to before the latest upgrade: All writes go into /tmp so after the upgrade and on the next reboot the system is back to its pre-upgraded state again Right now its a tool to help test if your version of ubuntu can upgrade to the next version of ubuntu without errors. It does a full regular upgrade from 8.10 to 9.04 but instead of writing it to the system disk it writes all changes to a directory in /tmp Doe the above always write to /tmp? If so does it clear upon restart automatically? Yes, after the upgrade the system will be jaunty until the next reboot, then the writable overlay is removed and the system is exactly in the same state as before the upgrade. Is there somewhere where i can get more information on it, a wiki or, blueprint or something? Unfortunately not right now. I created as a stub wiki page: https://wiki.ubuntu.com/AufsBasedUpgrades and we will probably talk about it at the next UDS and create a more formal plan. The currently version is build to get experience with the system and fint bugs and limitations with the aufs based approach, this is why its relatively complicated to enable it. Cheers, Michael -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On Sun, Mar 15, 2009 at 11:51:52AM +0100, Vincenzo Ciancia wrote: When doing something like this one should be careful because here you have a copy of all files that are modified during the upgrade. Applications keeping these files open will write to the old copies, and applications which reopen the file after the upgrade will not see this data. This may be dangerous and lead to unexpected behaviour. Thanks for the feedback. This is a problem we are aware of. The best ways to fix it will probably we discussed at UDS. One approach (that is available in the code as well) is to just create the overlay for the dpkg child, this means that the regular desktop stuff (including firefox) keeps working during the upgrade. But in my tests it has been less of a practical problem than I anticiapted, i.e. I have not had any issues because of that in my tests (but of course this is all young and has not been tested that much yet). Apart from that, as I ranted in the past, let me say that this is a very important change and I am really happy that ubuntu developers are making it happen. Thanks, if it works out in the way we hope it will mean more robustness to upgrades and that is certainly a good thing. Cheers, Michael -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
On 03/14/2009 10:13 AM, (``-_-´´) -- BUGabundo wrote: Olá Michael e a todos. On Friday 13 March 2009 18:19:28 Michael Vogt wrote: during the last UDS we talked informally about using the aufs overlay filesystem layer for release upgrade testing. I build a prototype implementation of this now that should be ok for public testing. The idea discussed with Evan Dandrea (and others) was to create a writable overlay into /tmp on top the systemdirs in / and then run the release upgrade. This way we can test easily if the system would upgrade cleanly (if no dpkg errors/maintainer script failures happen). All writes go into /tmp so after the upgrade and on the next reboot the system is back to its pre-upgraded state again (modulo /home, that is not overlayed). It also means the next boot takes a *long* time to clean /tmp - when I did test it on one of my production machines that wait made me *really* nervous :) But its ok, it just takes long (up to ~20 minutes or so). Feedback is welcome This idea seems like a really nice idea, and one that in some other form is requested by users/testers. I would like to add to points: * if all tests go OK, and we end up with this on koala (to late for FFe on jaunty, right?), a checkbox when using update-manager -d / cli question on do-release-upgrade to use Sandbox would be much nicer then running all that code. * to save the system state prior to upgrade, so that a user can restore the system if even after successful package upgrade, some application/kernel/driver upgrade doesnt go as good. I am a bit on the short end of this topic due to trouble with having this set to digest mode. What exactly is this going to do. It sounds very interesting. is this similar to system restore in windows? The following quote makes it sound like after reboot it is going to restore itself to before the latest upgrade: All writes go into /tmp so after the upgrade and on the next reboot the system is back to its pre-upgraded state again Doe the above always write to /tmp? If so does it clear upon restart automatically? Is there somewhere where i can get more information on it, a wiki or, blueprint or something? -- Sincerely Yours, John Vivirito https://launchpad.net/~gnomefreak https://wiki.ubuntu.com/JohnVivirito Linux User# 414246 How can i get lost, if i have no where to go -- Metallica from Unforgiven III signature.asc Description: OpenPGP digital signature -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
When doing something like this one should be careful because here you have a copy of all files that are modified during the upgrade. Applications keeping these files open will write to the old copies, and applications which reopen the file after the upgrade will not see this data. This may be dangerous and lead to unexpected behaviour. Apart from that, as I ranted in the past, let me say that this is a very important change and I am really happy that ubuntu developers are making it happen. Vincenzo -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: aufs based upgrade tests
Olá Michael e a todos. On Friday 13 March 2009 18:19:28 Michael Vogt wrote: during the last UDS we talked informally about using the aufs overlay filesystem layer for release upgrade testing. I build a prototype implementation of this now that should be ok for public testing. The idea discussed with Evan Dandrea (and others) was to create a writable overlay into /tmp on top the systemdirs in / and then run the release upgrade. This way we can test easily if the system would upgrade cleanly (if no dpkg errors/maintainer script failures happen). All writes go into /tmp so after the upgrade and on the next reboot the system is back to its pre-upgraded state again (modulo /home, that is not overlayed). It also means the next boot takes a *long* time to clean /tmp - when I did test it on one of my production machines that wait made me *really* nervous :) But its ok, it just takes long (up to ~20 minutes or so). Feedback is welcome This idea seems like a really nice idea, and one that in some other form is requested by users/testers. I would like to add to points: * if all tests go OK, and we end up with this on koala (to late for FFe on jaunty, right?), a checkbox when using update-manager -d / cli question on do-release-upgrade to use Sandbox would be much nicer then running all that code. * to save the system state prior to upgrade, so that a user can restore the system if even after successful package upgrade, some application/kernel/driver upgrade doesnt go as good. -- Hi, I'm BUGabundo, and I am Ubuntu (whyubuntu.com) (``-_-´´) http://LinuxNoDEI.BUGabundo.net Linux user #443786GPG key 1024D/A1784EBB http://BUGabundo.net ps. My emails tend to sound authority and aggressive. I'm sorry in advance. I'll try to be more assertive as time goes by... signature.asc Description: This is a digitally signed message part. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
