[Sugar-devel] [PATCH] Journal Volumes Backup and Restore
Add a basic backup and restore feature for the Sugar Journal.
It provides:
- Generic Backup and Restore dialog GUI.
- Process manager class as an abstraction layer between the dialog and
backup/restore scripts. (Allowing to work with many backup and restore
technologies, using the same GUI, with no need for script rewrite).
- Basic file system Volume Restore and Backup scripts implemented in Python.
- New backup and restore options for journal volumes palettes.
This patch is based on Esteban Arias (Plan Ceibal) Volume Backup and Restore
patch, with a few changes:
- Refactor original Backup dialog class into a generic dialog class.
- Create specialized VolumeBackupDialog and VolumeRestoreDialog subclasses.
- Rewrite backup and restore scripts in python for an easier sugar interaction.
- Add backup identification helpers to jarabe.journal.misc.
---
bin/Makefile.am |4 +-
bin/journal-backup-volume | 57
bin/journal-restore-volume| 67 +
src/jarabe/journal/Makefile.am|3 +-
src/jarabe/journal/misc.py| 27
src/jarabe/journal/processdialog.py | 248 +
src/jarabe/journal/volumestoolbar.py |5 +-
src/jarabe/model/Makefile.am |3 +-
src/jarabe/model/processmanagement.py | 98 +
src/jarabe/view/palettes.py | 44 ++
10 files changed, 551 insertions(+), 5 deletions(-)
create mode 100644 bin/journal-backup-volume
create mode 100644 bin/journal-restore-volume
create mode 100644 src/jarabe/journal/processdialog.py
create mode 100644 src/jarabe/model/processmanagement.py
diff --git a/bin/Makefile.am b/bin/Makefile.am
index 05a9215..8cc87b5 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -5,7 +5,9 @@ python_scripts =\
sugar-install-bundle\
sugar-launch\
sugar-session \
- sugar-ui-check
+ sugar-ui-check \
+ journal-backup-volume \
+ journal-restore-volume
bin_SCRIPTS = \
sugar \
diff --git a/bin/journal-backup-volume b/bin/journal-backup-volume
new file mode 100644
index 000..4f3ec8a
--- /dev/null
+++ b/bin/journal-backup-volume
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+# Copyright (C) 2010, Paraguay Educa tecnolo...@paraguayeduca.org
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+#
+
+import os
+import sys
+import subprocess
+import logging
+
+from sugar import env
+#from sugar.datastore import datastore
+
+backup_identifier = sys.argv[2]
+volume_path = sys.argv[1]
+
+if len(sys.argv) != 3:
+print 'Usage: %s volume_path backup_identifier' % sys.argv[0]
+exit(1)
+
+logging.debug('Backup started')
+
+backup_path = os.path.join(volume_path, 'backup', backup_identifier)
+
+if not os.path.exists(backup_path):
+os.makedirs(backup_path)
+
+#datastore.freeze()
+subprocess.call(['pkill', '-9', '-f', 'python.*datastore-service'])
+
+result = 0
+try:
+cmd = ['tar', '-C', env.get_profile_path(), '-czf', \
+ os.path.join(backup_path, 'datastore.tar.gz'), 'datastore']
+
+subprocess.check_call(cmd)
+
+except Exception, e:
+logging.error('Backup failed: %s', str(e))
+result = 1
+
+#datastore.thaw()
+
+logging.debug('Backup finished')
+exit(result)
diff --git a/bin/journal-restore-volume b/bin/journal-restore-volume
new file mode 100644
index 000..aa14ad0
--- /dev/null
+++ b/bin/journal-restore-volume
@@ -0,0 +1,67 @@
+#!/usr/bin/env python
+# Copyright (C) 2010, Paraguay Educa tecnolo...@paraguayeduca.org
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+#
+
+import os
+import sys
+import shutil
+import logging
+import subprocess
+
+from sugar import env
+#from
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
A few screenshots of the actual work. Please, if there is some other text I should add or change let me know. http://img25.imageshack.us/i/screenshotlym.png/ http://img41.imageshack.us/i/screenshot1dk.png/ http://img534.imageshack.us/i/screenshot2wl.png/ http://img37.imageshack.us/i/screenshot3cfv.png/ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
Hi Martin, On 3 Jul 2010, at 18:13, Martin Abente mabe...@paraguayeduca.org wrote: A few screenshots of the actual work. Thanks for the screen shots! Please, if there is some other text I should add or change let me know. http://img25.imageshack.us/i/screenshotlym.png/ Perhaps, in a future pass, we can make some better icons for the backup/restore menu items, rather than using the upload/download arrow icons; but don't let that block the patch. http://img41.imageshack.us/i/screenshot1dk.png/ Can the dialogue buttons be named Restore/Cancel, and Backup/Cancel for the backup case? http://img534.imageshack.us/i/screenshot2wl.png/ http://img37.imageshack.us/i/screenshot3cfv.png/ Regards, --Gary ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Sat, 3 Jul 2010 20:53:52 +0100, Gary Martin garycmar...@googlemail.com wrote: Hi Martin, On 3 Jul 2010, at 18:13, Martin Abente mabe...@paraguayeduca.org wrote: A few screenshots of the actual work. Thanks for the screen shots! Please, if there is some other text I should add or change let me know. http://img25.imageshack.us/i/screenshotlym.png/ Perhaps, in a future pass, we can make some better icons for the backup/restore menu items, rather than using the upload/download arrow icons; but don't let that block the patch. http://img41.imageshack.us/i/screenshot1dk.png/ Can the dialogue buttons be named Restore/Cancel, and Backup/Cancel for the backup case? of course :) http://img534.imageshack.us/i/screenshot2wl.png/ http://img37.imageshack.us/i/screenshot3cfv.png/ Regards, --Gary ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On 29 Jun 2010, at 19:56, Martin Abente mabe...@paraguayeduca.org wrote: Martin and James, In case you haven't noticed, we know how it could be better and we know how the high level pseudo code would look like. We just can't do it, because we have no more time to spend on this. The backup and restore script are just a little part of this whole patch, and it would be _very_ helpful if someone could actually test it and review the code. Thanks for your comments, hopefully we or someone else will have the time to improve it, in the near future. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel On Thu, 1 Jul 2010 19:50:02 +0100, Gary Martin garycmar...@googlemail.com wrote: On 1 Jul 2010, at 14:02, Tomeu Vizoso to...@sugarlabs.org wrote: On Wed, Jun 30, 2010 at 03:57, Bernie Innocenti ber...@codewiz.org wrote: == New Features == sugar/backup-0001-Volumes-Backup-and-Restore.patch sugar/backup-0002-Journal-XS-backup-and-restore.patch There are concerns about restore deleting new entries since the last backup. I agree, but since nobody seems to have the time to implement and test a more sophisticated procedure, at this time this is the best restore feature we have for Sugar. Do we know any other deployment willing to deploy this? If we decide to merge it, I think it should be disabled by default and have a gconf setting, because knowingly shipping a feature that causes data loss may not be a good idea. Sounds fair. I was going to suggest making sure there was at least a second user action needed after hitting a backup or restore button (I skimmed through the patch code but couldn't see a conformation warning step). A warning notification with Cancel/Backup, and Cancel/Restore could help avoid some accidents. Just posting on this email thread as well. Not being sarcastic here :-p genuine question, is there a trac ticket for this one? Some patches do, some patches don't. Regards, --Gary ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
Martin and James, In case you haven't noticed, we know how it could be better and we know how the high level pseudo code would look like. We just can't do it, because we have no more time to spend on this. The backup and restore script are just a little part of this whole patch, and it would be _very_ helpful if someone could actually test it and review the code. Thanks for your comments, hopefully we or someone else will have the time to improve it, in the near future. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Tue, Jun 29, 2010 at 2:56 PM, Martin Abente mabe...@paraguayeduca.org wrote: In case you haven't noticed, we know how it could be better and we know how the high level pseudo code would look like. We just can't do it, because we have no more time to spend on this. I understand this -- and I am in a similar quandary. However, I am bothering you with this because I am a bit worried -- because two things have combined in an unfortunate way: - the restore action is destructive - it is now in a UI button that is present in the everyday UI kids are going to try things. They try things all the time; and they'll try the restore button. One of the key design concerns in Sugar is that operations should not be destructive. You can imagine how a dangerous button, made so easily available, can be... well... dangerous. Thanks for your comments, hopefully we or someone else will have the time to improve it, in the near future. And that may well be me. I hope at least. m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Tue, Jun 29, 2010 at 02:56:57PM -0400, Martin Abente wrote: The backup and restore script are just a little part of this whole patch, and it would be _very_ helpful if someone could actually test it and review the code. I've reviewed it just now. You have my support for merging; because the situation is still greatly improved over what was present previously. Reviewed-by: James Cameron qu...@laptop.org -- James Cameron http://quozl.linux.org.au/ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
[Sugar-devel] [PATCH] Journal Volumes Backup and Restore
Add a basic backup and restore feature for the Sugar Journal.
It provides:
- Generic Backup and Restore dialog GUI.
- Process manager class as an abstraction layer between the dialog and
backup/restore scripts. (Allowing to work with many backup and restore
technologies, using the same GUI, with no need for script rewrite).
- Basic file system Volume Restore and Backup scripts implemented in Python.
- New backup and restore options for journal volumes palettes.
This patch is based on Esteban Arias (Plan Ceibal) Volume Backup and Restore
patch, with a few changes:
- Refactor original Backup dialog class into a generic dialog class.
- Create specialized VolumeBackupDialog and VolumeRestoreDialog subclasses.
- Rewrite backup and restore scripts in python for an easier sugar interaction.
- Add backup identification helpers to jarabe.journal.misc.
---
bin/Makefile.am |4 +-
bin/journal-backup-volume | 56
bin/journal-restore-volume| 65 +
src/jarabe/journal/Makefile.am|3 +-
src/jarabe/journal/misc.py| 27
src/jarabe/journal/processdialog.py | 233 +
src/jarabe/journal/volumestoolbar.py |5 +-
src/jarabe/model/Makefile.am |3 +-
src/jarabe/model/processmanagement.py | 98 ++
src/jarabe/view/palettes.py | 44 ++
10 files changed, 533 insertions(+), 5 deletions(-)
create mode 100644 bin/journal-backup-volume
create mode 100644 bin/journal-restore-volume
create mode 100644 src/jarabe/journal/processdialog.py
create mode 100644 src/jarabe/model/processmanagement.py
diff --git a/bin/Makefile.am b/bin/Makefile.am
index 05a9215..8cc87b5 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -5,7 +5,9 @@ python_scripts =\
sugar-install-bundle\
sugar-launch\
sugar-session \
- sugar-ui-check
+ sugar-ui-check \
+ journal-backup-volume \
+ journal-restore-volume
bin_SCRIPTS = \
sugar \
diff --git a/bin/journal-backup-volume b/bin/journal-backup-volume
new file mode 100644
index 000..10bdba9
--- /dev/null
+++ b/bin/journal-backup-volume
@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+# Copyright (C) 2010, Paraguay Educa tecnolo...@paraguayeduca.org
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+#
+
+import os
+import sys
+import subprocess
+import logging
+
+from sugar import env
+#from sugar.datastore import datastore
+
+backup_identifier = sys.argv[2]
+volume_path = sys.argv[1]
+
+if len(sys.argv) != 3:
+print 'Usage: %s volume_path backup_identifier' % sys.argv[0]
+exit(1)
+
+logging.debug('Backup started')
+
+backup_path = os.path.join(volume_path, 'backup', backup_identifier)
+
+if not os.path.exists(backup_path):
+os.makedirs(backup_path)
+
+#datastore.freeze()
+
+result = 0
+try:
+cmd = ['tar', '-C', env.get_profile_path(), '-czf', \
+ os.path.join(backup_path, 'datastore.tar.gz'), 'datastore']
+
+subprocess.check_call(cmd)
+
+except Exception, e:
+logging.error('Backup failed: %s', str(e))
+result = 1
+
+#datastore.thaw()
+
+logging.debug('Backup finished')
+exit(result)
diff --git a/bin/journal-restore-volume b/bin/journal-restore-volume
new file mode 100644
index 000..3150fca
--- /dev/null
+++ b/bin/journal-restore-volume
@@ -0,0 +1,65 @@
+#!/usr/bin/env python
+# Copyright (C) 2010, Paraguay Educa tecnolo...@paraguayeduca.org
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/.
+#
+
+import os
+import sys
+import shutil
+import logging
+import subprocess
+
+from sugar import env
+#from sugar.datastore import datastore
+
+backup_identifier = sys.argv[2]
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Mon, 28 Jun 2010 15:11:04 -0400, Martin Langhoff martin.langh...@gmail.com wrote: Hi Martin, great to see action on this front. Thanks for working on this. I have a few questions - Why is the UI in the main Journal UI? A manual backup should be a very infrequent op. (And same with a full restore). Well, originally I added a XS backup and restore option (Journal Management) into the Control Panel. Even when it was not intended for upstreaming, I sent it to the mailing list, therefore a discussion started about where these backup and restore options should go. Finally, this idea of having the options in the resouces (volumes) palette was the most wanted. The arguments favoring this idea were based on user friendliness: It seems easier and makes more sense to find those options in the same context of the resource that provides them. - Is the 'restore' option a 'full restore' or does it allow to browse existing backup snapshots? Not yet, it does a full restore from the one that is available on the storage device. For the Schoolserver backup, we plan to let users choose from the list of available backups, in the future. ...If it is a 'full restore', is it destructive? What does it do with documents that already exist but have been modified locally? What does it do with space concerns (ie: if the backup won't fit on disk) ? Yes, the entries added after the last backup will be lost. At first sight it seemed easy to just append data, but there are many hard to cover scenarios (Corner cases). Mostly the ones when you have to consider the very few space available. So for the time being we don't merge backups, still better than loosing all data :). cheers, -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Mon, Jun 28, 2010 at 3:58 PM, Martin Abente mabe...@paraguayeduca.org wrote: Well, originally I added a XS backup and restore option (Journal Management) into the Control Panel. Even when it was not intended for upstreaming, I sent it to the mailing list, therefore a discussion started about where these backup and restore options should go. Finally, this idea of having the options in the resouces (volumes) palette was the most wanted. By whom? ... what is the scenario where you need to click on the backup button? [ The only one *I* know of is just before an upgrade... once a year? ] Not yet, it does a full restore from the one that is available on the storage device. What worries me with this is that it _destroys anything done since the last backup_. At first sight it seemed easy to just append data It _is_ easy... to cover scenarios (Corner cases). Mostly the ones when you have to consider the very few space available But the full restore case also has to handle low-disk-space. What do you do when the data to restore does not fit in the disk? (For example, because the user has downloaded installed large activities...) cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [PATCH] Journal Volumes Backup and Restore
On Mon, Jun 28, 2010 at 05:21:39PM -0400, Martin Abente wrote: On Mon, 28 Jun 2010 16:13:33 -0400, Martin Langhoff martin.langh...@gmail.com wrote: On Mon, Jun 28, 2010 at 3:58 PM, Martin Abente mabe...@paraguayeduca.org wrote: Well, originally I added a XS backup and restore option (Journal Management) into the Control Panel. Even when it was not intended for upstreaming, I sent it to the mailing list, therefore a discussion started about where these backup and restore options should go. Finally, this idea of having the options in the resouces (volumes) palette was the most wanted. By whom? ... what is the scenario where you need to click on the backup button? The ones that replied to the previous emails sent by Esteban Arias a few months ago, James Cameron, Bernie and myself. http://lists.sugarlabs.org/archive/sugar-devel/2010-April/023273.html My expectation was that the child user would use backup daily, or more frequently, and a full restore only if a laptop was reimaged or all content was lost. Not yet, it does a full restore from the one that is available on the storage device. What worries me with this is that it _destroys anything done since the last backup_. Yes, it is a problem to loose _some_ data. But it is a bigger problem to not being able to backup and restore, loosing _all_ data. For now this is the state of the art :) I may not understand the design yet. To lose all data sounds simple; child does backup of empty journal, child fills journal with valuable content, child does restore. Result empty journal? I think that could be solved by a merge restore. This is just for discussion, I'm not offering code or suggesting you defer your work just for this. Merge restore logic would be: for each item in backup if item is not in journal attempt to restore it Failure to proceed due to lack of disk space should be handled; by advising the user that there is not enough available space to restore the whole backup, and that only a portion of the backup was restored. Being able to view the backup content in the journal would allow per item restore. -- James Cameron http://quozl.linux.org.au/ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
