On Fri, Apr 15, 2016 at 10:25:59AM +0800, Tony Anderson wrote:
Hi Manash
The registration process is awkward but not the problem.
This is unfair scope creep. Manash began by asking about bug #362 and
has been working to fix that. Now you're asking him to consider a
much larger task; not a coding task, but a redesign of Sugar Journal
and Backup interaction. This is huge.
And as far as I can tell, students aren't even accepted yet [1].
1. https://en.wikipedia.org/wiki/Google_Summer_of_Code#2016
What you propose is from a set of tasks [2] you added to the Wiki,
which have not undergone any design review according to Sugar Labs
design practice and feature policy. I do not see any consensus on
these; we're yet to build a consensus.
2. https://wiki.sugarlabs.org/go/Summer_of_Code/2016#Sugar_on_the_Ground
Or, it looks like you're trying to make your own fork of Sugar, which
I'm fine with, it's open source after all, but to push that on others
without their input is wrong.
If you proceed without consensus as a sole designer, then OLPC will
fork Sugar (as we already have so that XO-1s will go faster), and
you'll be making your own builds.
The problem is that rsync is used to create backups of the Journal
and no effective means is offered to restore.
Agreed. We have no restore from server feature in Sugar 0.108, along
with no way to start a backup to server, and no selective restore.
(We have backup to media, restore from media, but no selective
restore from media. Also, restore from media replaces Journal!)
However, the ultimate problem is thinking of the problem as one of
backup. If you try to solve the wrong problem, often the result is a
wasted effort.
The Journal is single place where Sugar users save their documents.
This is done by the Sugar activities when they close. The majority
of XOs are still XO-1s with a 1GB store.
This point in your argument is void, because XO-1 are 45% of the XO
laptops manufactured so far. I have the numbers.
Also, many XO-1 have been upgraded with an SD card.
If the available store is less than 50GB,
No, that's 50 MB, not 50 GB. See _SPACE_TRESHOLD (sic) in
sugar:src/jarabe/journal/journalactivity.py [3].
3.
https://github.com/sugarlabs/sugar/blob/master/src/jarabe/journal/journalactivity.py#L56
Sugar effectively shuts down.
This point in your argument is void, because this has been fixed [4,
5, 6], please upgrade to Sugar 0.108 which is in OLPC OS 13.2.7 [6].
4. http://dev.laptop.org/ticket/9623
5. https://bugs.sugarlabs.org/ticket/1720
6. http://wiki.laptop.org/go/Release_notes/13.2.7#Fixes
This typically results in the deployment reflashing the XO erasing
all of the documents created by that user - a tragedy.
It was a known bug, so that's a training issue. You previously
proposed to train a teacher to use "rm -rf" to delete a known_hosts
file instead of Manash coding up an "ssh-keygen -R" command. It is
inconsistent to be able to do one and not the other.
What I am proposing is to use the school server as the primary store
for the Journal with its effectively unlimited storage capacity. The
ds_backup script needs to read the datastore uploading any new or
modified documents. The local datastore can then be viewed as a
cache for current working documents.
I'm favour of this ideal in principle, but it remains a huge design
and consensus challenge, not a coding challenge.
However, with the XO-1, XO-1.5 and XO-1.75 using IEEE 802.11g the
local wireless network will collapse sooner due to this new load.
On the XO, the datastore is shown in the Journal. The 'keep' star
There's no such thing. There's a favorite star [7]. It has a defined
purpose. Are you proposing to destroy that purpose, or add another
column to the journal? There's even less room now that the multiple
selection checkbox was added.
7. https://help.sugarlabs.org/en/journal.html#journal-features
could be used to show whether there is a local copy of that document
or not. If the document is not needed locally, the user can clear
the star. In this case, the backup script could delete the local
copy. If there is no local copy of the document, then the user could
set the star. In this case the backup script could download the
document.
My preference would be for the flag to be in the Journal detail
view [8], where there is available display space.
8. https://help.sugarlabs.org/en/journal.html#journal-detail-view
This capability could be used to set a quota on the amount of space
used by the Journal. If the space is exceeded, the 'backup' script
could delete local copies of document by LRU until the quota is met.
Similarly, there should be a quota on Sugar activities which could
also automatically be pruned back LRU. Managing the store
automatically is consistent with keeping the Sugar UI as simple as
possible.
This should be built into Sugar rather than in the non-Sugar backup
script. They should be maintained together.
This would be a code change to git repository sugar-datastore and the
Journal activity in repository sugar.
As always, there are complications. The original OLPC concept was
that there would be one XO per user. As a result the software was
designed for a single user identified by the XO serial number.
The multiple user feature is supported by Fedora and Sugar, but we
removed it for OLPC OS.
Today, many XO deployments provide enough XOs for a classroom.
During the day, different students use the XO as their class goes to
the computer lab or as the computers are distributed from classroom
to classroom. However, all of the documents created are in a single
Journal with only the user's memory to indicate which document goes
with which user.
OLPC did not design OLPC OS to be used in this scenario, so no
surprise you've hit that. But it's not a Sugar problem. Don't
conflate Sugar with OLPC OS.
The OLPC Ubuntu Sugar 14.04 Trusty LTS (to use its official name)
solves this problem at the laptop side by using standard gnu/linux
logins.
The multiple user feature is supported by Ubuntu and Sugar, and I
haven't removed it yet. I know how to; small configuration change to
lightdm package.
Don't forget SoaS. The Fedora 23 SoaS is easily installed to disk and
has multiple user capability. The Fedora 24 SoaS is shaping up to be
just as good or better, since it is based on Sugar 0.108.
Each user has her own username and password. The Sugar activities
have been moved to common space in the file system so only one copy
is needed to support multiple users. Users are not 'olpc' but
identified by their username. However, the datastore is part of the
user space (one datastore per user).
Yes. ODPU.
This is problematic since the backup script uploads to
/library/user/serial-number on the school server.
No, you're wrong. In the Ubuntu scenario, the register_laptop
function will invent a serial number because it won't find Open
Firmware [1]. So it wouldn't be a problem. It doesn't sound like
you've tested this.
1.
https://github.com/sugarlabs/sugar/blob/master/src/jarabe/desktop/schoolserver.py#L110
So, one strategy would be to upload to /library/user/username. This
requires that usernames be unique across all laptops using a given
schoolserver. This could be enforced at registration on the school
server.
Starting to sound very complicated. Single-sign-on (SSO) across a
school. These are truly amazing teachers with lots of free
administration time.
(There are deployments using Sugar with SSO already, but as it's
outside the scope of Sugar we don't hear about them at Sugar Labs, and
we don't provide the facility in OLPC OS, but that doesn't stop them.)
However, the Sugar releases for the XO
We call that OLPC OS, which includes Sugar and Gnome desktops.
still maintains Sugar activities in /home/olpc/Activities. So, one
requirement is to restructure Sugar as was done for OLPC Ubuntu
Sugar 14.04 Trusty LTS.
That would not block implementing a server datastore, since the
implementation would not care what $HOME is set to.
(And besides, it's already done for SoaS, so the Fedora activity
packages can be used immediately.)
Another approach might be to create directories for each user of a
single XO (e.g. /library/user/serial-number/user1).
That would require authentication service by the server datastore.
Another complication is that the Browse activity downloads files
from the school server to the Journal (e.g. pdfs, mp3). These
documents do not need to be saved to the users Journal backup on the
school server since they can be restored from the school server
'library'. Also, such documents when downloaded should be stored in
a common space available to all users of that laptop. Fortunately,
the source of a document is provided in the metadata.
What you describe here can also be solved by deduplication.
The Journal Git backend proposed by Martin and Walter could help with
deduplication of journal objects across multiple journals.
https://wiki.sugarlabs.org/go/Summer_of_Code/2016#Sugar_Core
One approach would be to divide the datastore into two directories
on the laptop, one in common space and the other local to the user.
The Journal could show both sets of objects.
Or the server datastore would recognise content hashes of server
artefacts and know it need not send the content from the client to the
server before LRU local deletion. It could hard link it.
Finally, each Journal object consists of a metadata file and an
optional document. The metadata files tend to clutter the Journal
display (mine has hundreds of Terminal activity and Log activity
entries). I would propose that the Journal show only objects which
have a document with a user-supplied name (a metadata flag). The
script should backup the metadata files for those objects without a
document to a 'log' on the school server for statistical analysis
but delete them from the local datastore. Journal objects saved
without a user-supplied name (but something like Write.activity)
should have their document deleted. As part of GSOC there is an
initiative to require users to supply a name for documents they wish
to save - so this problem may not be part of the 'backup' scheme.
Whether a document is saved or deleted, the metadata can be saved to
the log and displayed by the existing statistical tools.
I'm against any classification of journal objects in this way. We
cannot know how useful a Terminal and Log activity object is to the
learner.
However, I would like a way for expert users to terminate an activity
without saving a journal object.
As an old crumudgeon, I still believe design precedes coding.
Reading the existing code is always a good idea:
Sugar
*
/usr/lib/python2.7/site-packages/jarabe/desktop/schoolserver.py
#registers server - notice transition from gconf to gsettings
* /usr/bin/ds_backup.sh #primarily decides if backup can be run
#backup logic is needed
because an rsync can use a lot of bandwidth in a local network
* /usr/bin/ds_backup.py #actually does the backup using rsync
(note: -d option AFAIK deletes an object from the backup if it is
deleted in the source,
#this has the effect of
limiting the size of the datastore to the available space on the XO
not on the school server).
Server (xsce6)
* /usr/libexec/idmgr #contains a number of utilities
used in registration
* /library/users #contains a directory per
serial-number of registered user
#use ls -a to see files
created. The idmgr creates a public/private key pair which is used
by sftp to authenticate - avoiding password
Note: if you look at the server code, you can see why registering
the laptop on each connection works (and can avoid any need for a
registration menu item).
When you get to know your way around the existing process, I'll send
you a copy of the ds_backup.py code I use to implement the item by
item backup.
You should start using GitHub like the rest of us.
