Re: Recent Updates to Sugar Almanac
Hi Faisal, sorry to get to this so late, are there any outstanding doubts? The DS is right now a big mess due to long-standing bugs and basic features still unimplemented. I hope that during the next release significant resources will be allocated to this, if in the meantime you could suggest better APIs (both high and low level), that would be awesome. Thanks, Tomeu ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Recent Updates to Sugar Almanac
Hi Tomeu, No problem about the late reply ... I am documenting improvements as I document the current state of the system. So I'll try to suggest some more coherent api as well. Let's keep in touch about this. Faisal On Tue, Jul 1, 2008 at 4:49 AM, Tomeu Vizoso [EMAIL PROTECTED] wrote: Hi Faisal, sorry to get to this so late, are there any outstanding doubts? The DS is right now a big mess due to long-standing bugs and basic features still unimplemented. I hope that during the next release significant resources will be allocated to this, if in the meantime you could suggest better APIs (both high and low level), that would be awesome. Thanks, Tomeu ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Recent Updates to Sugar Almanac
Hi James,
Thanks for the feedback. I had two follow up questions for you or others who
are in the know with datastore:
1. You said: 'However, if your application is like Read, Read Etexts, or
View Slides, which are always resumed from existing Journal entries, there
is no need to write a file. You can still save metadata, even if you don't
write a file.'
I've written the following sample code:
def write_file(self, file_path):
logging.debug('WRITING FILE ...')
self.metadata['current_page'] = '3'
#f = open(file_path, 'w')
#try:
#f.write(Hello World)
#finally:
#f.close()
Now, when the part about writing to a file is commented out, I get the
following error when I run my activity in the terminal:
1213618380.556841 DEBUG root: Error saving activity object to datastore:
org.freedesktop.DBus.Python.IOError: Traceback (most recent call last):
File /usr/lib/python2.5/site-packages/dbus/service.py, line 655, in
_message_cb
retval = candidate_method(self, *args, **keywords)
File
/home/fanwar/sugar-jhbuild/sugar-jhbuild/install/lib/python2.5/site-packages/olpc/datastore/datastore.py,
line 475, in update
completion=lambda *args: self._update_completion_cb(async_cb,
async_err_cb, content, *args))
File
/home/fanwar/sugar-jhbuild/sugar-jhbuild/install/lib/python2.5/site-packages/olpc/datastore/backingstore.py,
line 583, in update_async
filelike = open(filelike, r)
IOError: [Errno 2] No such file or directory:
dbus.String(u'/home/fanwar/.sugar/default/org.laptop.AnnotateActivity/instance/1213618380')
So I guess the question is how do you just save metadata without writing to
a file? It seems that sugar errors out when I try to do that. Do you have
some sample code I could try out?
2. What exactly is the behavior (or at least the intended behavior) of
metadata and files in the datastore? What are the rules for persisting
metadata - is it a bug that it does not persist through reboots, or is this
intended functionality? Is the metadata-file model one where you always have
a single file in the datastore that maps to a specific activity instance and
metadata is always attached to this file? Or is there some other model for
how the metadata, files and activitities interact?
I guess at a fundamental level, I am wondering how the conception of journal
activities outlined at
http://wiki.laptop.org/go/OLPC_Human_Interface_Guidelines/The_Laptop_Experience/The_Journal#The_Journaltranslates
to some technical model for how stuff is stored. My understanding
(from talking to other sugar developers and looking at what you've written
at Beyond Hello World) has been that datastore has files and metadata for
each activity- but it seems that if everything is supposed to be abstracted
as an activity, then there should be some data structure that represents
this and then the files and metadata are there to give additional
information about the activity itself. This would avoid confusion as to
whether there must be a file or metadata or some other ancillary information
to save an activity.
Faisal
On Mon, Jun 16, 2008 at 10:13 AM, James Simmons [EMAIL PROTECTED]
wrote:
Faisal,
What you wrote looks good, but I did see a possible error in it. You say:
The following simple write_file() method shows how both metadata and
files are written. Currently, write_file() will throw an error unless
somewhere you actually write an actual file to the file_path that is
passed to write_file. The code below writes a dummy file within the
body of write_file itself (you can do this elsewhere as long as you
have a handle on the file_path variable used by write_file).
I think this is not always true. Now if you're creating a new Journal
entry for your app and have implemented write_file it *might* be true.
However, if your application is like Read, Read Etexts, or View Slides,
which are always resumed from existing Journal entries, there is no need to
write a file. You can still save metadata, even if you don't write a file.
Another thing worth pointing out is that metadata doesn't work very well
right now. If I was in the position where I needed to write a file but
didn't have anything that needed to go into it I wouldn't use metadata at
all. Instead I would put any saved options, etc. in the file, because
*that* would persist across reboots but metadata currently does not. I
think several activities already do this.
I wrote a wiki page titled Beyond Hello World that covers much the same
ground as yours, but I like yours better.
Thanks,
James Simmons
Date: Fri, 13 Jun 2008 17:34:43 -0400
From: Faisal Anwar [EMAIL PROTECTED]
Subject: Recent Updates to Sugar Almanac
To: devel@lists.laptop.org, [EMAIL PROTECTED]
Message-ID:
[EMAIL PROTECTED]
Content-Type: text/plain; charset=iso-8859-1
Hello All,
As many of you know, I'm writing up a sugar almanac to help new
sugar/python
developers get up and running with
Re: Recent Updates to Sugar Almanac
Faisal, Look at the code for the Read activity. It never creates a Journal entry itself. What it does is to resume from a directory entry created by downloading a PDF through the Browse activity or perhaps one copied from a USB key using the Journal activity. So there *is* a file in the Journal entry, but there is no desire to update that file. If you have that situation you can save metadata without writing a file, just like Read does. No special code is needed. Metadata not being preserved across a reboot is a known bug. Metadata not being preserved for entries on SD cards and USB keys is a *feature*. These devices are not considered part of the Journal proper, although the current UI makes them *look* like they should be. There are good reasons to treat files on USB devices differently from Journal entries. I understand that the USB and SD parts of the Journal may be given a different look to avoid this kind of confusion. I don't know when this will happen. The developers who could do this have a lot on their plates. If you have no data file, it would be a good idea to use a file to store persistent information about options, etc. The only time you wouldn't do that is when you have a file in the datastore that must not be modified, as in the Read activity. James Simmons Faisal Anwar wrote: Hi James, Thanks for the feedback. I had two follow up questions for you or others who are in the know with datastore: 1. You said: 'However, if your application is like Read, Read Etexts, or View Slides, which are always resumed from existing Journal entries, there is no need to write a file. You can still save metadata, even if you don't write a file.' ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Recent Updates to Sugar Almanac
Hi James, Based on your feedback, I've changed to wording for the write_file() entry to the following: *The following simple write_file() method shows how both metadata and files are written. write_file() must point to a physical file. This file reference can be created in several ways: * 1. * Your program can explicitly create a file in write_file() and write to it. * 2. * You can download or save a file to a particular location and then change the self._jobject.file_path variable (as the Read activity does).* However, I still am hesitant about writing metadata to a physical file if the activity doesn't use a physical file for anything else. At a conceptual level, this approach (to me) seems like it would confuse what is metadata and what is actual file data and puts unnecessary burden on a developer to keep track. Ideally, I'm wondering if there should just be an abstraction to an activity as having both metadata and file data and it is the prerogative of the developer to put stuff where it is appropriate (without the two necessarily interacting, as is now the case where you have to put metadata about an activity in a physical file if none exists). If I were to write a wrapper for datastore to build this abstraction, it would look something like: class ActivityInstanceInDatastore: def write_metadata_entry(self, key, value) def read_metadata_entry(self, key) def write_file(self, file_path, stuff_to_write) def read_file(self, file_path) These exist already to some extent in datastore, but it seems from this thread that the functional behavior doesn't yet follow a strict pattern of behavior. Also, there is the issue of activities having multiple data files (eg. for a file diff activity) which I believe is dealt with by setting special metadata mappings to keep track of all the files (correct me if I'm wrong). Practically, I understand there will be many cases where the current setup won't be a huge deal. However, it will be a big problem when people develop activities that may or may not be associated with a data file at different times. For example, I could envision a music activity that has specific preset music settings (such as frequency settings, volume, etc.) as well as optional files that are being manipulated by the activity. If the user doesn't have a data file open, then you would just want to save metadata, but then later if the user does open an audio file, then you will want to have both metadata and physical data. Rather than keeping track of what state you are in, it seems more coherent and natural that the developer only worries about specific calls to write metadata when he wants to write metadata (like volume and playback settings) and only worries about file-specific calls when he wants to manipulate files. Thanks Faisal On Mon, Jun 16, 2008 at 1:35 PM, James Simmons [EMAIL PROTECTED] wrote: Faisal, Look at the code for the Read activity. It never creates a Journal entry itself. What it does is to resume from a directory entry created by downloading a PDF through the Browse activity or perhaps one copied from a USB key using the Journal activity. So there *is* a file in the Journal entry, but there is no desire to update that file. If you have that situation you can save metadata without writing a file, just like Read does. No special code is needed. Metadata not being preserved across a reboot is a known bug. Metadata not being preserved for entries on SD cards and USB keys is a *feature*. These devices are not considered part of the Journal proper, although the current UI makes them *look* like they should be. There are good reasons to treat files on USB devices differently from Journal entries. I understand that the USB and SD parts of the Journal may be given a different look to avoid this kind of confusion. I don't know when this will happen. The developers who could do this have a lot on their plates. If you have no data file, it would be a good idea to use a file to store persistent information about options, etc. The only time you wouldn't do that is when you have a file in the datastore that must not be modified, as in the Read activity. James Simmons Faisal Anwar wrote: Hi James, Thanks for the feedback. I had two follow up questions for you or others who are in the know with datastore: 1. You said: 'However, if your application is like Read, Read Etexts, or View Slides, which are always resumed from existing Journal entries, there is no need to write a file. You can still save metadata, even if you don't write a file.' ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Recent Updates to Sugar Almanac
Faisal, /* You can download or save a file to a particular location and then change the self._jobject.file_path variable (as the Read activity does). */The Read activity only does the self.jobject.file_path variable when receiving a shared document. I wish I had a really old version of Read Etexts to show you. Unfortunately, by the time I got my git repository I already had the sharing code in there. But basically you don't need to do anything special to do a write_file() that doesn't write a file. You just need a Journal entry that exists and has a MIME type. Then you have an Activity that can resume any Journal entry with that MIME type. The core Read activity doesn't make PDFs. It just reads them. Then, when it closes down write_file() it saves the current page number as metadata. No file is saved. self._jobject has nothing to do with it. Activities that only read Journal entries are a bit unusual. I know a bit about them because I've written two (Read Etexts and View Slides). And there are occasions when you need to write a file. You cannot create a Journmal entry without writing a file. Even in the Read activity if you press the Save button on the Edit toolbar you'll get an error because Read is being asked to create a new Journal entry and it has no code to do this. As for using a file to store meta data, I'm not pushing it as a best practice, but a practical expedient until metadata starts working better. Several Activities already do this. In your Music activity I would save an empty file and use metadata, as you suggest. Eventually that file would be replaced with one that had music in it. In an adventure game program I would save all the state information in a file, and could easily put other kinds of meta data in there too. James Simmons ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Journal, Clipboard and activity instances vs. files (was Re: Recent Updates to Sugar Almanac)
2008/6/16 Faisal Anwar [EMAIL PROTECTED]: Rather than keeping track of what state you are in, it seems more coherent and natural that the developer only worries about specific calls to write metadata when he wants to write metadata (like volume and playback settings) and only worries about file-specific calls when he wants to manipulate files. Hey, I've had this read_file and write_file confusion before too. I think it's an issue in the design of Sugar, where there has been an attempt to mix files and activity instances. This leads to problems like when you plug external storage into the XO and all these fake activity instances appear in the Journal and have to be filtered into the datastore, even though they are really just files, and the usability issues with the Journal spending time indexing, removing file extensions, flattening path hierarchy, etc. Or the need to pop up the Journal Object chooser when we have a perfectly function clipboard on the frame. (If I were designing Sugar from scratch) I would consider linking the concepts of file with object on the clipboard. That is, you cannot start an activity from a file, you can only paste a file into a running activity instance from the clipboard (or choose the Open With item on the clipboard context menu). This takes care of the Start with issue quite nicely. And Read would not be a special case activity, you could either launch it and Paste, or launch it from the Clipboard context menu. The clipboard would become an arbitrary storage pool of mime objects. It would persist across reboots, and would provide the user interface to access external storage. In fact, it would really just be a simplified UI to a folder tree in /home/olpc/ (and /media/). This would leave the Journal as the collection of activity instances only, while the clipboard would be the collection of plain mime objects only. When you download an item in Browse, rather than going to the Journal, it would go to the clipboard. Downloaded files could be easily located in Terminal by checking /home/olpc/.sugar/clipboard/ or similar. The issue with starting activities in Develop could be reduced to the context menu of an activity having a Copy Bundle option which would copy the .xo file to the clipboard, to be pasted into a new Develop instance. The effect of all this on the API would be that activities would not have files to read and write at all, they would deal solely with metadata, which should clear up the confusion. The current APIs would be left for backwards compatibility, but whatever was written would be packaged up with the metadata and stored in the activity instance. Hope all this blue sky brainstorming isn't annoying anyone too much :) Best regards, Wade ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Journal, Clipboard and activity instances vs. files (was Re: Recent Updates to Sugar Almanac)
Wade, What you propose sounds difficult for a kid to master. For myself, I'd like external storage to be treated differently from Journal entries and to *look* like its being treated differently. External storage could have files and directories, you could navigate through them with a Midnight Commander-like interface, and you could lauch Activities from files in them if they had the needed MIME type. This interface would be for the grownups (teachers), who would be the main users of external storage. The current Journal user interface would be left pretty much alone, because it *does* work for kids. Meta data would be fixed so that it persisted across reboots. Journal entries would *not* be made for Activities that don't really need them (Terminal, Log Viewer, etc. The Activity would specify that it did not want to be resumeable). James Simmons Wade Brainerd wrote: Hey, I've had this read_file and write_file confusion before too. I think it's an issue in the design of Sugar, where there has been an attempt to mix files and activity instances. This leads to problems like when you plug external storage into the XO and all these fake activity instances appear in the Journal and have to be filtered into the datastore, even though they are really just files, and the usability issues with the Journal spending time indexing, removing file extensions, flattening path hierarchy, etc. Or the need to pop up the Journal Object chooser when we have a perfectly function clipboard on the frame. ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Journal, Clipboard and activity instances vs. files (was Re: Recent Updates to Sugar Almanac)
Hey James, It actually sounds a bit like we're talking about the same thing... On Mon, Jun 16, 2008 at 3:26 PM, James Simmons [EMAIL PROTECTED] wrote: What you propose sounds difficult for a kid to master. For myself, I'd like external storage to be treated differently from Journal entries and to *look* like its being treated differently. External storage could have files and directories, you could navigate through them with a Midnight Commander-like interface, and you could lauch Activities from files in them if they had the needed MIME type. This interface would be for the grownups (teachers), who would be the main users of external storage. Right, I was moving the external storage into the clipboard UI (extending the clipboard to handle it basically) as an alternative to introducing a third UI for managing data objects. Since the clipboard deals in MIME types already, it seemed natural to put external storage there too, and to route more data to the clipboard by default, for example downloaded files. The current Journal user interface would be left pretty much alone, because it *does* work for kids. Meta data would be fixed so that it persisted across reboots. Journal entries would *not* be made for Activities that don't really need them (Terminal, Log Viewer, etc. The Activity would specify that it did not want to be resumeable). Agreed completely, I just want to remove the files from the Journal and put them somewhere else. Also, I think Terminal could use a decent Journal implementation, that would save the environment and scrollback buffer. The problem is more that the current home UI encourages creation of new blank activity instances over resuming old ones. This is somewhat addressed in the new UI design, at the cost of some clarity in the separation between activities and activity instances. I also wrote up some ideas on how to deal with it at [[User:Wade/Ideas/Activity_Management]]. Regards, Wade ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
