Re: Data Storage and User-facing System Requirements [was Re: [sugar] 9.1 Proposal: Files]
On Thu, Oct 30, 2008 at 03:09:16PM -0400, Benjamin M. Schwartz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Erik Garrison wrote: > > It seems from my reading of mailing lists, IRC logs, and listening to > > conversations with people that we are trying to resolve all of these > > issues by implementing more code to get around difficulties imposed by > > our current data storage implementation and security model. > > > > My argument is that we can do less work and get an improved result from > > the user's perspective by removing the layers of code (datastore and > > security restrictions) which prevent applications from behaving as they > > normally do on other systems. > > Erik: If you want applications to behave as they do on other systems, > then why not just use an other system? > > I am not being facetious, and I hope I don't seem disrespectful. If you > are not interested in Sugar's goal of rearchitecting the computer > experience to optimize for our students, then don't use Sugar. It sounds > to me like your goals would be achieved, for example, by running Andres's > debxo-LXDE or the Fedora XO spin, perhaps with minor UI customizations. > Sugar is far more than its data storage system and a security model. It is a community of people trying to build a very usable computing environment specifically geared toward use in an educational setting. I think that this community should continue; I don't want to derail it! I want to help it be more effective at its core goals. If writing our own data storage systems and upstream-incompatible security models frustrates the fundamental things that Sugar wants to do (among which I believe lie the requirements which I listed), then we should reconsider devoting resources to such tasks. I think there is ample evidence that this is the case. > Sugar is not nearly finished, but it is headed for a realm of new > features, including an entirely new metaphor for stored data. You seem to > be proposing that we stop that development process because our current > betas (Sugar is still in beta) are not up to the quality of mature > systems. This upsets me. Please don't derail this train just because it > has not yet reached its destination. I don't think we should be coupling important features of computer use, such as data storage and retrieval, to systems that are relatively untested or still under heavy development and design. By making the datastore and Journal lynchpins in the system we have caused serious issues for users. There are small changes we can make to the system design which resolve this issue by decoupling data browsing and data storage. The obvious system to use is the underlying POSIX filesystem. That said, I can find no clear reason why the unique work which we want to do cannot happen on top of such a stable base layer. Why does the use of files preclude the Journal and our unique metaphor for stored data? Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Data Storage and User-facing System Requirements [was Re: [sugar] 9.1 Proposal: Files]
On Thu, 30 Oct 2008, Benjamin M. Schwartz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Erik Garrison wrote: >> It seems from my reading of mailing lists, IRC logs, and listening to >> conversations with people that we are trying to resolve all of these >> issues by implementing more code to get around difficulties imposed by >> our current data storage implementation and security model. >> >> My argument is that we can do less work and get an improved result from >> the user's perspective by removing the layers of code (datastore and >> security restrictions) which prevent applications from behaving as they >> normally do on other systems. > > Erik: If you want applications to behave as they do on other systems, > then why not just use an other system? if Sugar is only the Journal then I would definantly say to not ship Sugar until it's ready, and even then I would question it's value. however Sugar is a lot more than just the Journal. The problem is the attitude that some people seem to have that becouse Sugar is new and innovative, all existing software should be re-written to use all the new Sugar goodies. this means that until Sugar is complete and re-implements all the worlds software, there will be a significant number of people who cannot use it becouse it can't run something that they need for a system that's running Linux on a x86 cpu, the fact that it can't use the vast majority of the software available today (including the opensource software) is embarrasing. but beyond that, it limits how effective the OLPC can be by limiting the users (and as a side effect, limiting developers) it's time for everyone to give up the dream that Sugar will be the biggest OS in the world and everyone will adapt to it. (some people have acknowledged this publicly, most have accepted it privatly, but some people still seem to think that they can force the rest of the world to adapt to Sugar) Switching from 'the Journal is the main API, and we will implement some other mechansims to simulate a filesystem' to 'provide a POSIX filesystem and make the Journal be a view to that filesystem' would change the Journal from a liability to an asset. it would eliminate one of the biggest barriers for existing software (the window manager stuff would just about cover the remainder), and with the Journal being optional it can be used where it makes sense, and bypassed where it doesn't make sense (or where it's not ready yet) note that the POSIX filesystem does NOT mean that the user needs to directly access the filesystem on the nand, it could be that the file access is done through FUSE so that additional metadata can be stored along with the file. they key is that it needs to be transparent to the software so that existing software doesn't need to change. David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Data Storage and User-facing System Requirements [was Re: [sugar] 9.1 Proposal: Files]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Erik Garrison wrote: > It seems from my reading of mailing lists, IRC logs, and listening to > conversations with people that we are trying to resolve all of these > issues by implementing more code to get around difficulties imposed by > our current data storage implementation and security model. > > My argument is that we can do less work and get an improved result from > the user's perspective by removing the layers of code (datastore and > security restrictions) which prevent applications from behaving as they > normally do on other systems. Erik: If you want applications to behave as they do on other systems, then why not just use an other system? I am not being facetious, and I hope I don't seem disrespectful. If you are not interested in Sugar's goal of rearchitecting the computer experience to optimize for our students, then don't use Sugar. It sounds to me like your goals would be achieved, for example, by running Andres's debxo-LXDE or the Fedora XO spin, perhaps with minor UI customizations. Sugar is not nearly finished, but it is headed for a realm of new features, including an entirely new metaphor for stored data. You seem to be proposing that we stop that development process because our current betas (Sugar is still in beta) are not up to the quality of mature systems. This upsets me. Please don't derail this train just because it has not yet reached its destination. If you would like to argue that OLPC should not be shipping Sugar, because it is not ready, then I am happy to listen. If you would like to argue that the datastore and Journal, once their implementation catches up with our designs for the future, will still be inferior to traditional filesystems for our target users, then that is an architectural discussion we can have, though you will meet a great deal of resistance. I think you should take a look at Ubuntu Netbook Remix. It's a very stable, simple platform that presents a classic Linux desktop environment in a UI optimized for small computers. You might find that you prefer it over Sugar. There's nothing wrong with that. - --Ben -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkkKBlwACgkQUJT6e6HFtqQrBACdGR27QDeNyHkxYb58CsJjya4l gOQAni6nBGlJZ9E9/a0uqpatCaFV/cyE =O/tm -END PGP SIGNATURE- ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Data Storage and User-facing System Requirements [was Re: [sugar] 9.1 Proposal: Files]
Thank you to the repliers to my original proposal. I missed a crucial
point--- which is that I must clearly draw the lines of connection between
user-facing requirements and the solutions provided by my proposal. I
am rewriting it with this pattern in mind. If you wish to reply on the
mailing list, please write *one* email per point, lest things get too
crazy. (I will post on the OLPC wiki shortly and reply with a link.)
Proposal:
Allow activities which run under Sugar to write regular POSIX files.
Where sensible and possible with respect to user-facing requirements,
restructure our systems to expect them to behave as such.
I am basing this proposal off the idea that the system has at very least
the following requirements. The system (Sugar on the XO) should:
1) Be Bug-free, stable
2) Allow users to save data
3) Allow users to find data later
4) Allow users to share data (collaborate)
5) Afford users wide choice in the applications they can run
6) Allow users to modify the system
It seems from my reading of mailing lists, IRC logs, and listening to
conversations with people that we are trying to resolve all of these
issues by implementing more code to get around difficulties imposed by
our current data storage implementation and security model.
My argument is that we can do less work and get an improved result from
the user's perspective by removing the layers of code (datastore and
security restrictions) which prevent applications from behaving as they
normally do on other systems. For practical reasons this has immediate
rammifications for the requirements 1-5 listed above. I will now
discuss the use of file with respect to these requirements:
1) Be bug-free, stable:
Every line of new code written must be learned (both in effect and in
document) by everyone who wants to work within the framework it creates.
Provided this, it is preferable to write less code on our own, and share
existing solutions and design patterns with the larger open source
software development community. I base this in the commonly accepted
FLOSS theory that, provided feedback systems which connect users and
developers, bugs are a function of users, time, code complexity, and
changes to the codebase over time. Roughly:
complexity * changes
== number of bugs
users * time in use
Following this theory, writing our own systems decreases (users * time
in use) and increases (complexity * changes), thus increasing the number
of bugs. Using an existing system applies roughly the same bug function
found in that system to our case, provided our specific use doesn't
markedly increase complexity in code paths.
2) Allow users to save data:
Our users have potentially unique sub-requirements on this point. I
have gleaned the following from listening to discussions on olpc-related
mailing lists and in IRC logs. The system must:
a. Automatically save data.
b. Encourage use of names to make it easier to find things.
c. Don't require names for things for which naming is tedious or
unnecessary, such as photos.
Currently we meet all these features, but the implementation which we
use to do so has caused issues for users in some regards:
a. Automatically saving everything confuses users by mixing things
which they find important and things which they don't care about (the
'journal spam' problem).
b. We don't encourage naming strongly enough, even though doing so is
incredibly simple in the current UI. This confuses users when they try
to find things later.
c. We do this very well, but the lack of distinction in how we display
items in the Journal has caused issues for users (e.g. it is necessary
to click on photos to see previews of them instead of having the photos
visible in the top-level view).
Generally I have heard that most people thing we need to resolve (a) and
(b) by more strongly encouraging naming. (c) is little discussed, but
is something that 'traditional' file browsers such as Nautilus seem to
do quite well.
My impression is that if we switch to a 'stronger enforcement of naming'
scheme we may resolve (a) and (b). But also I note that switching to a
stronger enforcement of naming brings us much closer to the 'save named
files' pattern under which non-sugar applications function.
3) Allow users to find data later
Sugar's Journal is built around the idea that it should be easy for
users to find data they have produced. The exposition of this
requirement would be:
a. Provide application(s) which allow for the browsing and location of
past work in unique ways (search, tagging, metadata). The current
Sugar Journal is an example. Other similar examples are Beagle,
Tracker, and Pinot.
To help users find data later we must provide a way to browse all data
creation events on the system. This implies a need to index all data
events. We have chosen a design in which we log all user data events
by providing a
Re: [sugar] 9.1 Proposal: Files
2008/10/30 Luke Faraone <[EMAIL PROTECTED]>: > On Wed, Oct 29, 2008 at 20:50, Bill Bogstad <[EMAIL PROTECTED]> wrote: >> >> > the fact that they will quickly disappear off the screen, and may be >> > auto-deleted by the system greatly limits their value. >> >> Only if they don't get used. In which case, those entries should >> scroll off the bottom and get auto-deleted. The same way that email >> client users usually delete the 'welcome to the system' email that >> many systems generate. Once a user learns the basics of the >> activities, they will spend less and less time consulting the manual >> and the screen real-estate it takes up in the Journal is better suited >> for other things. > > Until they need to consult their manual, which sugar decided they didn't > need and expunged. The activity itself would still be accessible from Home View, just not the dummy journal entry that would launch it... Regards Morgan ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
On Wed, 29 Oct 2008, Bill Bogstad wrote: > On Wed, Oct 29, 2008 at 8:11 PM, <[EMAIL PROTECTED]> wrote: >> On Wed, 29 Oct 2008, Bill Bogstad wrote: >> > >>> Alternative proposal to encourage Journal use: >>> >>> When the system boots, start in the Activity view with the Journal; >>> not in the Home view. Pre-populate Journal with entries for the >>> recently written help manual and selected (3 or 4) entries for the >>> most commonly used/popular activities. This would emphasize the >>> importance of the Journal in tying the whole system together. By >>> putting the help manual at the top, I believe the discoverability of >>> many features/activities would be greatly increased. >> >> the fact that they will quickly disappear off the screen, and may be >> auto-deleted by the system greatly limits their value. > > Only if they don't get used. don't get used or don't get updated? if I'm viewing a document why would it move in the journal? David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
On Wed, Oct 29, 2008 at 20:50, Bill Bogstad <[EMAIL PROTECTED]> wrote: > > the fact that they will quickly disappear off the screen, and may be > > auto-deleted by the system greatly limits their value. > > Only if they don't get used. In which case, those entries should > scroll off the bottom and get auto-deleted. The same way that email > client users usually delete the 'welcome to the system' email that > many systems generate. Once a user learns the basics of the > activities, they will spend less and less time consulting the manual > and the screen real-estate it takes up in the Journal is better suited > for other things. Until they need to consult their manual, which sugar decided they didn't need and expunged. -lf ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
On Wed, Oct 29, 2008 at 8:11 PM, <[EMAIL PROTECTED]> wrote: > On Wed, 29 Oct 2008, Bill Bogstad wrote: > >> Alternative proposal to encourage Journal use: >> >> When the system boots, start in the Activity view with the Journal; >> not in the Home view. Pre-populate Journal with entries for the >> recently written help manual and selected (3 or 4) entries for the >> most commonly used/popular activities. This would emphasize the >> importance of the Journal in tying the whole system together. By >> putting the help manual at the top, I believe the discoverability of >> many features/activities would be greatly increased. > > the fact that they will quickly disappear off the screen, and may be > auto-deleted by the system greatly limits their value. Only if they don't get used. In which case, those entries should scroll off the bottom and get auto-deleted. The same way that email client users usually delete the 'welcome to the system' email that many systems generate. Once a user learns the basics of the activities, they will spend less and less time consulting the manual and the screen real-estate it takes up in the Journal is better suited for other things. The system will have adapted to their sophistication/usage pattern. All the activities and the manual will remain available in the Home view which currently has a lot more screen space anyway. The idea is to give a nudge towards a more effective way of working with the system. Right now the Home view (activity picker) is the starting point for the system. This encourages people to start activities (creating new Journal entries) every time they want to Chat or Browse the web. If the Journal was the first screen they would have a tendency to actually reuse entries instead of creating new ones. Why wasn't a key on the keyboard dedicated to the Journal? It's currently treated as not much more important then any other Activity. I'm not sure why people are surprised that people ignore it/misuse it. As for people not bothering to change/set the name on entries. Here's another simple idea. Why not change the default name of a Journal entry to be "CLICK TO NAME" (localized of course)? The icons already tells you the Activity (filetype). Repeating that info as text is a waste of screen space. Bill Bogstad ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
On Wed, 29 Oct 2008, Bill Bogstad wrote: > On Wed, Oct 29, 2008 at 1:21 AM, Benjamin M. Schwartz > <[EMAIL PROTECTED]> wrote: > >>> 2) Data access and user-perceived reliability: >> >> Problem: Users can't find things because they're not providing enough >> metadata for search to work, and there's too much stuff for them to find >> it without search. >> >> ... >> - Reduce the number of unwanted objects that accumulate >> - by providing a "draft mode" option that does not produce any object >> - but still produces a listing in a separate Actions view (see >> http://wiki.laptop.org/go/Designs/Journal) >> - by resuming previous instances by default, so as to encourage >> continued work on existing objects > > Alternative proposal to encourage Journal use: > > When the system boots, start in the Activity view with the Journal; > not in the Home view. Pre-populate Journal with entries for the > recently written help manual and selected (3 or 4) entries for the > most commonly used/popular activities. This would emphasize the > importance of the Journal in tying the whole system together. By > putting the help manual at the top, I believe the discoverability of > many features/activities would be greatly increased. the fact that they will quickly disappear off the screen, and may be auto-deleted by the system greatly limits their value. David Lang > Note: the pre-populated Journal entries would be no different then any > other entry. If the user doesn't want them anymore, they can delete > them without any other consequence. This is much like the initial > mail messages that some mail clients use to demonstrate features to > new users. > > Bill Bogstad > ___ > Devel mailing list > Devel@lists.laptop.org > http://lists.laptop.org/listinfo/devel > ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
On Wed, Oct 29, 2008 at 1:21 AM, Benjamin M. Schwartz <[EMAIL PROTECTED]> wrote: >> 2) Data access and user-perceived reliability: > > Problem: Users can't find things because they're not providing enough > metadata for search to work, and there's too much stuff for them to find > it without search. > >... > - Reduce the number of unwanted objects that accumulate > - by providing a "draft mode" option that does not produce any object > - but still produces a listing in a separate Actions view (see > http://wiki.laptop.org/go/Designs/Journal) > - by resuming previous instances by default, so as to encourage > continued work on existing objects Alternative proposal to encourage Journal use: When the system boots, start in the Activity view with the Journal; not in the Home view. Pre-populate Journal with entries for the recently written help manual and selected (3 or 4) entries for the most commonly used/popular activities. This would emphasize the importance of the Journal in tying the whole system together. By putting the help manual at the top, I believe the discoverability of many features/activities would be greatly increased. Note: the pre-populated Journal entries would be no different then any other entry. If the user doesn't want them anymore, they can delete them without any other consequence. This is much like the initial mail messages that some mail clients use to demonstrate features to new users. Bill Bogstad ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [sugar] 9.1 Proposal: Files
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mel Chua wrote: > I'm going to try to reword this as a list of features that would potentially > be good for the XO to have Me too. I should make my bias clear: I believe we should put our efforts into improving our existing datastore/Journal implementations, focusing on pure-userspace solutions with carefully considered semantics, robustness, and performance properties. I have yet to be convinced that any filesystem-like proposal can provide the behaviors that I believe we require. If we give up on these behaviors, then there is hardly any reason left to develop Sugar, and we should instead just ship a standard lightweight desktop. > 1) Compatibility with existing applications Problem: In current Sugar, Linux/X applications that are ported without source-level changes cannot retrieve and store data to the Journal. Erik's solution: Throw out the current datastore and API, and replace it with a straight POSIX filesystem. Remove the Rainbow layer that prevents Activities from accessing files to which access has not been granted explicitly by the user. Other solutions: - Throw out the current API and replace it with an API implementing similar semantics but routed through the VFS layer instead of DBUS. - Write a userspace wrapper that copies files out of the datastore using the current API to an appropriate location, and then copies the modified files back into the datastore when the file is updated (e.g. using inotify). - Replace the GTK filechooser with a Datastore-aware chooser (e.g. http://wiki.laptop.org/go/Journal%2C_reloaded) > 2) Data access and user-perceived reliability: Problem: Users can't find things because they're not providing enough metadata for search to work, and there's too much stuff for them to find it without search. Erik's solution: Only keep objects by saving them as files. This forces users either to choose a unique name for each item they produce or to close an Activity instance without producing a file. Unique names provide searchable metadata that may improve the effectiveness of search, and requiring users to type in a unique name in order to save may reduce the number of objects that are saved, allowing easier non-search navigation. Other solutions: - Use modal dialogs to encourage users to provide better metadata such as titles. (We may then easily experiment with making titles optional or mandatory.) - Reduce the number of unwanted objects that accumulate - by providing a "draft mode" option that does not produce any object - but still produces a listing in a separate Actions view (see http://wiki.laptop.org/go/Designs/Journal) - by resuming previous instances by default, so as to encourage continued work on existing objects - by automatically "starring" only those objects that have been titled - and automatically deleting the oldest unstarred objects when necessary to free space or declutter the Journal - and emphasizing or only showing starred objects in the default view of the Journal > 3) Resting on upstream stability: Problem: Our current datastore is buggy. Erik's solution: Replace the datastore with a standard Linux filesystem that already has millions of users and is maintained by upstream developers. Other solutions: - Write a dramatically simplified datastore that is less likely to have serious bugs. - Join with Gnome and other projects to produce an upstream datastore that serves our needs, is maintained independently, and has many stakeholders. - Devote more OLPC developer time to the datastore. > 4) Collaboration Problem: Users can't send Objects to each other directly over the network. Erik's solution: store Objects directly in a Linux filesystem, and then use a standard file-sharing mechanism such as an HTTP server to list these files, possibly with some access restrictions. Other solutions: - Use Scott's HTTP-based networked object sharing system that runs over Xapian and the datastore - Implement the push-to-share plan drawn up at http://wiki.laptop.org/go/Specifications/Object_Transfers > 5) Hackability Problem: A sophisticated Journal-like interface abstracts away the behaviors of the underlying filesystem. This means that using the interface does not provide children with knowledge required to understand and alter its implementation. Erik's solution: Provide a direct view into the underlying filesystem as a first-class component of the interface. In this way, children will gain familiarity with POSIX filesystem semantics and so be better prepared to modify Sugar. Other solutions: - Extend the Journal so that it is easy to write Activities without touching the filesystem. - Accept the two systems as a necessary cost in order to provide a richer user experience. - Regard the filesystem as a low-level implementation detail, as we regard a raw block device. > 6) System modularity Problem: The datastore is a linchpin of the system. If it fai
