Re: [Sugar-devel] Datastore rewrite
As far as I know, you could use fuse-mounted olpcfs to store the journal files, and obtain this space savings (which is technically known as deduplication) with no extra effort. You could also store the files in git for the same effect. --scott On Sat, Jun 12, 2010 at 3:07 PM, Frederick Grose fgr...@gmail.com wrote: On Sat, Jun 12, 2010 at 12:57 PM, Bernie Innocenti ber...@codewiz.org wrote: El Sat, 12-06-2010 a las 11:40 -0400, Benjamin M. Schwartz escribió: It is one thing to say that we need a new datastore, and another to say what the new datastore should look like. I believe we have consensus on the first part, and I'm fairly sure we don't have consensus on the second. I tend to agree with you. For the record, I am pushing a proposal in which no deltas are computed. Files are stored as whole files. Instead, I want each datastore object version to consist of an entire directory. To save space, files that are identical inside multiple objects would only be stored once on disk. This allows us to store and launch Activity Bundles directly from the journal. It also allows slight modifications to objects (including activities) to be stored efficiently if the object consists of multiple files and not all of them are changed. Sounds like a good approach, please ping me to review the spec when it's available. Some references here: http://wiki.sugarlabs.org/go/Design_Team/Proposals/Journal Sascha Silbe's Datastore redesign draft with embedded comments from Eben, Tomeu, Sascha: https://docs.google.com/a/sugarlabs.org/Doc?docid=0AUl2E5uTm959ZGd3N3FucXdfMWhzbjVjeGhthl=en (Sugar Labs account holders may edit this document.) As an optimization to reduce the number of inodes and vfs syscalls, perhaps it might be worthwhile to let the activity specify whether it needs to store one file or a directory with multiple files. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- ( http://cscott.net/ ) ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore rewrite
El Sat, 12-06-2010 a las 11:40 -0400, Benjamin M. Schwartz escribió: It is one thing to say that we need a new datastore, and another to say what the new datastore should look like. I believe we have consensus on the first part, and I'm fairly sure we don't have consensus on the second. I tend to agree with you. For the record, I am pushing a proposal in which no deltas are computed. Files are stored as whole files. Instead, I want each datastore object version to consist of an entire directory. To save space, files that are identical inside multiple objects would only be stored once on disk. This allows us to store and launch Activity Bundles directly from the journal. It also allows slight modifications to objects (including activities) to be stored efficiently if the object consists of multiple files and not all of them are changed. Sounds like a good approach, please ping me to review the spec when it's available. As an optimization to reduce the number of inodes and vfs syscalls, perhaps it might be worthwhile to let the activity specify whether it needs to store one file or a directory with multiple files. -- // Bernie Innocenti - http://codewiz.org/ \X/ Sugar Labs - http://sugarlabs.org/ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Datastore rewrite
On Sat, Jun 12, 2010 at 12:57 PM, Bernie Innocenti ber...@codewiz.orgwrote: El Sat, 12-06-2010 a las 11:40 -0400, Benjamin M. Schwartz escribió: It is one thing to say that we need a new datastore, and another to say what the new datastore should look like. I believe we have consensus on the first part, and I'm fairly sure we don't have consensus on the second. I tend to agree with you. For the record, I am pushing a proposal in which no deltas are computed. Files are stored as whole files. Instead, I want each datastore object version to consist of an entire directory. To save space, files that are identical inside multiple objects would only be stored once on disk. This allows us to store and launch Activity Bundles directly from the journal. It also allows slight modifications to objects (including activities) to be stored efficiently if the object consists of multiple files and not all of them are changed. Sounds like a good approach, please ping me to review the spec when it's available. Some references here: http://wiki.sugarlabs.org/go/Design_Team/Proposals/Journal Sascha Silbe's Datastore redesign draft with embedded comments from Eben, Tomeu, Sascha: https://docs.google.com/a/sugarlabs.org/Doc?docid=0AUl2E5uTm959ZGd3N3FucXdfMWhzbjVjeGhthl=en (Sugar Labs account holders may edit this document.) As an optimization to reduce the number of inodes and vfs syscalls, perhaps it might be worthwhile to let the activity specify whether it needs to store one file or a directory with multiple files. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
