On 4/29/05, Jesse Ross <[EMAIL PROTECTED]> wrote: > > In my "what about" emails serie, here's the persistence one... > > Great idea... I just have a few comments. > > > *that* much space per document, and most documents aren't that big... > > even for videos that should be doable... > > Have you done a lot of work with raw audio tracks and video footage? We're > talking about hundreds of megs or even gigs worth of data in a _single > file_. While versioning might be trivial for text documents or even > images, the amount of space that would be consumed in versioning other > media is overwhelming. It's not an excuse not to make it happen (since we > will likely not have pro quality video/audio editing apps for Etoile for a > few years, and in that time HD storage cost and RAM cost will only go > down, and processors will get faster), but just something to consider > while thinking about implementation.
Yes, I know -- but for Video, you'd likely save the list of actions rather than the whole video all the time. I guess there's probably intelligent possible strategies. It will be up to the application (or component) to actually choose what it store, anyway. Etoile should just provides something to store and retrieve the data... Something I didn't mention (well, a bit, about the overlapping projects) is that having unique files (in our "storage area") and just references to them will be extremely handy for implementing proper component/document composition and link ... > > > Now, the question (for me) is not "is it a good idea" but more, "how > > the hell are we supposed to do it" :-) > > Ummm.... can we cast a 9th-level Wizard spell "rIO's Persistent and > Versioning Documents"? I wish ! :-D > > > First problem: do we want it to be "compatible" with existing software ? > > Our native (editable) formats should be whatever work best for our system > and developers. As long as we can export files to common formats (PDF, > JPG, TXT, MP3, etc), then I think we're okay. indeed. > > Second problem: ok, it's nice to have such a "local" system.. but what > > about exchanging information with other people ? I see two solutions: > > -> have a "raw" export of your document if it's a standard (jpeg...), > > without any metadata or versioning info... > > -> have a "full" export of your document (in case you want to send it > > to another étoilé's user) > > Yep and yep, although I wouldn't call it a "raw" format... maybe a > "published" format? Yes, "published" is probably a better term. > > Now, my proposal for the technical part: use a database (PostgreSQL). > > We could either store the files directly in the database, or store > > them on the file system > > in a special "storage area". > > > > Using a database will let us easily have persistence, versioning, > > scalability and it will be extremely easy to have meta data and do > > research ... Saving the data to the database could just be saving > > binary blobs (NSData) or possibly create separate databases for > > handling documents... (using GDL2 + EOModeller)... > > > > The other point of having only one storage area (using a database or > > using multiple files on the file system) is that it will be really > > handy for our documents and projects management; the files you'd "see" > > in your project / folder will be references to the real data stored. > > Why so ? to prevent multiple copies, allow multiple view on the same > > data (David's shared/overlapping projects point). > > Seems like a good idea to me... I'm not completely sure if we want to store the files in postgresql or on the filesystem.. postgresql could be neat, although hackers will probably prefer to have their files stored on their filesystem, not on a "closed" database... ie, on the filesystem you can use normal tools. The point anyway is more the concept of a storage area where documents are stored with a unique id, and versionned. -- Nicolas Roard "Any sufficiently advanced technology is indistinguishable from magic." -Arthur C. Clarke
