Dear Richard, On Thu, 2007-04-19 at 04:23, Richard Rodgers wrote: > Richard: > > I'm putting up a prototype implementation of (inter alia) an S3 backend > on the DSpace wiki. (see 'PluggableStorage' page). Would love volunteers > to vet it (not ready for production). > > Thanks, > > Richard R.
Without wanting to sound overly effusive, I'd just like to say how deeply grateful I am that you are working on the Amazon S3 bitstore. This is all very exciting and I hope to experiment with S3BitStore once I am finished migrating Indica et Buddhica to Joyent/TextDrive, hopefully by the end of the month.** ... Something I'd like to ask before then though. Presently all the material I hold on S3 consists of encrypted compressed tar balls (Solaris 10: gtar, bzip2, encrypt). These can be created using UNIX pipes, similar to producing encrypted tape backups. How hard would it be, then, to use S3BitStore to send encrypted, possibly compressed, data to an assetstore on S3? I already send and retrieve all material using SSL. It seems to me that the addition of data encryption and compression would certainly go some way to reassuring an institution wishing to archive sensitive material, cost effectively. Would all of this be non-trivial? Any thoughts. Kind regards, Richard M. ** I think I recall reading a while ago on this list about firms, notably TextDrive, being unwilling to host Java apps. It seemed that if one wished to run DSpace one needed a dedicated machine. This is no longer the case. See Joyent/TextDrive's Accelerators: http://radiant.joyent.com/accelerator/ > On Thu, 2007-04-12 at 09:49 +1200, Richard MAHONEY wrote: > > Dear Robert et al., > > > > On Thu, 2007-04-12 at 07:15, Robert Tansley wrote: > > > We considered this way back when (2001); we decided on using the > > > filesystem because some files might be very very large, there might be > > > lots of them and in general it's easier to split filesystem-based > > > asset stores across multiple drives/machines than a big relational > > > database. > > > > > > That said, the intention was that storage would be made pluggable -- > > > so you could have RDBMS, SRB/iRODs, open-source GoogleFileSystem, > > > LOCKSS-ish etc. storage. That pluggability ended up being one of the > > > many non-critical-for-version-1 features we had to drop to get DSpace > > > 1.0 finished :-) There are some projects (e.g. the MIT ones) looking > > > at how to really accomplish this. > > > > Over the past few weeks I've been using Amazon's Simple Storage Service > > (S3): > > > > http://www.amazon.com/gp/browse.html?node=16427261 > > > > At this point I've merely been using it to backup web servers and > > development directories. This has involved the simple upload of > > compressed tarballs (using the Java app. jSh3ll) but also the > > synchronising of file systems (using the Ruby app. s3sync). > > > > In all, I've been pleasantly surprised by the results. It would seem > > that the S3 storage system promises to be more resilient than anything > > I could build at a reasonable cost. > > > > Although I've only been using S3 for remote backup, it seems that it > > can also be used as a live file system for storing and retrieving data > > for web apps. I am wondering then, if anyone, may be able to suggest > > how it might be possible to configure (cajole) DSpace-1.4 into using S3 > > as an assetstore. The Amazon blurb says that S3: > > > > `Uses standards-based REST and SOAP interfaces designed to work with any > > Internet-development toolkit.' > > > > > > Best regards, > > > > Richard MAHONEY > > > > > > -- Richard MAHONEY | internet: http://indica-et-buddhica.org/ Littledene | telephone/telefax (man.): +64 3 312 1699 Bay Road | cellular: +64 27 482 9986 OXFORD, NZ | email: [EMAIL PROTECTED] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Indica et Buddhica: Materials for Indology and Buddhology Repositorium: http://indica-et-buddhica.org/repositorium/ Philologica: http://indica-et-buddhica.org/philologica/ Subscriptions: http://subscriptions.indica-et-buddhica.org/ ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ DSpace-tech mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-tech
