Hi Terrell, On Mon, Dec 5, 2011 at 10:27 PM, Terrell Russell <[email protected]> wrote: > Hi Jimmy, > > I'm also sitting at the intersection of these two pieces of grid-aware > software. I've copied the iRODS list on this reply. > > I suspect the performance of Tahoe-LAFS will be a major bottleneck for > using the mass storage driver. But that's just a hunch. I'd love to > see real numbers for this kind of thing instead of my idle speculation. > If you've already got your head around the pieces to build, give it a > go and see if it's usable. >
I too suspect it will be a bottleneck, I've gotten git-annex to work with tahoe-lafs in the past with git-annex's hooks (which wouldn't be too different to the iRODS equivalent) but performance wasn't great, but it did do what I needed it for, I just needed a little bit more confirmation that performance wise it would be a bit bad. I think Zooko made some suggestions in the past to access the tahoe-lafs system in a particular way to get good performance, I will need to dig up some posts and comments from git-annex's wiki to find out what they were. > Accessing web-accessible data through iRODS microservices is another option. > > However, a third option, released in iRODS 3.0, is the already-included > microservice object (MSO) drivers: > > > Microservice Object (MSO) Drivers [1] > > msiobjget_http - Gets an HTTP/HTTPS/FTP object [2] > msiobjput_http - Puts an HTTP/HTTPS/FTP object [3] > > > Please keep in touch with what you decide - iRODS currently does not > provide encryption at the file-level, so a Tahoe-LAFS integration has > been interesting to me for a while as well. > In relation to implementing a microservice I would still need to write some rules and workflows to put data on a tahoe-grid on top of creating a new microservice, I guess it is not the hardest thing to do. I haven't been keeping too up to date with iRODS development recently but I will make sure I have a look at the MSO drivers. I'm more interested in the erase-coding capabilities of tahoe-lafs rather than the crypto/security/protecting the data from prying eyes aspect as I am interested in preserving data as opposed to privacy issues. I think I need to look at the iRODS WOS driver/implementation to see if that is a good way of approaching the problem or not. I'm both personally and professionally interested in seeing some iRODS and tahoe-lafs integration as well. Anyway it's just an idea for now and I am still in the process of cooking up ideas for evaluating and analysing if its worth doing. Thanks, Jimmy. -- http://www.sgenomics.org/~jtang/ _______________________________________________ tahoe-dev mailing list [email protected] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
