On May 26, 2009 12:44 -0700, Jeffrey Bennett wrote: > Is it technically possible to implement this without using the > Lustre client on the OSS and accessing directly to the objects on the > OSS? I guess this would imply the development of some sort of Lustre > Client/Gridftp server all together in the same code. Not that I am going > to implement it, but I am curious...
There is no such thing as "direct" access to the object. What you are describing is essentially just putting a GridFTP server accessing Lustre via the Lustre client. If you want to put a "DSI module" that just writes to a mounted client filesystem on the OSS that is fine too, but I don't see how this is significantly different than just running multiple GridFTP servers, one on each OSS. Yujun Wu <[email protected]> wrote: > As the frontend and backend are separated, the security risk is > not big. On the other hand, I have another question: if I install a > gridftp server on an OSS, it will write to that OSS ONLY or data can go > to other OSSs as well? The client would be a normal client - it can access all of the OSS storage in the Lustre filesystem. While it might be possible to create a DSI that knows the internal structure of the Lustre striping and drive the traffic directly to a specific OST, this would require a non-trivial amount of ongoing effort. It would be much simpler to just run multiple GridFTP servers on Lustre clients (whether local to the OSS nodes or on standalone clients). It would seem that even a small number of Lustre clients could accept all of the GridFTP traffic that your WAN link could handle (10Gbit/s?) The biggest obstacle to performance is the TCP data copy overhead, which is probably independent of Lustre itself. > > -----Original Message----- > > From: [email protected] [mailto:[email protected]] > > On Behalf Of Andreas Dilger > > Sent: Monday, May 25, 2009 4:14 PM > > To: Jeffrey Bennett > > Cc: Yujun Wu; [email protected] > > Subject: Re: [Lustre-discuss] gridftp and Lustre > > > > On May 22, 2009 10:33 -0700, Jeffrey Bennett wrote: > > > I guess you could implement some sort of gridftp plugin, a > > DSI module > > > is what they call it, to write data directly to the OSS and > > then you > > > could install gridftp servers on the OSS. This is basically > > what the > > > HPSS DSI module for Gridftp does, it writes data directly > > to the HPSS > > > system using the HPSS API. However, I am not sure it's technically > > > possible with Lustre. > > > > It is possible to mount Lustre clients on the OSS nodes, if > > you are willing to make at least the GridFTP port accessible > > to the outside network on > > those nodes. That said, this exposes the back-end storage > > to attack by > > a remote client, if the GridFTP server is not secure. > > > > Cheers, Andreas > > -- > > Andreas Dilger > > Sr. Staff Engineer, Lustre Group > > Sun Microsystems of Canada, Inc. > > > > Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
