Jim: In our next release, 2.8.6, the user library will be automatically generated into the lib directory and is called (at least right now) libofs.a
Becky On Wed, Mar 14, 2012 at 9:55 AM, Becky Ligon <[email protected]> wrote: > Jim, > > The pvfs2.conf file can be anywhere you want it to be as long as the > server machine has access to it. When you start the server, you specify > where the conf file is located. Example: > > ./pvfs2-server /home/jim/pvfs2.conf > > The pvfs2tab file can also be in a different location. In this case, you > define an environment variable, PVFS2TAB_FILE, and set it to the location. > > We have JUST added this feature, so I'm not sure if it builds > automatically or not. I will find out for you and get back to you on that. > > Becky > > > On Tue, Mar 13, 2012 at 8:34 PM, Jim Kusznir <[email protected]> wrote: > >> Thanks; I'll work on the rpm spec file in a bit. >> >> One more question on the library for preload (well, 2 questions): >> >> Is there a way to have the pvfs2.conf file in a location other than >> /etc (eg, on systems where the admin doesn't want to modify the >> filesystem of all the clusters, but we can toss stuff in our own home >> directory)? >> >> Second, is there anything "special" I have to do to get the user >> library, or is that just built by default? I'll look at the spec file >> and see if there's any rpm-specific answers... >> >> -- >> >> On Tue, Mar 13, 2012 at 2:23 PM, Becky Ligon <[email protected]> wrote: >> > Jim: >> > >> > Attached is a copy of a spec file that we use here at Clemson. Please >> read >> > the comments that I provided. Let me know if you have any questions. >> > >> > The user module is not a module but a library. So, the ld_preload >> variable >> > would stipulate where this library is located. When a user makes a >> call to >> > write(), for example, the OrangeFS pre-loaded library will check the >> call, >> > if a unix file is used, then the OrangeFS code is bypassed, if a >> OrangeFS >> > file is passed in, then the write request is processed by OrangeFS. On >> the >> > node where the user is running his program, /etc/pvfs2tab would have to >> be >> > defined and include the normal mount point information (but the >> filesystem >> > does not have to be mounted). >> > >> > Hope this helps! >> > >> > Becky >> > >> > >> > On Tue, Mar 13, 2012 at 2:32 PM, Jim Kusznir <[email protected]> >> wrote: >> >> >> >> I am interested in the rpm spec file. >> >> >> >> I'm also a bit confused about how the user module would be used...Does >> >> the user just have a config file for where to find the pvfs2 servers >> >> and a "mount point" and if the module is included in the ld_preload >> >> env variable, anytime they "ls" or otherwise do anything with that >> >> path, the module grabs it and runs it through pvfs? >> >> >> >> --Jim >> >> >> >> On Mon, Mar 12, 2012 at 3:40 PM, Becky Ligon <[email protected]> >> wrote: >> >> > Jim: >> >> > >> >> > See responses below: >> >> > >> >> > On Mon, Mar 12, 2012 at 3:44 PM, Jim Kusznir <[email protected]> >> wrote: >> >> >> >> >> >> Hi all: >> >> >> >> >> >> I'm still on PVFS 2.8.2 on my cluster, but enough things have been >> >> >> going wrong that I'm finally getting some traction toward upgrading. >> >> >> Furthermore, we're beginning work on an interoperability project >> that >> >> >> will make our pvfs2 storage available on another cluster. As I look >> >> >> into OrangeFS, I'm finding that I'm full of questions, and so far >> >> >> haven't found much useful documentation. There's the "high level >> >> >> features", the list of open projects, but I haven't found much with >> >> >> what currently works and how to build for various platforms. >> >> >> >> >> >> For example, on my cluster, all packages must be rpms. I've got a >> >> >> spec file that I used to build both the pvfs2 userspace and a >> seperate >> >> >> one for the kernel modules, but for pvfs-2.8.2 (It was actually for >> >> >> several versions earlier that I bumped up a few times). At this >> >> >> point, I suspect I need a new .spec file, and I'm not very good at >> >> >> writing them. As I recall, there used to be a .spec file in the >> pvfs2 >> >> >> source tarball, but I couldn't find one this time. Is there a .spec >> >> >> file for building OrangeFS into rpm(s)? >> >> > >> >> > >> >> > OrangeFS does not provide an rpm SPEC file. Here at Clemson, we >> created >> >> > our >> >> > own rpm and I have a spec file for that; however, it is tailored to >> our >> >> > specific environment. If you're interested, I can send you my SPEC >> file >> >> > to >> >> > use as a model. >> >> > >> >> >> >> >> >> >> >> >> Second question: fuse support >> >> >> It seems that the vast majority of my difficulty has come with the >> >> >> kernel module. Furthermore, the cluster I'm trying to intertie with >> >> >> has stated that they will NOT load a kernel module on that cluster. >> >> >> My users of course have no clue about MPI-IO, and have no desire to >> >> >> rewrite portions of their code to make use of it; they do all their >> >> >> I/O via standard filesystem calls. So, is the kernel module still >> my >> >> >> best/only option, or is there a fuse or other module? Is the fuse >> >> >> module "faster/better" than the kernel module (by staying in >> >> >> userspace)? >> >> >> >> >> >> Of cousre, the other cluster owner would prefer that I just provide >> an >> >> >> NFS export for him to add to the automount table...Is that fessable? >> >> > >> >> > >> >> > I don't recommend using NFS. It will slow down OrangeFS and not all >> >> > commands work properly with an NFS mount. >> >> > >> >> >> >> >> >> >> >> >> Presently my cluster headnode is a pvfs2-client, and it >> >> >> crashes/reboots frequently as a result of I/O intensive activities >> on >> >> >> it (eg, sftps, tarball creation/expansion on a pvfs2 volume, etc). >> So >> >> >> re-exporting my pvfs2 volume as NFS from this node seems like a bad >> >> >> idea (especially as it also hosts the user home directories for the >> >> >> cluster). Also, in general it seems that my pvfs2 performance has >> >> >> been pretty poor in general, especially from this node, and as such >> >> >> I'm quite concerned about that. >> >> > >> >> > >> >> > In OrangeFS, we now have a user library that you can preload that >> allows >> >> > you >> >> > to use the system like you would with the kernel module. It bypasses >> >> > the >> >> > kernel module AND the client core. >> >> > >> >> > >> >> > >> >> > >> >> >> >> >> >> >> >> >> Finally, anything special I should be aware of with OrangeFS upgrade >> >> >> and interoperability? >> >> >> >> >> >> --Jim >> >> >> _______________________________________________ >> >> >> Pvfs2-users mailing list >> >> >> [email protected] >> >> >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > Becky Ligon >> >> > OrangeFS Support and Development >> >> > Omnibond Systems >> >> > Anderson, South Carolina >> >> > >> >> > >> > >> > >> > >> > >> > -- >> > Becky Ligon >> > OrangeFS Support and Development >> > Omnibond Systems >> > Anderson, South Carolina >> > >> > >> > > > > -- > Becky Ligon > OrangeFS Support and Development > Omnibond Systems > Anderson, South Carolina > > > -- Becky Ligon OrangeFS Support and Development Omnibond Systems Anderson, South Carolina
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
