Jim, The pvfs2.conf file can be anywhere you want it to be as long as the server machine has access to it. When you start the server, you specify where the conf file is located. Example:
./pvfs2-server /home/jim/pvfs2.conf The pvfs2tab file can also be in a different location. In this case, you define an environment variable, PVFS2TAB_FILE, and set it to the location. We have JUST added this feature, so I'm not sure if it builds automatically or not. I will find out for you and get back to you on that. Becky On Tue, Mar 13, 2012 at 8:34 PM, Jim Kusznir <[email protected]> wrote: > Thanks; I'll work on the rpm spec file in a bit. > > One more question on the library for preload (well, 2 questions): > > Is there a way to have the pvfs2.conf file in a location other than > /etc (eg, on systems where the admin doesn't want to modify the > filesystem of all the clusters, but we can toss stuff in our own home > directory)? > > Second, is there anything "special" I have to do to get the user > library, or is that just built by default? I'll look at the spec file > and see if there's any rpm-specific answers... > > -- > > On Tue, Mar 13, 2012 at 2:23 PM, Becky Ligon <[email protected]> wrote: > > Jim: > > > > Attached is a copy of a spec file that we use here at Clemson. Please > read > > the comments that I provided. Let me know if you have any questions. > > > > The user module is not a module but a library. So, the ld_preload > variable > > would stipulate where this library is located. When a user makes a call > to > > write(), for example, the OrangeFS pre-loaded library will check the > call, > > if a unix file is used, then the OrangeFS code is bypassed, if a OrangeFS > > file is passed in, then the write request is processed by OrangeFS. On > the > > node where the user is running his program, /etc/pvfs2tab would have to > be > > defined and include the normal mount point information (but the > filesystem > > does not have to be mounted). > > > > Hope this helps! > > > > Becky > > > > > > On Tue, Mar 13, 2012 at 2:32 PM, Jim Kusznir <[email protected]> wrote: > >> > >> I am interested in the rpm spec file. > >> > >> I'm also a bit confused about how the user module would be used...Does > >> the user just have a config file for where to find the pvfs2 servers > >> and a "mount point" and if the module is included in the ld_preload > >> env variable, anytime they "ls" or otherwise do anything with that > >> path, the module grabs it and runs it through pvfs? > >> > >> --Jim > >> > >> On Mon, Mar 12, 2012 at 3:40 PM, Becky Ligon <[email protected]> > wrote: > >> > Jim: > >> > > >> > See responses below: > >> > > >> > On Mon, Mar 12, 2012 at 3:44 PM, Jim Kusznir <[email protected]> > wrote: > >> >> > >> >> Hi all: > >> >> > >> >> I'm still on PVFS 2.8.2 on my cluster, but enough things have been > >> >> going wrong that I'm finally getting some traction toward upgrading. > >> >> Furthermore, we're beginning work on an interoperability project that > >> >> will make our pvfs2 storage available on another cluster. As I look > >> >> into OrangeFS, I'm finding that I'm full of questions, and so far > >> >> haven't found much useful documentation. There's the "high level > >> >> features", the list of open projects, but I haven't found much with > >> >> what currently works and how to build for various platforms. > >> >> > >> >> For example, on my cluster, all packages must be rpms. I've got a > >> >> spec file that I used to build both the pvfs2 userspace and a > seperate > >> >> one for the kernel modules, but for pvfs-2.8.2 (It was actually for > >> >> several versions earlier that I bumped up a few times). At this > >> >> point, I suspect I need a new .spec file, and I'm not very good at > >> >> writing them. As I recall, there used to be a .spec file in the > pvfs2 > >> >> source tarball, but I couldn't find one this time. Is there a .spec > >> >> file for building OrangeFS into rpm(s)? > >> > > >> > > >> > OrangeFS does not provide an rpm SPEC file. Here at Clemson, we > created > >> > our > >> > own rpm and I have a spec file for that; however, it is tailored to > our > >> > specific environment. If you're interested, I can send you my SPEC > file > >> > to > >> > use as a model. > >> > > >> >> > >> >> > >> >> Second question: fuse support > >> >> It seems that the vast majority of my difficulty has come with the > >> >> kernel module. Furthermore, the cluster I'm trying to intertie with > >> >> has stated that they will NOT load a kernel module on that cluster. > >> >> My users of course have no clue about MPI-IO, and have no desire to > >> >> rewrite portions of their code to make use of it; they do all their > >> >> I/O via standard filesystem calls. So, is the kernel module still my > >> >> best/only option, or is there a fuse or other module? Is the fuse > >> >> module "faster/better" than the kernel module (by staying in > >> >> userspace)? > >> >> > >> >> Of cousre, the other cluster owner would prefer that I just provide > an > >> >> NFS export for him to add to the automount table...Is that fessable? > >> > > >> > > >> > I don't recommend using NFS. It will slow down OrangeFS and not all > >> > commands work properly with an NFS mount. > >> > > >> >> > >> >> > >> >> Presently my cluster headnode is a pvfs2-client, and it > >> >> crashes/reboots frequently as a result of I/O intensive activities on > >> >> it (eg, sftps, tarball creation/expansion on a pvfs2 volume, etc). > So > >> >> re-exporting my pvfs2 volume as NFS from this node seems like a bad > >> >> idea (especially as it also hosts the user home directories for the > >> >> cluster). Also, in general it seems that my pvfs2 performance has > >> >> been pretty poor in general, especially from this node, and as such > >> >> I'm quite concerned about that. > >> > > >> > > >> > In OrangeFS, we now have a user library that you can preload that > allows > >> > you > >> > to use the system like you would with the kernel module. It bypasses > >> > the > >> > kernel module AND the client core. > >> > > >> > > >> > > >> > > >> >> > >> >> > >> >> Finally, anything special I should be aware of with OrangeFS upgrade > >> >> and interoperability? > >> >> > >> >> --Jim > >> >> _______________________________________________ > >> >> Pvfs2-users mailing list > >> >> [email protected] > >> >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > >> > > >> > > >> > > >> > > >> > -- > >> > Becky Ligon > >> > OrangeFS Support and Development > >> > Omnibond Systems > >> > Anderson, South Carolina > >> > > >> > > > > > > > > > > > -- > > Becky Ligon > > OrangeFS Support and Development > > Omnibond Systems > > Anderson, South Carolina > > > > > -- Becky Ligon OrangeFS Support and Development Omnibond Systems Anderson, South Carolina
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
