Raul: The reason why we don't support locking is because clients on different machines accessing the same file do NOT share the knowledge that the file is locked. The file is "locked" at the kernel and NOT at the server, which means that another client could access/modify the file regardless of the "locking" at the kernel. If you are okay with that scenario, then you could change the kernel module to allow the locking. It's your choice.
Becky On Fri, Mar 14, 2014 at 2:13 PM, Kist <[email protected]> wrote: > :) > > Rob sent this commit in the code > http://www.orangefs.org/fisheye/orangefs/changelog/orangefs?cs=6613 > > What is supposed to happen if I... locally revert... that change... and... > recompile? > :D > > > > > > On Fri, Mar 14, 2014 at 3:07 PM, Becky Ligon <[email protected]> wrote: > >> Just saw the previous two responses. Please ignore my suggestion. >> >> Becky >> >> >> On Fri, Mar 14, 2014 at 2:05 PM, Becky Ligon <[email protected]> wrote: >> >>> Raul: >>> >>> Check the software that you are using and see if you can disable the >>> locking. What software are you using? >>> >>> Becky >>> >>> >>> On Fri, Mar 14, 2014 at 10:07 AM, Kist <[email protected]> wrote: >>> >>>> Ya! >>>> Progress! =] >>>> >>>> I looked for PVFS_ENOSYS and its alias in the code, printed some stuff >>>> and recompiled it... but I couldn't find anything. >>>> >>>> Then I managed to use the strace as Sam Sampson suggested and found my >>>> ENOSYS: >>>> (...) >>>> fcntl(4, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = -1 >>>> ENOSYS (Function not implemented) >>>> (...) >>>> fcntl(5, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = -1 >>>> ENOSYS (Function not implemented) >>>> (...) >>>> >>>> So, it seems that Rob Latham was right from the beginning. >>>> >>>> >>>> The question now is: how do I fix it? Or any idea of any workaround? >>>> >>>> Thanks a lot! >>>> >>>> >>>> On Thu, Mar 13, 2014 at 2:32 PM, Sam Sampson <[email protected]>wrote: >>>> >>>>> The best explanation is the lack of lock support. The kernel module >>>>> may also return "not implemented" for a statfs call with certain >>>>> parameters. >>>>> >>>>> >>>>> >>>>> Locking support may be added in the future, but not in the near term. >>>>> >>>>> >>>>> >>>>> You might study your software settings and seeing if locking can be >>>>> disabled. >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Sam Sampson >>>>> >>>>> Omnibond LLC >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *From:* [email protected] [mailto: >>>>> [email protected]] *On Behalf Of *Kist >>>>> *Sent:* Wednesday, March 12, 2014 8:08 PM >>>>> *To:* Becky Ligon >>>>> *Cc:* [email protected] >>>>> *Subject:* Re: [Pvfs2-users] I need some help with a "Function not >>>>> implemented" message >>>>> >>>>> >>>>> >>>>> Ya! >>>>> >>>>> Me too! >>>>> >>>>> This is why I was intrigued. >>>>> >>>>> >>>>> >>>>> This code works in the local filesystem and in the NFS. >>>>> >>>>> But it is not my code, as I was saying in the first email. :( >>>>> >>>>> >>>>> >>>>> On Wed, Mar 12, 2014 at 7:16 PM, Becky Ligon <[email protected]> >>>>> wrote: >>>>> >>>>> Raul: >>>>> >>>>> I'm not seeing any reference to OrangeFS/PVFS. Are you sorting an >>>>> OrangeFS/PVFS file? Does this code work correctly against the local >>>>> filesystem? >>>>> >>>>> Becky >>>>> >>>>> >>>>> >>>>> On Wed, Mar 12, 2014 at 5:55 PM, Kist <[email protected]> wrote: >>>>> >>>>> So, here we go: >>>>> >>>>> nothing on /var/log/messages, pvfs2-client.log, pvfs2-server.log. >>>>> >>>>> I am using the kernel module. >>>>> >>>>> Using 2.8.7 version. >>>>> >>>>> And about the strace... this client I'm using is queueing (does this >>>>> word exist?) some HPC jobs in the cluster... it seems to prepare a python >>>>> script to submit them, I'll try to edit the script so it includes the >>>>> strace in the invocation. It's a little bit messy =] >>>>> >>>>> My output in the node is this >>>>> >>>>> =========== >>>>> error in startExecution() for tool Output (2) : Function not >>>>> implemented >>>>> >>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> +++++++ Signal Handler Invoked +++++++ >>>>> +++++++ Signal : SIGSEGV +++++++ >>>>> +++++++ Meaning: Invalid memory reference +++++++ >>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> >>>>> >>>>> Stack trace is: >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libbluefin_tools.so: >>>>> ogi::backtraceutil::get(int)+0x82 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libbluefin_tools.so: >>>>> ogi::backtraceutil::maybePrint(std::ostream&)+0x62 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> ogi::SignalHandling::handleSignal(int)+0x1e1 >>>>> /lib64/libc.so.6() [0x309f032900] >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> void std::__insertion_sort<__gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, ogi::IndexComparator<unsigned >>>>> long, int> >(__gnu_cxx::__normal_iterator<long*, std::vector<long, >>>>> std::allocator<long> > >, __gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, ogi::IndexComparator<unsigned >>>>> long, int>)+0x31 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> void std::__merge_sort_with_buffer<__gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, long*, >>>>> ogi::IndexComparator<unsigned long, int> >>>>> >(__gnu_cxx::__normal_iterator<long*, std::vector<long, >>>>> std::allocator<long> > >, __gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, long*, >>>>> ogi::IndexComparator<unsigned long, int>)+0x64 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> void std::__stable_sort_adaptive<__gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, long*, long, >>>>> ogi::IndexComparator<unsigned long, int> >>>>> >(__gnu_cxx::__normal_iterator<long*, std::vector<long, >>>>> std::allocator<long> > >, __gnu_cxx::__normal_iterator<long*, >>>>> std::vector<long, std::allocator<long> > >, long*, long, >>>>> ogi::IndexComparator<unsigned long, int>)+0xd3 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> ogi::OutputMSeis::flush()+0x119 >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libopencps_engine.so: >>>>> ogi::OutputMSeis::finishExecution()+0x1c >>>>> >>>>> /net/gis/tools/opencps/opencps-time-3.1-2013-06-26/bin/linux64/lib/libbluefin_tools.so: >>>>> ogi::Tool::finishExecutionInternal()+0x17 >>>>> ========== >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Mar 11, 2014 at 12:54 PM, Mike Marshall <[email protected]> >>>>> wrote: >>>>> >>>>> Here is our implementation of lock in the kernel module: >>>>> >>>>> >>>>> >>>>> int pvfs2_lock(struct file *f, int flags, struct file_lock *lock) >>>>> >>>>> { >>>>> >>>>> return -ENOSYS; >>>>> >>>>> } >>>>> >>>>> >>>>> >>>>> Some things (sqllite is one I noticed) pitch a big fit if there is no >>>>> file lock >>>>> >>>>> mechanism supported in the kernel... >>>>> >>>>> >>>>> >>>>> -Mike >>>>> >>>>> >>>>> >>>>> On Tue, Mar 11, 2014 at 11:29 AM, Kist <[email protected]> wrote: >>>>> >>>>> Hi!! >>>>> >>>>> Thanks for the answers! >>>>> >>>>> I was just looking at the /var/log/messages, yesterday, when a small >>>>> fire started in the building's energy board and we had to evacuate it. >>>>> >>>>> I'll have access to the cluster again tomorrow (I hope). >>>>> >>>>> Then I'll check for everything you suggested. >>>>> >>>>> Sorry for the delay. >>>>> :) >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Mar 11, 2014 at 9:50 AM, Sam Sampson <[email protected]> >>>>> wrote: >>>>> >>>>> You might try running strace on the client application, and seeing >>>>> what file system call results in the error. >>>>> >>>>> Thanks, >>>>> Sam Sampson >>>>> >>>>> Omnibond Systems >>>>> >>>>> >>>>> >>>>> On Mon, Mar 10, 2014 at 1:56 PM, Becky Ligon <[email protected]> >>>>> wrote: >>>>> >>>>> Raul: >>>>> >>>>> Can you give me the exact error message that you are seeing? Is the >>>>> error message showing up in /var/log/messages, pvfs2-client.log, >>>>> pvfs2-server.log, or stdout from your program? >>>>> >>>>> Also, are you linking the application with the pvfs2 libraries or just >>>>> using the kernel module? >>>>> >>>>> Which version of OrangeFS are you using? >>>>> >>>>> Becky >>>>> >>>>> >>>>> >>>>> On Mon, Mar 10, 2014 at 12:26 PM, Rob Latham <[email protected]> wrote: >>>>> >>>>> >>>>> >>>>> On 03/10/2014 08:45 AM, Kist wrote: >>>>> >>>>> Hi! >>>>> >>>>> I'm testing OrangeFS (pvfs2) in a small virtual cluster before we try >>>>> it >>>>> on a big cluster for seismic processing. >>>>> I am running 8 CentOS with 2.6 kernel. All of them are servers (io) and >>>>> clients and the headnode is the metadata server. >>>>> >>>>> So far it's the best DFS I found and our standalone tools are working >>>>> alright on it but when I try to use our libraries with a different >>>>> client (as a plugin in another tool) I get a "Function not implemented" >>>>> error. >>>>> >>>>> I have no access to the source code of the client I'm using and, now, >>>>> I'm looking for the pvfs2 source code for "PVFS_ENOSYS" ("Function not >>>>> implemented") references but it seems a little bit more work than it >>>>> should be. >>>>> >>>>> So, I would like to know if you guys have some ideas of what could it >>>>> be >>>>> (lock?) or some magic flag I can set to check what is this. >>>>> >>>>> >>>>> >>>>> If you are using the kernel interface, the one routine we've >>>>> explicitly disabled is fcntl() with the F_SETLKW and related flags. >>>>> >>>>> slang did this years and years ago >>>>> >>>>> http://www.orangefs.org/fisheye/orangefs/changelog/orangefs?cs=6613 >>>>> >>>>> The other thing to look at is certain kinds of mmap operations, but I >>>>> never, even at my peak, remembered all the various details about that >>>>> one... >>>>> >>>>> ==rob >>>>> >>>>> -- >>>>> Rob Latham >>>>> Mathematics and Computer Science Division >>>>> Argonne National Lab, IL USA >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>>> >>>>> -- >>>>> Becky Ligon >>>>> OrangeFS Support and Development >>>>> Omnibond Systems >>>>> Anderson, South Carolina >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Raul Kist >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Raul Kist >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Raul Kist >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pvfs2-users mailing list >>>>> [email protected] >>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>>> >>>>> >>>> >>>> >>>> -- >>>> Raul Kist >>>> >>>> >>>> _______________________________________________ >>>> Pvfs2-users mailing list >>>> [email protected] >>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >>>> >>>> >>> >> > > > -- > Raul Kist > >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
