Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Marc Eshel 
I see, I tried your fix and it did work for the case that I had problems 
with.
Thanks, Marc.



From:   Frank Filz 
To: Marc Eshel/Almaden/IBM@IBMUS
Cc: nfs-ganesha-devel 
Date:   09/07/2016 09:14 PM
Subject:Re: [Nfs-ganesha-devel] NLM async locking



Request is resubmitted to the FSAL in the cookie processing (sorry don't 
have code handy. Follow the whole chain. It's several steps...)

Frank

Sent from my iPhone

> On Sep 7, 2016, at 8:35 PM, Marc Eshel  wrote:
> 
> It will not help if the request is not resubmitted to the FSAL.
> Marc.
> 
> 
> 
> From:   "Frank Filz" 
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "'nfs-ganesha-devel'" 
> Date:   09/07/2016 04:18 PM
> Subject:RE: [Nfs-ganesha-devel] NLM async locking
> 
> 
> 
> I changed it so we don't take the lock off the list when it find the 
> blocked
> lock entry in the blocked lock list.
> 
> It then removes it from the list only if sending the async grant 
succeeds.
> 
> It probably needs some tweaking, but it SHOULD help.
> 
> Frank
> 
>> -Original Message-
>> From: Marc Eshel [mailto:es...@us.ibm.com]
>> Sent: Wednesday, September 7, 2016 4:11 PM
>> To: Frank Filz 
>> Cc: 'nfs-ganesha-devel' 
>> Subject: RE: [Nfs-ganesha-devel] NLM async locking
>> 
>> Just looking at the code I don't see where you retry the lock request
> from
>> the FSAL which is required to add it back to the FSAL queue.
>> Am I missing something?
>> Marc.
>> 
>> 
>> 
>> From:   "Frank Filz" 
>> To: Marc Eshel/Almaden/IBM@IBMUS






--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Frank Filz 
Request is resubmitted to the FSAL in the cookie processing (sorry don't have 
code handy. Follow the whole chain. It's several steps...)

Frank

Sent from my iPhone

> On Sep 7, 2016, at 8:35 PM, Marc Eshel  wrote:
> 
> It will not help if the request is not resubmitted to the FSAL.
> Marc.
> 
> 
> 
> From:   "Frank Filz" 
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "'nfs-ganesha-devel'" 
> Date:   09/07/2016 04:18 PM
> Subject:RE: [Nfs-ganesha-devel] NLM async locking
> 
> 
> 
> I changed it so we don't take the lock off the list when it find the 
> blocked
> lock entry in the blocked lock list.
> 
> It then removes it from the list only if sending the async grant succeeds.
> 
> It probably needs some tweaking, but it SHOULD help.
> 
> Frank
> 
>> -Original Message-
>> From: Marc Eshel [mailto:es...@us.ibm.com]
>> Sent: Wednesday, September 7, 2016 4:11 PM
>> To: Frank Filz 
>> Cc: 'nfs-ganesha-devel' 
>> Subject: RE: [Nfs-ganesha-devel] NLM async locking
>> 
>> Just looking at the code I don't see where you retry the lock request
> from
>> the FSAL which is required to add it back to the FSAL queue.
>> Am I missing something?
>> Marc.
>> 
>> 
>> 
>> From:   "Frank Filz" 
>> To: Marc Eshel/Almaden/IBM@IBMUS


--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Marc Eshel 
It will not help if the request is not resubmitted to the FSAL.
Marc.



From:   "Frank Filz" 
To: Marc Eshel/Almaden/IBM@IBMUS
Cc: "'nfs-ganesha-devel'" 
Date:   09/07/2016 04:18 PM
Subject:RE: [Nfs-ganesha-devel] NLM async locking



I changed it so we don't take the lock off the list when it find the 
blocked
lock entry in the blocked lock list.

It then removes it from the list only if sending the async grant succeeds.

It probably needs some tweaking, but it SHOULD help.

Frank

> -Original Message-
> From: Marc Eshel [mailto:es...@us.ibm.com]
> Sent: Wednesday, September 7, 2016 4:11 PM
> To: Frank Filz 
> Cc: 'nfs-ganesha-devel' 
> Subject: RE: [Nfs-ganesha-devel] NLM async locking
> 
> Just looking at the code I don't see where you retry the lock request 
from
> the FSAL which is required to add it back to the FSAL queue.
> Am I missing something?
> Marc.
> 
> 
> 
> From:   "Frank Filz" 
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "'nfs-ganesha-devel'" 
> Date:   09/07/2016 03:57 PM
> Subject:RE: [Nfs-ganesha-devel] NLM async locking
> 
> 
> 
> Marc,
> 
> Could you try the top commit in this branch:
> 
> https://github.com/ffilz/nfs-ganesha/commits/async
> 
> It may not be the complete solution, but I think it will help your
> scenario.
> 
> I need to do more work on async blocking locks...
> 
> And it looks like without async blocking lock support, Ganesha doesn't
> handle the case where a lock blocks on a conflicting lock from outside 
the
> Ganesha instance. I will be looking at implementing my thread pool idea
> that
> I modeled in the multilock tool.
> 
> Frank
> 
> > -Original Message-
> > From: Frank Filz [mailto:ffilz...@mindspring.com]
> > Sent: Wednesday, September 7, 2016 9:42 AM
> > To: 'Marc Eshel' 
> > Cc: 'nfs-ganesha-devel' 
> > Subject: Re: [Nfs-ganesha-devel] NLM async locking
> >
> > Ok, I'm not sure this ever worked right...
> >
> > With the lock available upcall, we never put the lock back on the
> blocked
> lock
> > list if an attempt to acquire the lock from the FSAL fails...
> >
> > So the way the lock available upcall is supposed to work:
> >
> > Client requests conflicting lock
> > Blocked lock gets registered by FSAL
> > SAL puts lock on blocked lock list
> > Time passes
> > FSAL makes lock available upcall
> > SAL finds the blocked lock entry in the blocked lock list SAL makes a
> call
> to
> > FSAL to attempt to acquire the lock Assume that fails (in the example,
> > because multiple conflicting locks got
> > notified)
> > SAL puts the lock BACK on the blocked lock list (this step is missing)
> and
> all is
> > well
> > Time passes
> > FSAL makes lock available upcall
> > SAL finds the blocked lock entry in the blocked lock list SAL makes a
> call
> to
> > FSAL to attempt to acquire the lock Lock is granted by FSAL SAL makes
> async
> > call back to client If THAT fails, SAL releases the lock from the FSAL
> and
> > disposes of the lock entry and all is well If THAT succeeds, the lock 
is
> > completely granted and all is well
> >
> > I also see that if the client retries the lock before it is granted, 
we
> don't
> > remove the lock entry from the blocked lock list... I don't think that
> will ever
> > cause a problem but we should clean that up also...
> >
> > Let me try a patch to fix...
> >
> > Frank
> >
> > > -Original Message-
> > > From: Marc Eshel [mailto:es...@us.ibm.com]
> > > Sent: Tuesday, September 6, 2016 9:34 PM
> > > To: Frank Filz 
> > > Cc: 'nfs-ganesha-devel' 
> > > Subject: RE: NLM async locking
> > >
> > > Did you get a chance to look at this problem?
> > > Marc.
> > >
> > >
> > >
> > > From:   "Frank Filz" 
> > > To: Marc Eshel/Almaden/IBM@IBMUS
> > > Cc: "'nfs-ganesha-devel'"
> 
> > > Date:   08/29/2016 02:37 PM
> > > Subject:RE: NLM async locking
> > >
> > >
> > >
> > > > I see the following failure:
> > > > 1. Get conflicting locks from 3 clients
> > > > cli 1 gets 0-100
> > > > cli 2 is blocked on 0-1000
> > > > cli 3 is blocked on 0-1
> > > > 2. cli 1 unlocks
> > > > up-call for cli 2 and 3 to retry
> > > > cli 2 gets 0-1000
> > > > cli 3 is blocked on 0-1000
> > > > 3. cli 2 unlocks
> > > > up-call for cli 3 but Ganesha fails
> > > >
> > > > /* We must be out of sync with FSAL, this is fatal */
> > > > LogLockDesc(COMPONENT_STATE, NIV_MAJ, "Blocked Lock Not
> > > > Found for"
> > > > ,
> > > > obj, owner, lock);
> > > > LogFatal(COMPONENT_STATE, "Locks out of sync with FSAL");
> > > >
> > > > I think the problem

[Nfs-ganesha-devel] Blocking locks in FSALs

2016-09-07 Thread Frank Filz 
As I have dug into things, I realize blocking locks (used by NFS v3 NLM
clients) are broken for FSALs that don't support async blocking locks.

It looks like libgfapi and libcephfs both support blocking locks (libgfapi
with F_SETLKW and libcephfs by passing the last param (sleep) to
ceph_ll_setlk as true). If this is the case, I propose adding pseudo-async
blocking lock support to SAL for when support_ex is true but
lock_support_async_block is false. 

The idea is the following:

There will be a configurable number of blocking lock threads.

Any time a blocking lock is not able to be granted immediately (we will
first use a non-blocking lock request to the FSAL), and the FSAL doesn't
support async blocking locks, it gets put on a queue.
 
The blocked lock threads will pull items off the queue and do a blocking
lock request to the FSAL. This will have to be cancelable somehow (that may
be an issue, I need thoughts from GLUSTER and CEPH for that, for FSAL_VFS,
fcntl can be interrupted by a SIGIO to the thread).

Another single thread will poll any blocked locks not handled by an async
capable FSAL or a blocking lock thread. It will poll each lock at some
modest interval (with a non-blocking lock).

This all is modeled in the multilock tool in ml_posix_client. I don't know
if I actually tested it, but it's also modeled for libcephfs with
ml_ceph_client.

I'd appreciate any input on this, most especially from Gluster folks.

Thanks

Frank


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Frank Filz 
I changed it so we don't take the lock off the list when it find the blocked
lock entry in the blocked lock list.

It then removes it from the list only if sending the async grant succeeds.

It probably needs some tweaking, but it SHOULD help.

Frank

> -Original Message-
> From: Marc Eshel [mailto:es...@us.ibm.com]
> Sent: Wednesday, September 7, 2016 4:11 PM
> To: Frank Filz 
> Cc: 'nfs-ganesha-devel' 
> Subject: RE: [Nfs-ganesha-devel] NLM async locking
> 
> Just looking at the code I don't see where you retry the lock request from
> the FSAL which is required to add it back to the FSAL queue.
> Am I missing something?
> Marc.
> 
> 
> 
> From:   "Frank Filz" 
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "'nfs-ganesha-devel'" 
> Date:   09/07/2016 03:57 PM
> Subject:RE: [Nfs-ganesha-devel] NLM async locking
> 
> 
> 
> Marc,
> 
> Could you try the top commit in this branch:
> 
> https://github.com/ffilz/nfs-ganesha/commits/async
> 
> It may not be the complete solution, but I think it will help your
> scenario.
> 
> I need to do more work on async blocking locks...
> 
> And it looks like without async blocking lock support, Ganesha doesn't
> handle the case where a lock blocks on a conflicting lock from outside the
> Ganesha instance. I will be looking at implementing my thread pool idea
> that
> I modeled in the multilock tool.
> 
> Frank
> 
> > -Original Message-
> > From: Frank Filz [mailto:ffilz...@mindspring.com]
> > Sent: Wednesday, September 7, 2016 9:42 AM
> > To: 'Marc Eshel' 
> > Cc: 'nfs-ganesha-devel' 
> > Subject: Re: [Nfs-ganesha-devel] NLM async locking
> >
> > Ok, I'm not sure this ever worked right...
> >
> > With the lock available upcall, we never put the lock back on the
> blocked
> lock
> > list if an attempt to acquire the lock from the FSAL fails...
> >
> > So the way the lock available upcall is supposed to work:
> >
> > Client requests conflicting lock
> > Blocked lock gets registered by FSAL
> > SAL puts lock on blocked lock list
> > Time passes
> > FSAL makes lock available upcall
> > SAL finds the blocked lock entry in the blocked lock list SAL makes a
> call
> to
> > FSAL to attempt to acquire the lock Assume that fails (in the example,
> > because multiple conflicting locks got
> > notified)
> > SAL puts the lock BACK on the blocked lock list (this step is missing)
> and
> all is
> > well
> > Time passes
> > FSAL makes lock available upcall
> > SAL finds the blocked lock entry in the blocked lock list SAL makes a
> call
> to
> > FSAL to attempt to acquire the lock Lock is granted by FSAL SAL makes
> async
> > call back to client If THAT fails, SAL releases the lock from the FSAL
> and
> > disposes of the lock entry and all is well If THAT succeeds, the lock is
> > completely granted and all is well
> >
> > I also see that if the client retries the lock before it is granted, we
> don't
> > remove the lock entry from the blocked lock list... I don't think that
> will ever
> > cause a problem but we should clean that up also...
> >
> > Let me try a patch to fix...
> >
> > Frank
> >
> > > -Original Message-
> > > From: Marc Eshel [mailto:es...@us.ibm.com]
> > > Sent: Tuesday, September 6, 2016 9:34 PM
> > > To: Frank Filz 
> > > Cc: 'nfs-ganesha-devel' 
> > > Subject: RE: NLM async locking
> > >
> > > Did you get a chance to look at this problem?
> > > Marc.
> > >
> > >
> > >
> > > From:   "Frank Filz" 
> > > To: Marc Eshel/Almaden/IBM@IBMUS
> > > Cc: "'nfs-ganesha-devel'"
> 
> > > Date:   08/29/2016 02:37 PM
> > > Subject:RE: NLM async locking
> > >
> > >
> > >
> > > > I see the following failure:
> > > > 1. Get conflicting locks from 3 clients
> > > > cli 1 gets 0-100
> > > > cli 2 is blocked on 0-1000
> > > > cli 3 is blocked on 0-1
> > > > 2. cli 1 unlocks
> > > > up-call for cli 2 and 3 to retry
> > > > cli 2 gets 0-1000
> > > > cli 3 is blocked on 0-1000
> > > > 3. cli 2 unlocks
> > > > up-call for cli 3 but Ganesha fails
> > > >
> > > > /* We must be out of sync with FSAL, this is fatal */
> > > > LogLockDesc(COMPONENT_STATE, NIV_MAJ, "Blocked Lock Not
> > > > Found for"
> > > > ,
> > > > obj, owner, lock);
> > > > LogFatal(COMPONENT_STATE, "Locks out of sync with FSAL");
> > > >
> > > > I think the problem is in step 2, after cli 3 failed for the second
> > > > time
> > > it is not
> > > > put back in queue, the sbd_list.
> > > >
> > > > Can you please confirm this logic is very complicated.
> > >
> > > That sounds like a likely problem. I'd have to dig into the code to
> > > see
> > why...
> > > May take me a day or two to

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Marc Eshel 
Just looking at the code I don't see where you retry the lock request from 
the FSAL which is required to add it back to the FSAL queue.
Am I missing something?
Marc.



From:   "Frank Filz" 
To: Marc Eshel/Almaden/IBM@IBMUS
Cc: "'nfs-ganesha-devel'" 
Date:   09/07/2016 03:57 PM
Subject:RE: [Nfs-ganesha-devel] NLM async locking



Marc,

Could you try the top commit in this branch:

https://github.com/ffilz/nfs-ganesha/commits/async

It may not be the complete solution, but I think it will help your 
scenario.

I need to do more work on async blocking locks...

And it looks like without async blocking lock support, Ganesha doesn't
handle the case where a lock blocks on a conflicting lock from outside the
Ganesha instance. I will be looking at implementing my thread pool idea 
that
I modeled in the multilock tool.

Frank

> -Original Message-
> From: Frank Filz [mailto:ffilz...@mindspring.com]
> Sent: Wednesday, September 7, 2016 9:42 AM
> To: 'Marc Eshel' 
> Cc: 'nfs-ganesha-devel' 
> Subject: Re: [Nfs-ganesha-devel] NLM async locking
> 
> Ok, I'm not sure this ever worked right...
> 
> With the lock available upcall, we never put the lock back on the 
blocked
lock
> list if an attempt to acquire the lock from the FSAL fails...
> 
> So the way the lock available upcall is supposed to work:
> 
> Client requests conflicting lock
> Blocked lock gets registered by FSAL
> SAL puts lock on blocked lock list
> Time passes
> FSAL makes lock available upcall
> SAL finds the blocked lock entry in the blocked lock list SAL makes a 
call
to
> FSAL to attempt to acquire the lock Assume that fails (in the example,
> because multiple conflicting locks got
> notified)
> SAL puts the lock BACK on the blocked lock list (this step is missing) 
and
all is
> well
> Time passes
> FSAL makes lock available upcall
> SAL finds the blocked lock entry in the blocked lock list SAL makes a 
call
to
> FSAL to attempt to acquire the lock Lock is granted by FSAL SAL makes
async
> call back to client If THAT fails, SAL releases the lock from the FSAL 
and
> disposes of the lock entry and all is well If THAT succeeds, the lock is
> completely granted and all is well
> 
> I also see that if the client retries the lock before it is granted, we
don't
> remove the lock entry from the blocked lock list... I don't think that
will ever
> cause a problem but we should clean that up also...
> 
> Let me try a patch to fix...
> 
> Frank
> 
> > -Original Message-
> > From: Marc Eshel [mailto:es...@us.ibm.com]
> > Sent: Tuesday, September 6, 2016 9:34 PM
> > To: Frank Filz 
> > Cc: 'nfs-ganesha-devel' 
> > Subject: RE: NLM async locking
> >
> > Did you get a chance to look at this problem?
> > Marc.
> >
> >
> >
> > From:   "Frank Filz" 
> > To: Marc Eshel/Almaden/IBM@IBMUS
> > Cc: "'nfs-ganesha-devel'" 

> > Date:   08/29/2016 02:37 PM
> > Subject:RE: NLM async locking
> >
> >
> >
> > > I see the following failure:
> > > 1. Get conflicting locks from 3 clients
> > > cli 1 gets 0-100
> > > cli 2 is blocked on 0-1000
> > > cli 3 is blocked on 0-1
> > > 2. cli 1 unlocks
> > > up-call for cli 2 and 3 to retry
> > > cli 2 gets 0-1000
> > > cli 3 is blocked on 0-1000
> > > 3. cli 2 unlocks
> > > up-call for cli 3 but Ganesha fails
> > >
> > > /* We must be out of sync with FSAL, this is fatal */
> > > LogLockDesc(COMPONENT_STATE, NIV_MAJ, "Blocked Lock Not
> > > Found for"
> > > ,
> > > obj, owner, lock);
> > > LogFatal(COMPONENT_STATE, "Locks out of sync with FSAL");
> > >
> > > I think the problem is in step 2, after cli 3 failed for the second
> > > time
> > it is not
> > > put back in queue, the sbd_list.
> > >
> > > Can you please confirm this logic is very complicated.
> >
> > That sounds like a likely problem. I'd have to dig into the code to
> > see
> why...
> > May take me a day or two to investigate.
> >
> > Frank
> >
> >
> >
> > ---
> > This email has been checked for viruses by Avast antivirus software.
> > https://www.avast.com/antivirus
> >
> >
> >
> 
> 
> 
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
> 
> 
>

--
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus






--

Re: [Nfs-ganesha-devel] lock_op

2016-09-07 Thread Frank Filz 
Owner is an opaque value to the FSAL. It happens to be the pointer to a 
state_lock_owner_t so Ganesha does expect it back on call backs when async 
blocking locks are supported (see other e-mail chain for issues that are 
arising without that support). Ganesha never dereferences the owner value from 
the FSAL however, it just searches its blocked locks list for locks with the 
same owner.



The FSAL may use it in any way it chooses to differentiate lock owners.



I should note that all new FSAL development should use the support_ex API, 
where there will be a state_t object for each lock owner/file pair. This 
actually allows the FSAL to define it’s own lock owner in whatever way makes 
sense (but the opaque void * lock owner is also still available).



I hope that helps,



Frank



From: Sagar M D [mailto:sagar...@gmail.com]
Sent: Wednesday, September 7, 2016 3:37 PM
To: nfs-ganesha-devel@lists.sourceforge.net
Subject: [Nfs-ganesha-devel] lock_op



Hi All,



could someone please shed light on the usage of "owner" in fsal side ? Is 
"owner" implemented as comments says "it is not yet".



Also "owner" looks  opaque to FSAL. Will NFS-Ganesha expects this back in any 
call ?



 how fsal should use this "owner" ?



/**

* @brief Perform a lock operation

*

* This function performs a lock operation (lock, unlock, test) on a

* file.

*

* @param[in] obj_hdl File on which to operate

* @param[in] owner Lock owner (Not yet implemented)

* @param[in] lock_op Operation to perform

* @param[in] request_lock Lock to take/release/test



* @param[out] conflicting_lock Conflicting lock

*

* @return FSAL status.

*/

fsal_status_t (*lock_op)(struct fsal_obj_handle *obj_hdl,

void *owner,

fsal_lock_op_t lock_op,

fsal_lock_param_t *request_lock,

fsal_lock_param_t *conflicting_lock);



Thanks,

Sagar



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Frank Filz 
Marc,

Could you try the top commit in this branch:

https://github.com/ffilz/nfs-ganesha/commits/async

It may not be the complete solution, but I think it will help your scenario.

I need to do more work on async blocking locks...

And it looks like without async blocking lock support, Ganesha doesn't
handle the case where a lock blocks on a conflicting lock from outside the
Ganesha instance. I will be looking at implementing my thread pool idea that
I modeled in the multilock tool.

Frank

> -Original Message-
> From: Frank Filz [mailto:ffilz...@mindspring.com]
> Sent: Wednesday, September 7, 2016 9:42 AM
> To: 'Marc Eshel' 
> Cc: 'nfs-ganesha-devel' 
> Subject: Re: [Nfs-ganesha-devel] NLM async locking
> 
> Ok, I'm not sure this ever worked right...
> 
> With the lock available upcall, we never put the lock back on the blocked
lock
> list if an attempt to acquire the lock from the FSAL fails...
> 
> So the way the lock available upcall is supposed to work:
> 
> Client requests conflicting lock
> Blocked lock gets registered by FSAL
> SAL puts lock on blocked lock list
> Time passes
> FSAL makes lock available upcall
> SAL finds the blocked lock entry in the blocked lock list SAL makes a call
to
> FSAL to attempt to acquire the lock Assume that fails (in the example,
> because multiple conflicting locks got
> notified)
> SAL puts the lock BACK on the blocked lock list (this step is missing) and
all is
> well
> Time passes
> FSAL makes lock available upcall
> SAL finds the blocked lock entry in the blocked lock list SAL makes a call
to
> FSAL to attempt to acquire the lock Lock is granted by FSAL SAL makes
async
> call back to client If THAT fails, SAL releases the lock from the FSAL and
> disposes of the lock entry and all is well If THAT succeeds, the lock is
> completely granted and all is well
> 
> I also see that if the client retries the lock before it is granted, we
don't
> remove the lock entry from the blocked lock list... I don't think that
will ever
> cause a problem but we should clean that up also...
> 
> Let me try a patch to fix...
> 
> Frank
> 
> > -Original Message-
> > From: Marc Eshel [mailto:es...@us.ibm.com]
> > Sent: Tuesday, September 6, 2016 9:34 PM
> > To: Frank Filz 
> > Cc: 'nfs-ganesha-devel' 
> > Subject: RE: NLM async locking
> >
> > Did you get a chance to look at this problem?
> > Marc.
> >
> >
> >
> > From:   "Frank Filz" 
> > To: Marc Eshel/Almaden/IBM@IBMUS
> > Cc: "'nfs-ganesha-devel'" 
> > Date:   08/29/2016 02:37 PM
> > Subject:RE: NLM async locking
> >
> >
> >
> > > I see the following failure:
> > > 1. Get conflicting locks from 3 clients
> > > cli 1 gets 0-100
> > > cli 2 is blocked on 0-1000
> > > cli 3 is blocked on 0-1
> > > 2. cli 1 unlocks
> > > up-call for cli 2 and 3 to retry
> > > cli 2 gets 0-1000
> > > cli 3 is blocked on 0-1000
> > > 3. cli 2 unlocks
> > > up-call for cli 3 but Ganesha fails
> > >
> > > /* We must be out of sync with FSAL, this is fatal */
> > > LogLockDesc(COMPONENT_STATE, NIV_MAJ, "Blocked Lock Not
> > > Found for"
> > > ,
> > > obj, owner, lock);
> > > LogFatal(COMPONENT_STATE, "Locks out of sync with FSAL");
> > >
> > > I think the problem is in step 2, after cli 3 failed for the second
> > > time
> > it is not
> > > put back in queue, the sbd_list.
> > >
> > > Can you please confirm this logic is very complicated.
> >
> > That sounds like a likely problem. I'd have to dig into the code to
> > see
> why...
> > May take me a day or two to investigate.
> >
> > Frank
> >
> >
> >
> > ---
> > This email has been checked for viruses by Avast antivirus software.
> > https://www.avast.com/antivirus
> >
> >
> >
> 
> 
> 
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
> 
> 
>

--
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] lock_op

2016-09-07 Thread Sagar M D 
Hi All,


could someone please shed light on the usage of "owner" in fsal side ? Is
"owner" implemented as comments says "it is not yet".


Also "owner" looks  opaque to FSAL. Will NFS-Ganesha expects this back in
any call ?


 how fsal should use this "owner" ?


/**

* @brief Perform a lock operation

*

* This function performs a lock operation (lock, unlock, test) on a

* file.

*

* @param[in] obj_hdl File on which to operate

* @param[in] owner Lock owner (Not yet implemented)

* @param[in] lock_op Operation to perform

* @param[in] request_lock Lock to take/release/test



* @param[out] conflicting_lock Conflicting lock

*

* @return FSAL status.

*/

fsal_status_t (*lock_op)(struct fsal_obj_handle *obj_hdl,

void *owner,

fsal_lock_op_t lock_op,

fsal_lock_param_t *request_lock,

fsal_lock_param_t *conflicting_lock);


Thanks,

Sagar
--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Prepend root path to pathname for RQUOTA

2016-09-07 Thread GerritHub 
>From :

madhu.punj...@in.ibm.com has uploaded a new change for review.

  https://review.gerrithub.io/290509

Change subject: Prepend root path to pathname for RQUOTA
..

Prepend root path to pathname for RQUOTA

Currently on NFS client the rpc.quotad sends pathnames
of NFSv4 mountpoints without leading slash in path.
But rquota_getquota() and do_rquota_setquota() are
not able to identify the export for a pathname with
missing leading slash.
Modified rquota_getquota() and do_rquota_setquota()
functions to check if leading slash is missing, and then
prepend the root path to the pathname.

Change-Id: I13bf3752fa076cf20718f6c3cd846ca6f9717fc5
Signed-off-by: Madhu Thorat 
---
M src/Protocols/RQUOTA/rquota_getquota.c
M src/Protocols/RQUOTA/rquota_setquota.c
2 files changed, 28 insertions(+), 12 deletions(-)


  git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha 
refs/changes/09/290509/1
-- 
To view, visit https://review.gerrithub.io/290509
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I13bf3752fa076cf20718f6c3cd846ca6f9717fc5
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: madhu.punj...@in.ibm.com

--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: fsal_rgw: fix compilation with librgw2-devel 10.2.2

2016-09-07 Thread GerritHub 
>From :

ka...@redhat.com has uploaded a new change for review.

  https://review.gerrithub.io/290492

Change subject: fsal_rgw: fix compilation with librgw2-devel 10.2.2
..

fsal_rgw: fix compilation with librgw2-devel 10.2.2

$subject

Change-Id: I49c175ae85c9a068f9c0f26e5573300833149b4e
Signed-off-by: Kaleb S. KEITHLEY 
---
M src/FSAL/FSAL_RGW/handle.c
1 file changed, 6 insertions(+), 10 deletions(-)


  git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha 
refs/changes/92/290492/1
-- 
To view, visit https://review.gerrithub.io/290492
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I49c175ae85c9a068f9c0f26e5573300833149b4e
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: ka...@redhat.com

--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: fsal_rgw: size_t format string warning/error

2016-09-07 Thread GerritHub 
>From :

ka...@redhat.com has uploaded a new change for review.

  https://review.gerrithub.io/290482

Change subject: fsal_rgw: size_t format string warning/error
..

fsal_rgw: size_t format string warning/error

$subject

Change-Id: I5314f01b1cd2d0dbfca0ee32e51bd8f4a9f3bf08
Signed-off-by: Kaleb S. KEITHLEY 
---
M src/FSAL/FSAL_RGW/handle.c
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha 
refs/changes/82/290482/1
-- 
To view, visit https://review.gerrithub.io/290482
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5314f01b1cd2d0dbfca0ee32e51bd8f4a9f3bf08
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: ka...@redhat.com

--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] NLM async locking

2016-09-07 Thread Frank Filz 
Ok, I'm not sure this ever worked right...

With the lock available upcall, we never put the lock back on the blocked
lock list if an attempt to acquire the lock from the FSAL fails...

So the way the lock available upcall is supposed to work:

Client requests conflicting lock
Blocked lock gets registered by FSAL
SAL puts lock on blocked lock list
Time passes
FSAL makes lock available upcall
SAL finds the blocked lock entry in the blocked lock list
SAL makes a call to FSAL to attempt to acquire the lock
Assume that fails (in the example, because multiple conflicting locks got
notified)
SAL puts the lock BACK on the blocked lock list (this step is missing) and
all is well
Time passes
FSAL makes lock available upcall
SAL finds the blocked lock entry in the blocked lock list
SAL makes a call to FSAL to attempt to acquire the lock
Lock is granted by FSAL
SAL makes async call back to client
If THAT fails, SAL releases the lock from the FSAL and disposes of the lock
entry and all is well
If THAT succeeds, the lock is completely granted and all is well

I also see that if the client retries the lock before it is granted, we
don't remove the lock entry from the blocked lock list... I don't think that
will ever cause a problem but we should clean that up also...

Let me try a patch to fix...

Frank

> -Original Message-
> From: Marc Eshel [mailto:es...@us.ibm.com]
> Sent: Tuesday, September 6, 2016 9:34 PM
> To: Frank Filz 
> Cc: 'nfs-ganesha-devel' 
> Subject: RE: NLM async locking
> 
> Did you get a chance to look at this problem?
> Marc.
> 
> 
> 
> From:   "Frank Filz" 
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "'nfs-ganesha-devel'" 
> Date:   08/29/2016 02:37 PM
> Subject:RE: NLM async locking
> 
> 
> 
> > I see the following failure:
> > 1. Get conflicting locks from 3 clients
> > cli 1 gets 0-100
> > cli 2 is blocked on 0-1000
> > cli 3 is blocked on 0-1
> > 2. cli 1 unlocks
> > up-call for cli 2 and 3 to retry
> > cli 2 gets 0-1000
> > cli 3 is blocked on 0-1000
> > 3. cli 2 unlocks
> > up-call for cli 3 but Ganesha fails
> >
> > /* We must be out of sync with FSAL, this is fatal */
> > LogLockDesc(COMPONENT_STATE, NIV_MAJ, "Blocked Lock Not Found
> > for"
> > ,
> > obj, owner, lock);
> > LogFatal(COMPONENT_STATE, "Locks out of sync with FSAL");
> >
> > I think the problem is in step 2, after cli 3 failed for the second
> > time
> it is not
> > put back in queue, the sbd_list.
> >
> > Can you please confirm this logic is very complicated.
> 
> That sounds like a likely problem. I'd have to dig into the code to see
why...
> May take me a day or two to investigate.
> 
> Frank
> 
> 
> 
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
> 
> 
> 



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] [ntirpc] refcount issue ?

2016-09-07 Thread Swen Schillig 
On Di, 2016-09-06 at 16:16 -0400, Matt Benjamin wrote:
> Hi,
> 
> inline
> 
> - Original Message -
> > 
> > From: "Swen Schillig" 
> > To: "Matt Benjamin" , "Daniel Gryniewicz"  > n...@redhat.com>
> > Cc: "nfs-ganesha-devel" 
> > Sent: Tuesday, September 6, 2016 4:10:32 PM
> > Subject: [ntirpc] refcount issue ?
> > 
> > Matt, Dan.
> > 
> > Could you please have a look at the following code areas and verify
> > what I think is a refcount issue.
> > 
> > clnt_vc_ncreate2()
> > {
> > ...
> > if ((oflags & RPC_DPLX_LKP_OFLAG_ALLOC) || (!rec->hdl.xd)) {
> > xd = rec->hdl.xd = alloc_x_vc_data();
> > ...
> > } else {
> > xd = rec->hdl.xd;
> > ++(xd->refcnt);     <=== this is not right. we're not
> > taking an addtl' ref
> Aren't we?  We now share a ref to previously allocated rec->hdl.xd.
No, we're just locally using the reference from rec
which is already refcounted and rec is protected to not change
by the REFLOCK.
> 
> > 
> > here.
> > }
> > ...
> > clnt->cl_p1 = xd;           <=== but here we should increment
> > the refocunt.
but this reference is not counted.
> > }
> > 
> > another code section with the same handling.
> > 
> > makefd_xprt()
> > {
> > ...
> > if ((oflags & RPC_DPLX_LKP_OFLAG_ALLOC) || (!rec->hdl.xd)) {
> > newxd = true;
> > xd = rec->hdl.xd = alloc_x_vc_data();
> > ...
> > } else {
> > xd = (struct x_vc_data *)rec->hdl.xd;
> > /* dont return destroyed xprts */
> > if (!(xd->flags & X_VC_DATA_FLAG_SVC_DESTROYED)) {
> > if (rec->hdl.xprt) {
> > xprt = rec->hdl.xprt;
> > /* inc xprt refcnt */
> > SVC_REF(xprt, SVC_REF_FLAG_NONE);
> > } else
> > ++(xd->refcnt);    < not right, no
> > addtl' ref to xd taken.
> so this looks more likely to be incorrect, need to review
> 
> > 
> > }
> > /* return extra ref */
> > rpc_dplx_unref(rec,
> >    RPC_DPLX_FLAG_LOCKED |
> > RPC_DPLX_FLAG_UNLOCK);
> > *allocated = FALSE;
> > 
> > /* return ref'd xprt */
> > goto done_xprt;
> > }
> > ...
> > xprt->xp_p1 = xd;    < but here we should increment the
> > refcount
> > 
> > ...
> > }
> > 
> > Both areas handle the refcount'ing wrong, but it might balance out
> > sometimes.
> > 
> > 
> > What do you think ?
> > 
> > Cheers Swen
> > 
> > 


--
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel