Re: [Gluster-devel] Attackers hitting vulnerable HDFS installations

[Ira Cooper]

Honestly:

>From what I know of Gluster, and my experience with Samba.

If they target Gluster, you are going to get pwned.  HARD.

Many, many, many, many times.

Trust me... On the Samba Team, we try to avoid security issues VERY actively, 
and we still get a few here and there.

You can walk up to a gluster brick today with no auth... and goto town, never 
mind any of the "standard" attack vectors, like buffer overflows, bad data, etc.

I'd like to believe I'm wrong.  I want to be PROVEN wrong.

But I suspect... You got it right, Gluster isn't big enough to attack today.

... I want to be proven wrong!

-Ira

- Original Message -
> (warning, slightly offensive language)
> 
> https://www.theregister.co.uk/2017/02/09/hadoop_clusters_fked/
> 
> Similar attacks have occurred against MongoDB and ElasticSearch.  How long
> before they target us?  How will we do?
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] 'Reviewd-by' tag for commits

[Ira Cooper]

"Feedback-given-by: "

Cheers,

-Ira

- Original Message -
> On 2016-09-30 at 17:52 +0200, Niels de Vos wrote:
> > On Fri, Sep 30, 2016 at 08:50:12PM +0530, Ravishankar N wrote:
> > > On 09/30/2016 06:38 PM, Niels de Vos wrote:
> > > > On Fri, Sep 30, 2016 at 07:11:51AM +0530, Pranith Kumar Karampuri
> > > > wrote:
> > ...
> > > > Maybe we can add an additional tag that mentions all the people that
> > > > did do reviews of older versions of the patch. Not sure what the tag
> > > > would be, maybe just CC?
> > > It depends on what tags would be processed to obtain statistics on review
> > > contributions.
> > 
> > Real statistics would come from Gerrit, not from the 'git log' output.
> > We do have a ./extras/who-wrote-glusterfs/ in the sources, but that is
> > only to get an idea about the changes that were made and should not be
> > used for serious statistics.
> > 
> > It is possible to feed the Gerrit comment-stream into things like
> > Elasticsearch and get an accurate impression how many reviews people do
> > (and much more). I hope we can get some contribution diagrams from
> > someting like this at one point.
> > 
> > Would some kind of Gave-feedback tag for people that left a comment on
> > earlier versions of the patch be appreciated by others? It will show in
> > the 'git log' who was involved in some way or form.
> 
> I think this would be fair.
> 
> Reviewed-by tags should imho be reserved for the final
> incarnation of the patch. Those mean that the person named
> in the tag has aproved this version of the patch for getting
> into the official tree. A previous version of the patch can
> have been entirely different, so a reviewed-by for that
> previous version may not actually apply to the new version at all
> and hence create a false impression!
> 
> It is also difficult to track all activities by tags,
> and anyone who wants to measure performance and contributions
> only by looking at git commit tags will not be doing several
> people justice. We could add 'discussed-with' or 'designed-by'
> tags, etc ... ;-)
> 
> On a serious note, in Samba we use 'Pair-programmed-with' tags,
> because we do pair-programming a lot, but only one person can
> be an author of a git commit ...
> 
> The 'Gave-feedback' tag I do like. even though it does
> not quite match with the foobar-by pattern of other tags.
> 
> Michael
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] 'Reviewd-by' tag for commits

[Ira Cooper]

"Feedback-given-by: "

Cheers,

-IRa

- Original Message -
> On 2016-09-30 at 17:52 +0200, Niels de Vos wrote:
> > On Fri, Sep 30, 2016 at 08:50:12PM +0530, Ravishankar N wrote:
> > > On 09/30/2016 06:38 PM, Niels de Vos wrote:
> > > > On Fri, Sep 30, 2016 at 07:11:51AM +0530, Pranith Kumar Karampuri
> > > > wrote:
> > ...
> > > > Maybe we can add an additional tag that mentions all the people that
> > > > did do reviews of older versions of the patch. Not sure what the tag
> > > > would be, maybe just CC?
> > > It depends on what tags would be processed to obtain statistics on review
> > > contributions.
> > 
> > Real statistics would come from Gerrit, not from the 'git log' output.
> > We do have a ./extras/who-wrote-glusterfs/ in the sources, but that is
> > only to get an idea about the changes that were made and should not be
> > used for serious statistics.
> > 
> > It is possible to feed the Gerrit comment-stream into things like
> > Elasticsearch and get an accurate impression how many reviews people do
> > (and much more). I hope we can get some contribution diagrams from
> > someting like this at one point.
> > 
> > Would some kind of Gave-feedback tag for people that left a comment on
> > earlier versions of the patch be appreciated by others? It will show in
> > the 'git log' who was involved in some way or form.
> 
> I think this would be fair.
> 
> Reviewed-by tags should imho be reserved for the final
> incarnation of the patch. Those mean that the person named
> in the tag has aproved this version of the patch for getting
> into the official tree. A previous version of the patch can
> have been entirely different, so a reviewed-by for that
> previous version may not actually apply to the new version at all
> and hence create a false impression!
> 
> It is also difficult to track all activities by tags,
> and anyone who wants to measure performance and contributions
> only by looking at git commit tags will not be doing several
> people justice. We could add 'discussed-with' or 'designed-by'
> tags, etc ... ;-)
> 
> On a serious note, in Samba we use 'Pair-programmed-with' tags,
> because we do pair-programming a lot, but only one person can
> be an author of a git commit ...
> 
> The 'Gave-feedback' tag I do like. even though it does
> not quite match with the foobar-by pattern of other tags.
> 
> Michael
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Support to reclaim locks (posix) provided lkowner & range matches

[Ira Cooper]

SMB3 with persistent handles has very similar needs.

Michael,

You may also want to review this.

Cheers,

-Ira

- Original Message -
> Hi,
> 
> In certain scenarios (esp.,in highly available environments), the
> application may have to fail-over/connect to a different glusterFS
> client while the I/O is happening. In such cases until there is a ping
> timer expiry and glusterFS server cleans up the locks held by the older
> glusterFS client, the application will not be able to reclaim their lost
> locks. To avoid that we need support in Gluster to let clients reclaim
> the existing locks provided lkwoner and the lock range matches.
> 
> One of the applications which shall benefit from this support is
> NFS-Ganesha. NFS clients try to reclaim their post server reboot.
> 
> I have made relevant changes (WIP) on the server side to have this
> support [1]. The changes include -
> 
> * A new CLI option is provided "features.locks-reclaim-lock" to enable
> this support.
> 
> * Assuming below is done on the client-side (gfapi) - TODO
> While re-trying the lock request, application has to notify the
> glusterFS client that it is a reclaim request. Client on receiving such
> request should set a boolean "reclaim-lock" in the xdata passed to lock
> request.
> 
> * On the server-side -
>- A new field 'reclaim' is added to 'posix_lock_t' to note if it is
> to be reclaimed.
>- While processing LOCK fop, if the "reclaim-lock" is set in the
> xdata received, reclaim field will be enabled in the new posix lock created.
>- While checking for conflicting locks (in 'same_owner()'), if the
> reclaim field is set, comparison will be done for lkowner and lock
> ranges instead of comparing both lkwoner and client UID.
>- Later it will fall through '__insert_and_merge' and the old lock
> will be updated with the details of the new lock created (along with
> client details).
> 
> For client-side support, I am thinking if we can integrate with the new
> lock API being introduced as part of mandatory lock support in gfapi [2]
> 
> Kindly take a look and provide your comments/suggestions.
> 
> The changes seemed minimal and hence I haven't added it as 3.9 release
> feature. But if you feel it is a feature candidate, please let me know.
> I shall open up a feature page.
> 
> Thanks,
> Soumya
> 
> [1] http://review.gluster.org/#/c/14986/
> [2] http://review.gluster.org/11177
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] samba vfs module for debian jessie?

[Ira Cooper]

Roman  writes:

> Oh, thanks for pointing out. My bad.
> It was really there, so IN CASE if someone else would like to use this vfs
> module with debian jessie it is as simple as download the source from its
> repository and just build the samba package from debian sources. Then
> install it. Seems like everything is working.

No problem, and I'm glad you are up and running!

Cheers,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] samba vfs module for debian jessie?

[Ira Cooper]

That version of Samba has vfs_glusterfs.c as part of the samba source tree.

source3/modules/vfs_glusterfs.c is the source file.

Cheers,

-Ira

- Original Message -
> 2016-04-06 16:36 GMT+03:00 Ira Cooper <i...@redhat.com>:
> 
> > Roman <rome...@gmail.com> writes:
> >
> > > Hi,
> > >
> > > does anyone know, where do I
> > > get /usr/lib/x86_64-linux-gnu/samba/vfs/glusterfs.so or where do I get
> > it's
> > > source to compile it for debian jessie?
> >
> > What version of samba is in jessie?
> >
> > -Ira
> >
> 
> Its
> root@glstor1:/# smbd  --version
> Version 4.1.17-Debian
> 
> 
> --
> Best regards,
> Roman.
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

[Ira Cooper]

Pranith Kumar Karampuri  writes:

> On 02/03/2016 07:54 PM, Jeff Darcy wrote:
>>> Problem is with workloads which know the files that need to be read
>>> without readdir, like hyperlinks (webserver), swift objects etc. These
>>> are two I know of which will have this problem, which can't be improved
>>> because we don't have metadata, data co-located. I have been trying to
>>> think of a solution for past few days. Nothing good is coming up :-/
>> In those cases, caching (at the MDS) would certainly help a lot.  Some
>> variation of the compounding infrastructure under development for Samba
>> etc. might also apply, since this really is a compound operation.
> Even with compound fops It will still require two sequential network 
> operations from dht2. One to MDC and one to DC So I don't think it helps.

You can do better.

You control the MDC.

The MDC goes ahead and forwards the request, under the client's GUID.

That'll cut 1/2 a RTT.

Cheers,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD portability (was Re: NetBSD tests not running to completion)

[Ira Cooper]

Kaleb KEITHLEY  writes:

> On 01/07/2016 08:47 AM, Jeff Darcy wrote:
 I am not that big a fan of portable software development at all- more
 so  for system software.
>>>
>>> This can be discussed for server side, but for client side, making
>>> glusterfs linux-only would make it a bit irrelevant. Don't you want to
>>> build universal storage?
>> 
>> Getting a bit philosophical here, so I'm splitting this off from the 
>> previous topic.
>> 
>> As long as NFS and SMB clients can access our storage, and we have GFAPI as 
>> well, the additional value of a native (e.g. FUSE) client is actually rather 
>> small.  I personally think it's still a better way to do things on Linux 
>> where it's already well supported, but it's a bit harder to make the same 
>> argument on OSes where our dependencies are themselves in worse shape.
>
> I was in the middle of composing a reply along pretty much the same
> lines when I saw Jeff's reply land in my inbox.
>
> We are migrating from gnfs to NFS-Ganesha for NFS; NFS-Ganesha uses
> GFAPI. If Samba isn't already using a GFAPI-based VFS, it will be soon.
>
> FUSE isn't the best answer, IMO, to the "universal storage" question.
>
> We should really be focusing on making sure GFAPI, NFS-Ganesha, and
> Samba remain portable across multiple platforms.

We've been using a vfs based on gfapi since before we starting working
on Ganesha ;).

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] compound fop design first cut

[Ira Cooper]

Jeff Darcy  writes:

> However, I’d be even more comfortable with an even simpler approach that
> avoids the need to solve what the database folks (who have dealt with
> complex transactions for years) would tell us is a really hard problem.
> Instead of designing for every case we can imagine, let’s design for the
> cases that we know would be useful for improving performance.  Open plus
> read/write plus close is an obvious one.  Raghavendra mentions
> create+inodelk as well.  For each of those, we can easily define a
> structure that contains the necessary fields, we don’t need a
> client-side translator, and the server-side translator can take care of
> “forwarding” results from one sub-operation to the next.  We could even
> use GF_FOP_IPC to prototype this.  If we later find that the number of
> “one-off” compound requests is growing too large, then at least we’ll
> have some experience to guide our design of a more general alternative.
> Right now, I think we’re trying to look further ahead than we can see
> clearly.

Actually, I'm taking the design, I've seen another network protocol use,
SMB2, and proposing it here, I'd be shocked if NFS doesn't behave in the
same way.

Interestingly, all the cases, really deal with a single file, and a
single lock, and a single...

There's a reason I talked about a single sentinel value, and not
multiple ones.  Because I wanted to keep it simple.  Yes, the extensions
you mention are obvious, but they lead to a giant mess, that we may not
want initially.  (But that we CAN extend into if we want them.  I made
the choice not to go there because honestly, I found the complexity too
much for me.)

A simple "abort on failure" and let the higher levels clean it up is
probably right for the type of compounding I propose.  It is what SMB2
does.  So, if you get an error return value, cancel the rest of the
request, and have it return ECOMPOUND as the errno.

Note: How you keep the list to be compounded doesn't matter much to me.
the semantics matter, because those are what I can ask for later, and
allow us to create ops the original desginers hadn't thought of, which
is usually the hallmark of a good design.

I think you should look for a simple design you can "grow into" instead
of creating one off ops, to satisfy a demand today.

My thoughts,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] compound fop design first cut

[Ira Cooper]

Raghavendra Gowdappa  writes:

> From what I can see, new compound ops will _evolve_ in future based on 
> requirements unseen as of now.

Yes,

That is the one thing you can count on here ;)

The compounding architecture proposed here, scares me to be honest.

The complexity it can create is pretty immense.

I'm going to suggest a simpler scheme to you, there is no API provided,
but I think you'll see the idea, which is the key.  Then APIs and
whatnot can follow.

In the proposal today, if I want to compound op A and B, I have to write
compound_A_and_B basically.

That will create headaches for anyone who comes up with neat ideas :(.
Or needs to do longer and longer compounds.

I propose that we define a "compound op" that contains ops.

Within each op, there are fields that can be "inherited" from the
previous op, via use of a sentinel value.

Sentinel is -1, for all of these examples.

So:

LOOKUP (1, "foo") (Sets the gfid value to be picked up by compounding, 1
is the root directory, as a gfid, by convention.)
OPEN(-1, O_RDWR) (Uses the gfid value, sets the glfd compound value.)
WRITE(-1, "foo", 3) (Uses the glfd compound value.)
CLOSE(-1) (Uses the glfd compound value)

Note, that we can define what fields can take "sentinel" values, so
things like leases, locks etc, can all be handled properly.

The other trick is, if we return an error, we MUST stop the compound,
and return the rest of the return results as ECOMPOUND or some similar
value.  The actual thing that errored should return proper error codes.

Now, the cute thing about this is that a translator can look at a compound
stream, element by element, and decide what to do with it, or if you
need to break the stream and handle the compound semantics yourself or
what.

So this actually fits well with gluster's architecture, of being very
composable :).

I'm interested in your thoughts on where the edges of this proposal may
be, and if it meets your needs.

Thanks,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] libgfapi changes to add lk_owner and lease ID

[Ira Cooper]

Niels de Vos <nde...@redhat.com> writes:

> On Thu, Dec 03, 2015 at 06:34:56PM -0500, Ira Cooper wrote:
>> Kaleb KEITHLEY <kkeit...@redhat.com> writes:
>> 
>> > On 12/03/2015 03:10 PM, Ira Cooper wrote:
>> >> Poornima Gurusiddaiah <pguru...@redhat.com> writes:
>> >> 
>> >>> Hi, 
>> >>>
>> >>> Brief Background: 
>> >>>  
>> >>> For the below two features, we need ligfapi to take 2 other parameters 
>> >>> from the applications for most number of fops. 
>> >>> http://review.gluster.org/#/c/12014/ 
>> >>> http://review.gluster.org/#/c/11980/ 
>> >>>
>> >>> For leases to work as explained in the above doc, every file data 
>> >>> read/write fop needs to be associated with a lease ID. This is specially 
>> >>> required for Samba and NFS-Ganesha as they inturn serve other clients 
>> >>> who request for leases. For Gluster to identify the end client (which is 
>> >>> not Samba/NFS Ganesha) we need lease ID to be filled by Samba/NFS 
>> >>> Ganesha. 
>> >>>
>> >>> For mandatory locks feature to work as explained, every file data 
>> >>> read/write fop needs to be associated with a lk_owner. In linux Kernel 
>> >>> VFS takes care of filling the lk_ownere for the file system. In libgfapi 
>> >>> case, the applications calling into libgfapi should be providing 
>> >>> lk_owner with every fop. This is again required mainly for Samba and NFS 
>> >>> Ganesha, as they serve multiple clients. 
>> >>>
>> >>> Possible solutions: 
>> >>> = 
>> >>> 1. Modify all the required APIs to take 2 other parameter, lease ID and 
>> >>> lk_owner. But that would mean backward compatibility issues and is a 
>> >>> programming overhead for applications not interested in Leases and 
>> >>> mandatory lock feature. 
>> >>> 2. Add an API called glfs_set_fop_attrs (lease ID, lk_owner) which works 
>> >>> similar to glfs_set_uid(uid). The API sets a thread local storage 
>> >>> (pthread_key) with the values provided, the further fops on that thread 
>> >>> will pick the lease ID and lk_owner from the thread local storage 
>> >>> (pthread_key). There are few minor details that needs to be worked out: 
>> >>> - In case of async API will end up using lease ID and lk_owner from 
>> >>> wrong thread. 
>> >>> - unset lease ID and lk_owner after every fop to ensure there is no 
>> >>> stale lease ID or lk_owner set? 
>> >>> - For fd based fops we can store the lease ID and lk_owner in glfd, so 
>> >>> that the application writed need not set it for every fop. But for 
>> >>> handle based fops lease ID and lk_owner needs to be set explicitly 
>> >>> every-time. 
>> >>>
>> >>> Solution 2 is more preferable except for that it adds overhead of 
>> >>> calling another API, for the libgfapi users who intends to use these 
>> >>> features. 
>> >>> A prototype of solution 2 can be found at 
>> >>> http://review.gluster.org/#/c/12876/ 
>> >>>
>> >
>> > why not use storage in the client_t instead of thread local?
>> 
>> It comes down to the use case.  For Samba, the right spot is almost
>> certainly the fd, because lease keys are a per-handle (which we map to
>> fd) property.
>> 
>> client_t is a horror show, due to race conditions between threads, IMHO.
>
> If there are known races, should we not address that? Got a bug that
> explains it in more detail?

Niels,

For samba, if we do multi-threaded open, Kaleb's proposal is a
race-condition.  I haven't gone through every use of client_t and seen
if it is racy.

The race here is pretty simple:

Thread 1: Sets lease_id
Thread 2: Sets lease_id
Thread 1: Opens file. (wrong lease_id)

If these two threads represent requests from different clients, client_t
won't work, unless there's a client_t per-thread.

For global things on the connection, client_t is fine, and appropriate.
For this?  No.

This is a property per-open, and belongs in the glfs_fd and glfs_object,
IMHO.

Thanks,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] libgfapi changes to add lk_owner and lease ID

[Ira Cooper]

Poornima Gurusiddaiah  writes:

> Hi, 
>
> Brief Background: 
>  
> For the below two features, we need ligfapi to take 2 other parameters from 
> the applications for most number of fops. 
> http://review.gluster.org/#/c/12014/ 
> http://review.gluster.org/#/c/11980/ 
>
> For leases to work as explained in the above doc, every file data read/write 
> fop needs to be associated with a lease ID. This is specially required for 
> Samba and NFS-Ganesha as they inturn serve other clients who request for 
> leases. For Gluster to identify the end client (which is not Samba/NFS 
> Ganesha) we need lease ID to be filled by Samba/NFS Ganesha. 
>
> For mandatory locks feature to work as explained, every file data read/write 
> fop needs to be associated with a lk_owner. In linux Kernel VFS takes care of 
> filling the lk_ownere for the file system. In libgfapi case, the applications 
> calling into libgfapi should be providing lk_owner with every fop. This is 
> again required mainly for Samba and NFS Ganesha, as they serve multiple 
> clients. 
>
> Possible solutions: 
> = 
> 1. Modify all the required APIs to take 2 other parameter, lease ID and 
> lk_owner. But that would mean backward compatibility issues and is a 
> programming overhead for applications not interested in Leases and mandatory 
> lock feature. 
> 2. Add an API called glfs_set_fop_attrs (lease ID, lk_owner) which works 
> similar to glfs_set_uid(uid). The API sets a thread local storage 
> (pthread_key) with the values provided, the further fops on that thread will 
> pick the lease ID and lk_owner from the thread local storage (pthread_key). 
> There are few minor details that needs to be worked out: 
> - In case of async API will end up using lease ID and lk_owner from wrong 
> thread. 
> - unset lease ID and lk_owner after every fop to ensure there is no stale 
> lease ID or lk_owner set? 
> - For fd based fops we can store the lease ID and lk_owner in glfd, so that 
> the application writed need not set it for every fop. But for handle based 
> fops lease ID and lk_owner needs to be set explicitly every-time. 
>
> Solution 2 is more preferable except for that it adds overhead of calling 
> another API, for the libgfapi users who intends to use these features. 
> A prototype of solution 2 can be found at 
> http://review.gluster.org/#/c/12876/ 
>
> Please let me know if you have any suggestions.

Poornima,

What calls are impacted by solution #1?

Assume we can stash data in glfs_fd. and glfs_object.

Thanks,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] libgfapi changes to add lk_owner and lease ID

[Ira Cooper]

Kaleb KEITHLEY <kkeit...@redhat.com> writes:

> On 12/03/2015 03:10 PM, Ira Cooper wrote:
>> Poornima Gurusiddaiah <pguru...@redhat.com> writes:
>> 
>>> Hi, 
>>>
>>> Brief Background: 
>>>  
>>> For the below two features, we need ligfapi to take 2 other parameters from 
>>> the applications for most number of fops. 
>>> http://review.gluster.org/#/c/12014/ 
>>> http://review.gluster.org/#/c/11980/ 
>>>
>>> For leases to work as explained in the above doc, every file data 
>>> read/write fop needs to be associated with a lease ID. This is specially 
>>> required for Samba and NFS-Ganesha as they inturn serve other clients who 
>>> request for leases. For Gluster to identify the end client (which is not 
>>> Samba/NFS Ganesha) we need lease ID to be filled by Samba/NFS Ganesha. 
>>>
>>> For mandatory locks feature to work as explained, every file data 
>>> read/write fop needs to be associated with a lk_owner. In linux Kernel VFS 
>>> takes care of filling the lk_ownere for the file system. In libgfapi case, 
>>> the applications calling into libgfapi should be providing lk_owner with 
>>> every fop. This is again required mainly for Samba and NFS Ganesha, as they 
>>> serve multiple clients. 
>>>
>>> Possible solutions: 
>>> = 
>>> 1. Modify all the required APIs to take 2 other parameter, lease ID and 
>>> lk_owner. But that would mean backward compatibility issues and is a 
>>> programming overhead for applications not interested in Leases and 
>>> mandatory lock feature. 
>>> 2. Add an API called glfs_set_fop_attrs (lease ID, lk_owner) which works 
>>> similar to glfs_set_uid(uid). The API sets a thread local storage 
>>> (pthread_key) with the values provided, the further fops on that thread 
>>> will pick the lease ID and lk_owner from the thread local storage 
>>> (pthread_key). There are few minor details that needs to be worked out: 
>>> - In case of async API will end up using lease ID and lk_owner from wrong 
>>> thread. 
>>> - unset lease ID and lk_owner after every fop to ensure there is no stale 
>>> lease ID or lk_owner set? 
>>> - For fd based fops we can store the lease ID and lk_owner in glfd, so that 
>>> the application writed need not set it for every fop. But for handle based 
>>> fops lease ID and lk_owner needs to be set explicitly every-time. 
>>>
>>> Solution 2 is more preferable except for that it adds overhead of calling 
>>> another API, for the libgfapi users who intends to use these features. 
>>> A prototype of solution 2 can be found at 
>>> http://review.gluster.org/#/c/12876/ 
>>>
>
> why not use storage in the client_t instead of thread local?

It comes down to the use case.  For Samba, the right spot is almost
certainly the fd, because lease keys are a per-handle (which we map to
fd) property.

client_t is a horror show, due to race conditions between threads, IMHO.

Thanks,

-Ira
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Lease-lock Design notes

[Ira Cooper]

1. Yes, it is intentional.  The internals of gluster should be able to 
lease-lks.  We discussed using them in the read ahead translator and the write 
behind translator.
2. This has been discussed, and proposed, but there is actually a need for a 
lease fop also, because clients can request the renewal or reinstatement of 
a lease.  (Actually for Samba having it all be one call and atomic is very 
interesting.)
3. This I can't answer... I haven't been in the most recent discussions.  But 
the intent of this work, when I started, was to be useful to the whole of 
Gluster, not just Samba or Ganesha.

Thanks,

-Ira

- Original Message -
 Hi,
 
 I have some questions or gaps in understanding the current lease design,
 request some clarification on the same.
 
 1) Is there a notion of a *gluster* client holding a lease on a file/dir?
 - As per your Barcelona gluster summit presentation the client side
 caching needs this notion
 - DHT/EC/AFR can leverage such a lease to prevent eager or any form of
 locking when attempting to hold consistency of data being operated on
- Please note that this requires not each xlator requesting and
 holding the lease, but a *gluster* client holding the lease, assuming
 that one can do local in memory locking to prevent different
 fd/connection/applications performing operations on the same file
 against the *same* gluster client and not have to coordinate this with
 the brick
 
 I see some references to this in the design but wanted to understand if
 the thought is there.
 
 IOW, if an NFS client requests a lease, Ganesha requests the lease from
 Gluster, in which case the gfapi client that requested the lease first
 gets the lease, and then re-leases it to Ganesha, now Ganesha is free to
 lease it to any client on its behalf and recall leases etc. as the case
 may be and the gluster client does not care. When another gluster client
 (due to Ganesha or otherwise (say self heal or rebalance)) attempts to
 get a lease, that is when the lease is broken all across.
 
 Is my understanding right, is the design along these lines?
 
 2) Why is the lease not piggy backed with the open? Why 2 network FOPs
 instead of 1 in this case? How can we compound this lease with other
 requests? Or do you have thoughts around the same?
 
 Isn't NFS and SMB requesting leases with its open calls
 
 3) If there were discussions around some designs that were rejected,
 could you also update the document with the same, so the one can
 understand the motivation behind the current manner of implementing leases.
 
 Thanks,
 Shyam
 
 On 04/16/2015 07:37 AM, Soumya Koduri wrote:
  Hi,
 
  Below link contains the lease-lock design notes (as per the latest
  discussion we had). Thanks to everyone involved (CC'ed).
 
  http://www.gluster.org/community/documentation/index.php/Features/Upcall-infrastructure#delegations.2Flease-locks
 
 
  Kindly go through the same and provide us your inputs.
 
  Thanks,
  Soumya
  ___
  Gluster-devel mailing list
  Gluster-devel@gluster.org
  http://www.gluster.org/mailman/listinfo/gluster-devel
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Proposal: Using LLVM clang-analyzer in gluster development

[Ira Cooper]

Prasanna, have you compared the results to the ones we see via coverity?

-Ira

- Original Message -
 Hi gluster team,
 
 Proposal:
 
 Using Clang static analyzer tool for gluster project
 
 
 
 About Clang:
 
 From a very high level view, Clang has two features
 
 1. Clang as a compiler
 2. Clang as a code analyzer
 
 The Idea hear is to use second point i.e Clang as code analyzer and still gcc
 will be our default compiler.
 
 The Clang Static Analyzer is a source code analysis tool that finds bugs in
 C,
 C++, and Objective-C programs. Given the exact same code base, clang-analyzer
 reported ~70 potential issues. clang is an awesome and free tool.
 
 The reports from clang-analyzer are in HTML and there’s a single file for
 each
 issue and it generates a nice looking source code with embedded comments
 about
 which flow that was followed all the way down to the problem.
 
 
 
 Why Clang-Analyzer: (Advantages)
 
 Since its is an open source tool:

Available to all the developers

Easy Access, we can run the tool while we compile the code (say $
scan-build make )

No restrictions on Number of Runs per week/day/hour/min ..

Defects are Identified before submitting a patch, thus very less
chance
of defect injection into project
 
 
 The Html view of clang is very impressive with all the source code including
 comments of clang-analyzer, which lead to defect line number directly .
 
 I have attached a sample clang results for geo-replication module run on
 latest 3.7+ glusterfs code, please find them above.
 
 Thanks for your time.
 
 Best Regards,
 Prasanna Kumar K.
 
 
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Unable to start samba service (smb) - due to /run/samba/ncalrpc - No such file or directory

[Ira Cooper]

Was this ever solved?

-Ira

- Original Message -
 Hi,
 
 Can you post the output of #strace smbd and the logs(log.smbd)
 Was it an upgrade from older samba version? If yes, try clearing the
 /var/lib/samba/*.tdb files.
 
 Regards,
 Poornima
 
 
 
 
 From: Omkar Sharma om...@fractalio.com
 To: gluster-devel@gluster.org
 Sent: Thursday, December 25, 2014 3:17:44 PM
 Subject: [Gluster-devel] Unable to start samba service (smb) - due to
 /run/samba/ncalrpc - No such file or directory
 
 Hi,
 
 I have installed samba-4.1.11-2.el6.x86_64.rpm and its dependencies taken
 from the below link:
 
 http://download.gluster.org/pub/gluster/glusterfs/samba/CentOS/epel-6.5/x86_64/
 
 The samba package got installed but unable to start the smb service. And I
 have smb file inside /etc/init.d/ directory.
 
 $ sudo service smb status ...not showing any output
 $ sudo service smb start ...not showing any output
 
 $ sudo /etc/init.d/smb status ...not showing any output
 $ sudo /etc/init.d/smb start ...not showing any output
 
 And I went to see the log files inside /var/log/samba/.
 
 $ vi /var/log/samba/log.smbd
 
 smbd version 4.1.11 started.
 Copyright Andrew Tridgell and the Samba Team 1992-2013
 [2014/12/24 06:36:44.225371, 0] ../source3/smbd/server.c:1289(main)
 standard input is not a socket, assuming -D option
 [2014/12/24 06:36:44.472002, 0]
 ../lib/util/util.c:216(directory_create_or_exist)
 mkdir failed on directory /run/samba/ncalrpc: No such file or directory
 [2014/12/24 06:36:44.472317, 0] ../source3/smbd/server.c:1471(main)
 Failed to create pipe directory /run/samba/ncalrpc - No such file or
 directory
 
 Why the samba rpm is not creating /run/samba directory and place the file
 ncalrpc ?
 
 
 With Regards,
 Omkar MN
 
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
 
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] User-serviceable snapshots design

[Ira Cooper]

Anand, I also have a concern regarding the user-serviceable snapshot feature.

You rightfully call out the lack of scaling caused by maintaining the gfid - 
gfid mapping tables, and correctly point out that this will limit the use cases 
this feature will be applicable to, on the client side.

If in fact gluster generates its gfids randomly, and has always done so, I 
propose that we can change the algorithm used to determine the mapping, to 
eliminate the lack of scaling of our solution.

We can create a fixed constant per-snapshot.  (Can be in just the client's 
memory, or stored on disk, that is an implementation detail here.)  We will 
call this constant n.

I propose we just add the constant to the gfid determine the new gfid.  It 
turns out that this new gfid has the same chance of collision as any random 
gfid.  (It will take a moment for you to convince yourself of this, but the 
argument is fairly intuitive.)  If we do this, I'd suggest we do it on the 
first 32 bits of the gfid, because we can use simple unsigned math, and let it 
just overflow.  (If we get up to 2^32 snapshots, we can revisit this aspect of 
the design, but we'll have other issues at that number.)

By using addition this way, we also allow for subtraction to be used for a 
later purpose.

Note: This design relies on our random gfid generator not turning out a linear 
range of numbers.  If it has in the past, or will in the future, clearly this 
design has flaws.  But, I know of no such plans.  As long as the randomness is 
sufficient, there should be no issue.  (IE: It doesn't turn out linear results.)

Thanks,

-Ira / ira@(redhat.com|samba.org)

PS: +1 to Jeff here.  He's spotting major issues, that should be looked at, 
above the issue above.

- Original Message -
  Attached is a basic write-up of the user-serviceable snapshot feature
  design (Avati's). Please take a look and let us know if you have
  questions of any sort...
 
 A few.
 
 The design creates a new type of daemon: snapview-server.
 
 * Where is it started?  One server (selected how) or all?
 
 * How do clients find it?  Are we dynamically changing the client
   side graph to add new protocol/client instances pointing to new
   snapview-servers, or is snapview-client using RPC directly?  Are
   the snapview-server ports managed through the glusterd portmapper
   interface, or patched in some other way?
 
 * Since a snap volume will refer to multiple bricks, we'll need
   more brick daemons as well.  How are *those* managed?
 
 * How does snapview-server manage user credentials for connecting
   to snap bricks?  What if multiple users try to use the same
   snapshot at the same time?  How does any of this interact with
   on-wire or on-disk encryption?
 
 I'm sure I'll come up with more later.  Also, next time it might
 be nice to use the upstream feature proposal template *as it was
 designed* to make sure that questions like these get addressed
 where the whole community can participate in a timely fashion.
 ___
 Gluster-users mailing list
 gluster-us...@gluster.org
 http://supercolony.gluster.org/mailman/listinfo/gluster-users
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel