Re: [Gluster-devel] pluggability of some aspects in afr/nsr/ec

2015-10-29 Thread Jeff Darcy 
> >> I want to understand if there is a possibility of exposing these as
> >> different modules that we can mix and match, using options.

It’s not only possible, but it’s easier than you might think.  If an
option is set (cluster.nsr IIRC) then we replace cluster/afr with
cluster/nsr-client and then add some translators to the server-side
stack.  A year ago that was just one nsr-server translator.  The
journaling part has already been split out, and I plan to do the same
with the leader-election parts (making them usable for server-side AFR
or EC) as well.  It shouldn’t be hard to control the addition and
removal of these and related translators (e.g. index) with multiple
options instead of just one.  The biggest stumbling block I’ve actually
hit when trying to do this with AFR on the server side is the *tests*,
many of which can’t handle delays on the client side while the server
side elects leaders and cross-connects peers.  That’s all solvable.  It
just would have taken more time than I had available for the experiment.

> precisely. I think switching is not that difficult once we make sure
> healing is complete. Switching is a rare operation IMO so we can
> probably ask the users to do stop/choose-new-value/start the volume
> after choosing the options. This way is simpler than to migrate
> between the volumes where you have to probably copy the data.

The two sets of metadata are *entirely* disjoint, which puts us in a
good position compared e.g. to DHT/tiering which had overlaps.  As long
as the bricks are “clean” switching back and forth should be simple.  In
fact I expect to do this a lot when we get to characterizing performance
etc.

> >> choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future
> >> if we come up with better metadata journals/stores it should be
> >> easy to plug them is what I'm thinking. The idea I have is based on
> >> the workload, users should be able to decide which pair of
> >> synchronization/metadata works best for them (Or we can also
> >> recommend based on our tests). Wanted to seek your inputs.

Absolutely.  As I’m sure you’re tired of hearing, I believe NSR will
outperform AFR by a significant margin for most workloads and
configurations.  I wouldn’t be the project’s initiator/leader if I
didn’t believe that, but I’m OK if others disagree.  We’ll find out
eventually.  ;)  More importantly, “most” is still not “all”.  Even by
my own reckoning, there are cases in which AFR will perform better or be
preferable for other reasons.  EC’s durability and space-efficiency
advantages make an even stronger case for preserving both kinds of data
paths and metadata arrangements.  That’s precisely why I want to make
the journaling and leader-election parts more generic.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] pluggability of some aspects in afr/nsr/ec

2015-10-29 Thread Shyam 

On 10/29/2015 02:06 AM, Pranith Kumar Karampuri wrote:

hi,
  I want to understand how are you guys planning to integrate
NSR volumes to the existing CLIs. Here are some thoughts I had, wanted
to know your thoughts:
At the heart of both the replication/ec schemes we have
1) synchronization mechanisms
a) afr,ec does it using locks
b) nsr does it using leader election
2) Metadata to figure out the healing/reconciliation aspects
a) afr,ec does it using xattrs
b) nsr does it using journals

I want to understand if there is a possibility of exposing these as
different modules that we can mix and match, using options. If the users
choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future if we
come up with better metadata journals/stores it should be easy to plug
them is what I'm thinking. The idea I have is based on the workload,
users should be able to decide which pair of synchronization/metadata
works best for them (Or we can also recommend based on our tests).


I wanted to add some more *insights* (or stir up the mess) based on the 
DHT2 MDS/DS split. With this case, we could (that may even change to a 
would) have NSR/AFR *only* on the MDS side of things and no EC there, 
and have any of EC/AFR/NSR on the DS side of things.


The theory being, EC provides data space efficiency and replicates 
meta-data anyway, so instead of having that on the MDS side of things, 
which does not have any data, we may want to stick to replication 
methods for availability reasons.


I am adding this here, although this is orthogonal to the current 
discussion, just to provide a perspective into our thought process.



Wanted to seek your inputs.

Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs mainline BZs to close...

2015-10-29 Thread Niels de Vos 
On Thu, Oct 29, 2015 at 05:38:28PM +0530, Bipin Kunal wrote:
> Hello Niels/Humble,
> 
> I would like to help you here and by this I can even start collaborating in
> upstream.
> 
> I will try to send first pull request very soon, late by next week.

Great, thanks! Let us know if you have any questions.

Niels


> 
> Thanks,
> Bipin Kunal
> 
> On Wed, Oct 28, 2015 at 5:04 PM, Niels de Vos  wrote:
> 
> > On Wed, Oct 28, 2015 at 04:02:16PM +0530, Humble Devassy Chirammal wrote:
> > > Hi Niels,
> > >
> > > >
> > >   Our shiny docs (or my bookmarks?) are broken again...
> > >
> > >
> > http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20report%20Life%20Cycle/
> > >   http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20Triage/
> > >
> > > >
> > > As you know, the "Developer Guide" is now part of glusterfs source code (
> > > https://github.com/gluster/glusterfs/tree/master/doc/developer-guide)
> > [1].
> > > The decision to split the glusterfs documentation  into three parts (
> > > developer spec/feature , Administration ) came late and it caused the
> > > bookmark to break.
> >
> > Right, but I do not think the documentation of procedures and workflows
> > should be part of the developers guide. This counts for topics related
> > to bugs, releases and probably other things. Can we have a section on
> > readthedocs where procedures and workflows can be placed?
> >
> > > Being Media Wiki  READONLY, we thought this type of documents can be part
> > > of Developer Guide for now. May be we need to sort the "developer guide"
> > in
> > > source code repo  again and put it into correct buckets, but this needs
> > > some time and effort.
> >
> > Yeah, we definitely should do that. And, also make sure that the old
> > wiki gets replaced by appropriate redirects to prevent confusion.
> >
> > Thanks,
> > Niels
> >
> > >
> > >
> > > [1]
> > >
> > > Because of the gerrit plugin issue, the commits in gerrit has not been
> > > synced to github since Sep 10. However you can see the the change here
> > > http://review.gluster.org/#/c/12227/
> > >
> > > --Humble
> > >
> > >
> > > On Mon, Oct 26, 2015 at 2:44 PM, Niels de Vos  wrote:
> > >
> > > > On Wed, Oct 21, 2015 at 05:22:49AM -0400, Nagaprasad Sathyanarayana
> > wrote:
> > > > > I came across the following BZs which are still open in mainline.
> > But
> > > > > they are fixed and made available in a upstream release.  Planning to
> > > > > close them this week, unless there are any objections.
> > > >
> > > > We have a policy to close bugs when their patches land in a released
> > > > version. The bugs against mainline will get closed when a release is
> > > > made that contains those fixes. For many of the current mainline bugs,
> > > > this would be the case when glusterfs-3.8 is released.
> > > >
> > > > What is the concern of having bugs against the mainline version open
> > > > until a release contains that particular fix? There are many bugs that
> > > > also get backports to stable versions (3.7, 3.6 and 3.5). Those bugs
> > get
> > > > closed with each minor releases for that stable version.
> > > >
> > > > Of course we can change our policy to close mainline bugs earlier. But,
> > > > we need to be consistent in setting the rules for that, and document
> > > > them well. There is an ongoing task about automatically changing the
> > > > status of bugs when patches get posted/merged and releases made.
> > Closing
> > > > mainline bugs should be part of that process.
> > > >
> > > > When do you suggest that a bug against mainline should be closed, when
> > > > one of the stable releases containst the fix, or when all of them do?
> > > > What version with fix should we point the users at when we closed a
> > > > mainline bug?
> > > >
> > > >   Our shiny docs (or my bookmarks?) are broken again...
> > > >
> > > >
> > http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20report%20Life%20Cycle/
> > > >
> > http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20Triage/
> > > >
> > > >   This is the old contents:
> > > >
> > > >
> > http://www.gluster.org/community/documentation/index.php/Bug_report_life_cycle
> > > >   http://www.gluster.org/community/documentation/index.php/Bug_triage
> > > >
> > > > I was about to suggest to send a pull request so that we can discuss
> > > > your proposal during the weekly Bug Triage meeting on Tuesdays.
> > > > Unfortunately I don't know where the latest documents moved to, so
> > > > please send your changes by email.
> > > >
> > > > Could you explain what Bugzilla query you used to find these bugs? We
> > > > have some keywords (like "Tracking") that should be handled with care,
> > > > and probably should not get closed even when patches were merged in
> > > > mainline and stable releases.
> > > >
> > > > Thanks,
> > > > Niels
> > > >
> > > >
> > > > >
> > > > >
> > > >
> >

Re: [Gluster-devel] Troubleshooting and Diagnostic tools for Gluster

2015-10-29 Thread Shyam 

On 10/27/2015 11:12 PM, Sankarshan Mukhopadhyay wrote:

On Mon, Oct 26, 2015 at 7:04 PM, Shyam  wrote:

Older idea on this was, to consume the logs and filter based on the message
IDs for those situations that can be remedied. The logs are hence the point
where the event for consumption is generated.

Also, the higher level abstraction uses these logs, it can *watch* based on
message ID filters that are of interest to it, than parse the log message
entirely to gain insight on the issue.


Are all situations usually atomic? Is it expected to have specific
mapping between an event recorded in a log from one part of an
installed system to a possible symptom? Or, do a collection of events
lead up to an observed failure (which, in turn, is recorded as a
series of events on the logs)?




logs are from a point in the code, at a point in time (just stating the 
obvious). So to detect a genuine failure or that an event has occurred, 
it may need multiple messages to detect the same. Pattern for such 
events need to be called out, for it to make sense.


But, *some* log messages from higher layers, do denote failure as a 
collection point.


Overall to answer your question, it is a combination of all, depending 
on the event/situation.


But I did not understand the _atomic_ part in the question and also I am 
not sure I answered what you are thinking about.

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs mainline BZs to close...

2015-10-29 Thread Bipin Kunal 
Hello Niels/Humble,

I would like to help you here and by this I can even start collaborating in
upstream.

I will try to send first pull request very soon, late by next week.

Thanks,
Bipin Kunal

On Wed, Oct 28, 2015 at 5:04 PM, Niels de Vos  wrote:

> On Wed, Oct 28, 2015 at 04:02:16PM +0530, Humble Devassy Chirammal wrote:
> > Hi Niels,
> >
> > >
> >   Our shiny docs (or my bookmarks?) are broken again...
> >
> >
> http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20report%20Life%20Cycle/
> >   http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20Triage/
> >
> > >
> > As you know, the "Developer Guide" is now part of glusterfs source code (
> > https://github.com/gluster/glusterfs/tree/master/doc/developer-guide)
> [1].
> > The decision to split the glusterfs documentation  into three parts (
> > developer spec/feature , Administration ) came late and it caused the
> > bookmark to break.
>
> Right, but I do not think the documentation of procedures and workflows
> should be part of the developers guide. This counts for topics related
> to bugs, releases and probably other things. Can we have a section on
> readthedocs where procedures and workflows can be placed?
>
> > Being Media Wiki  READONLY, we thought this type of documents can be part
> > of Developer Guide for now. May be we need to sort the "developer guide"
> in
> > source code repo  again and put it into correct buckets, but this needs
> > some time and effort.
>
> Yeah, we definitely should do that. And, also make sure that the old
> wiki gets replaced by appropriate redirects to prevent confusion.
>
> Thanks,
> Niels
>
> >
> >
> > [1]
> >
> > Because of the gerrit plugin issue, the commits in gerrit has not been
> > synced to github since Sep 10. However you can see the the change here
> > http://review.gluster.org/#/c/12227/
> >
> > --Humble
> >
> >
> > On Mon, Oct 26, 2015 at 2:44 PM, Niels de Vos  wrote:
> >
> > > On Wed, Oct 21, 2015 at 05:22:49AM -0400, Nagaprasad Sathyanarayana
> wrote:
> > > > I came across the following BZs which are still open in mainline.
> But
> > > > they are fixed and made available in a upstream release.  Planning to
> > > > close them this week, unless there are any objections.
> > >
> > > We have a policy to close bugs when their patches land in a released
> > > version. The bugs against mainline will get closed when a release is
> > > made that contains those fixes. For many of the current mainline bugs,
> > > this would be the case when glusterfs-3.8 is released.
> > >
> > > What is the concern of having bugs against the mainline version open
> > > until a release contains that particular fix? There are many bugs that
> > > also get backports to stable versions (3.7, 3.6 and 3.5). Those bugs
> get
> > > closed with each minor releases for that stable version.
> > >
> > > Of course we can change our policy to close mainline bugs earlier. But,
> > > we need to be consistent in setting the rules for that, and document
> > > them well. There is an ongoing task about automatically changing the
> > > status of bugs when patches get posted/merged and releases made.
> Closing
> > > mainline bugs should be part of that process.
> > >
> > > When do you suggest that a bug against mainline should be closed, when
> > > one of the stable releases containst the fix, or when all of them do?
> > > What version with fix should we point the users at when we closed a
> > > mainline bug?
> > >
> > >   Our shiny docs (or my bookmarks?) are broken again...
> > >
> > >
> http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20report%20Life%20Cycle/
> > >
> http://gluster.readthedocs.org/en/latest/Developer-guide/Bug%20Triage/
> > >
> > >   This is the old contents:
> > >
> > >
> http://www.gluster.org/community/documentation/index.php/Bug_report_life_cycle
> > >   http://www.gluster.org/community/documentation/index.php/Bug_triage
> > >
> > > I was about to suggest to send a pull request so that we can discuss
> > > your proposal during the weekly Bug Triage meeting on Tuesdays.
> > > Unfortunately I don't know where the latest documents moved to, so
> > > please send your changes by email.
> > >
> > > Could you explain what Bugzilla query you used to find these bugs? We
> > > have some keywords (like "Tracking") that should be handled with care,
> > > and probably should not get closed even when patches were merged in
> > > mainline and stable releases.
> > >
> > > Thanks,
> > > Niels
> > >
> > >
> > > >
> > > >
> > >
> 1211836,1212398,1211132,1215486,1213542,1212385,1210687,1209818,1210690,1215187,1215161,1214574,1218120,1217788,1216067,1213773,1210684,1209104,1217937,1200262,1204651,1211913,1211594,1163561,1176062,
> > > >
> > >
> 1219784,1176837,1208131,1200704,1220329,1221095,1172430,1219732,1219738,1213295,1212253,1211808,1207615,1216931,1224290,1217701,1223213,1223889,1223385,1221104,1221696,1219442,1224596,1165041,1225491,
> > > >
> > >
>

Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

2015-10-29 Thread Shyam 

On 10/29/2015 01:42 AM, Avra Sengupta wrote:

My 2 cents on this:

The decision we take on this should have certain rationale, and I see
two key things affecting that decision.
1. How much of the code we are planning to write now, is going to make
it to the final product. If we are sure that a sizeable amount of code
we are writing now, is gonna change over the course of time, then it
makes sense to have that kinda development in experimental branch.
2. Is the code we are planning to deliver meddle with existing
infrastructure. If so, then it should definitely go into experimental

Now analyzing NSR based on the above two constraints :
1. We definitely plan to use more than 80% of the code we write now, and
plan to go about it in an incremental module by module kinda way.
2. We hardly have any overlap with existing infrastructure, and we can
easily integrate with the current glusterd now, and move on to Glusterd
2, as and when it is ready for consumption.

Based on the above analysis, I would say NSR can go right into master,
and we can easily make sure that it's not built as part of any release.
Now what NSR would follow, isn;t necessary for other translators to
follow. For eg. I would think Glusterd2 would have to be in experimental
coz it might meddle with current glusterd (but Atin would know better).
The point being, the decision we take need not be a collective decision
for all components, that would be enforced as a process, but rather
should be a decision taken by individual components based on merit.


Will code that NSR puts up in master be ready to ship when 3.8 is branched?

I ask the above, as I think we need a *process*, and not an open ended 
"put it where you want option", for the following reasons,
- Something that ships experimental is not to be consumed in production, 
so till the point an/any effort is ready it can be housed in experimental
- It is not about how much of infra code is impacted, or how much code 
would change, it is about readiness of the functionality

- The intention is also to keep such WIP xlators segregated in master
- It is easy to control the "don't build/ship" directive for everything 
under experimental, rather than make these decisions at a per 
directory/xlator/module level
- It is a clear message for anyone picking up these pieces to deploy and 
play with etc.


Coming to DHT2, irrespective of reason (1) above, all other reasons for 
which NSR can stay outside of experimental holds good for DHT2 as well.


I would leave others to comment if NSR gets into experimental or not, 
and what is the due diligence we need in this case.


So are we going ahead with experimental as a process? I am punting this 
to Vijay and Jeff :)





On 10/28/2015 08:32 PM, Shyam wrote:

Sending this along again.

- Are we decided on *experimental*?
- If so, what else needs attention in the patch below?
- (Re)views please... (views as in "What are your views on this?", not
"Have you viewed this?" ;) )

Shyam

On 10/12/2015 02:18 PM, Shyam wrote:

In an effort to push this along, update the change with suggested edits
and comments till now, request a review and further comments so that we
make this official at some (sooner) point in time.

http://review.gluster.org/#/c/12321/2
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Glusterfs mainline BZs to close...

2015-10-29 Thread Niels de Vos 
On Mon, Oct 26, 2015 at 08:42:34AM -0400, Kaleb S. KEITHLEY wrote:
> On 10/26/2015 05:14 AM, Niels de Vos wrote:
> > On Wed, Oct 21, 2015 at 05:22:49AM -0400, Nagaprasad Sathyanarayana wrote:
> >> I came across the following BZs which are still open in mainline.  But
> >> they are fixed and made available in a upstream release.  Planning to
> >> close them this week, unless there are any objections.
> > 
> > We have a policy to close bugs when their patches land in a released
> > version. The bugs against mainline will get closed when a release is
> > made that contains those fixes. For many of the current mainline bugs,
> > this would be the case when glusterfs-3.8 is released.
> 
> Some of those bugs are pretty old. We haven't been good about closing
> them when a 3.X.0 release has occurred. I sent email to the lists about
> closing some of the older ones with a note to reopen them if they're
> still valid.
> 
> The bugs with RFE in the subject, or RFE or FutureFeature as a keyword
> need to be left open after a 3.X.0 release if appropriate.
> 
> I'm going to take a stab at guessing — based on the date the bug was
> filed — at changing the version of the non-RFE bugs filed against mainline.

In addition to this, I have a script that

1. checks bugzilla for a list of open bugs
2. queries Gerrit for each bug
3. checks the git repository for tags of merged changes

A run earlier today, gave this as result: http://termbin.com/puiz

We seem to have quite some bugs that have all their changes merged in a
branch, before a tagged release was made. Those bugs should have been
closed when the stable release was announced. You can identify them when
you search for "Bug should be CLOSED".

There are also quite some bugs that had patches posted, but may have
been abandoned, or the patch does not have a "BUG: 0123456" tag or
correct topic. Those bugs are marked with "No change posted, but " ...
These all need some review, and either get closed as a duplicate, moved
back to "NEW" or "ASSIGNED", or anything else that helps the reporter to
understand the status.

  For example: https://bugzilla.redhat.com/show_bug.cgi?id=1213821#c4


I'll probably add my script to https://github.com/gluster/release-tools
so that others can improve it where needed. But for the time being,
would it be helpful to have the output emailed to gluster-devel once
every week?

Niels
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

2015-10-29 Thread Shyam 

On 10/29/2015 07:29 PM, Jeff Darcy wrote:

On October 29, 2015 at 9:12:46 AM, Shyam (srang...@redhat.com) wrote:

Will code that NSR puts up in master be ready to ship when 3.8 is
branched?


Do we know when 3.8 will be branched?


This is an example not absolute, what I mean is when the next release is 
made.





I ask the above, as I think we need a *process*, and not an open ended
"put it where you want option", for the following reasons, - Something
that ships experimental is not to be consumed in production, so till
the point an/any effort is ready it can be housed in experimental - It
is not about how much of infra code is impacted, or how much code
would change, it is about readiness of the functionality


I disagree.  From an operational perspective, unavoidable impact (not
just on infrastructure but on any existing functionality) is *the* thing
we want to avoid.  Experimental code that sits on disk, but isn’t even
loaded into memory without an explicit request to enable it, carries
little risk.  Experimental code that’s loaded and executed every time
Gluster is run carries more risk, even if it’s just one line.


I assume this is about infra changes (as the first 2 points are for some 
reason squashed in my reader). I think what you state is infra (or other 
non-experimental) code impact due to changes by experimental/inprogress 
code, should be dealt with clearly and carefully so as to not impact 
regular functionality. In which case I *agree* and do not mean otherwise.


I think this sort of goes back to what Niels commented on my squashing a 
.patch and proposed using #define EXPERIMENTAL in this thread (or such 
methods).





- The intention is also to keep such WIP xlators segregated in master
- It is easy to control the "don't build/ship" directive for
everything under experimental, rather than make these decisions at a
per directory/xlator/module level


They’re likely to make those decisions at the finer granularity anyway.
Most people will probably only want to try out one experimental
translator at a time.  Making them edit two makefiles (to enable
“experimental” and then a specific translator within it) instead of just
one doesn’t seem to get us very far.  Either way they’ll have to edit
the specfile, and they’ll see a list of all the experimental translators
they could enable.


- It is a clear message for anyone picking up these pieces to deploy
and play with etc.


If having to edit makefiles and specfiles by hand isn’t enough, there
are other things we could do to send such a clear message.  For example,
we could require a special configure flag or glusterd startup option to
enable management support for experimental features - not just whole
translators but options within existing translators as well.


Coming to DHT2, irrespective of reason (1) above, all other reasons
for which NSR can stay outside of experimental holds good for DHT2 as
well.


Perhaps.  I think that’s pretty clearly true for the DHT2 translator
itself.  For the associated posix changes it’s less clearly true, but
the modified version could go in storage/posix2 as easily as
experimental/posix or experimental/posix2.


Yes, posix2 is where the new posix code would land, hence the comment on 
DHT2 being mostly similar in nature.




IMO the main reason to have an “experimental” directory is one not
mentioned yet - to put them in a separate RPM.  This is not the same as
your “don’t build/ship” point above because they’ll get *built* anyway.


I think I should word my response more clearly, as "don't build/ship" is 
taken literally.


The choice of experimental as a separate entity, makes some of the 
choices presented below easier to implement/follow IMHO, which is what I 
am getting at.


Another thing I am getting at is, we *should* define a clear way to do 
such things, and not leave it open ended, which is where we seem to be 
headed below.



They’ll just be separately installable - or not, for people who aren’t
interested in experimental code.  Then we could combine that with the
special glusterd startup option mentioned above to make the wole process
of enabling or disabling experimental translators pretty seamless.
Install the RPM, tweak the option, and you can use experimental code.
Otherwise you can’t, and you get a nice clear error message if you try.


So, here's what I think we should do (right now - subject to change).

  1. We should create an "experimental" directory, and update makefiles
 accordingly.


I am going to point back to the patch around this here, as it contains a 
README as well, which we can put these specifics into, 
http://review.gluster.org/#/c/12321/2


We can further that, or create a new one, I am fine either way.



  2. The main Gluster 4.0 translators - dht2+posix, nsr* - should go into
 the experimental directory and the specfile should put those
 translators into a new RPM.

  3. All experimental functionality, whether in its own translator or
 otherwise,

Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

2015-10-29 Thread Jeff Darcy 
On October 29, 2015 at 9:12:46 AM, Shyam (srang...@redhat.com) wrote:
> Will code that NSR puts up in master be ready to ship when 3.8 is
> branched?

Do we know when 3.8 will be branched?

> I ask the above, as I think we need a *process*, and not an open ended
> "put it where you want option", for the following reasons, - Something
> that ships experimental is not to be consumed in production, so till
> the point an/any effort is ready it can be housed in experimental - It
> is not about how much of infra code is impacted, or how much code
> would change, it is about readiness of the functionality

I disagree.  From an operational perspective, unavoidable impact (not
just on infrastructure but on any existing functionality) is *the* thing
we want to avoid.  Experimental code that sits on disk, but isn’t even
loaded into memory without an explicit request to enable it, carries
little risk.  Experimental code that’s loaded and executed every time
Gluster is run carries more risk, even if it’s just one line.

> - The intention is also to keep such WIP xlators segregated in master
> - It is easy to control the "don't build/ship" directive for
> everything under experimental, rather than make these decisions at a
> per directory/xlator/module level

They’re likely to make those decisions at the finer granularity anyway.
Most people will probably only want to try out one experimental
translator at a time.  Making them edit two makefiles (to enable
“experimental” and then a specific translator within it) instead of just
one doesn’t seem to get us very far.  Either way they’ll have to edit
the specfile, and they’ll see a list of all the experimental translators
they could enable.

> - It is a clear message for anyone picking up these pieces to deploy
> and play with etc.

If having to edit makefiles and specfiles by hand isn’t enough, there
are other things we could do to send such a clear message.  For example,
we could require a special configure flag or glusterd startup option to
enable management support for experimental features - not just whole
translators but options within existing translators as well.

> Coming to DHT2, irrespective of reason (1) above, all other reasons
> for which NSR can stay outside of experimental holds good for DHT2 as
> well.

Perhaps.  I think that’s pretty clearly true for the DHT2 translator
itself.  For the associated posix changes it’s less clearly true, but
the modified version could go in storage/posix2 as easily as
experimental/posix or experimental/posix2.

IMO the main reason to have an “experimental” directory is one not
mentioned yet - to put them in a separate RPM.  This is not the same as
your “don’t build/ship” point above because they’ll get *built* anyway.
They’ll just be separately installable - or not, for people who aren’t
interested in experimental code.  Then we could combine that with the
special glusterd startup option mentioned above to make the wole process
of enabling or disabling experimental translators pretty seamless.
Install the RPM, tweak the option, and you can use experimental code.
Otherwise you can’t, and you get a nice clear error message if you try.


So, here's what I think we should do (right now - subject to change).

 1. We should create an "experimental" directory, and update makefiles
    accordingly.

 2. The main Gluster 4.0 translators - dht2+posix, nsr* - should go into
    the experimental directory and the specfile should put those
    translators into a new RPM.

 3. All experimental functionality, whether in its own translator or
    otherwise, should be controlled by an option which is off by
    default.

 4. Configuring an experimental feature should require a special
    glusterd flag (plus installation of the experimental RPM).
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] pluggability of some aspects in afr/nsr/ec

2015-10-29 Thread Pranith Kumar Karampuri 

hi,
 I want to understand how are you guys planning to integrate 
NSR volumes to the existing CLIs. Here are some thoughts I had, wanted 
to know your thoughts:

At the heart of both the replication/ec schemes we have
1) synchronization mechanisms
   a) afr,ec does it using locks
   b) nsr does it using leader election
2) Metadata to figure out the healing/reconciliation aspects
   a) afr,ec does it using xattrs
   b) nsr does it using journals

I want to understand if there is a possibility of exposing these as 
different modules that we can mix and match, using options. If the users 
choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future if we 
come up with better metadata journals/stores it should be easy to plug 
them is what I'm thinking. The idea I have is based on the workload, 
users should be able to decide which pair of synchronization/metadata 
works best for them (Or we can also recommend based on our tests). 
Wanted to seek your inputs.


Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] pluggability of some aspects in afr/nsr/ec

2015-10-29 Thread Venky Shankar 
On Thu, Oct 29, 2015 at 11:36 AM, Pranith Kumar Karampuri
 wrote:
> hi,
>  I want to understand how are you guys planning to integrate NSR
> volumes to the existing CLIs. Here are some thoughts I had, wanted to know
> your thoughts:
> At the heart of both the replication/ec schemes we have
> 1) synchronization mechanisms
>a) afr,ec does it using locks
>b) nsr does it using leader election
> 2) Metadata to figure out the healing/reconciliation aspects
>a) afr,ec does it using xattrs
>b) nsr does it using journals
>
> I want to understand if there is a possibility of exposing these as
> different modules that we can mix and match, using options. If the users

Do you mean abstracting it out during volume creation? At a high level
this could be in the form of client or server
side replication. Not that AFR cannot be used on the server side
(you'd know better than me), but, if at all this level
of abstraction is used, we'd need to default to what fits best in what
use case (as you already mentioned below)
but still retaining the flexibility to override it.

> choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future if we come
> up with better metadata journals/stores it should be easy to plug them is
> what I'm thinking. The idea I have is based on the workload, users should be
> able to decide which pair of synchronization/metadata works best for them
> (Or we can also recommend based on our tests). Wanted to seek your inputs.
>
> Pranith
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] pluggability of some aspects in afr/nsr/ec

2015-10-29 Thread Pranith Kumar Karampuri 



On 10/29/2015 12:18 PM, Venky Shankar wrote:

On Thu, Oct 29, 2015 at 11:36 AM, Pranith Kumar Karampuri
 wrote:

hi,
  I want to understand how are you guys planning to integrate NSR
volumes to the existing CLIs. Here are some thoughts I had, wanted to know
your thoughts:
At the heart of both the replication/ec schemes we have
1) synchronization mechanisms
a) afr,ec does it using locks
b) nsr does it using leader election
2) Metadata to figure out the healing/reconciliation aspects
a) afr,ec does it using xattrs
b) nsr does it using journals

I want to understand if there is a possibility of exposing these as
different modules that we can mix and match, using options. If the users

Do you mean abstracting it out during volume creation? At a high level
this could be in the form of client or server
side replication. Not that AFR cannot be used on the server side
(you'd know better than me), but, if at all this level
of abstraction is used, we'd need to default to what fits best in what
use case (as you already mentioned below)
but still retaining the flexibility to override it.
precisely. I think switching is not that difficult once we make sure 
healing is complete. Switching is a rare operation IMO so we can 
probably ask the users to do stop/choose-new-value/start the volume 
after choosing the options. This way is simpler than to migrate between 
the volumes where you have to probably copy the data.


Pranith



choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future if we come
up with better metadata journals/stores it should be easy to plug them is
what I'm thinking. The idea I have is based on the workload, users should be
able to decide which pair of synchronization/metadata works best for them
(Or we can also recommend based on our tests). Wanted to seek your inputs.

Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

2015-10-29 Thread Jeff Darcy 
On October 29, 2015 at 8:42:50 PM, Shyam (srang...@redhat.com) wrote:
> I assume this is about infra changes (as the first 2 points are for
> some reason squashed in my reader). I think what you state is infra
> (or other non-experimental) code impact due to changes by
> experimental/inprogress code, should be dealt with clearly and
> carefully so as to not impact regular functionality. In which case I
> *agree* and do not mean otherwise.
>
> I think this sort of goes back to what Niels commented on my squashing
> a .patch and proposed using #define EXPERIMENTAL in this thread (or
> such methods).

Sort of, although I would prefer that the distinction be run-time
instead of compile-time whenever possible.

> > 3. All experimental functionality, whether in its own translator or
> > otherwise, should be controlled by an option which is off by
> > default.
>
> Ah! I think this is something akin to the "#define EXPERIMENTAL"
> suggestion and non-experimental code impact I guess, right?

I think so.  Also, since I wasn’t clear before, I think there should be
*separate* options per feature, not one blanket “experimental” option.
For example, if you want to play with DHT you’d need to:

(1) Install the gluster-experimental RPM

(2) Tweak the glusterd script or volfile to allow experimental features

(3) Set cluster.dht2 (or whatever) on your volume

Note the absence of steps to download, hand-edit, or build anything
yourself.  I think that’s key: no risk if you don’t go out of your way
to enable experimental code, but you don’t have to be a full-time
Gluster developer to walk on the wild side.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel