Re: [Gluster-devel] Gluster volume snapshot - Plugin architecture proposal

[Rajesh Joseph]

On Mar 22, 2017 8:00 PM,  wrote:

Hi Amar,

On Wed, Mar 22, 2017, at 04:26 AM, Amar Tumballi wrote:

Hi Sriram,
Thanks for sharing this. Just one comment below.

On Tue, Mar 21, 2017 at 10:12 AM,  wrote:


Hi Raghavendra,

My name is Sriram I'd been working with Rajesh on creating a plugin
structure for snapshot functionality. Below is the document which Rajesh'd
created and I've edited the same with ideas and problems. Could you have a
look and review so that we could take it forward?

https://docs.google.com/document/d/1dHij_oy8V8CF2I7WfdYqKXFT
Gw0SKGzGlngDBpSwSGc/edit?usp=sharing


I am not sure if any 'code' has been written already for this. If not,
great, because we want any improvements in 'glusterd' space to come through
approval of new GlusterD 
design phase, so we can move away with the current glusterd.
But, looking at the design, the plugin can be pretty much be independent of
the glusterd architecture, even then better to run it through everyone
involved first.


There was a series of patches which I'd sent initially but, then Rajesh and
Avra proposed that we revisit the design once. So we'd stopped writing any
code.

https://review.gluster.org/#/c/16138/ This was the series I'd posted.  I'm
not really sure who'd be helping on this activity to get them involved in
the mail loop. I'll have a look at the GlusterD design phase approval wiki.
Let me know as how we proceed on this.



I think the Snapshot plugin architecture can be more or less independent of
glusterd design. The snapshot management module which will interact with
glusterD should be aware of the new design. With new GlusterD design the
existing snapshot code will any way need some refactoring. Therefore I
think the new plugin architecture will help the future migration to new
glusterd design.

It would be good if GlusterD team can aslo review the approach so that it
will be in sync with the new design. Will be good to hear more from
GlusterD team here.

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Snapshot maintainer

[Rajesh Joseph]

Hi all,

I have taken an opportunity which will take considerable amount of my
time. Therefore I may not be able to spend the desired time as a
maintainer. Because of which I would require a new maintainer for
snapshot to replace me. I will continue as a contributor to the
feature and Gluster in general.

Best Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Logging in a multi-brick daemon

[Rajesh Joseph]

On Thu, Feb 16, 2017 at 9:46 AM, Ravishankar N  wrote:
> On 02/16/2017 04:09 AM, Jeff Darcy wrote:
>>
>> One of the issues that has come up with multiplexing is that all of the
>> bricks in a process end up sharing a single log file.  The reaction from
>> both of the people who have mentioned this is that we should find a way to
>> give each brick its own log even when they're in the same process, and make
>> sure gf_log etc. are able to direct messages to the correct one.  I can
>> think of ways to do this, but it doesn't seem optimal to me.  It will
>> certainly use up a lot of file descriptors.  I think it will use more
>> memory.  And then there's the issue of whether this would really be better
>> for debugging.  Often it's necessary to look at multiple brick logs while
>> trying to diagnose this problem, so it's actually kind of handy to have them
>> all in one file.  Which would you rather do?
>>
>> (a) Weave together entries in multiple logs, either via a script or in
>> your head?
>>
>> (b) Split or filter entries in a single log, according to which brick
>> they're from?
>>
>> To me, (b) seems like a much more tractable problem.  I'd say that what we
>> need is not multiple logs, but *marking of entries* so that everything
>> pertaining to one brick can easily be found.  One way to do this would be to
>> modify volgen so that a brick ID (not name because that's a path and hence
>> too long) is appended/prepended to the name of every translator in the
>> brick.  Grep for that brick ID, and voila!  You now have all log messages
>> for that brick and no other.  A variant of this would be to leave the names
>> alone and modify gf_log so that it adds the brick ID automagically (based on
>> a thread-local variable similar to THIS).  Same effect, other than making
>> translator names longer, so I'd kind of prefer this approach.  Before I
>> start writing the code, does anybody else have any opinions, preferences, or
>> alternatives I haven't mentioned yet?
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
> My vote is for having separate log files per brick. Even in separate log
> files
> that we have today, I find it difficult to mentally ignore irrelevant
> messages
> in a single log file as I am sifting through it to look for errors that are
> related to the problem at hand. Having entries from multiple bricks and then
> grepping it would only make things harder. I cannot think of a case where
> having
> entries from all bricks in one file would particularly be beneficial for
> debugging since what happens in one brick is independent of the other bricks
> (at least until we move client xlators to server side and run them in the
> brick process).
> As for file descriptor count/memory usage, I think we should be okay
> as it is not any worse than that in the non-multiplexed approach we have
> today.
>
> On a side-note, I think the problem is not having too many log files but
> having
> them in multiple nodes. Having a log-aggregation solution where all messages
> are
> logged to a single machine (but still in separate files) would make it
> easier to
> monitor/debug issues.
> -Ravi
>

I believe the logs are not just from one volume but from all. In that
case merging them
into a single log file may not be great for debugging. Especially in
container use cases
there can be multiple volumes. Yes, with some tagging and scripting we
can separate
the logs and still live with it.

What about the log levels? Each volume can configure different log
levels. Will you carve
out a separate process in case log levels are changed for a volume?
How is this handled
here?

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Invitation: Re: Question on merging zfs snapshot supp... @ Tue Dec 20, 2016 2:30pm - 3:30pm (IST) (sri...@marirs.net.in)

[Rajesh Joseph]

Sure, will setup it from next week onward.

-Rajesh

On Mon, Jan 2, 2017 at 4:38 PM, <sri...@marirs.net.in> wrote:

> Hi Rajesh,
>
> Right now bi-weekly should be ok, with progress we could decide. I'll
> continue to rework the initial patch set and post it for review. We'll take
> it from there, is that ok with you?
>
> Sriram
>
>
> On Mon, Jan 2, 2017, at 03:32 PM, Rajesh Joseph wrote:
>
>
>
> On Mon, Jan 2, 2017 at 3:19 PM, <sri...@marirs.net.in> wrote:
>
>
> Hi Avra,
>
> Is the below request ok with you?
>
> Sriram
>
>
> On Wed, Dec 21, 2016, at 10:00 AM, sri...@marirs.net.in wrote:
>
> Hi Avra/Rajesh,
>
> In continuation to the discussion we'd yesterday, I'd be working on the
> change we'd initiated sometime back for pluggable FS specific snapshot
> implementation. We'd be moving our gluster deployements to "master"
> (stable) once this feature goes in. Since, glusterd2.0 release is scheduled
> release next year, I'd be happy if some of the work done here is re-usable
> to glusterd2.0 as well.
>
> Let me know, if this is ok. Like Rajesh mentioned in the call, could we've
> a weekly meeting for the same feature?
>
>
> Hi Sriram,
> I was on vacation so could not reply to your mail.
> I am OK with having a regular sync-up on this issue. Let's take this to
> conclusion.
> Do we need a weekly meeting or bi-weekly meeting is fine?
> Best Regards,
> Rajesh
>
>
>
>
>
> Sriram
>
> On Mon, Dec 19, 2016, at 03:55 PM, aseng...@redhat.com wrote:
>
>
>
> more details »
> <https://www.google.com/calendar/event?action=VIEW=YWJoMmtwNHAzc3EyNDgybTRjb2llaW1jNm8gc3JpcmFtQG1hcmlycy5uZXQuaW4=MTkjYXNlbmd1cHRAcmVkaGF0LmNvbTYyNWZlYjFmYzg2NWRkNGI2YzAyY2FlYmVkMTIwM2VlZmMxZTY0Mzg=Asia/Calcutta=en>
> Re: [Gluster-devel] Question on merging zfs snapshot support into the
> mainline glusterfs
> Hi Sriram,
>
> Could you please join the hangout, so that we can discuss snapshot
> plugabbility. Thanks
>
> Meeting Link: https://bluejeans.com/u/asengupt/
> <https://www.google.com/url?q=https%3A%2F%2Fbluejeans.com%2Fu%2Fasengupt%2F=D=2=AFQjCNHgp0xCwA9DqgdbAc9s2OxthUEHRA>
>
> Regards,
> Avra
>
> On 12/19/2016 01:38 PM, sri...@marirs.net.in wrote:
> > Hi Avra,
> >
> > Could you help on the below request? May I abandon the previous
> submitted patches, and could we consider the latest one?
> >
> > Sriram
> >
> >
> > On Thu, Dec 15, 2016, at 12:57 PM, sri...@marirs.net.in wrote:
> >> Hi Avra,
> >>
> >> Thanks for the reply,
> >>
> >> But the problem I see here is the previous patch set sent would'nt
> compile individually. So, I merged the changes into a single patch , which
> i'd posted today. Is it ok to drop all the previous posted patches and
> consider from the new one? Please suggest.
> >>
> >> Sriram
> >>
> >>
> >> On Thu, Dec 15, 2016, at 12:45 PM, Avra Sengupta wrote:
> >>> Hi Sriram,
> >>>
> >>> I have already provided comments on the new patch. It seems this new
> patch while addressing merge cloflicts, has undone some previous patches. I
> suggest you send this patch on top of the previous patchset(
> http://review.gluster.org/#/c/15554/1
> <https://www.google.com/url?q=http%3A%2F%2Freview.gluster.org%2F%23%2Fc%2F15554%2F1=D=2=AFQjCNE4gL3TlKKImxMOU_yOKoCFnP27BA>)
> instead of creating a new one. This will allow you to view the diff between
> the new version and the previous version, and will give u an idea if the
> diff is something that you added in the patch or got added as part of merge
> conflict.
> >>>
> >>> Regards,
> >>> Avra
> >>>
> >>> On 12/15/2016 12:09 PM, sri...@marirs.net.in wrote:
> >>>> Hi Avra,
> >>>>
> >>>> I've update the patch according to the comments below. And created a
> single patch which does the initial modularization. Fixed the tab->space
> issue as well. I've raised a new review request for the same bug ID here:
> >>>> http://review.gluster.org/#/c/16138/
> <https://www.google.com/url?q=http%3A%2F%2Freview.gluster.org%2F%23%2Fc%2F16138%2F=D=2=AFQjCNGST3yFW0o5r4X5DiUXR0GOvgUUUQ>
> >>>>
> >>>> Added, Rajesh and You as the reviewers, let me know if I need to do
> anything else.
> >>>>
> >>>> Could you have a look and let me know?
> >>>>
> >>>> (Sorry for the delay in creating this)
> >>>>
> >>>> Sriram
> >>>>
> >>>> On Thu, Oct 13, 2016, at 12:15 PM, Avra S

Re: [Gluster-devel] Invitation: Re: Question on merging zfs snapshot supp... @ Tue Dec 20, 2016 2:30pm - 3:30pm (IST) (sri...@marirs.net.in)

[Rajesh Joseph]

On Mon, Jan 2, 2017 at 3:19 PM,  wrote:

> Hi Avra,
>
> Is the below request ok with you?
>
> Sriram
>
>
> On Wed, Dec 21, 2016, at 10:00 AM, sri...@marirs.net.in wrote:
>
> Hi Avra/Rajesh,
>
> In continuation to the discussion we'd yesterday, I'd be working on the
> change we'd initiated sometime back for pluggable FS specific snapshot
> implementation. We'd be moving our gluster deployements to "master"
> (stable) once this feature goes in. Since, glusterd2.0 release is scheduled
> release next year, I'd be happy if some of the work done here is re-usable
> to glusterd2.0 as well.
>
> Let me know, if this is ok. Like Rajesh mentioned in the call, could we've
> a weekly meeting for the same feature?
>
>
Hi Sriram,

I was on vacation so could not reply to your mail.

I am OK with having a regular sync-up on this issue. Let's take this to
conclusion.
Do we need a weekly meeting or bi-weekly meeting is fine?

Best Regards,
Rajesh



>
> Sriram
>
> On Mon, Dec 19, 2016, at 03:55 PM, aseng...@redhat.com wrote:
>
>
>
> more details »
> 
> Re: [Gluster-devel] Question on merging zfs snapshot support into the
> mainline glusterfs
> Hi Sriram,
>
> Could you please join the hangout, so that we can discuss snapshot
> plugabbility. Thanks
>
> Meeting Link: https://bluejeans.com/u/asengupt/
> 
>
> Regards,
> Avra
>
> On 12/19/2016 01:38 PM, sri...@marirs.net.in wrote:
> > Hi Avra,
> >
> > Could you help on the below request? May I abandon the previous
> submitted patches, and could we consider the latest one?
> >
> > Sriram
> >
> >
> > On Thu, Dec 15, 2016, at 12:57 PM, sri...@marirs.net.in wrote:
> >> Hi Avra,
> >>
> >> Thanks for the reply,
> >>
> >> But the problem I see here is the previous patch set sent would'nt
> compile individually. So, I merged the changes into a single patch , which
> i'd posted today. Is it ok to drop all the previous posted patches and
> consider from the new one? Please suggest.
> >>
> >> Sriram
> >>
> >>
> >> On Thu, Dec 15, 2016, at 12:45 PM, Avra Sengupta wrote:
> >>> Hi Sriram,
> >>>
> >>> I have already provided comments on the new patch. It seems this new
> patch while addressing merge cloflicts, has undone some previous patches. I
> suggest you send this patch on top of the previous patchset(http://review.
> gluster.org/#/c/15554/1
> )
> instead of creating a new one. This will allow you to view the diff between
> the new version and the previous version, and will give u an idea if the
> diff is something that you added in the patch or got added as part of merge
> conflict.
> >>>
> >>> Regards,
> >>> Avra
> >>>
> >>> On 12/15/2016 12:09 PM, sri...@marirs.net.in wrote:
>  Hi Avra,
> 
>  I've update the patch according to the comments below. And created a
> single patch which does the initial modularization. Fixed the tab->space
> issue as well. I've raised a new review request for the same bug ID here:
>  http://review.gluster.org/#/c/16138/
> 
> 
>  Added, Rajesh and You as the reviewers, let me know if I need to do
> anything else.
> 
>  Could you have a look and let me know?
> 
>  (Sorry for the delay in creating this)
> 
>  Sriram
> 
>  On Thu, Oct 13, 2016, at 12:15 PM, Avra Sengupta wrote:
> > Hi Sriram,
> >
> > The point I was trying to make is, that we want that each patch
> should compile by itself, and pass regression. So for that to happen, we
> need to consolidate these patches(the first three) into one patch, and have
> the necessary make file changes into that patch too.
> >
> > http://review.gluster.org/#/c/15554/
> 
> > http://review.gluster.org/#/c/1/
> 
> > http://review.gluster.org/#/c/15556/
> 
> >
> > That will give us one single patch, that contains the changes of
> having the current code moved into separate files, and it should get
> compiled on it's own, and should pass regression. Also, we use spaces, and
> not tabs in the code. So we will need to get those changed too. Thanks.
> >
> > Regards,
> > Avra

Re: [Gluster-devel] Release 3.10 feature proposal:: Statedump for libgfapi

[Rajesh Joseph]

On Mon, Dec 12, 2016 at 11:34 PM, Shyam <srang...@redhat.com> wrote:
> On 12/12/2016 12:26 AM, Niels de Vos wrote:
>>
>> On Fri, Dec 09, 2016 at 06:20:22PM +0530, Rajesh Joseph wrote:
>>>
>>> Gluster should have some provision to take statedump of gfapi
>>> applications.
>>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1169302
>>
>>
>> A part of this feature should be to find out how applications that use
>> libgfapi expect to trigger debugging like this. Doing a statedump from
>> the gluster-cli should not be the main/only option. I agree that it
>> helps developers that work on Gluster, but we can not expect users to
>> trigger statedumps like that.
>>
>> I think there would be a huge benefit in having an option to communicate
>> with libgfapi through some minimal form of local IPC. It will allow
>> doing statedumps, and maybe even set/get configuration options for
>> applications that do not offer these in their usage (yet).
>>
>> The communication should be as simple and stable as possible. This could
>> be the only working interface towards getting something done inside
>> gfapi (worst case scenario). There is no need to have this a full
>> featured interface, possibly a named pipe (fifo) where libgfapi is the
>> reader is sufficient. A simple (text) command written to it can create
>> statedumps and eventually other files on request.
>>
>> Enabling/disabling or even selecting the possibilities for debugging
>> could be confiured through new functions in libgfapi, and even
>> environment variables.
>>
>> What do others think? Would this be useful?
>
>
> Would this be useful, yes.
>
> Would this work in cases like a container deployment, where such debugging
> maybe sought at scale, maybe not. I prefer options here, and also the
> ability to drive this from the storage admin perspective, i.e the
> server/brick end of things, identifying the client/connection against which
> we need the statedump and get that information over.
>

We were thinking something on the same line. Where statedumps can be
initiated by
glusterd by an admin. The option mentioned by Niels is also helpful
but that means
we should either provide some tool or the application has to do some
amount of changes
to make use of this feature.

> I guess the answer here is, this should not be the only option, but we
> can/should have other options as you describe above.
>

Make sense.

Thanks & Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Release 3.10 feature proposal: gfapi fix memory leak during graph switch

[Rajesh Joseph]

On Mon, Dec 12, 2016 at 11:42 PM, Shyam <srang...@redhat.com> wrote:
> On 12/09/2016 08:06 AM, Rajesh Joseph wrote:
>>
>> During graph switch we do not perform much cleanup operation on the
>> old graph, leading to memory leak.
>>
>> + Inode table of old graph needs cleanup.
>>- Fix inode leaks
>>- Fix forget of each xl to free inode ctx properly
>> + The xl objects itself (xlator_t)
>> + The mem_accnt structure in every xl object.
>>   - Fix all the leaks so that the ref count of mem_accnt structure is
>> 0
>> + Implement fini() in every xlator
>>
>> Would need support from all other xlator owner to bring down memory
>> leak in each xlator.
>
>
> Rajesh, I have marked this for 3.10 tentatively. As you call out a per
> xlator task, could you elaborate what is needed from others?


Basically we need proper fini implementation in all xlators. That means all
xlators must free up all resources allocated from init to fini.

>
> We need to possibly get ack's from the various component maintainers so that
> this can happen for 3.10. So if you could share a list of things that need
> to be done/checked for each component and drive the same, in terms of
> timelines, we could get this feature complete.
>

We have most of the big ticket items but each xlators must come up with their
list of leaks. May be for this release we can target only most frequently used
xlators. I will update the list on the github project planning card.

Thanks & Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Release 3.10 feature proposal: gfapi fix memory leak during graph switch

[Rajesh Joseph]

During graph switch we do not perform much cleanup operation on the
old graph, leading to memory leak.

+ Inode table of old graph needs cleanup.
   - Fix inode leaks
   - Fix forget of each xl to free inode ctx properly
+ The xl objects itself (xlator_t)
+ The mem_accnt structure in every xl object.
  - Fix all the leaks so that the ref count of mem_accnt structure is 0
+ Implement fini() in every xlator

Would need support from all other xlator owner to bring down memory
leak in each xlator.

Thanks & Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Release 3.10 feature proposal:: Statedump for libgfapi

[Rajesh Joseph]

Gluster should have some provision to take statedump of gfapi applications.

https://bugzilla.redhat.com/show_bug.cgi?id=1169302


Best Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Merge window closed for the approaching GlusterFS-3.7.18 release

[Rajesh Joseph]

On Mon, Nov 28, 2016 at 12:24 PM, Mohammed Rafi K C 
wrote:

>
> On 11/27/2016 12:34 PM, Samikshan Bairagya wrote:
> >
> > On 11/25/2016 05:24 PM, Samikshan Bairagya wrote:
> >> Hi all,
> >>
> >> GlusterFS-3.7.18 is on target to be released on November 30th
> >> (Wednesday).
> >>
> >> In preparation for the release, maintainers would need to merge all
> >> changes into release-3.7 before Saturday (November 26). This would give
> >> us ~4 days to test and verify the build.
> >>
> >
> > Hi all,
> >
> > Just a reminder, that maintainers would need to stop merging any more
> > changes into release-3.7 for the upcoming 3.7.18 release. In case
> > there are more changes that need to be merged, please reply to this
> > thread before Nov 28.
>

Following patch fixes libgfapi crash in a libvirt environment. Therefore I
would like to propose it for 3.7.

http://review.gluster.org/15910

The patch is pending for quite some time now. If its OK to merge this patch
then i can chase the corresponding
maintainers for it.

Thanks & Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-Maintainers] Gluster Test Thursday - Release 3.9

[Rajesh Joseph]

On Wed, Oct 26, 2016 at 8:04 PM, Aravinda  wrote:

> Gluster 3.9.0rc2 tarball is available here
> http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-
> 3.9.0rc2.tar.gz
>
> regards
> Aravinda
>
>

Tested snapshots with 3.9.0rc2 build. Everything seems to be working fine.
Ran all the snapshot specific regressions tests.
Also created a 2x2 setup and ran most of the basic snapshots
functionalities against it. Roughly covered following scenarios:
+ Snapshot creation
+ Snapshot deletion
+ Snapshot restore
+ Snapshot config
+ activate-on-create
+ auto-delete
+ brick down cases
+ clone
+ USS


+1 to this release candidate from snapshot perspective.


>
> On Tuesday 25 October 2016 04:12 PM, Aravinda wrote:
>
>> Hi,
>>
>> Since Automated test framework for Gluster is in progress, we need help
>> from Maintainers and developers to test the features and bug fixes to
>> release Gluster 3.9.
>>
>> In last maintainers meeting Shyam shared an idea about having a Test day
>> to accelerate the testing and release.
>>
>> Please participate in testing your component(s) on Oct 27, 2016. We will
>> prepare the rc2 build by tomorrow and share the details before Test day.
>>
>> RC1 Link: http://www.gluster.org/pipermail/maintainers/2016-September/
>> 001442.html
>> Release Checklist: https://public.pad.fsfe.org/p/
>> gluster-component-release-checklist
>>
>>
>> Thanks and Regards
>> Aravinda and Pranith
>>
>>
> ___
> maintainers mailing list
> maintain...@gluster.org
> http://www.gluster.org/mailman/listinfo/maintainers
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [gluster-devel] Documentation Tooling Review

[Rajesh Joseph]

On Thu, Sep 22, 2016 at 10:05 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:

> 2016-09-22 18:34 GMT+02:00 Amye Scavarda :
> > Nope! RHGS is the supported version and gluster.org is the open source
> > version. We'd like to keep the documentation reflecting that, but good
> > catch.
>
> Ok but features should be the same, right?
>

Yes, features would be same, infact upstream would be ahead of RHGS.
Similarly our upstream documentation will be ahead of Red Hat documentation.

I should have been more clear. In Red Hat documentation there are lot of
references to version specific changes which will not be applicable to
Gluster releases. Also there is branding difference Red Hat like to call
the product "Red Hat Gluster Storage" (RHGS) and we "Gluster"

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [gluster-devel] Documentation Tooling Review

[Rajesh Joseph]

On Thu, Sep 8, 2016 at 5:00 AM, Amye Scavarda  wrote:

>
>
> On Tue, Aug 23, 2016 at 4:42 PM, Joe Julian  wrote:
>
>>
>>
>> On 08/23/2016 12:27 PM, Justin Clift wrote:
>>
>>> On 11 Aug 2016, at 21:23, Amye Scavarda  wrote:
>>>
 The Red Hat Gluster Storage documentation team and I had a conversation
 about how we can our upstream documentation more consistent and improved
 for our users, and they're willing to work with us to find where the
 major
 gaps are in our documentation. This is awesome! But it's going to take
 some
 work on our side to make this a reality.

 One piece that's come up is that we should probably look towards
 changing
 current tooling for this. It turns out that our ReadTheDocs instance
 search
 is failing because we're using markdown, and this is a known issue. It
 doesn't look like it's going to be fixed anytime soon.

 Rather than continue to try to make RTD serve our needs, I'd like to
 propose the following changes to where our documentation lives and in
 what
 language:
 I'd much rather pattern after docs.openshift.org, move to ASCIIdoc and
 use
 ASCIIbinder as our engine to power this. What that does is give us
 control
 over our overall infrastructure underneath our documentation, maintain
 our
 existing git workflow for adding to documentation, and matches with
 other
 communities that we work closely with. I'm mindful that there's a
 burden of
 migration again, but we'll be able to resolve a lot of the challenges we
 have with documentation currently: more control over layout, ability to
 change the structure to make it more user friendly, use our own search
 however we see fit.

 I'm happy to take comments on this proposal. Over the next week, I'll be
 reviewing the level of effort it would take to migrate to ASCIIdocs and
 ASCIIbinder, with the goal being to have this in place by end of
 September.

 Thoughts?

>>> It's probably worth considering GitBook instead:
>>>
>>>https://www.gitbook.com
>>>
>>> Example here:
>>>
>>>http://tutorial.djangogirls.org/en/index.html
>>>
>>> Pros:
>>>
>>>* Works with Markdown & ASCIIdoc
>>>
>>>  No need to convert the existing docs to a new format,
>>>  and the already learned Markdown skills don't need relearning
>>>
>>>* Also fully Open Source
>>>
>>>  https://github.com/GitbookIO/gitbook/
>>>
>>>* Searching works very well
>>>
>>>  Try searching on the Django Girls tutorial above for "Python".
>>>
>>>  Correct results are returned in small fractions of a second.
>>>
>>>* Has well developed plugins to enable things like inline
>>>  videos, interactive exercises (and more)
>>>
>>>  https://plugins.gitbook.com
>>>
>>>* Can be self hosted, or hosted on the GitBooks infrastructure
>>>
>>>* Doesn't require Ruby, unlike ASCIIbinder which is written
>>>  in it.
>>>
>>> Cons:
>>>
>>>* It's written in Node.js instead
>>>
>>>  Not sure that's any better than Ruby
>>>
>>> It seems a better polished solution than docs.openshift.org is using,
>>> and would probably require less effort for the Gluster docs to be adapted
>>> to.
>>>
>>> Thoughts? :)
>>>
>>> + Justin
>>>
>>> +1 This seems like the most sane solution
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
> I'll jump back in here:
> https://github.com/gluster/glusterdocs/pulse/monthly shows not a ton of
> activity currently, as Humble pointed to earlier. That's not a bad thing,
> but this may speak to Niels' point that contributing to Docs isn't
> something that's currently well known.
>
> My sense is that we should separate this into two different conversations:
> improving the user + admin guides with contributions from the Red Hat
> Gluster Storage team, and the second conversation about how we take
> developer contribution. That gets Gluster.org back to a place where we can
> contribute the developer guides with what's under active development.
>
> Unfortunately, the tooling part is where we start with this, because the
> contribution of the actively maintained documentation also depends on
> ascii. As soon as I have a link to a proof of concept for what ASCIIdocs +
> asciibinder could look like, I'll post it here for review.
>
> My goal in pushing this is to get us to a place where we're contributing
> the things that we know best, the features we're actively working on, and
> how to help contribute to the project, while using the resources we have to
> improve the user experience.
>
> --
> Amye Scavarda | a...@redhat.com | Gluster Community Lead
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> 

Re: [Gluster-devel] 3.9. feature freeze status check

[Rajesh Joseph]

On Fri, Aug 26, 2016 at 9:38 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> hi,
>   Now that we are almost near the feature freeze date (31st of Aug),
> want to get a sense if any of the status of the features.
>
> Please respond with:
> 1) Feature already merged
> 2) Undergoing review will make it by 31st Aug
> 3) Undergoing review, but may not make it by 31st Aug
> 4) Feature won't make it for 3.9.
>
> I added the features that were not planned(i.e. not in the 3.9 roadmap
> page) but made it to the release and not planned but may make it to release
> at the end of this mail.
> If you added a feature on master that will be released as part of 3.9.0
> but forgot to add it to roadmap page, please let me know I will add it.
>
> Here are the features planned as per the roadmap:
> 1) Throttling
> Feature owner: Ravishankar
>
> 2) Trash improvements
> Feature owners: Anoop, Jiffin
>
> 3) Kerberos for Gluster protocols:
> Feature owners: Niels, Csaba
>
> 4) SELinux on gluster volumes:
> Feature owners: Niels, Manikandan
>
> 5) Native sub-directory mounts:
> Feature owners: Kaushal, Pranith
>
> 6) RichACL support for GlusterFS:
> Feature owners: Rajesh Joseph
>
>
Sorry, this feature won't make it for 3.9, hopefully fill be done by next
release.




> 7) Sharemodes/Share reservations:
> Feature owners: Raghavendra Talur, Poornima G, Soumya Koduri, Rajesh
> Joseph, Anoop C S
>
> 8) Integrate with external resource management software
> Feature owners: Kaleb Keithley, Jose Rivera
>
> 9) Python Wrappers for Gluster CLI Commands
> Feature owners: Aravinda VK
>
> 10) Package and ship libgfapi-python
> Feature owners: Prashant Pai
>
> 11) Management REST APIs
> Feature owners: Aravinda VK
>
> 12) Events APIs
> Feature owners: Aravinda VK
>
> 13) CLI to get state representation of a cluster from the local glusterd
> pov
> Feature owners: Samikshan Bairagya
>
> 14) Posix-locks Reclaim support
> Feature owners: Soumya Koduri
>
> 15) Deprecate striped volumes
> Feature owners: Vijay Bellur, Niels de Vos
>
> 16) Improvements in Gluster NFS-Ganesha integration
> Feature owners: Jiffin Tony Thottan, Soumya Koduri
>
> *The following need to be added to the roadmap:*
>
> Features that made it to master already but were not palnned:
> 1) Multi threaded self-heal in EC
> Feature owner: Pranith (Did this because serkan asked for it. He has 9PB
> volume, self-healing takes a long time :-/)
>
> 2) Lock revocation (Facebook patch)
> Feature owner: Richard Wareing
>
> Features that look like will make it to 3.9.0:
> 1) Hardware extension support for EC
> Feature owner: Xavi
>
> 2) Reset brick support for replica volumes:
> Feature owner: Anuradha
>
> 3) Md-cache perf improvements in smb:
> Feature owner: Poornima
>
> --
> Pranith
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

On Thu, Jul 21, 2016 at 3:07 AM, Vijay Bellur <vbel...@redhat.com> wrote:

> On 07/19/2016 11:01 AM, Atin Mukherjee wrote:
>
>>
>>
>> On Tue, Jul 19, 2016 at 7:29 PM, Rajesh Joseph <rjos...@redhat.com
>> <mailto:rjos...@redhat.com>> wrote:
>>
>>
>>
>> On Tue, Jul 19, 2016 at 11:23 AM, <sri...@marirs.net.in
>> <mailto:sri...@marirs.net.in>> wrote:
>>
>> __
>> Hi Rajesh,
>>
>> I'd thought about moving the zfs specific implementation to
>> something like
>>
>> xlators/mgmt/glusterd/src/plugins/zfs-specifs-stuffs for the
>> inital go. Could you let me know if this works or in sync with
>> what you'd thought about?
>>
>> Sriram
>>
>>
>> Hi Sriram,
>>
>> Sorry, I was not able to send much time on this. I would prefer you
>> move the code to
>>
>> xlators/mgmt/glusterd/plugins/src/zfs-specifs-stuffs
>>
>>
>>
>> How about having it under
>> xlators/mgmt/glusterd/plugins/snapshot/src/zfs-specifs-stuffs such that
>> in future if we have to write plugins for other features they can be
>> segregated?
>>
>>
> It would be nicer to avoid "specific-stuff" or similar from the naming. We
> can probably leave it at xlators/mgmt/glusterd/plugins/snapshot/src/zfs.
> The naming would be sufficient to indicate that code is specific to zfs
> snapshots.
>

I don't think the directory would be named "zfs-specific_stuffs, instead
zfs specific source file will come directly under
"xlators/mgmt/glusterd/plugins/snapshot/src/".
I think I should have been more clear, my bad.

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

On Tue, Jul 19, 2016 at 11:23 AM, <sri...@marirs.net.in> wrote:

> Hi Rajesh,
>
> I'd thought about moving the zfs specific implementation to something like
>
> xlators/mgmt/glusterd/src/plugins/zfs-specifs-stuffs for the inital go.
> Could you let me know if this works or in sync with what you'd thought
> about?
>
> Sriram
>

Hi Sriram,

Sorry, I was not able to send much time on this. I would prefer you move
the code to

xlators/mgmt/glusterd/plugins/src/zfs-specifs-stuffs

atinm and others do let us know if you have any objection to this.

I captured our initial discussions on an etherpad (
https://public.pad.fsfe.org/p/gluster-volume-snapshots). I will update it
further, meanwhile you can also
capture more details in the etherpad if needed.

Best Regards,
Rajesh



>
> On Tue, Jul 12, 2016, at 03:52 PM, sri...@marirs.net.in wrote:
>
> Hi Rajesh,
>
> Sure thanks.
>
> Sriram
>
>
> On Tue, Jul 12, 2016, at 03:07 PM, Rajesh Joseph wrote:
>
> Hi Sriram,
> The interface is not yet finalized. May be this is the right time to
> re-ignite discussion on this.
> I can create an etherpad which will explain the initial thoughts and
> design ideas on the same.
> Thanks & Regards,
> Rajesh
>
> On Mon, Jul 11, 2016 at 11:57 PM, <sri...@marirs.net.in> wrote:
>
>
> Hi Rajesh,
>
> Could you let us know the idea on how to go about this?
>
> Sriram
>
>
> On Wed, Jul 6, 2016, at 03:18 PM, Pranith Kumar Karampuri wrote:
>
> I believe Rajesh already has something here. May be he can post an outline
> so that we can take it from there?
>
> On Tue, Jul 5, 2016 at 10:52 PM, <sri...@marirs.net.in> wrote:
>
>
> Hi,
>
> I tried to go through the patch and find the reason behind the question
> posted. But could'nt get any concrete details about the same.
>
> When going through the mail chain, there were mentions of generic snapshot
> interface. I'd be interested in doing the changes if you guys could fill me
> with some initial information. Thanks.
>
> Sriram
>
>
> On Mon, Jul 4, 2016, at 01:59 PM, B.K.Raghuram wrote:
>
> Hi Rajesh,
> I did not want to respond to the question that you'd posed on the zfs
> snapshot code (about the volume backend backup) as I am not too familiar
> with the code and the person who's coded it is not with us anymore. This
> was done in bit of a hurry so it could be that it was just kept for later..
>
> However, Sriram who is cc'd on this email, has been helping us by starting
> to look at the gluster code  and has expressed an interest in taking the
> zfs code changes on. So he can probably dig out an answer to your question.
> Sriram, Rajesh had a question on one of the zfs related patches - (
> https://github.com/fractalio/glusterfs/commit/39a163eca338b6da146f72f380237abd4c671db2#commitcomment-18109851
> )
>
> Sriram is also interested in contributing to the process of creating a
> generic snapshot interface in the gluster code which you and Pranith
> mentioned above. If this is ok with you all, could you fill him in on what
> your thoughts are on that and how he could get started?
> Thanks!
> -Ram
>
> On Wed, Jun 22, 2016 at 11:45 AM, Rajesh Joseph <rjos...@redhat.com>
> wrote:
>
>
>
> On Tue, Jun 21, 2016 at 4:24 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
> hi,
>   Is there a plan to come up with an interface for snapshot
> functionality? For example, in handling different types of sockets in
> gluster all we need to do is to specify which interface we want to use and
> ib,network-socket,unix-domain sockets all implement the interface. The code
> doesn't have to assume anything about underlying socket type. Do you guys
> think it is a worthwhile effort to separate out the logic of interface and
> the code which uses snapshots? I see quite a few of if (strcmp ("zfs",
> fstype)) code which can all be removed if we do this. Giving btrfs
> snapshots in future will be a breeze as well, this way? All we need to do
> is implementing snapshot interface using btrfs snapshot commands. I am not
> talking about this patch per se. Just wanted to seek your inputs about
> future plans for ease of maintaining the feature.
>
>
>
> As I said in my previous mail this is in plan and we will be doing it. But
> due to other priorities this was not taken in yet.
>
>
>
>
>
> On Tue, Jun 21, 2016 at 11:46 AM, Atin Mukherjee <amukh...@redhat.com>
> wrote:
>
>
>
> On 06/21/2016 11:41 AM, Rajesh Joseph wrote:
> > What kind of locking issues you see? If you can provide some more
> > information I can be able to help you.
>
> That's related to stale lock issues on Glust

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

Hi Sriram,

The interface is not yet finalized. May be this is the right time to
re-ignite discussion on this.
I can create an etherpad which will explain the initial thoughts and design
ideas on the same.

Thanks & Regards,
Rajesh

On Mon, Jul 11, 2016 at 11:57 PM, <sri...@marirs.net.in> wrote:

> Hi Rajesh,
>
> Could you let us know the idea on how to go about this?
>
> Sriram
>
>
> On Wed, Jul 6, 2016, at 03:18 PM, Pranith Kumar Karampuri wrote:
>
> I believe Rajesh already has something here. May be he can post an outline
> so that we can take it from there?
>
> On Tue, Jul 5, 2016 at 10:52 PM, <sri...@marirs.net.in> wrote:
>
>
> Hi,
>
> I tried to go through the patch and find the reason behind the question
> posted. But could'nt get any concrete details about the same.
>
> When going through the mail chain, there were mentions of generic snapshot
> interface. I'd be interested in doing the changes if you guys could fill me
> with some initial information. Thanks.
>
> Sriram
>
>
> On Mon, Jul 4, 2016, at 01:59 PM, B.K.Raghuram wrote:
>
> Hi Rajesh,
> I did not want to respond to the question that you'd posed on the zfs
> snapshot code (about the volume backend backup) as I am not too familiar
> with the code and the person who's coded it is not with us anymore. This
> was done in bit of a hurry so it could be that it was just kept for later..
>
> However, Sriram who is cc'd on this email, has been helping us by starting
> to look at the gluster code  and has expressed an interest in taking the
> zfs code changes on. So he can probably dig out an answer to your question.
> Sriram, Rajesh had a question on one of the zfs related patches - (
> https://github.com/fractalio/glusterfs/commit/39a163eca338b6da146f72f380237abd4c671db2#commitcomment-18109851
> )
>
> Sriram is also interested in contributing to the process of creating a
> generic snapshot interface in the gluster code which you and Pranith
> mentioned above. If this is ok with you all, could you fill him in on what
> your thoughts are on that and how he could get started?
> Thanks!
> -Ram
>
> On Wed, Jun 22, 2016 at 11:45 AM, Rajesh Joseph <rjos...@redhat.com>
> wrote:
>
>
>
> On Tue, Jun 21, 2016 at 4:24 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
> hi,
>   Is there a plan to come up with an interface for snapshot
> functionality? For example, in handling different types of sockets in
> gluster all we need to do is to specify which interface we want to use and
> ib,network-socket,unix-domain sockets all implement the interface. The code
> doesn't have to assume anything about underlying socket type. Do you guys
> think it is a worthwhile effort to separate out the logic of interface and
> the code which uses snapshots? I see quite a few of if (strcmp ("zfs",
> fstype)) code which can all be removed if we do this. Giving btrfs
> snapshots in future will be a breeze as well, this way? All we need to do
> is implementing snapshot interface using btrfs snapshot commands. I am not
> talking about this patch per se. Just wanted to seek your inputs about
> future plans for ease of maintaining the feature.
>
>
>
> As I said in my previous mail this is in plan and we will be doing it. But
> due to other priorities this was not taken in yet.
>
>
>
>
>
> On Tue, Jun 21, 2016 at 11:46 AM, Atin Mukherjee <amukh...@redhat.com>
> wrote:
>
>
>
> On 06/21/2016 11:41 AM, Rajesh Joseph wrote:
> > What kind of locking issues you see? If you can provide some more
> > information I can be able to help you.
>
> That's related to stale lock issues on GlusterD which are there in 3.6.1
> since the fixes landed in the branch post 3.6.1. I have already provided
> the workaround/way to fix them [1]
>
> [1]
> http://www.gluster.org/pipermail/gluster-users/2016-June/thread.html#26995
>
> ~Atin
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
>
>
>
> --
> Pranith
>
>
>
>
>
>
>
>
>
> --
> Pranith
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

On Tue, Jun 21, 2016 at 4:24 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> hi,
>   Is there a plan to come up with an interface for snapshot
> functionality? For example, in handling different types of sockets in
> gluster all we need to do is to specify which interface we want to use and
> ib,network-socket,unix-domain sockets all implement the interface. The code
> doesn't have to assume anything about underlying socket type. Do you guys
> think it is a worthwhile effort to separate out the logic of interface and
> the code which uses snapshots? I see quite a few of if (strcmp ("zfs",
> fstype)) code which can all be removed if we do this. Giving btrfs
> snapshots in future will be a breeze as well, this way? All we need to do
> is implementing snapshot interface using btrfs snapshot commands. I am not
> talking about this patch per se. Just wanted to seek your inputs about
> future plans for ease of maintaining the feature.
>

As I said in my previous mail this is in plan and we will be doing it. But
due to other priorities this was not taken in yet.


>
> On Tue, Jun 21, 2016 at 11:46 AM, Atin Mukherjee <amukh...@redhat.com>
> wrote:
>
>>
>>
>> On 06/21/2016 11:41 AM, Rajesh Joseph wrote:
>> > What kind of locking issues you see? If you can provide some more
>> > information I can be able to help you.
>>
>> That's related to stale lock issues on GlusterD which are there in 3.6.1
>> since the fixes landed in the branch post 3.6.1. I have already provided
>> the workaround/way to fix them [1]
>>
>> [1]
>> http://www.gluster.org/pipermail/gluster-users/2016-June/thread.html#26995
>>
>> ~Atin
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Pranith
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

On Mon, Jun 20, 2016 at 5:58 PM, B.K.Raghuram <bkr...@gmail.com> wrote:

> Thanks Rajesh,
>
> I was looking at 3.6 only to check on some locking issues that we were
> seeing.
>

What kind of locking issues you see? If you can provide some more
information I can be able to help you.


> However, we would like to see this in master. Please feel free to suggest
> modifications/modify the code as you see fit.
>

Sure, I will review the code and let you know what needs to be changed.


> Are there plans of having a more general way of integrating other
> underlying snapshotting mechanisms such as btrfs/lxd as well?
>

We do have this in our backlog, but due to manpower and other priorities it
was never picked up. Hope this get sorted in the
coming future and also it would be great to get contributions from other
community members in this area.

Best Regards,
Rajesh


>
> On Mon, Jun 20, 2016 at 3:16 PM, Rajesh Joseph <rjos...@redhat.com> wrote:
>
>>
>>
>> On Mon, Jun 20, 2016 at 12:33 PM, Kaushal M <kshlms...@gmail.com> wrote:
>>
>>> On Mon, Jun 20, 2016 at 11:38 AM, B.K.Raghuram <bkr...@gmail.com> wrote:
>>> > We had hosted some changes to an old version of glusterfs (3.6.1) in
>>> order
>>> > to incorporate ZFS snapshot support for gluster snapshot commands.
>>> These
>>> > have been done quite a while back and were not forward ported to newer
>>> > versions of glusterfs. I have a couple of questions on this :
>>> >
>>> > 1. If one needs to incorporate these changes in their current or
>>> modified
>>> > form into the glusterfs master, what is the procedure to do so?
>>> >
>>> > 2. Since the above process may take longer to roll in, we would like
>>> to get
>>> > the changes into at least the latest version of the 3.6 branch. In
>>> order to
>>> > do this, I tried the following and needed some help :
>>> >
>>> > I tried to apply the two ZFS relates commits
>>> > (https://github.com/fractalio/glusterfs/commits/release-3.6) to the
>>> latest
>>> > gluster code in the  guster-3.6 branch. I hit  one merge conflict per
>>> > commit, both in xlators/mgmt/glusterd/src/glusterd-snapshot.c. The
>>> attached
>>> > glusterd-snapshot.c_1 is the file with the merge conflicts after
>>> applying
>>> > the first commit and  glusterd-snapshot.c_2 is the one applying the
>>> second
>>> > commit. In order to process, I removed the HEAD changes in each of the
>>> merge
>>> > conflicts and proceeded just to see if anything else breaks but it went
>>> > through. glusterd-snapshot.c_1_corrected and
>>> glusterd-snapshot.c_2_corrected
>>> > and the corresponding files after removing the merge conflicts.
>>> >
>>> > The question I had is, are the changes that I made to correct the merge
>>> > conflicts safe? If not, could someone provide some suggestions on how
>>> to
>>> > correct the two conflicts?
>>> >
>>> > The file cmd_log contains the history of commands that I went through
>>> in the
>>> > process..
>>> >
>>>
>>> Thanks for sharing this Ram!
>>>
>>> Rajesh is the right person to answer your questions. As a GlusterD
>>> maintainer, I'll go through this and see if I can answer as well.
>>>
>>>
>> Overall the merge resolution seems fine, except few mistakes. e.g. in
>> glusterd-snapshot.c_2 you missed
>> to add "(unmount == _gf_true)" in the while loop in function
>> "glusterd_do_lvm_snapshot_remove".
>>
>> In function "glusterd_lvm_snapshot_remove" wrong chunk of code added. The
>> "if" condition should break here
>> instead of continuing from here.
>>
>> Also I think it would be better to rebase the change against master
>> instead of 3.6.
>>
>> Apart from this I am yet to review the complete change. I have taken an
>> initial look and seems like
>> we do need some amount of cleanup to the code before it can be taken in.
>> I also need to see how well it will
>> work the existing framework. I will go through it and provide a detailed
>> comments later.
>>
>> Thanks & Regards,
>> Rajesh
>>
>>
>>
>>> > Thanks,
>>> > -Ram
>>> >
>>> > ___
>>> > Gluster-devel mailing list
>>> > Gluster-devel@gluster.org
>>> > http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] snapshot tests cause coredump in dmeventd

[Rajesh Joseph]

On Tue, May 10, 2016 at 2:25 PM, Michael Scherer <msche...@redhat.com>
wrote:

> Le lundi 09 mai 2016 à 16:16 +0530, Rajesh Joseph a écrit :
> > On Wed, May 4, 2016 at 7:19 PM, Michael Scherer <msche...@redhat.com>
> wrote:
> >
> > > Le mercredi 04 mai 2016 à 10:42 +0530, Rajesh Joseph a écrit :
> > > > On Mon, May 2, 2016 at 3:04 AM, Niels de Vos <nde...@redhat.com>
> wrote:
> > > >
> > > > > It seems that a snaphot regression test managed to trigger a core
> dump
> > > > > in dmeventd. Because our regression tests check for cores, the
> test was
> > > > > marked as a failure (mostly a 'good' thing).
> > > > >
> > > > > I'd appreciate it if one of the developers of snapshot can have a
> look
> > > > > at the core from dmeventd and maybe check with the LVM developers
> if
> > > > > this is a known bug. The core can be downloaded from the bottom of
> this
> > > > > link:
> > > > >
> > > > >
> > >
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/20315/console
> > > > >
> > > > >
> > > > The slave machine is not accessible therefore could not download the
> core
> > > > file. Is it
> > > > possible to get access to this machine?
> > >
> > > I opened the firewall for that port, didn't see it was not before.
> > > So you can now download the tarball.
> > >
> > > I will add it to the automation to open the others too.
> > >
> >
> > Thanks Michael for looking into this. The machine is still not
> accessible.
>
> Seems something did clean the firewall (potentially a reboot).
>
> i was on PTO for the last days, so I didn't see it earlier.
>
> I opened it and will fix the root cause.
>
>
Thanks Michael. Its working now.

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] snapshot tests cause coredump in dmeventd

[Rajesh Joseph]

On Wed, May 4, 2016 at 7:19 PM, Michael Scherer <msche...@redhat.com> wrote:

> Le mercredi 04 mai 2016 à 10:42 +0530, Rajesh Joseph a écrit :
> > On Mon, May 2, 2016 at 3:04 AM, Niels de Vos <nde...@redhat.com> wrote:
> >
> > > It seems that a snaphot regression test managed to trigger a core dump
> > > in dmeventd. Because our regression tests check for cores, the test was
> > > marked as a failure (mostly a 'good' thing).
> > >
> > > I'd appreciate it if one of the developers of snapshot can have a look
> > > at the core from dmeventd and maybe check with the LVM developers if
> > > this is a known bug. The core can be downloaded from the bottom of this
> > > link:
> > >
> > >
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/20315/console
> > >
> > >
> > The slave machine is not accessible therefore could not download the core
> > file. Is it
> > possible to get access to this machine?
>
> I opened the firewall for that port, didn't see it was not before.
> So you can now download the tarball.
>
> I will add it to the automation to open the others too.
>

Thanks Michael for looking into this. The machine is still not accessible.

Thanks & Regards,
Rajesh


> --
> Michael Scherer
> Sysadmin, Community Infrastructure and Platform, OSAS
>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] snapshot tests cause coredump in dmeventd

[Rajesh Joseph]

On Mon, May 2, 2016 at 3:04 AM, Niels de Vos  wrote:

> It seems that a snaphot regression test managed to trigger a core dump
> in dmeventd. Because our regression tests check for cores, the test was
> marked as a failure (mostly a 'good' thing).
>
> I'd appreciate it if one of the developers of snapshot can have a look
> at the core from dmeventd and maybe check with the LVM developers if
> this is a known bug. The core can be downloaded from the bottom of this
> link:
>
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/20315/console
>
>
The slave machine is not accessible therefore could not download the core
file. Is it
possible to get access to this machine?

Thanks & Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Need help diagnosing regression-test crashes

[Rajesh Joseph]

On Sat, Apr 9, 2016 at 2:05 AM, Jeff Darcy  wrote:

> Upon further investigation, I've been able to determine that the problem
> lies in this line of our generic cleanup routine.
>
> type cleanup_lvm &>/dev/null && cleanup_lvm || true;
>
> This works great if snapshot.rc we're at the end of a test that included
> snapshot.rc (which defines cleanup_lvm), but we've generally been moving
> away from that in favor of calling it only at the beginning.  Thus, when
> we go from a snapshot test to a non-snapshot test, the cleanup at the
> beginning of the latter does *not* clean up any LVM stuff that's left
> over.  What might have been a simple and correctly attributed failure in
> the snapshot test can instead show up later.  In this case, the sequence
> of events is as follows:
>
>  1) bug-1322772 (snapshot) test starts glusterd
>
>  2) bug-1322772 exits while the new glusterd is still initializing
>
>  3) run-tests.sh looks for new core files and finds none
>
>  4) run-tests.sh starts bug-1002207 (stripe) test
>
>  5) glusterd from bug-1322772 dumps core
>
>  6) bug-1002207 test completes
>
>  7) run-tests.sh sees new core and misattributes it to bug-1002207
>
> The question is what to do about this.  Unconditionally calling
> lvm_cleanup from generic cleanup is simple, but might make regression
> tests noticeably slower.  Another possibility would be to change all
> snapshot tests to call cleanup (or at least cleanup_lvm) at the end, or
> use bash's "trap" mechanism to ensure the same.  I'm not wild about any
> of those, but lean toward the "trap" approach.  Anyone else have any
> opinions?
>

I think each snapshot test script should call cleanup_lvm and trap is a
great suggestion.

atinm: Can you please look into the crash in the following test case?
bugs/snapshot/bug-1322772-real-path-fix-for-snapshot.t



> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Handling locks in NSR

[Rajesh Joseph]

- Original Message -
> From: "Atin Mukherjee" 
> To: "Avra Sengupta" 
> Cc: "Gluster Devel" 
> Sent: Wednesday, March 2, 2016 4:03:11 PM
> Subject: Re: [Gluster-devel] Handling locks in NSR
> 
> 
> 
> 
> 
> -Atin
> Sent from one plus one
> On 02-Mar-2016 3:41 pm, "Avra Sengupta" < aseng...@redhat.com > wrote:
> > 
> > On 03/02/2016 02:55 PM, Venky Shankar wrote:
> >> 
> >> On Wed, Mar 02, 2016 at 02:29:26PM +0530, Avra Sengupta wrote:
> >>> 
> >>> On 03/02/2016 02:02 PM, Venky Shankar wrote:
>  
>  On Wed, Mar 02, 2016 at 01:40:08PM +0530, Avra Sengupta wrote:
> > 
> > Hi,
> > 
> > All fops in NSR, follow a specific workflow as described in this UML(
> > https://docs.google.com/presentation/d/1lxwox72n6ovfOwzmdlNCZBJ5vQcCaONvZva0aLWKUqk/edit?usp=sharing
> > ).
> > However all locking fops will follow a slightly different workflow as
> > described below. This is a first proposed draft for handling locks, and
> > we
> > would like to hear your concerns and queries regarding the same.
> > 
> > 1. On receiving the lock, the leader will Journal the lock himself, and
> > then
> > try to actually acquire the lock. At this point in time, if it fails to
> > acquire the lock, then it will invalidate the journal entry, and return
> > a
> > -ve ack back to the client. However, if it is successful in acquiring
> > the
> > lock, it will mark the journal entry as complete, and forward the fop
> > to the
> > followers.
>  
>  So, does a contending non-blocking lock operation check only on the
>  leader
>  since the followers might have not yet ack'd an earlier lock operation?
> >>> 
> >>> A non-blocking lock follows the same work flow, and thereby checks on the
> >>> leader first. In this case, it would be blocked on the leader, till the
> >>> leader releases the lock. Then it will follow the same workflow.
> >> 
> >> A non-blocking lock should ideally return EAGAIN if the region is already
> >> locked.
> >> Checking just on the leader (posix/locks on the leader server stack) and
> >> returning
> >> EAGAIN is kind of incomplete as the earlier lock request might not have
> >> been granted
> >> (due to failure on followers).
> >> 
> >> or does it even matter if we return EAGAIN during the transient state?
> >> 
> >> We could block the lock on the leader until an earlier lock request is
> >> satisfied
> >> (in which case return EAGAIN) or in case of failure try to satisfy the
> >> lock request.
> > 
> > That is what I said, it will be blocked on the leader till the leader
> > releases the already held lock.
> > 
> >> 
> > 2. The followers on receiving the fop, will journal it, and then try to
> > actually acquire the lock. If it fails to acquire the lock, then it
> > will
> > invalidate the journal entry, and return a -ve ack back to the leader.
> > If it
> > is successful in acquiring the lock, it will mark the journal entry as
> > complete,and send a +ve ack to the leader.
> > 
> > 3. The leader on receiving all acks, will perform a quorum check. If
> > quorum
> > meets, it will send a +ve ack to the client. If the quorum fails, it
> > will
> > send a rollback to the followers.
> > 
> > 4. The followers on receiving the rollback, will journal it first, and
> > then
> > release the acquired lock. It will update the rollback entry in the
> > journal
> > as complete and send an ack to the leader.
>  
>  What happens if the rollback fails for whatever reason?
> >>> 
> >>> The leader receives a -ve rollback ack, but there's little it can do
> >>> about
> >>> it. Depending on the failure, it will be resolved during reconciliation
> > 
> > 5. The leader on receiving the rollback acks, will journal it's own
> > rollback, and then release the acquired lock. It will update the
> > rollback
> > entry in the journal, and send a -ve ack to the client.
> > 
> > Few things to be noted in the above workflow are:
> > 1. It will be a synchronous operation, across the replica volume.
> > 
> > Atin, I am not sure how AFR handles it.
> If AFR/EC handle them asynchronously do you see any performance bottleneck
> with NSR for this case?

I assume when you say synchronous it means you will send lock request to
leader and when it succeed then only you will send to other followers. And
the request to followers will not be serialized.

AFAIK, AFR sends non-blocking lock request asynchronously and blocking locks
in a serialized way.

> > 
> > 2. Reconciliation will take care of nodes who have missed out the
> > locks.

Will the bricks be available before the reconciliation is complete?

> > 3. On a client disconnect, there will be a lock-timeout on whose
> > expiration
> > all locks held by that particular client will be 

Re: [Gluster-devel] Tips and Tricks for Gluster Developer

[Rajesh Joseph]

- Original Message -
> From: "Niels de Vos" <nde...@redhat.com>
> To: "Rajesh Joseph" <rjos...@redhat.com>
> Cc: "Richard Wareing" <rware...@fb.com>, "Gluster Devel" 
> <gluster-devel@gluster.org>
> Sent: Monday, January 25, 2016 6:30:53 PM
> Subject: Re: [Gluster-devel] Tips and Tricks for Gluster Developer
> 
> On Mon, Jan 25, 2016 at 06:41:50AM -0500, Rajesh Joseph wrote:
> > 
> > 
> > - Original Message -
> > > From: "Richard Wareing" <rware...@fb.com>
> > > To: "Raghavendra Talur" <rta...@redhat.com>
> > > Cc: "Gluster Devel" <gluster-devel@gluster.org>
> > > Sent: Monday, January 25, 2016 8:12:53 AM
> > > Subject: Re: [Gluster-devel] Tips and Tricks for Gluster Developer
> > > 
> > > Here's my tips:
> > > 
> > > 1. General C tricks
> > > - learn to use vim or emacs & read their manuals; customize to suite your
> > > style
> > > - use vim w/ pathogen plugins for auto formatting (don't use tabs!) &
> > > syntax
> > > - use ctags to jump around functions
> > > - Use ASAN & valgrind to check for memory leaks and heap corruption
> > > - learn to use "git bisect" to quickly find where regressions were
> > > introduced
> > > & revert them
> > > - Use a window manager like tmux or screen
> > > 
> > > 2. Gluster specific tricks
> > > - Alias "ggrep" to grep through all Gluster source files for some string
> > > and
> > > show you the line numbers
> > > - Alias "gvim" or "gemacs" to open any source file without full path, eg.
> > > "gvim afr.c"
> > > - GFS specific gdb macros to dump out pretty formatting of various
> > > structs
> > > (Jeff Darcy has some of these IIRC)
> > 
> > I also use few macros for printing dictionary and walking through the list
> > structures.
> > I think it would be good to collect these macros, scripts and tool in a
> > common place
> > so that people can use them. Can we include them in "extras/dev" directory
> > under Gluster source tree?
> 
> Yes, but please call it "extras/devel-tools" or something descriptive
> like that. "extras/dev" sounds like some device under /dev :)

Yes, sure :-)

> 
> Thanks,
> Niels
> 
> 
> > 
> > > - Write prove tests...for everything you write, and any bug you fix.
> > > Make
> > > them deterministic (timing/races shouldn't matter).
> > > - Bugs/races and/or crashes which are hard or impossible to repro often
> > > require the creation of a developer specific feature to simulate the
> > > failure
> > > and efficiently code/test a fix.  Example: "monkey-unlocking" in the lock
> > > revocation patch I just posted.
> > > - That edge case you are ignoring because you think it's
> > > impossible/unlikely?
> > > We will find/hit it in 48hrs at large scale (seriously we will)
> > > handle
> > > it correctly or at a minimum write a (kernel style) "OOPS" log type
> > > message.
> > > 
> > > That's all I have off the top of my head.  I'll give example aliases in
> > > another reply.
> > > 
> > > Richard
> > > 
> > > Sent from my iPhone
> > > 
> > > > On Jan 22, 2016, at 6:14 AM, Raghavendra Talur <rta...@redhat.com>
> > > > wrote:
> > > > 
> > > > HI All,
> > > > 
> > > > I am sure there are many tricks hidden under sleeves of many Gluster
> > > > developers.
> > > > I realized this when speaking to new developers. It would be good have
> > > > a
> > > > searchable thread of such tricks.
> > > > 
> > > > Just reply back on this thread with the tricks that you have and I
> > > > promise
> > > > I will collate them and add them to developer guide.
> > > > 
> > > > 
> > > > Looking forward to be amazed!
> > > > 
> > > > Thanks,
> > > > Raghavendra Talur
> > > > 
> > > > ___
> > > > Gluster-devel mailing list
> > > > Gluster-devel@gluster.org
> > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.gluster.org_mailman_listinfo_gluster-2Ddevel=CwICAg=5VD0RTtNlTh3ycd41b3MUw=qJ8Lp7ySfpQklq3QZr44Iw=wVrGhYdkvCanDEZF0xOyVbFg0am_GxaoXR26Cvp7H2U=JOrY0up51BoZOq2sKaNJQHPzqKiUS3Bwgn7fr5VPXjw=
> > > ___
> > > Gluster-devel mailing list
> > > Gluster-devel@gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-devel
> > > 
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Tips and Tricks for Gluster Developer

[Rajesh Joseph]

- Original Message -
> From: "Richard Wareing" 
> To: "Raghavendra Talur" 
> Cc: "Gluster Devel" 
> Sent: Monday, January 25, 2016 8:12:53 AM
> Subject: Re: [Gluster-devel] Tips and Tricks for Gluster Developer
> 
> Here's my tips:
> 
> 1. General C tricks
> - learn to use vim or emacs & read their manuals; customize to suite your
> style
> - use vim w/ pathogen plugins for auto formatting (don't use tabs!) & syntax
> - use ctags to jump around functions
> - Use ASAN & valgrind to check for memory leaks and heap corruption
> - learn to use "git bisect" to quickly find where regressions were introduced
> & revert them
> - Use a window manager like tmux or screen
> 
> 2. Gluster specific tricks
> - Alias "ggrep" to grep through all Gluster source files for some string and
> show you the line numbers
> - Alias "gvim" or "gemacs" to open any source file without full path, eg.
> "gvim afr.c"
> - GFS specific gdb macros to dump out pretty formatting of various structs
> (Jeff Darcy has some of these IIRC)

I also use few macros for printing dictionary and walking through the list 
structures.
I think it would be good to collect these macros, scripts and tool in a common 
place
so that people can use them. Can we include them in "extras/dev" directory
under Gluster source tree?

> - Write prove tests...for everything you write, and any bug you fix.  Make
> them deterministic (timing/races shouldn't matter).
> - Bugs/races and/or crashes which are hard or impossible to repro often
> require the creation of a developer specific feature to simulate the failure
> and efficiently code/test a fix.  Example: "monkey-unlocking" in the lock
> revocation patch I just posted.
> - That edge case you are ignoring because you think it's impossible/unlikely?
> We will find/hit it in 48hrs at large scale (seriously we will) handle
> it correctly or at a minimum write a (kernel style) "OOPS" log type message.
> 
> That's all I have off the top of my head.  I'll give example aliases in
> another reply.
> 
> Richard
> 
> Sent from my iPhone
> 
> > On Jan 22, 2016, at 6:14 AM, Raghavendra Talur  wrote:
> > 
> > HI All,
> > 
> > I am sure there are many tricks hidden under sleeves of many Gluster
> > developers.
> > I realized this when speaking to new developers. It would be good have a
> > searchable thread of such tricks.
> > 
> > Just reply back on this thread with the tricks that you have and I promise
> > I will collate them and add them to developer guide.
> > 
> > 
> > Looking forward to be amazed!
> > 
> > Thanks,
> > Raghavendra Talur
> > 
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.gluster.org_mailman_listinfo_gluster-2Ddevel=CwICAg=5VD0RTtNlTh3ycd41b3MUw=qJ8Lp7ySfpQklq3QZr44Iw=wVrGhYdkvCanDEZF0xOyVbFg0am_GxaoXR26Cvp7H2U=JOrY0up51BoZOq2sKaNJQHPzqKiUS3Bwgn7fr5VPXjw=
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] NetBSD regression fixes

[Rajesh Joseph]

- Original Message -
> From: "Emmanuel Dreyfus" 
> To: "Niels de Vos" 
> Cc: gluster-in...@gluster.org, gluster-devel@gluster.org
> Sent: Sunday, January 17, 2016 10:23:16 AM
> Subject: Re: [Gluster-devel] [Gluster-infra] NetBSD regression fixes
> 
> Niels de Vos  wrote:
> 
> > > 2) Spurious failures
> > > I added a retry-failed-test-once feature so that we get less regression
> > > failures because of spurious failures. It is not used right now because
> > > it does not play nicely with bad tests blacklist.
> > > 
> > > This will be fixed by that changes:
> > > http://review.gluster.org/13245
> > > http://review.gluster.org/13247
> > > 
> > > I have been looping failure-free regression for a while with that trick.
> > 
> > Nice, thanks for these improvements!
> 
> But I just realized the change is wrong, since running tests "new way"
> stops on first failed test. My change just retry the failed test and
> considers the regression run to be good on success, without running next
> tests.
> 
> I will post an update shortly.
> 

I think we should not take this approach. If the tests are not reliable then 
there
is no guarantee that it will pass in the next retry. In fact we should not rely 
on 
luck here. Lets not run those tests which are spurious in nature. Anyway we 
don't
consider the result of those tests. Therefore I think we should consider the 
patch 
sent by Talur (http://review.gluster.org/13173).

> > Could you send a pull request for the regression.sh script on
> > https://github.com/gluster/glusterfs-patch-acceptance-tests/ ? Or, if
> > you dont use GitHub, send the patch by email and we'll take care of
> > pushing it for you.
> 
> Sure, but let me settle on something that works first.
> 
> --
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] freebsd smoke failure

[Rajesh Joseph]

- Original Message -
> From: "Niels de Vos" 
> To: "Atin Mukherjee" 
> Cc: "Gluster Devel" , "Gluster Infra" 
> 
> Sent: Tuesday, January 12, 2016 1:37:50 PM
> Subject: Re: [Gluster-devel] [Gluster-infra] freebsd smoke failure
> 
> On Tue, Jan 12, 2016 at 11:19:30AM +0530, Atin Mukherjee wrote:
> > I've been observing freebsd smoke failure for all the patches for last
> > few days with the following error:
> > 
> > mkdir: /usr/local/lib/python2.7/site-packages/gluster: Permission denied
> > mkdir: /usr/local/lib/python2.7/site-packages/gluster: Permission denied
> > 
> > Can any one from infra team can help here?
> 
> This is not an infra issue, it is a build/installation issue in Gluster
> that the infra caught.
> 
> http://review.gluster.org/13208 should fix it, but I was at loss why
> only smoke tests got triggered and regression tests not.
> 

I am just wondering why are we seeing this smoke failure now and not on earlier 
patches?
Is it a regression? I believe smoke was passing sometime back.

Anyway, if this is the fix then can we fast track the patch merge? Thanks Niels 
for sending
the patch.

Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] freebsd smoke failure

[Rajesh Joseph]

- Original Message -
> From: "Niels de Vos" <nde...@redhat.com>
> To: "Rajesh Joseph" <rjos...@redhat.com>
> Cc: "Atin Mukherjee" <amukh...@redhat.com>, "Gluster Devel" 
> <gluster-devel@gluster.org>, "Gluster Infra"
> <gluster-in...@gluster.org>
> Sent: Tuesday, January 12, 2016 2:48:36 PM
> Subject: Re: [Gluster-devel] [Gluster-infra] freebsd smoke failure
> 
> On Tue, Jan 12, 2016 at 03:52:43AM -0500, Rajesh Joseph wrote:
> > 
> > 
> > - Original Message -
> > > From: "Niels de Vos" <nde...@redhat.com>
> > > To: "Atin Mukherjee" <amukh...@redhat.com>
> > > Cc: "Gluster Devel" <gluster-devel@gluster.org>, "Gluster Infra"
> > > <gluster-in...@gluster.org>
> > > Sent: Tuesday, January 12, 2016 1:37:50 PM
> > > Subject: Re: [Gluster-devel] [Gluster-infra] freebsd smoke failure
> > > 
> > > On Tue, Jan 12, 2016 at 11:19:30AM +0530, Atin Mukherjee wrote:
> > > > I've been observing freebsd smoke failure for all the patches for last
> > > > few days with the following error:
> > > > 
> > > > mkdir: /usr/local/lib/python2.7/site-packages/gluster: Permission
> > > > denied
> > > > mkdir: /usr/local/lib/python2.7/site-packages/gluster: Permission
> > > > denied
> > > > 
> > > > Can any one from infra team can help here?
> > > 
> > > This is not an infra issue, it is a build/installation issue in Gluster
> > > that the infra caught.
> > > 
> > > http://review.gluster.org/13208 should fix it, but I was at loss why
> > > only smoke tests got triggered and regression tests not.
> > > 
> > 
> > I am just wondering why are we seeing this smoke failure now and not on
> > earlier patches?
> > Is it a regression? I believe smoke was passing sometime back.
> 
> I think/suspect that the FreeBSD slaves are now managed by Salt and
> manual changes might have been undone. If people suggested (like now)
> that this was an issue on the slave, one of the admins might have opened
> up the permissions on the directories on request.
> 
> Things like this are really bugs in the build system though.
> 
> > Anyway, if this is the fix then can we fast track the patch merge?
> > Thanks Niels for sending the patch.
> 
> Well, the failing of the smoke test does not seem to add Verified=-1
> anymore (no idea why)? Whats the reason of the hurry?

I was in the impression that all smoke runs should pass. I used to check
result of smoke test before merging any patch. I guess I am wrong.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] snapshot/bug-1227646.t throws core [rev...@dev.gluster.org: Change in glusterfs[master]: hook-scripts: reconsile mount, fixing manual mount]

[Rajesh Joseph]

Yesterday, I looked into the core. From the core it looked like the rebalance 
process crashed
during cleanup_and_exit. The frame seems to be corrupted. Therefore it does not 
look like a snapshot
bug. The test case tests snapshot with tiered volume. If somebody from 
rebalance or tiering team can
also take a look then it would help.

Thanks & Regards,
Rajesh

- Original Message -
> From: "Vijay Bellur" <vbel...@redhat.com>
> To: "Michael Adam" <ob...@samba.org>, gluster-devel@gluster.org, "Rajesh 
> Joseph" <rjos...@redhat.com>
> Sent: Wednesday, January 6, 2016 6:23:10 AM
> Subject: Re: [Gluster-devel] snapshot/bug-1227646.t throws core 
> [rev...@dev.gluster.org: Change in glusterfs[master]:
> hook-scripts: reconsile mount, fixing manual mount]
> 
> Rajesh - can you please help with this?
> 
> Thanks,
> Vijay
> 
> On 01/04/2016 06:40 PM, Michael Adam wrote:
> > https://build.gluster.org/job/rackspace-regression-2GB-triggered/17291/console
> >
> > - Forwarded message from "Gluster Build System (Code Review)"
> > <rev...@dev.gluster.org> -
> >
> > Date: Mon, 4 Jan 2016 15:29:51 -0800
> > From: "Gluster Build System (Code Review)" <rev...@dev.gluster.org>
> > To: Michael Adam <ob...@samba.org>
> > Subject: Change in glusterfs[master]: hook-scripts: reconsile mount, fixing
> > manual mount
> > User-Agent: Gerrit/2.9.4
> >
> > Gluster Build System has posted comments on this change.
> >
> > Change subject: hook-scripts: reconsile mount, fixing manual mount
> > ..
> >
> >
> > Patch Set 1: Verified-1
> >
> > http://build.gluster.org/job/rackspace-regression-2GB-triggered/17291/consoleFull
> > : FAILED
> >
> >
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> >
> 
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] compound fop design first cut

[Rajesh Joseph]

- Original Message -
> From: "Ira Cooper" 
> To: "Jeff Darcy" , "Raghavendra Gowdappa" 
> , "Pranith Kumar Karampuri"
> 
> Cc: "Gluster Devel" 
> Sent: Wednesday, December 9, 2015 5:37:05 PM
> Subject: Re: [Gluster-devel] compound fop design first cut
> 
> Jeff Darcy  writes:
> 
> > However, I’d be even more comfortable with an even simpler approach that
> > avoids the need to solve what the database folks (who have dealt with
> > complex transactions for years) would tell us is a really hard problem.
> > Instead of designing for every case we can imagine, let’s design for the
> > cases that we know would be useful for improving performance.  Open plus
> > read/write plus close is an obvious one.  Raghavendra mentions
> > create+inodelk as well.  For each of those, we can easily define a
> > structure that contains the necessary fields, we don’t need a
> > client-side translator, and the server-side translator can take care of
> > “forwarding” results from one sub-operation to the next.  We could even
> > use GF_FOP_IPC to prototype this.  If we later find that the number of
> > “one-off” compound requests is growing too large, then at least we’ll
> > have some experience to guide our design of a more general alternative.
> > Right now, I think we’re trying to look further ahead than we can see
> > clearly.
> 
> Actually, I'm taking the design, I've seen another network protocol use,
> SMB2, and proposing it here, I'd be shocked if NFS doesn't behave in the
> same way.
> 
> Interestingly, all the cases, really deal with a single file, and a
> single lock, and a single...
> 
> There's a reason I talked about a single sentinel value, and not
> multiple ones.  Because I wanted to keep it simple.  Yes, the extensions
> you mention are obvious, but they lead to a giant mess, that we may not
> want initially.  (But that we CAN extend into if we want them.  I made
> the choice not to go there because honestly, I found the complexity too
> much for me.)
> 
> A simple "abort on failure" and let the higher levels clean it up is
> probably right for the type of compounding I propose.  It is what SMB2
> does.  So, if you get an error return value, cancel the rest of the
> request, and have it return ECOMPOUND as the errno.
> 
> Note: How you keep the list to be compounded doesn't matter much to me.
> the semantics matter, because those are what I can ask for later, and
> allow us to create ops the original desginers hadn't thought of, which
> is usually the hallmark of a good design.
> 
> I think you should look for a simple design you can "grow into" instead
> of creating one off ops, to satisfy a demand today.
> 

I agree with Ira here. This problem is already addressed by NFS and SMB.
So instead of reinventing the wheel lets pick the best bits from these
solutions and incorporate in Gluster.

From multi-protocol point of view we like to compound operations like
open + set_leaseID + lk and many more. With the current approach it would 
be really messy to have separate functions for each such combinations and a 
dedicated translator to handle them.

As others have mentioned I think it would be better to have a general
fop (fop_compound) which can handle compound fop. Each translator can
choose to implement it or not. Each translator can take a decision 
whether to compound more fops or de-compound them. e.g. currently
you can make the protocol server de-compound all the compound fops.

-Rajesh

> My thoughts,
> 
> -Ira
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] [Status]: RichACL support in GlusterFS

[Rajesh Joseph]

Hi all,

It's been long I sent any update on the RichACL work being done for GlusterFS. 
Following is the current state of the work being done.

Current State:
+ Initial patch uploaded at https://github.com/rajeshjoseph/glusterfs
+ Able to use setrichacl and getrichacl tool on gluster mount which set and 
retrieves RichACL for a file.
+ Currently using ext4 with RichACL support as backend.
+ Using the test cases provided by Andreas 
(https://github.com/andreas-gruenbacher/richacl) to validate the implementation

Some of the TODOs:
+ Provide support for chmod command
+ Move RichACL enforcement from Ext4 to Gluster.
+ Integrate this with NFS and SMB

Best Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Implementing Flat Hierarchy for trashed files

[Rajesh Joseph]

- Original Message -
 From: Prashanth Pai p...@redhat.com
 To: Anoop C S anoo...@redhat.com
 Cc: gluster-devel@gluster.org
 Sent: Tuesday, August 18, 2015 11:59:09 AM
 Subject: Re: [Gluster-devel] Implementing Flat Hierarchy for trashed files
 
 
 - Original Message -
  From: Anoop C S anoo...@redhat.com
  To: gluster-devel@gluster.org
  Sent: Monday, August 17, 2015 6:20:50 PM
  Subject: [Gluster-devel] Implementing Flat Hierarchy for trashed files
  
  Hi all,
  
  As we move forward, in order to fix the limitations with current trash
  translator we are planning to replace the existing criteria for trashed
  files inside trash directory with a general flat hierarchy as described
  in the following sections. Please have your thoughts on following
  design considerations.
  
  Current implementation
  ==
  * Trash translator resides on glusterfs server stack just above posix.
  * Trash directory (.trashcan) is created during volume start and is
visible under root of the volume.
  * Each trashed file is moved (renamed) to trash directory with an
appended time stamp in the file name.
 
 Do these files get moved during re-balance due to name change or do you
 choose file name according to the DHT regex magic to avoid that ?
 
  * Exact directory hierarchy (w.r.t the root of volume) is maintained
inside trash directory whenever a file is deleted/truncated from a
directory
  
  Outstanding issues
  ==
  * Since renaming occurs at the server side, client-side is unaware of
trash doing rename or create operations.
  * As a result files/directories may not be visible from mount point.
  * Files/Directories created from from trash translator will not have
gfid associated with it until lookup is performed.
  
  Proposed Flat hierarchy
  ===
  * Instead of creating the whole directory under trash, we will rename
the file and place it directly under trash directory (of course with
appended time stamp).
 
 The .trashcan directory might not scale with millions of such files placed
 under one directory. We had faced the same problem with gluster-swift
 project for object expiration feature and had decided to distribute our
 files across multiple directories in a deterministic way. And, personally,
 I'd prefer storing absolute timestamp, for example: as returned by `date
 +%s` command.
 
  * Directory hierarchy can be stored via either of the following two
approaches:
  (a) File name will contain the whole path with time stamp
  appended
 
 If this approach is taken, you might have trouble with choosing a magic
 letter representing slashes.
 
  (b) Store whole hierarchy as an xattr
  
  Other enhancements
  ==
  * Create the trash directory only
  when trash xlator is enabled.
 
 This is a needed enhancement. Upgrade to 3.7.* from older glusterfs versions
 caused undesired results in gluster-swift integration because .trashcan was
 visible by default on all glusterfs volumes.
 
  * Operations such as unlink, rename etc
  will be prevented on trash
directory only when trash xlator is
  enabled.
  * A new trash helper translator on client side(loaded only when
  trash
is enabled) to resolve split brain issues with truncation of
  files.
  * Restore files from trash with the help of an explicit setfattr
  call.
 
 You have to be very careful with races involved in re-creating the path when
 clients are accessing volume, also with over-writing if path exists.
 It's way easier (from implementer's perspective) if this is a manual process.
 
  


If the on-disk structure is changed how will upgrades are handled?


  Thanks  Regards,
  -Anoop C S
  -Jiffin Tony Thottan
  ___
  Gluster-devel mailing list
  Gluster-devel@gluster.org
  http://www.gluster.org/mailman/listinfo/gluster-devel
  
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-infra] NetBSD regressions not being triggered for patches

[Rajesh Joseph]

- Original Message -
 From: Kaushal M kshlms...@gmail.com
 To: Emmanuel Dreyfus m...@netbsd.org
 Cc: Gluster Devel gluster-devel@gluster.org, gluster-infra 
 gluster-in...@gluster.org
 Sent: Wednesday, 17 June, 2015 11:59:22 AM
 Subject: Re: [Gluster-devel] [Gluster-infra] NetBSD regressions not being 
 triggered for patches
 
 cloud.gluster.org is served by Rackspace Cloud DNS. AFAICT, there is
 no readily available option to do zone transfers from it. We might
 have to contact the Rackspace support to find out if they can do it as
 a special request.
 

If this is going to take time then I prefer not to block patches for NetBSD. We 
can address
any NetBSD regression caused by patches as a separate bug. Otherwise our 
regression queue will 
continue to grow.

 
 On Wed, Jun 17, 2015 at 11:50 AM, Emmanuel Dreyfus m...@netbsd.org wrote:
  Venky Shankar yknev.shan...@gmail.com wrote:
 
  If that's the case, then I'll vote for this even if it takes some time
  to get things in workable state.
 
  See my other mail about this: you enter a new slave VM in the DNS and it
  does not resolve, or somethimes you get 20s delays. I am convinced this
  is the reason why Jenkins bugs.
 
  --
  Emmanuel Dreyfus
  http://hcpnet.free.fr/pubz
  m...@netbsd.org
  ___
  Gluster-infra mailing list
  gluster-in...@gluster.org
  http://www.gluster.org/mailman/listinfo/gluster-infra
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] volume-snapshot.t spurious failure

[Rajesh Joseph]

- Original Message -
 From: Atin Mukherjee amukh...@redhat.com
 To: Avra Sengupta aseng...@redhat.com, Gluster Devel 
 gluster-devel@gluster.org
 Sent: Tuesday, June 16, 2015 5:38:36 PM
 Subject: [Gluster-devel] volume-snapshot.t spurious failure
 
 Avra,
 
 volume-snapshot.t failed in one of the regression run [1]. Could you
 take a look at it?

Thanks Atin for reporting it. I am looking into this.

 
 [1]
 http://build.gluster.org/job/rackspace-regression-2GB-triggered/10801/consoleFull
 --
 ~Atin
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] volume-snapshot.t spurious failure

[Rajesh Joseph]

- Original Message -
 From: Rajesh Joseph rjos...@redhat.com
 To: Atin Mukherjee amukh...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Wednesday, June 17, 2015 6:30:51 AM
 Subject: Re: [Gluster-devel] volume-snapshot.t spurious failure
 
 
 
 - Original Message -
  From: Atin Mukherjee amukh...@redhat.com
  To: Avra Sengupta aseng...@redhat.com, Gluster Devel
  gluster-devel@gluster.org
  Sent: Tuesday, June 16, 2015 5:38:36 PM
  Subject: [Gluster-devel] volume-snapshot.t spurious failure
  
  Avra,
  
  volume-snapshot.t failed in one of the regression run [1]. Could you
  take a look at it?
 
 Thanks Atin for reporting it. I am looking into this.
 

Avra, already root caused it. He will be sending a patch soon.

  
  [1]
  http://build.gluster.org/job/rackspace-regression-2GB-triggered/10801/consoleFull
  --
  ~Atin
  ___
  Gluster-devel mailing list
  Gluster-devel@gluster.org
  http://www.gluster.org/mailman/listinfo/gluster-devel
  
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Reduce regression runs wait time - New gerrit/review work flow

[Rajesh Joseph]

On Monday 15 June 2015 04:19 PM, Kaushal M wrote:

Hi all,

The recent rush of reviews being sent due to the release of 3.7 was a
cause of frustration for many of us because of the regression tests
(gerrit troubles themselves are another thing).

W.R.T regression 3 main sources of frustration were,
1. Spurious test failures
2. Long wait times
3. Regression slave troubles

We've already tackled the spurious failure issue and are quite stable
now. The trouble with the slave vms is related to the gerrit issues,
and is mainly due to the network issues we are having between the
data-centers hosting the slaves and gerrit/jenkins. People have been
looking into this, but we haven't had much success. This leaves the
issue of the long wait times.

The long wait times are because of the long queues of pending jobs,
some of which take days to get scheduled. Two things cause the long
queues,
1. Automatic regression job triggering for all submissions to gerrit
2. Long run time for regression (~2h)

The long queues coupled with the spurious failure and network
problems, meant that jobs would fail for no reason after a long wait,
and would have to be added to the back of the queue to be re-run. This
meant that developers would have to wait days for their changes to get
merged, and was one of the causes for the delay in the release of 3.7.

The solution reduce wait times for regression runs. To reduce wait
times we should,
1. Trigger runs only when required
2. Reduce regression run time.

Raghavendra Talur (rtalur/RaSTar) will soon send out a mail with his
findings on the regression run times, and we can continue discussion
on it on that thread.

Earlier, the regression runs used to be manually triggered by the
maintainers once they had determined that a change was ready for
submission. But as there were only two maintainers before (Vijay and
Avati) auto triggering was brought in to reduce their load. Auto
triggering worked fine when we had a lower volume of changes being
submitted to gerrit. But now, with the large volumes we see during the
release freeze dates, auto triggering just adds to problems.

I propose that we move back to the old model of starting regression
runs only once the maintainers are ready to merge. But instead of the
maintainers manually tiggering the runs, we could automate it.

We can model our new workflow on those of OpenStack[1] and
Wikimedia[2]. The existing Gerrit plugin for Jenkins doesn't provide
the features necessary to enable selective triggering based on Gerrit
flags. Both OpenStack and Wikimedia use a project gating tool called
Zuul[3], which provides a much better integration with Jenkins and
Gerrit and more features on top.

I propose the following work flow,

- Developer pushes change to Gerrit.
   - Zuul is notified by Gerrit of new change
- Zuul runs pre-review checks on Jenkins. This will be the current smoke tests.
   - Zuul reports back status of the checks to Gerrit.
 - If checks fail, developer will need to resend the change after
the required fixes. The process starts once more.
 - If the checks pass, the change is now ready for review
- The change is now reviewed by other developers and maintainers.
Non-maintainers will be able to give only a +1 review.
   - On a negative review, the developer will need to rework the change
and resend it. The process starts once more.
- The maintainer give a +2 review once he/she is satisfied. The
maintainers work is done here.
   - Zuul is notified of the +2 review
- Zuul runs the regression runs and reports back the status.
   - If the regression runs fail, the process starts over again.
   - If the runs pass, the change is ready for acceptance.
- Zuul will pick the change into the repository.
   - If the pick fails, Zuul will report back the failure, and the
process starts once again.

Following this flow should,
1. Reduce regression wait time
2. Improve change acceptance time
3. Reduce unnecessary  wastage of infra resources
4. Improve infra stability.

It also brings in drawbacks that we need to maintain one other piece
of infra (Zuul). This would be an additional maintenance overhead on
top of Gerrit, Jenkins and the current slaves. But I feel the
reduction in the upkeep efforts of the slaves would be enough to
offset this.

tl;dr
Current auto-triggering of regression runs is stupid and a waste of
time and resources. Bring in a project gating system, Zuul, which can
do a much more intelligent jobs triggering, and use it to
automatically trigger regression only for changes with Reviewed+2 and
automatically merge ones that pass.

What does the community think of this?


+1.



~kaushal

[1]: http://docs.openstack.org/infra/manual/developers.html#automated-testing
[2]: https://www.mediawiki.org/wiki/Continuous_integration/Workflow
[3]: http://docs.openstack.org/infra/zuul/
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression tests hanging after ./tests/basic/mgmt_v3-locks.t

[Rajesh Joseph]

On Monday 15 June 2015 05:21 PM, Kaushal M wrote:

The hang we observe is not something specific to Gluster. I've
observed this kind of hangs when a filesystem which is in use goes
offline.
For example I've accidently shutdown machines which were being used
for mounting nfs, which lead to the client systems hanging completely
and required a hard reboot.

If there are ways to avoid these kinds hangs when they eventually
occur, I'm all ears.


For these test cases can't we use the nfs soft mount option to prevent 
the hang?




On Mon, Jun 15, 2015 at 4:38 PM, Pranith Kumar Karampuri
pkara...@redhat.com wrote:

Emmanuel,
I am not sure of the feasibility but just wanted to ask you. Do you
think there is a possibility to error out operations on the mount when mount
crashes instead of hanging? That would prevent a lot of manual intervention
even in future.

Pranith.

On 06/15/2015 01:35 PM, Niels de Vos wrote:

Hi,

sometimes the NetBSD regression tests hang with messages like this:

  [12:29:07] ./tests/basic/mgmt_v3-locks.t
  ... ok79867 ms
  No volumes present
  mount_nfs: can't access /patchy: Permission denied
  mount_nfs: can't access /patchy: Permission denied
  mount_nfs: can't access /patchy: Permission denied

Most (if not all) of these hangs are caused by a crashing Gluster/NFS
process. Once the Gluster/NFS server is not reachable anymore,
unmounting fails.

The only way to recover is to reboot the VM and retrigger the test. For
rebooting, the http://build.gluster.org/job/reboot-vm job can be used,
and retriggering works by clicking the retrigger link in the left menu
once the test has been marked as failed/aborted.

When logging in on the NetBSD system that hangs, you can verify with
these steps:

1. check if there is a /glusterfsd.core file
2. run gdb on the core:

  # cd /build/install
  # gdb --core=/glusterfsd.core sbin/glusterfs
  ...
  Program terminated with signal SIGSEGV, Segmentation fault.
  #0  0xb9b94f0b in auth_cache_lookup (cache=0xb9aa2310, fh=0xb9044bf8,
  host_addr=0xb900e400 104.130.205.187, timestamp=0xbf7fd900,
  can_write=0xbf7fd8fc)
  at

/home/jenkins/root/workspace/rackspace-netbsd7-regression-triggered/xlators/nfs/server/src/auth-cache.c:164
  164 *can_write = lookup_res-item-opts-rw;

3. verify the lookup_res structure:

  (gdb) p *lookup_res
  $1 = {timestamp = 1434284981, item = 0xb901e3b0}
  (gdb) p *lookup_res-item
  $2 = {name = 0xff00 error: Cannot access memory at address
  0xff00, opts = 0x}


A fix for this has been sent, it is currently waiting for an update to
the prosed reference counting:

- http://review.gluster.org/11022
  core: add gf_ref_t for common refcounting structures
- http://review.gluster.org/11023
  nfs: refcount each auth_cache_entry and related data_t

Thanks,
Niels
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] NetBSD regression tests hanging after ./tests/basic/mgmt_v3-locks.t

[Rajesh Joseph]

On Monday 15 June 2015 06:34 PM, Emmanuel Dreyfus wrote:

On Mon, Jun 15, 2015 at 06:28:26PM +0530, Rajesh Joseph wrote:

For these test cases can't we use the nfs soft mount option to prevent the
hang?

soft mount will not be enough. I think you also need interruptible.


Correct me if I am wrong, but I think interruptible is good with hard 
mount. Which is good
in real deployment scenario. Since we are talking about test scripts, I 
thought soft mount

along with timeout period can be a good option to prevent hangs.


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Gluster Coreutils

[Rajesh Joseph]

On Monday 15 June 2015 03:14 AM, Joe Julian wrote:



On 06/14/2015 11:43 AM, Raghavendra Talur wrote:



On Sun, Jun 14, 2015 at 11:02 PM, chris holcombe 
chris.holco...@canonical.com mailto:chris.holco...@canonical.com 
wrote:


Welcome to the party Matthew!  Nice to see you're still keeping
an eye on on the list.  I'm excited to see this collaboration. 
This is going to turn out great :)


On 06/14/2015 01:58 AM, Matthew McKeen wrote:

Hey Craig and Chris:

I might be interested in collaborating on this as well.

Will be useful when I come back to FB in September.

Let me know where the public repository ends up being.

Thanks,
Matthew McKeen

P.S. Tell Richard I said hello

On Fri, Jun 12, 2015 at 11:29 AM chris holcombe
chris.holco...@canonical.com
mailto:chris.holco...@canonical.com
mailto:chris.holco...@canonical.com
mailto:chris.holco...@canonical.com wrote:

Yeah I have this repo but it's basically empty:
https://github.com/cholcombe973/GlusterUtils

On 06/12/2015 11:27 AM, Craig Cabrey wrote:
 Chris,

 That sounds good to me.

 I already have started on implementation, just to get
familiar
with the codebase and GFAPI.

 Is there a public repo that we can use for collaboration?

 Craig

 On Jun 12, 2015, at 10:46 AM, chris holcombe
chris.holco...@canonical.com
mailto:chris.holco...@canonical.com
mailto:chris.holco...@canonical.com
mailto:chris.holco...@canonical.com wrote:

 Craig,

 I was actually planning on building the same tool
set.  I would
like to work with you also on this if that's ok.

 -Chris

 On 06/12/2015 10:43 AM, Jeff Darcy wrote:
 Hi everyone,

 This summer I am an intern at Facebook working on
the Gluster
team. Part of
 my project for the summer includes developing a set of
coreutils that
 utilizes the Gluster C API natively.

 This project is similar in nature to the NFS
coreutils that
some of you may
 have heard about from the other Facebook engineers
at the
Gluster summit
 recently. I just wanted to reach out to the Gluster
community
to gather
 ideas, potential features, feedback, and direction.

 The initial set of utilities that I am developing
includes
the following:

 * cat
 * mkdir
 * put (read from stdin and write to a file)
 * mv
 * ls
 * rm
 * tail

 Again, any feedback will be welcome.
 Hi, Craig, and welcome to the project.  :)

 There seems to be some overlap with a proposal
Ragahavendra
Talur sent out
 a couple of days ago.



https://urldefense.proofpoint.com/v1/url?u=https://docs.google.com/document/d/1yuRLRbdccx_0V0UDAxqWbz4g983q5inuINHgM1YO040/edit?usp%3Dsharingk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0Am=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0As=8d55bb5770b8ed1d683a6908a05af32b79289735c537c660252fcaa7c690e162

 This seems like an excellent opportunity to
collaborate. Ideally, I think
 it would be useful to have both an FTP-client-like
shell and
a set of
 standalone one shot commands, based on as much
common code
as possible.

 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
mailto:Gluster-devel@gluster.org
mailto:Gluster-devel@gluster.org
mailto:Gluster-devel@gluster.org


https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-develk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0Am=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0As=28546cdc6fdf6f75f4cfa4b8260abc595eee96601a5f849ebb230ddbd1faf8b3

___
Gluster-devel mailing list
Gluster-devel@gluster.org mailto:Gluster-devel@gluster.org
mailto:Gluster-devel@gluster.org
mailto:Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel 

Re: [Gluster-devel] RichACL support in Gluster

[Rajesh Joseph]

- Original Message -
 From: Andreas Grünbacher andreas.gruenbac...@gmail.com
 To: Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Friday, May 22, 2015 2:27:20 PM
 Subject: Re: [Gluster-devel] RichACL support in Gluster
 
 Hi Rajesh,
 
 2015-05-22 5:34 GMT+02:00 Rajesh Joseph rjos...@redhat.com:
  I have created a feature page for RichACL:
  http://www.gluster.org/community/documentation/index.php/Features/RichACL
 
 looks like a good start, thanks. Meanwhile, Jeremy Allison has written
 an initial
 Samba vfs module that should allow to get us started there.

Oh that's great progress. I am following him on Samba Technical. I will keep
you updated on my progress as well.

 
  Please see my Gluster Summit slides for more details:
  http://www.slideshare.net/RajeshJoseph6/richacl-glusterfs
 
 Just a minor nit for page 16: the slide says Richacl supports EVERYONE@
 instead
 of OTHER class. The classes are actually owner, group, and other as in
 POSIX.1.
 EVERYONE@ acl entries apply to all three classes.

Thanks for the correction.

 
 Thanks,
 Andreas
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] RichACL support in Gluster

[Rajesh Joseph]

Hi all,

I am working towards providing RichACL support in Gluster. This is a 
crucial step towards multi-protocol support. Currently Andreas Gruenbacher 
is the driving force behind RichACL efforts in ext4 file-system and other
file-systems. See his earlier mail for more details:
http://www.gluster.org/pipermail/gluster-devel/2015-February/044008.html

Niels created a Fedora repo for the RichACL supported kernel, tools and 
the richacl library. 
https://copr.fedoraproject.org/coprs/devos/richacl/

My plan is to provide a new RichACL translator in gluster which will
store, retrieve and enforce RichACL irrespective of whether the lower
file-system supports RichACL or not. I will make use of librichacl
developed by Andreas for managing and enforcing ACLs. One important
point to note is that RichACL and POSIX ACL won't work together. The 
file-system can either support RichACL or POSIX ACL. Richacls share some 
design elements with POSIX ACLs, but they go beyond POSIX ACLs in 
several ways. Converting from POSIX ACLs to richacls is relatively easy, 
but converting back from richacls to POSIX ACLs is not possible without 
losing information.

I have created a feature page for RichACL:
http://www.gluster.org/community/documentation/index.php/Features/RichACL

Please see my Gluster Summit slides for more details:
http://www.slideshare.net/RajeshJoseph6/richacl-glusterfs

Please let me know your comments or suggestions.

Thanks  Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] metdata-volume for gluster

[Rajesh Joseph]

- Original Message -
 From: Niels de Vos nde...@redhat.com
 To: Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Wednesday, April 8, 2015 12:40:14 AM
 Subject: Re: [Gluster-devel] metdata-volume for gluster
 
 On Tue, Apr 07, 2015 at 06:26:23AM -0400, Rajesh Joseph wrote:
  Hi all,
  
  In gluster 3.7 multiple features (Snapshot scheduler, NFS Ganesha,
  Geo-rep) are planning to use additional volume to store metadata
  related to these features. This volume needs to be manually created
  and explicitly managed by an admin.
  
  I think creating and managing these many metadata volume would be an
  overhead for an admin. Instead of that I am proposing to have a single
  unified metata-volume which can be used by all these features.
 
 That sounds like a great idea!
 
  For simplicity and easier management we are proposing to have a
  pre-defined volume name.
 
 Who is the we you speak about? It would be nice to know who gave their
 opinions before you wrote this email. (I assume they would not respond
 to this email because they have agreed already?)

We means Snapshot team and RHSC team, which needs this for better 
synchronization.
Apart from this we had discussion with Geo-rep team and NFS team.

 
  If needed this name can be configured using a global gluster option.
 
 I would say that configuring is not needed, and surely not advised. A
 name that is easy to recognise would be good, like prepending with a _.

Personally I also prefer to keep it fixed and not configurable. Which will
reduce lot of communication and synchronization issues. 

As you suggested we can have the volume name start with _. Vijay also
had a similar suggestion. 

 
  Please let me know if you have any suggestions or comments.
 
 What would be the (default) name of this volume that you are thinking
 about?

We have not finalized the name yet, may be something like _gf_meta_volume_.

 
  Thanks  Regards,
  Rajesh
 
 Thanks for sharing,
 Niels
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] metdata-volume for gluster

[Rajesh Joseph]

- Original Message -
 From: Jeff Darcy jda...@redhat.com
 To: Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Wednesday, April 8, 2015 1:53:38 AM
 Subject: Re: [Gluster-devel] metdata-volume for gluster
 
  In gluster 3.7 multiple features (Snapshot scheduler, NFS Ganesha, Geo-rep)
  are planning to use
  additional volume to store metadata related to these features. This volume
  needs to be manually
  created and explicitly managed by an admin.
  
  I think creating and managing these many metadata volume would be an
  overhead
  for an admin. Instead
  of that I am proposing to have a single unified metata-volume which can be
  used by all these features.
  
  For simplicity and easier management we are proposing to have a pre-defined
  volume name.
  If needed this name can be configured using a global gluster option.
  
  Please let me know if you have any suggestions or comments.
 
 Do these metadata volumes already exist, or are they being added to designs
 as we speak?  There seem to be a lot of unanswered questions that suggest
 the latter.  For example...

This is being added as we speak.

 
 * What replication level(s) do we need?  What performance translators
   should be left out to ensure consistency?

Ideally here we need greater number of replication but since we only support
x3 replication we would be recommending x3 replication.

It is still under design therefore we have not finalized on the performance
translators part. But I think we can leave out of most of the performance
translators for this.

 
 * How much storage will we need?  How will it be provisioned and tracked?

As I said earlier in the current release it would be done manually by system
administrators. And therefore managed by them. In the subsequent release we
can work towards a management framework.

 
 * What nodes would this volume be hosted on?  Does the user have to
   (or get to) decide, or do we decide automatically?  What happens as
   the cluster grows or shrinks?
 
 * How are the necessary daemons managed?  From glusterd?  What if we
   want glusterd itself to use this facility?
 
 * Will there be an API, so the implementation can be changed to be
   compatible with similar facilities already scoped out for 4.0?
 
 I like the idea of this being shared infrastructure.  It would also be
 nice if it can be done with a minimum of administrative overhead.  To
 do that, though, I think we need a more detailed exploration of the
 problem(s) we're trying to solve and of the possible solutions.

I agree that we should do this with minimum administrative overhead, but
it really require more detailed investigation and a thorough design. we have
started exploring in that direction, but we have no workable solution as
of now.

My current proposition is to just reduce the number of manual metadata-volume
required to run gluster.

Thanks  Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] metdata-volume for gluster

[Rajesh Joseph]

- Original Message -
 From: Raghavendra Talur raghavendra.ta...@gmail.com
 To: Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Wednesday, April 8, 2015 12:16:54 AM
 Subject: Re: [Gluster-devel] metdata-volume for gluster
 
 On Tue, Apr 7, 2015 at 3:56 PM, Rajesh Joseph rjos...@redhat.com wrote:
 
  Hi all,
 
  In gluster 3.7 multiple features (Snapshot scheduler, NFS Ganesha,
  Geo-rep) are planning to use
  additional volume to store metadata related to these features. This volume
  needs to be manually
  created and explicitly managed by an admin.
 
  I think creating and managing these many metadata volume would be an
  overhead for an admin. Instead
  of that I am proposing to have a single unified metata-volume which can be
  used by all these features.
 
  For simplicity and easier management we are proposing to have a
  pre-defined volume name.
  If needed this name can be configured using a global gluster option.
 
  Please let me know if you have any suggestions or comments.
 
 
  Thanks  Regards,
  Rajesh
 
  ___
  Gluster-devel mailing list
  Gluster-devel@gluster.org
  http://www.gluster.org/mailman/listinfo/gluster-devel
 
 
 
 +1 for
 a. automatic creation of metadata volume over manual

Right now it would be manual because automatic means we not only
need to create the volume but manage them. Which would need an
entire management frame. I think the next phase should be to 
achieve this.

 b. configuration through gluster cli
 

What do you mean by configuration?

 few suggestions
 a. disable access through any mechanisms other than fuse/gfapi.(Samba and
 NFS should not export.)

Right now there is no restriction on export. I think it might be good to disable
NFS and Samba access.

 b. Restrict access to peer nodes.

I think we should consider this when we have the meta-volume management 
implementation.

 
 Question:
 What would be the replica count of the said volume and would that be ok for
 every use case
 mentioned above?

Since we support only 3-way replication we are recommending the same.

 
 Thanks,
 Raghavendra Talur
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Feature Freeze for 3.7

[Rajesh Joseph]

- Original Message -
 From: Vijay Bellur vbel...@redhat.com
 To: Kiran Patil ki...@fractalio.com, Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Friday, February 27, 2015 5:48:39 PM
 Subject: Re: [Gluster-devel] Feature Freeze for 3.7
 
 On 02/27/2015 05:05 PM, Kiran Patil wrote:
  Namaste Vijay,
 
  There is no mention of including ZFS snapshot feature support in
  Glusterfs v3.7.
 
  Any updates on it ?
 
 
 Adding Rajesh who might have an update on this.

Prakash is working on it. He will be able to provide the current update.

-Rajesh

 
 -Vijay
 
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Help needed with Coverity - How to remove tainted_data_argument?

[Rajesh Joseph]

- Original Message -
 From: Niels de Vos nde...@redhat.com
 To: Atin Mukherjee amukh...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Wednesday, December 17, 2014 2:21:50 PM
 Subject: Re: [Gluster-devel] Help needed with Coverity - How to remove 
 tainted_data_argument?
 
 On Wed, Dec 17, 2014 at 01:54:09PM +0530, Atin Mukherjee wrote:
  
  
  On 12/17/2014 01:01 PM, Lalatendu Mohanty wrote:
   On 12/17/2014 12:56 PM, Krishnan Parthasarathi wrote:
   I was looking into a Coverity issue (CID 1228603) in GlusterFS.
   I sent a patch[1] before I fully understood why this was an issue.
   After searching around in the internet for explanations, I identified
   that
   the core issue was that a character buffer, storing parts of a file
   (external I/O),
   was marked tainted. This taint spread wherever the buffer was used.
   This seems
   acceptable in the context of static analysis. How do we indicate to
   Coverity that
   the 'taint' would cause no harm as speculated?
  
   [1] - Coverity fix attempt: http://review.gluster.org/#/c/9286/
   [2] - CID 1228603:  Use of untrusted scalar value  (TAINTED_SCALAR):
  glusterd-utils.c: 2131 in glusterd_readin_file()
  
   thanks,
   kp
   ___
   Gluster-devel mailing list
   Gluster-devel@gluster.org
   http://supercolony.gluster.org/mailman/listinfo/gluster-devel
   KP,
   
   We can mark the CID in Coverity scan website that it is not an issue
   (i.e. as designed) and it would stop reporting it as a bug.
  Question is whether coverity will stop reporting on such occurrences in
  other places in future, my guess is no. Idea is to make coverity
  understand that this pattern should not be reported further.
 
 This pattern can be dangerous. I think we need to review all occurences
 and mark each occurence as 'intentional' or 'not a bug' if the usage is
 safe. The unsafe occurences would receive a patch.

+1. We should analyse each occurrence, there could be a potential unsafe usage
as well.

 
 Niels
 
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://supercolony.gluster.org/mailman/listinfo/gluster-devel
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] USS test cases failure with core

[Rajesh Joseph]

- Original Message -
 From: Atin Mukherjee amukh...@redhat.com
 To: Rajesh Joseph rjos...@redhat.com, Avra Sengupta 
 aseng...@redhat.com
 Cc: a-t...@redhat.com, gluster-devel@gluster.org
 Sent: Friday, November 21, 2014 2:07:28 PM
 Subject: USS test cases failure with core
 
 Rajesh/Avra,
 
 For one of the regression run [1] few uss testcases failed with core file.
 Can you please take a look?
 
 [1]
 http://build.gluster.org/job/rackspace-regression-2GB-triggered/2821/consoleFull
 

Thanks Atin for letting us know. We will look into this and revert back.

Rajesh 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] XML support for snapshot command

[Rajesh Joseph]

Hi all,

Following patch provides --xml support for all the snapshot command. I need
some reviewers to take a look at the patch and provide +1.
This patch is pending for quite long time.

http://review.gluster.org/#/c/7663

Thanks  Regards,
Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Build up of xfsmount dirs... ;)

[Rajesh Joseph]

Thanks Justin for letting me know. Its clearly a bug in the implementation. I 
will send a patch to fix this. Meanwhile if it is causing any test failures or 
other issues then you can have a temporary fix to delete them in the script.

Thanks  Regards,
Rajesh

- Original Message -
 From: Justin Clift jus...@gluster.org
 To: Rajesh Joseph rjos...@redhat.com
 Cc: Gluster Devel gluster-devel@gluster.org
 Sent: Friday, June 20, 2014 7:07:06 PM
 Subject: Build up of xfsmount dirs... ;)
 
 Hi Rajesh,
 
 Looking at the regression testing boxes this morning, they all have
 several thousand /tmp/xfsmount* dirs on them.
 
 Seems like they're coming from this:
 
   f1705e2d (Rajesh Joseph 2014-06-05 10:00:33 +0530 3988) char
   template [] = /tmp/xfsmountXX;
 
 (it's the only mention in the source for /tmp/xfsmount...)
 
 Could it be the case that the directory gets made, but never
 gets cleaned up?  If so, is there a feasible way to clean it
 up after use?
 
 We *can* just change the regression test scripting to nuke
 /tmp/xfsmount* after each run.  But wondering it's something
 Gluster should take care of itself. :)
 
 Regards and best wishes,
 
 Justin Clift
 
 --
 GlusterFS - http://www.gluster.org
 
 An open source, distributed file system scaling to several
 petabytes, and handling thousands of clients.
 
 My personal twitter: twitter.com/realjustinclift
 
 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] autodelete in snapshots

[Rajesh Joseph]

- Original Message -
From: M S Vishwanath Bhat msvb...@gmail.com
To: Vijay Bellur vbel...@redhat.com
Cc: Seema Naik sen...@redhat.com, Gluster Devel 
gluster-devel@gluster.org
Sent: Tuesday, June 3, 2014 1:02:08 AM
Subject: Re: [Gluster-devel] autodelete in snapshots




On 2 June 2014 20:22, Vijay Bellur  vbel...@redhat.com  wrote: 



On 04/23/2014 05:50 AM, Vijay Bellur wrote: 


On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote: 


On 04/16/2014 11:39 AM, Avra Sengupta wrote: 


The whole purpose of introducing the soft-limit is, that at any point 
of time the number of 
snaps should not exceed the hard limit. If we trigger auto-delete on 
hitting hard-limit, then 
the purpose itself is lost, because at that point we would be taking a 
snap, making the limit 
hard-limit + 1, and then triggering auto-delete, which violates the 
sanctity of the hard-limit. 
Also what happens when we are at hard-limit + 1, and another snap is 
issued, while auto-delete 
is yet to process the first delete. At that point we end up at 
hard-limit + 1. Also what happens 
if for a particular snap the auto-delete fails. 

We should see the hard-limit, as something set by the admin keeping in 
mind the resource consumption 
and at no-point should we cross this limit, come what may. If we hit 
this limit, the create command 
should fail asking the user to delete snaps using the snapshot 
delete command. 

The two options Raghavendra mentioned are applicable for the 
soft-limit only, in which cases on 
hitting the soft-limit 

1. Trigger auto-delete 

or 

2. Log a warning-message, for the user saying the number of snaps is 
exceeding the snap-limit and 
display the number of available snaps 

Now which of these should happen also depends on the user, because the 
auto-delete option 
is configurable. 

So if the auto-delete option is set as true, auto-delete should be 
triggered and the above message 
should also be logged. 

But if the option is set as false, only the message should be logged. 

This is the behaviour as designed. Adding Rahul, and Seema in the 
mail, to reflect upon the 
behaviour as well. 

Regards, 
Avra 

This sounds correct. However we need to make sure that the usage or 
documentation around this should be good enough , so that users 
understand the each of the limits correctly. 


It might be better to avoid the usage of the term soft-limit. 
soft-limit as used in quota and other places generally has an alerting 
connotation. Something like auto-deletion-limit might be better. 


I still see references to soft-limit and auto deletion seems to get triggered 
upon reaching soft-limit. 

Why is the ability to auto delete not configurable? It does seem pretty nasty 
to go about deleting snapshots without obtaining explicit consent from the 
user. 

I agree with Vijay here. It's not good to delete a snap (even though it is 
oldest) without the explicit consent from user. 

FYI It took me more than 2 weeks to figure out that my snaps were getting 
autodeleted after reaching soft-limit. For all I know I had not done anything 
and my snap restore were failing. 

I propose to remove the terms soft and hard limit. I believe there should 
be a limit (just limit) after which all snapshot creates should fail with 
proper error messages. And there can be a water-mark after which user should 
get warning messages. So below is my proposal. 

auto-delete + snap-limit: If the snap-limit is set to n , next snap create 
(n+1th) will succeed only if if auto-delete is set to on/true/1 and oldest snap 
will get deleted automatically. If autodelete is set to off/false/0 , (n+1)th 
snap create will fail with proper error message from gluster CLI command. But 
again by default autodelete should be off. 

snap-water-mark : This should come in picture only if autodelete is turned off. 
It should not have any meaning if auto-delete is turned ON. Basically it's 
usage is to give the user warning that limit almost being reached and it is 
time for admin to decide which snaps should be deleted (or which should be 
kept) 

*my two cents* 

-MS 


The reason for having a hard-limit is to stop snapshot creation once we reached 
this limit. This helps to have a control over the resource consumption. 
Therefore if we only have this limit (as snap-limit) then there is no question 
of auto-delete. Auto-delete can only be triggered once the count crosses the 
limit. Therefore we introduced the concept of soft-limit and a hard-limit. As 
the name suggests once the hard-limit is reached no more snaps will be created.

So the idea is to keep the number of snapshots always less than the hard-limit. 
To do so we introduced the concept of soft-limit, wherein we allow snapshots 
even when this limit is crossed and once the snapshot is taken we delete the 
oldest snap. If you consider this definition then the name soft-limit and 
hard-limit looks ok to me.

In phase II we are planning to have auto-delete feature configurable with 
different 

Re: [Gluster-devel] Fwd: Snapshot CLI question

[Rajesh Joseph]

The --xml option is in plan as a good to have. I am trying to
get this done by code freeze time. 

Best Regards,
Rajesh 

- Original Message -
From: James purplei...@gmail.com
To: gluster-devel gluster-devel@gluster.org
Sent: Tuesday, April 29, 2014 11:45:06 AM
Subject: Re: [Gluster-devel] Fwd: Snapshot CLI question

On Mon, Apr 28, 2014 at 11:39 PM, Paul Cuzner pcuz...@redhat.com wrote:
 Without --xml we're limiting the automation potential and resorting to
 'screen scraping' - so with that said, is --xml in plan, or do you need an
 RFE?


+1 for having --xml. Without it, it's quite hard (or insane) to do
automation for Puppet-Gluster. I imagine Paul might want this for
gluster deploy too.

Cheers,
James
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel