Re: [Gluster-devel] inode lru limit

[Raghavendra Bhat]

On Tuesday 03 June 2014 07:00 AM, Raghavendra Gowdappa wrote:


- Original Message -

From: "Raghavendra Bhat" 
To: gluster-devel@gluster.org
Cc: "Anand Avati" 
Sent: Monday, June 2, 2014 6:41:30 PM
Subject: Re: [Gluster-devel] inode lru limit

On Monday 02 June 2014 11:06 AM, Raghavendra G wrote:





On Fri, May 30, 2014 at 2:24 PM, Raghavendra Bhat < rab...@redhat.com >
wrote:



Hi,

Currently the lru-limit of the inode table in brick processes is 16384. There
is a option to configure it to some other value. The protocol/server uses
inode_lru_limit variable present in its private structure while creating the
inode table (whose default value is 16384). When the option is reconfigured
via volume set option the protocol/server's inode_lru_limit variable present
in its private structure is changed. But the actual size of the inode table
still remains same as old one. Only when the brick is restarted the newly
set value comes into picture. Is it ok? Should we change the inode table's
lru_limit variable also as part of reconfigure? If so, then probably we
might have to remove the extra inodes present in the lru list by calling
inode_table_prune.

Yes, I think we should change the inode table's lru limit too and call
inode_table_prune. From what I know, I don't think this change would cause
any problems.


But as of now the inode table is bound to bound_xl which is associated with
the client_t object for the client being connected. As part of fops we can
get the bound_xl (thus the inode table) from the rpc request
(req->trans->xl_private). But in reconfigure we get just the xlator pointer
of protocol/server and dict containing new options.

So what I am planning is this. If the xprt_list (transport list corresponding
to the clients mounted) is empty, then just set the private structure's
variable for lru limit (which will be used to create the inode table when a
client mounts). If xprt_list of protocol/server's private structure is not
empty, then get one of the transports from that list and get the client_t
object corresponding to the transport, from which bould_xl is obtained (all
the client_t objects share the same inode table) . Then from bound_xl
pointer to inode table is got and its variable for lru limit is also set to
the value specified via cli and inode_table_prune is called to purge the
extra inodes.

In the above proposal if there are no active clients, lru limit of itable is 
not reconfigured. Here are two options to improve correctness of your proposal.
If there are no active clients, then there will not be any itable. 
itable will be created when 1st client connects to the brick. And while 
creating the itable we use the inode_lru_limit variable present in 
protocol/server's private structure and inode table that is created also 
saves the same value.

1. On a successful handshake, you check whether the lru_limit of itable is 
equal to configured value. If not equal, set it to the configured value and 
prune the itable. The cost is that you check inode table's lru limit on every 
client connection.
On successful handshake, for the 1st client inode table will be created 
with lru_limit value saved in protocol/server's private. For further 
handshakes since inode table is already there, new inode tables will not 
be created. So instead of waiting for a new handshake to happen to set 
the lru_limit and purge the inode table, I think its better to do it as 
part of reconfigure itself.


2. Traverse through the list of all xlators (since there is no easy way of 
finding potential candidates for bound_xl other than peaking into options 
specific to authentication) and if there is an itable associated with that 
xlator, set its lru limit and prune it. The cost here is traversing the list of 
xlators. However, our xlator list in brick process is relatively small, this 
shouldn't have too much performance impact.

Comments are welcome.


Regards,
Raghavendra Bhat

Does it sound OK?

Regards,
Raghavendra Bhat

Regards,
Raghavendra Bhat






Please provide feedback


Regards,
Raghavendra Bhat
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel



--
Raghavendra G


___
Gluster-devel mailing list Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] autodelete in snapshots

[Rahul Hinduja]

+rajesh

- Original Message -
> From: "M S Vishwanath Bhat" 
> To: "Vijay Bellur" 
> Cc: "Lalatendu Mohanty" , "Avra Sengupta" 
> , "Raghavendra Bhat"
> , "Gluster Devel" , "Rahul 
> Hinduja" , "Seema
> Naik" 
> Sent: Tuesday, June 3, 2014 1:02:08 AM
> Subject: Re: [Gluster-devel] autodelete in snapshots
> 
> On 2 June 2014 20:22, Vijay Bellur  wrote:
> 
> > On 04/23/2014 05:50 AM, Vijay Bellur wrote:
> >
> >> On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:
> >>
> >>> On 04/16/2014 11:39 AM, Avra Sengupta wrote:
> >>>
>  The whole purpose of introducing the soft-limit is, that at any point
>  of time the number of
>  snaps should not exceed the hard limit. If we trigger auto-delete on
>  hitting hard-limit, then
>  the purpose itself is lost, because at that point we would be taking a
>  snap, making the limit
>  hard-limit + 1, and then triggering auto-delete, which violates the
>  sanctity of the hard-limit.
>  Also what happens when we are at hard-limit + 1, and another snap is
>  issued, while auto-delete
>  is yet to process the first delete. At that point we end up at
>  hard-limit + 1. Also what happens
>  if for a particular snap the auto-delete fails.
> 
>  We should see the hard-limit, as something set by the admin keeping in
>  mind the resource consumption
>  and at no-point should we cross this limit, come what may. If we hit
>  this limit, the create command
>  should fail asking the user to delete snaps using the "snapshot
>  delete" command.
> 
>  The two options Raghavendra mentioned are applicable for the
>  soft-limit only, in which cases on
>  hitting the soft-limit
> 
>  1. Trigger auto-delete
> 
>  or
> 
>  2. Log a warning-message, for the user saying the number of snaps is
>  exceeding the snap-limit and
>  display the number of available snaps
> 
>  Now which of these should happen also depends on the user, because the
>  auto-delete option
>  is configurable.
> 
>  So if the auto-delete option is set as true, auto-delete should be
>  triggered and the above message
>  should also be logged.
> 
>  But if the option is set as false, only the message should be logged.
> 
>  This is the behaviour as designed. Adding Rahul, and Seema in the
>  mail, to reflect upon the
>  behaviour as well.
> 
>  Regards,
>  Avra
> 
> >>>
> >>> This sounds correct. However we need to make sure that the usage or
> >>> documentation around this should be good enough , so that users
> >>> understand the each of the limits correctly.
> >>>
> >>>
> >> It might be better to avoid the usage of the term "soft-limit".
> >> soft-limit as used in quota and other places generally has an alerting
> >> connotation. Something like "auto-deletion-limit" might be better.
> >>
> >>
> > I still see references to "soft-limit" and auto deletion seems to get
> > triggered upon reaching soft-limit.
> >
> > Why is the ability to auto delete not configurable? It does seem pretty
> > nasty to go about deleting snapshots without obtaining explicit consent
> > from the user.
> >
> 
> I agree with Vijay here. It's not good to delete a snap (even though it is
> oldest) without the explicit consent from user.
> 
> FYI It took me more than 2 weeks to figure out that my snaps were getting
> autodeleted after reaching "soft-limit". For all I know I had not done
> anything and my snap restore were failing.
> 
> I propose to remove the terms "soft" and "hard" limit. I believe there
> should be a limit (just "limit") after which all snapshot creates should
> fail with proper error messages. And there can be a water-mark after which
> user should get warning messages. So below is my proposal.
> 
> *auto-delete + snap-limit:  *If the snap-limit is set to *n*, next snap
> create (n+1th) will succeed *only if* *if auto-delete is set to on/true/1*
> and oldest snap will get deleted automatically. If autodelete is set to
> off/false/0 , (n+1)th snap create will fail with proper error message from
> gluster CLI command.  But again by default autodelete should be off.
> 
> *snap-water-mark*: This should come in picture only if autodelete is turned
> off. It should not have any meaning if auto-delete is turned ON. Basically
> it's usage is to give the user warning that limit almost being reached and
> it is time for admin to decide which snaps should be deleted (or which
> should be kept)
> 
> *my two cents*
> 
> -MS
> 
> 
> >
> > Cheers,
> >
> > Vijay
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> >
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] op-version issues in the 3.5.1 beta release

[Kaushal M]

Niels,
This approach will work for well when the cluster is uniform, ie. of
the same (major) version. This could lead to problems in mixed
clusters when using volume set. Volume set compares the op-versions of
the options being set and will reject the set operation when the
op-versions are different. So, if a user were to run a mixed cluster
of gluster-3.5.1 and gluster-3.6.0, he wouldn't be able to set
server.manage-gids as its op-versions would be different.

But I don't expect anyone to be running such a cluster always. It
would mostly be during upgrades, during which users shouldn't be doing
volume operations.

~kaushal

On Mon, Jun 2, 2014 at 8:28 PM, Niels de Vos  wrote:
> Hi,
>
> today on IRC we has a discussion about the op-version for the current
> 3.5.1 beta. This beta includes a backport that introduces a new volume
> option (server.manage-gids) and needed to increase the op-version to
> prevent issues with systems that do not know about this new option.
>
> Currently, the op-version in 3.5.1 is (seems to be) hard-coded to '3':
>
>   libglusterfs/src/globals.h:#define GD_OP_VERSION_MAX  3
>
> Now, the new option required a glusterd with op-version=4. This worked
> fine when setting the option, and glusterd.info got updated too.
> Unfortunately, a restart of glusterd fails, because the op-version from
> the configuration is greater than the GD_OP_VERSION_MAX.
>
> Increasing GD_OP_VERSION_MAX is not really suitable, because
> op-version=4 would make other systems assume that the 3.5.1 release has
> all the op-version=4 features (incorrect, because the upcoming 3.6 has
> op-version=4).
>
> I see one option to fix this issue, that allows stable branches to
> include backports of volume options and similar, without conflicting
> with the development branch or newer versions:
>
> 1. define an op-version as multi-digit value, with gaps for stable
>releases
> 2. stable releases may only include backports of volume options that are
>in the development branch and newer versions
> 3. stable maintainers should pay extra care when new volume options are
>being backported
>
> The idea is the following:
>
> - update the hard-coded op-version in libglusterfs/src/globals.h in the
>   master branch to 360 (based on the 3.6 release for easier matching)
> - update any options that have op-version >= 4 to 360 (master branch)
> - update the op-version in libglusterfs/src/globals.h in the release-3.5
>   branch to 351
> - update the op-version of server.manage-gids option in 3.5.1 to 351
>
>
> The only issue would be that current 3.6 packages in testing have
> a lower op-version than the new 3.5.1 packages. I hope it is not
> a common practise to have systems installed with packages from the
> master-branch in the same environment as 3.5.1 servers.
>
> Any ideas, suggestions or thoughts?
>
> If this can not be solved in a similar easy way, I will be forced to
> revert the 3.5.1 server.manage-gids option. Users were expecting this to
> be present so that deployments with many (ca. 93+) secondary groups have
> permissions working as expected.
>
> Thanks,
> Niels
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] inode lru limit

[Raghavendra Gowdappa]

- Original Message -
> From: "Raghavendra Bhat" 
> To: gluster-devel@gluster.org
> Cc: "Anand Avati" 
> Sent: Monday, June 2, 2014 6:41:30 PM
> Subject: Re: [Gluster-devel] inode lru limit
> 
> On Monday 02 June 2014 11:06 AM, Raghavendra G wrote:
> 
> 
> 
> 
> 
> On Fri, May 30, 2014 at 2:24 PM, Raghavendra Bhat < rab...@redhat.com >
> wrote:
> 
> 
> 
> Hi,
> 
> Currently the lru-limit of the inode table in brick processes is 16384. There
> is a option to configure it to some other value. The protocol/server uses
> inode_lru_limit variable present in its private structure while creating the
> inode table (whose default value is 16384). When the option is reconfigured
> via volume set option the protocol/server's inode_lru_limit variable present
> in its private structure is changed. But the actual size of the inode table
> still remains same as old one. Only when the brick is restarted the newly
> set value comes into picture. Is it ok? Should we change the inode table's
> lru_limit variable also as part of reconfigure? If so, then probably we
> might have to remove the extra inodes present in the lru list by calling
> inode_table_prune.
> 
> Yes, I think we should change the inode table's lru limit too and call
> inode_table_prune. From what I know, I don't think this change would cause
> any problems.
> 
> 
> But as of now the inode table is bound to bound_xl which is associated with
> the client_t object for the client being connected. As part of fops we can
> get the bound_xl (thus the inode table) from the rpc request
> (req->trans->xl_private). But in reconfigure we get just the xlator pointer
> of protocol/server and dict containing new options.
> 
> So what I am planning is this. If the xprt_list (transport list corresponding
> to the clients mounted) is empty, then just set the private structure's
> variable for lru limit (which will be used to create the inode table when a
> client mounts). If xprt_list of protocol/server's private structure is not
> empty, then get one of the transports from that list and get the client_t
> object corresponding to the transport, from which bould_xl is obtained (all
> the client_t objects share the same inode table) . Then from bound_xl
> pointer to inode table is got and its variable for lru limit is also set to
> the value specified via cli and inode_table_prune is called to purge the
> extra inodes.

In the above proposal if there are no active clients, lru limit of itable is 
not reconfigured. Here are two options to improve correctness of your proposal.

1. On a successful handshake, you check whether the lru_limit of itable is 
equal to configured value. If not equal, set it to the configured value and 
prune the itable. The cost is that you check inode table's lru limit on every 
client connection.

2. Traverse through the list of all xlators (since there is no easy way of 
finding potential candidates for bound_xl other than peaking into options 
specific to authentication) and if there is an itable associated with that 
xlator, set its lru limit and prune it. The cost here is traversing the list of 
xlators. However, our xlator list in brick process is relatively small, this 
shouldn't have too much performance impact.

Comments are welcome.

> 
> Does it sound OK?
> 
> Regards,
> Raghavendra Bhat
> 
> Regards,
> Raghavendra Bhat
> 
> 
> 
> 
> 
> 
> Please provide feedback
> 
> 
> Regards,
> Raghavendra Bhat
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> 
> 
> 
> --
> Raghavendra G
> 
> 
> ___
> Gluster-devel mailing list Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> 
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] autodelete in snapshots

[M S Vishwanath Bhat]

On 3 June 2014 01:02, M S Vishwanath Bhat  wrote:

>
>
>
> On 2 June 2014 20:22, Vijay Bellur  wrote:
>
>> On 04/23/2014 05:50 AM, Vijay Bellur wrote:
>>
>>> On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:
>>>
 On 04/16/2014 11:39 AM, Avra Sengupta wrote:

> The whole purpose of introducing the soft-limit is, that at any point
> of time the number of
> snaps should not exceed the hard limit. If we trigger auto-delete on
> hitting hard-limit, then
> the purpose itself is lost, because at that point we would be taking a
> snap, making the limit
> hard-limit + 1, and then triggering auto-delete, which violates the
> sanctity of the hard-limit.
> Also what happens when we are at hard-limit + 1, and another snap is
> issued, while auto-delete
> is yet to process the first delete. At that point we end up at
> hard-limit + 1. Also what happens
> if for a particular snap the auto-delete fails.
>
> We should see the hard-limit, as something set by the admin keeping in
> mind the resource consumption
> and at no-point should we cross this limit, come what may. If we hit
> this limit, the create command
> should fail asking the user to delete snaps using the "snapshot
> delete" command.
>
> The two options Raghavendra mentioned are applicable for the
> soft-limit only, in which cases on
> hitting the soft-limit
>
> 1. Trigger auto-delete
>
> or
>
> 2. Log a warning-message, for the user saying the number of snaps is
> exceeding the snap-limit and
> display the number of available snaps
>
> Now which of these should happen also depends on the user, because the
> auto-delete option
> is configurable.
>
> So if the auto-delete option is set as true, auto-delete should be
> triggered and the above message
> should also be logged.
>
> But if the option is set as false, only the message should be logged.
>
> This is the behaviour as designed. Adding Rahul, and Seema in the
> mail, to reflect upon the
> behaviour as well.
>
> Regards,
> Avra
>

 This sounds correct. However we need to make sure that the usage or
 documentation around this should be good enough , so that users
 understand the each of the limits correctly.


>>> It might be better to avoid the usage of the term "soft-limit".
>>> soft-limit as used in quota and other places generally has an alerting
>>> connotation. Something like "auto-deletion-limit" might be better.
>>>
>>>
>> I still see references to "soft-limit" and auto deletion seems to get
>> triggered upon reaching soft-limit.
>>
>> Why is the ability to auto delete not configurable? It does seem pretty
>> nasty to go about deleting snapshots without obtaining explicit consent
>> from the user.
>>
>
> I agree with Vijay here. It's not good to delete a snap (even though it is
> oldest) without the explicit consent from user.
>
> FYI It took me more than 2 weeks to figure out that my snaps were getting
> autodeleted after reaching "soft-limit". For all I know I had not done
> anything and my snap restore were failing.
>
> I propose to remove the terms "soft" and "hard" limit. I believe there
> should be a limit (just "limit") after which all snapshot creates should
> fail with proper error messages. And there can be a water-mark after which
> user should get warning messages. So below is my proposal.
>
> *auto-delete + snap-limit:  *If the snap-limit is set to *n*, next snap
> create (n+1th) will succeed *only if* *if auto-delete is set to on/true/1*
> and oldest snap will get deleted automatically. If autodelete is set to
> off/false/0 , (n+1)th snap create will fail with proper error message from
> gluster CLI command.  But again by default autodelete should be off.
>
> *snap-water-mark*: This should come in picture only if autodelete is
> turned off. It should not have any meaning if auto-delete is turned ON.
> Basically it's usage is to give the user warning that limit almost being
> reached and it is time for admin to decide which snaps should be deleted
> (or which should be kept)
>
> *my two cents*
>
Adding gluster-users as well.

-MS

>
> -MS
>
>
>>
>> Cheers,
>>
>> Vijay
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] autodelete in snapshots

[M S Vishwanath Bhat]

On 2 June 2014 20:22, Vijay Bellur  wrote:

> On 04/23/2014 05:50 AM, Vijay Bellur wrote:
>
>> On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:
>>
>>> On 04/16/2014 11:39 AM, Avra Sengupta wrote:
>>>
 The whole purpose of introducing the soft-limit is, that at any point
 of time the number of
 snaps should not exceed the hard limit. If we trigger auto-delete on
 hitting hard-limit, then
 the purpose itself is lost, because at that point we would be taking a
 snap, making the limit
 hard-limit + 1, and then triggering auto-delete, which violates the
 sanctity of the hard-limit.
 Also what happens when we are at hard-limit + 1, and another snap is
 issued, while auto-delete
 is yet to process the first delete. At that point we end up at
 hard-limit + 1. Also what happens
 if for a particular snap the auto-delete fails.

 We should see the hard-limit, as something set by the admin keeping in
 mind the resource consumption
 and at no-point should we cross this limit, come what may. If we hit
 this limit, the create command
 should fail asking the user to delete snaps using the "snapshot
 delete" command.

 The two options Raghavendra mentioned are applicable for the
 soft-limit only, in which cases on
 hitting the soft-limit

 1. Trigger auto-delete

 or

 2. Log a warning-message, for the user saying the number of snaps is
 exceeding the snap-limit and
 display the number of available snaps

 Now which of these should happen also depends on the user, because the
 auto-delete option
 is configurable.

 So if the auto-delete option is set as true, auto-delete should be
 triggered and the above message
 should also be logged.

 But if the option is set as false, only the message should be logged.

 This is the behaviour as designed. Adding Rahul, and Seema in the
 mail, to reflect upon the
 behaviour as well.

 Regards,
 Avra

>>>
>>> This sounds correct. However we need to make sure that the usage or
>>> documentation around this should be good enough , so that users
>>> understand the each of the limits correctly.
>>>
>>>
>> It might be better to avoid the usage of the term "soft-limit".
>> soft-limit as used in quota and other places generally has an alerting
>> connotation. Something like "auto-deletion-limit" might be better.
>>
>>
> I still see references to "soft-limit" and auto deletion seems to get
> triggered upon reaching soft-limit.
>
> Why is the ability to auto delete not configurable? It does seem pretty
> nasty to go about deleting snapshots without obtaining explicit consent
> from the user.
>

I agree with Vijay here. It's not good to delete a snap (even though it is
oldest) without the explicit consent from user.

FYI It took me more than 2 weeks to figure out that my snaps were getting
autodeleted after reaching "soft-limit". For all I know I had not done
anything and my snap restore were failing.

I propose to remove the terms "soft" and "hard" limit. I believe there
should be a limit (just "limit") after which all snapshot creates should
fail with proper error messages. And there can be a water-mark after which
user should get warning messages. So below is my proposal.

*auto-delete + snap-limit:  *If the snap-limit is set to *n*, next snap
create (n+1th) will succeed *only if* *if auto-delete is set to on/true/1*
and oldest snap will get deleted automatically. If autodelete is set to
off/false/0 , (n+1)th snap create will fail with proper error message from
gluster CLI command.  But again by default autodelete should be off.

*snap-water-mark*: This should come in picture only if autodelete is turned
off. It should not have any meaning if auto-delete is turned ON. Basically
it's usage is to give the user warning that limit almost being reached and
it is time for admin to decide which snaps should be deleted (or which
should be kept)

*my two cents*

-MS


>
> Cheers,
>
> Vijay
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] All builds are failing with BUILD ERROR

[Justin Clift]

On 02/06/2014, at 7:04 AM, Kaleb KEITHLEY wrote:

> someone cleaned the loopback devices. I deleted 500 unix domain sockets in 
> /d/install/var/run and requeued the regressions.

Interesting.  The extra sockets problem is what prompted me
to rewrite the cleanup function.  The sockets are being
created by glusterd during each test startup, but aren't
removed by the existing cleanup function.  (so, substantial
build up over time)


> I'm not sure which of those two things was the solution.


_Probably_ the loopback device thing.  The extra sockets seem
to be "messy" but (so far) I haven't seen them break anything.

+ Justin

--
Open Source and Standards @ Red Hat

twitter.com/realjustinclift

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] op-version issues in the 3.5.1 beta release

[Niels de Vos]

Hi,

today on IRC we has a discussion about the op-version for the current 
3.5.1 beta. This beta includes a backport that introduces a new volume 
option (server.manage-gids) and needed to increase the op-version to
prevent issues with systems that do not know about this new option.

Currently, the op-version in 3.5.1 is (seems to be) hard-coded to '3':

  libglusterfs/src/globals.h:#define GD_OP_VERSION_MAX  3

Now, the new option required a glusterd with op-version=4. This worked 
fine when setting the option, and glusterd.info got updated too.  
Unfortunately, a restart of glusterd fails, because the op-version from 
the configuration is greater than the GD_OP_VERSION_MAX.

Increasing GD_OP_VERSION_MAX is not really suitable, because 
op-version=4 would make other systems assume that the 3.5.1 release has 
all the op-version=4 features (incorrect, because the upcoming 3.6 has 
op-version=4).

I see one option to fix this issue, that allows stable branches to 
include backports of volume options and similar, without conflicting 
with the development branch or newer versions:

1. define an op-version as multi-digit value, with gaps for stable 
   releases
2. stable releases may only include backports of volume options that are 
   in the development branch and newer versions
3. stable maintainers should pay extra care when new volume options are 
   being backported

The idea is the following:

- update the hard-coded op-version in libglusterfs/src/globals.h in the 
  master branch to 360 (based on the 3.6 release for easier matching)
- update any options that have op-version >= 4 to 360 (master branch)
- update the op-version in libglusterfs/src/globals.h in the release-3.5 
  branch to 351
- update the op-version of server.manage-gids option in 3.5.1 to 351


The only issue would be that current 3.6 packages in testing have 
a lower op-version than the new 3.5.1 packages. I hope it is not 
a common practise to have systems installed with packages from the 
master-branch in the same environment as 3.5.1 servers.

Any ideas, suggestions or thoughts?

If this can not be solved in a similar easy way, I will be forced to 
revert the 3.5.1 server.manage-gids option. Users were expecting this to 
be present so that deployments with many (ca. 93+) secondary groups have 
permissions working as expected.

Thanks,
Niels
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] autodelete in snapshots

[Vijay Bellur]

On 04/23/2014 05:50 AM, Vijay Bellur wrote:

On 04/20/2014 11:42 PM, Lalatendu Mohanty wrote:

On 04/16/2014 11:39 AM, Avra Sengupta wrote:

The whole purpose of introducing the soft-limit is, that at any point
of time the number of
snaps should not exceed the hard limit. If we trigger auto-delete on
hitting hard-limit, then
the purpose itself is lost, because at that point we would be taking a
snap, making the limit
hard-limit + 1, and then triggering auto-delete, which violates the
sanctity of the hard-limit.
Also what happens when we are at hard-limit + 1, and another snap is
issued, while auto-delete
is yet to process the first delete. At that point we end up at
hard-limit + 1. Also what happens
if for a particular snap the auto-delete fails.

We should see the hard-limit, as something set by the admin keeping in
mind the resource consumption
and at no-point should we cross this limit, come what may. If we hit
this limit, the create command
should fail asking the user to delete snaps using the "snapshot
delete" command.

The two options Raghavendra mentioned are applicable for the
soft-limit only, in which cases on
hitting the soft-limit

1. Trigger auto-delete

or

2. Log a warning-message, for the user saying the number of snaps is
exceeding the snap-limit and
display the number of available snaps

Now which of these should happen also depends on the user, because the
auto-delete option
is configurable.

So if the auto-delete option is set as true, auto-delete should be
triggered and the above message
should also be logged.

But if the option is set as false, only the message should be logged.

This is the behaviour as designed. Adding Rahul, and Seema in the
mail, to reflect upon the
behaviour as well.

Regards,
Avra


This sounds correct. However we need to make sure that the usage or
documentation around this should be good enough , so that users
understand the each of the limits correctly.



It might be better to avoid the usage of the term "soft-limit".
soft-limit as used in quota and other places generally has an alerting
connotation. Something like "auto-deletion-limit" might be better.



I still see references to "soft-limit" and auto deletion seems to get 
triggered upon reaching soft-limit.


Why is the ability to auto delete not configurable? It does seem pretty 
nasty to go about deleting snapshots without obtaining explicit consent 
from the user.


Cheers,
Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] inode lru limit

[Raghavendra Bhat]

On Monday 02 June 2014 11:06 AM, Raghavendra G wrote:



On Fri, May 30, 2014 at 2:24 PM, Raghavendra Bhat > wrote:



Hi,

Currently the lru-limit of the inode table in brick processes is
16384. There is a option to configure it to some other value. The
protocol/server uses inode_lru_limit variable present in its
private structure while creating the inode table (whose default
value is 16384). When the option is reconfigured via volume set
option the protocol/server's inode_lru_limit variable present in
its private structure is changed. But the actual size of the inode
table still remains same as old one. Only when the brick is
restarted the newly set value comes into picture. Is it ok? Should
we change the inode table's lru_limit variable also as part of
reconfigure? If so, then probably we might have to remove the
extra inodes present in the lru list by calling inode_table_prune.


Yes, I think we should change the inode table's lru limit too and call 
inode_table_prune. From what I know, I don't think this change would 
cause any problems.




But as of now the inode table is bound to bound_xl which is associated 
with the client_t object for the client being connected. As part of fops 
we can get the bound_xl  (thus the inode table) from the rpc request 
(req->trans->xl_private). But in reconfigure we get just the xlator 
pointer of protocol/server and dict containing new options.


So what I am planning is this. If the xprt_list (transport list 
corresponding to the clients mounted) is empty, then just set the 
private structure's variable for lru limit (which will be used to create 
the inode table when a client mounts). If xprt_list of protocol/server's 
private structure is not empty, then get one of the transports from that 
list and get the client_t object corresponding to the transport, from 
which bould_xl is obtained (all the client_t objects share the same 
inode table) . Then from bound_xl pointer to inode table is got and its 
variable for lru limit is also set to the value specified via cli and 
inode_table_prune is called to purge the extra inodes.


Does it sound OK?

Regards,
Raghavendra Bhat

Regards,
Raghavendra Bhat



Please provide feedback


Regards,
Raghavendra Bhat
___
Gluster-devel mailing list
Gluster-devel@gluster.org 
http://supercolony.gluster.org/mailman/listinfo/gluster-devel




--
Raghavendra G


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] [RFC] GlusterFS Operations Guide

[Vijay Bellur]

On 05/30/2014 04:50 PM, Lalatendu Mohanty wrote:


I think it is time to create  an operations/ops guide for GlusterFS.
Operating guide should address issues which administrators face while
running/maintaining GlusterFS storage nodes. Openstack project has an
operating guide [2] which try to address similar issues and it is pretty
cool.

IMO these are the typical/example questions which operating guide should
try to address.

  * /Maintenance, Failures, and Debugging/

  * What are steps for planned maintenance for GlusterFS node?

  * Steps for replacing a failed node?

  * Steps to decommission a brick?

  * /Logging and Monitoring/

  * Where are the log files?

  * How to find-out if self-heal is working properly?

  * Which log files to monitor for detecting failures?


Operating guide needs good amount of work, hence we all need to come
together for this. You can contribute for this by either of the following

  *   Let know others about the questions you want to get answered in
the operating guide. ( I have set-up a etherpad for this [1])
  * Answer the questions/issues raised by others.


Comments, suggestions?
Should this be part of gluster code base i.e. /doc or somewhere else?



+1 to this idea.

We can possibly add this as part of the gluster-docs project in forge. 
We can even create a project for this on gerrit for better review of 
incoming content and mirror it on forge.


-Vijay

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] struct dirent in snapview-server.c

[Emmanuel Dreyfus]

Raghavendra Bhat  wrote:

> I have raised a bug for it 
> (https://bugzilla.redhat.com/show_bug.cgi?id=1103591) and have sent a
> patch to handle the issue (http://review.gluster.org/#/c/7946/). Please
> let me know if this handles the issue.

It does. I reviewed your change +1 and abandoned mine.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] struct dirent in snapview-server.c

[Raghavendra Bhat]

On Monday 02 June 2014 06:47 AM, Harshavardhana wrote:

It is always possible to translate structures, the question is whether
it is useful of not. d_off is the offset of this struct dirent within
the buffer for the whole directory returned by getdents(2) system call.

Since we glusterfs does not use getdents(2) but upper level
opendir(3)/readdir(3), which use getdents(2) themselves, it never has
the whole buffer, and therefore I am not sure it can make any use of
d_off.

Understood that makes sense.


Hi Emmanuel,

I have raised a bug for it 
(https://bugzilla.redhat.com/show_bug.cgi?id=1103591) and have sent a 
patch to handle the issue (http://review.gluster.org/#/c/7946/). Please 
let me know if this handles the issue.


Regards,
Raghavendra Bhat
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel