Re: [Gluster-users] An other Gluster 3.8 Long-Term-Maintenance update with the 3.8.8 release

2017-01-17 Thread Lindsay Mathieson 
On 16 January 2017 at 03:35, Niels de Vos  wrote:
> The Gluster team has been busy over the end-of-year holidays and this
> latest update to the 3.8 [9]Long-Term-Maintenance release intends to fix
> quite a number of bugs. Packages have been built for [10]many different
> distributions and are available from the download server. The
> [11]release-notes for 3.8.8 have been included below for the ease of
> reference. All users on the 3.8 version are recommended to update to
> this current release.

I've been running this for a few days now, no issues. Did via a
rolling upgrade from 3.8.7 to 3.8.8

Thanks Devs,


-- 
Lindsay
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] rebalance and volume commit hash

2017-01-17 Thread Piotr Misiak 

17 sty 2017 17:10 Jeff Darcy  napisał(a):
>
> > Do you think that is wise to run rebalance process manually on every 
> > brick with the actual commit hash value? 
> > 
> > I didn't do anything with bricks after previous rebalance run and I have 
> > cluster.weighted-rebalance=off. 
> > 
> > My problem is that I have a very big directory structure (millions of 
> > directories and files) and I haven't ever completed rebalance process 
> > once, because it will take I guess weeks or months. I'd like to speed it 
> > up a bit by not generating new commit hash for volume during new 
> > rebalance run. Then directories rebalanced in the previous run will be 
> > untouched during the new run. Is it possible? 
>
> I'm pretty sure that trying to rebalance on each brick separately will 
> not work correctly.  Rebalancing smaller pieces of the directory 
> hierarchy separately, by issuing the appropriate setxattr calls on them 
> instead of using the CLI, *might* work.  Either way, I think the DHT 
> experts could provide a better answer. 
>

Is it possible to start rebalancing from a particular subdirecory? Do you know 
how? It would be very useful for me.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] volume start fails

2017-01-17 Thread Atin Mukherjee 
On Tue, Jan 17, 2017 at 9:10 PM, Paul Bickerstaff [DATACOM] <
pau...@datacom.co.nz> wrote:

> I have tried to add a few additional volumes but consistently get an error
> on executing
>
>
>
> gluster volume start 
>
>
>
> The error message is:
>
>
>
> volume start: : failed: Commit failed on localhost. Please
> check log file for details.
>
>
>
> What log file is being referred to here? I can find no errors anywhere
> except a repeat of this line in cmd_history.log.
>

You should be able to see glusterd log file in /var/log/glusterfs  which
needs to be referred at.


>
> This has worked for me in the past. I have a couple of dozen active
> gluster volumes.
>
>
>
> (These new volumes were created successfully and can be listed but show as
> not started.)
>
>
>
> Any pointers on to how to resolve this would be much appreciated.
>
>
>
> Thanks
>
> Paul
>
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>



-- 

~ Atin (atinm)
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] volume start fails

2017-01-17 Thread Paul Bickerstaff [DATACOM] 
I have tried to add a few additional volumes but consistently get an error on 
executing

gluster volume start 

The error message is:

volume start: : failed: Commit failed on localhost. Please check 
log file for details.

What log file is being referred to here? I can find no errors anywhere except a 
repeat of this line in cmd_history.log.

This has worked for me in the past. I have a couple of dozen active gluster 
volumes.

(These new volumes were created successfully and can be listed but show as not 
started.)

Any pointers on to how to resolve this would be much appreciated.

Thanks
Paul

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] rebalance and volume commit hash

2017-01-17 Thread Jeff Darcy 
> I don't understand why  new commit hash is generated for the volume during
> rebalance process? I think it should be generated only during add/remove
> brick events but not during rebalance.

The mismatch only becomes important during rebalance.  Prior to that, even
if we've added or removed a brick, the layouts haven't changed and the
optimization is still as valid as it was before.  If there are multiple
add/remove operations, we don't need or want to change the hash between
them.  Conversely, there are cases besides add/remove brick where we might
want to do a rebalance - e.g. after replace-brick with a brick of a
different size, or to change between total-space vs. free-space weighting.
Changing the hash in add/remove brick doesn't handle these cases, but
changing it at the start of rebalance does.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] rebalance and volume commit hash

2017-01-17 Thread Jeff Darcy 
> Can you tell me please why every volume rebalance generates a new value
> for the volume commit hash?
> 
> If I have fully rebalanced cluster (or almost) with millions of
> directories then rebalance has to change DHT xattr for every directory
> only because there is a new volume commit hash value. It is pointless in
> my opinion. Is there any reason behind this? As I observed, the volume
> commit hash is set at the rebalance beginning which totally destroys
> benefit of lookup optimization algorithm for directories not
> scanned/fixed yet by this rebalance run.

It disables the optimization because the optimization would no longer
lead to correct results.  There are plenty of distributed filesystems
that seem to have "fast but wrong" as a primary design goal; we're
not one of them.

The best way to think of the volume-commit-hash update is as a kind of
cache invalidation.  Lookup optimization is only valid as long as we
know that the actual distribution of files within a directory is
consistent with the current volume topology.  That ceases to be the
case as soon as we add or remove a brick, leaving us with three choices.

(1) Don't do lookup optimization at all.  *Every* time we fail to find
a file on the brick where hashing says it should be, look *everywhere*
else.  That's how things used to work, and still work if lookup
optimization is disabled.  The drawback is that every add/remove brick
operation causes a permanent and irreversible degradation of lookup
performance.  Even on a freshly created volume, lookups for files that
don't exist anywhere will cause every brick to be queried.

(2) Mark every directory as "unoptimized" at the very beginning of
rebalance.  Besides being almost as slow as fix-layout itself, this
would require blocking all lookups and other directory operations
*anywhere in the volume* while it completes.

(3) Change the volume commit hash, effectively marking every
directory as unoptimized without actually having to touch every one.
The root-directory operation is cheap and almost instantaneous.
Checking each directory commit hash isn't free, but it's still a
lot better than (1) above.  With upcalls we can enhance this even
further.

Now that you know a bit more about the tradeoffs, do "pointless"
and "destroys the benefit" still seem accurate?

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Minutes from Gluster Bug Triage meeting today

2017-01-17 Thread Saravanakumar Arumugam 

Hi,

Thanks all who joined!

Next week at the same time (Tuesday 12:00 UTC) we will have an other bug
triage meeting to catch the bugs that were not handled by developers and
maintainers yet. We'll stay repeating this meeting as safety net so that
bugs get the initial attention and developers can immediately start
working on the issues that were reported.

Bug triaging (in general, no need to only do it during the meeting) is
intended to help developers, in the hope that developers can focus on
writing bug fixes instead of spending much of their valued time
troubleshooting incorrectly/incompletely reported bugs.

More details about bug triaging can be found here:
http://gluster.readthedocs.io/en/latest/Contributors-Guide/Bug-Triage/

Meeting minutes below.

Thanks,
Saravana



Meeting summary
agenda:https://github.com/gluster/glusterfs/wiki/Bug-Triage-Meeting 
(Saravanakmr, 12:00:24)


Roll call (Saravanakmr, 12:00:31)
Next week’s meeting host (Saravanakmr, 12:03:39)

ndevos need to decide on how to provide/use debug builds (Saravanakmr, 
12:06:15)
ACTION: ndevos need to decide on how to provide/use debug builds 
(Saravanakmr, 12:06:27)


jiffin needs to send the changes to check-bugs.py also (Saravanakmr, 
12:06:47)
ACTION: jiffin needs to send the changes to check-bugs.py also 
(Saravanakmr, 12:07:23)


Group Triage (Saravanakmr, 12:08:13)
you can find the bugs to triage here 
inhttp://bit.ly/gluster-bugs-to-triage (Saravanakmr, 12:08:21)

http://bit.ly/gluster-bugs-to-triage (Saravanakmr, 12:08:53)

Open Floor (Saravanakmr, 12:16:01)


Meeting ended at 12:18:53 UTC (full logs).

Action items
ndevos need to decide on how to provide/use debug builds
jiffin needs to send the changes to check-bugs.py also


Action items, by person
jiffin needs to send the changes to check-bugs.py also
ndevos need to decide on how to provide/use debug builds


People present (lines said)
Saravanakmr (40)
ndevos (9)
kkeithley (5)
skoduri (3)
jiffin (3)
zodbot (3)

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] When to use striped volumes?

2017-01-17 Thread Dave Fan 
Hi Krutika,


The use-case would be storage for a parallel computing application with high 
concurrency.


How's striping fit into different use-case scenarios? We are targeting other 
applications as well, for example, Hadoop, Virtual Desktop, etc.


My Best,
Dave


 
-- Original --
From:  "Krutika Dhananjay";
Date:  Tue, Jan 17, 2017 05:22 PM
To:  "Dave Fan"; 
Cc:  "gluster-users"; 
Subject:  Re: [Gluster-users] When to use striped volumes?

 
Could you describe what use-case you intend to use striping for?



-Krutika



On Tue, Jan 17, 2017 at 12:52 PM, Dave Fan  wrote:
Hello everyone,


We are trying to set up a Gluster-based storage for best performance. On the 
official Gluster website. It says:


Striped – Striped volumes stripes data across bricks in the volume. For best 
results, you should use striped volumes only in high concurrency environments 
accessing very large files.


Is there a rule-of-thumb on what size qualifies as "very large files" here?


Many thanks,
Dave

___
 Gluster-users mailing list
 Gluster-users@gluster.org
 http://lists.gluster.org/mailman/listinfo/gluster-users___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] When to use striped volumes?

2017-01-17 Thread Kaushal M 
On Tue, Jan 17, 2017 at 12:52 PM, Dave Fan  wrote:
> Hello everyone,
>
> We are trying to set up a Gluster-based storage for best performance. On the
> official Gluster website. It says:
>
> Striped – Striped volumes stripes data across bricks in the volume. For best
> results, you should use striped volumes only in high concurrency
> environments accessing very large files.
>
> Is there a rule-of-thumb on what size qualifies as "very large files" here?

Joe Julian has an excellent post on this at
https://joejulian.name/blog/should-i-use-stripe-on-glusterfs/ .

>
> Many thanks,
> Dave
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] When to use striped volumes?

2017-01-17 Thread Krutika Dhananjay 
Could you describe what use-case you intend to use striping for?

-Krutika


On Tue, Jan 17, 2017 at 12:52 PM, Dave Fan  wrote:

> Hello everyone,
>
> We are trying to set up a Gluster-based storage for best performance. On
> the official Gluster website. It says:
>
> *Striped* – Striped volumes stripes data across bricks in the volume. For
> best results, you should use striped volumes only in high concurrency
> environments accessing very large files.
>
> Is there a rule-of-thumb on what size qualifies as "very large files" here?
>
> Many thanks,
> Dave
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] REMINDER: Gluster Community Bug Triage meeting at 12:00 UTC (today)

2017-01-17 Thread Saravanakumar Arumugam 

Hi,

This meeting is scheduled for anyone, who is interested in learning more
about, or assisting with the Bug Triage.

*This is the first bug triage meeting using hackmd.io (as 
http://public.pad.fsfe.org/ is going to be decommissioned).**

**Agenda and Group Triage links updated**
*
Meeting details:
- location: #gluster-meeting on Freenode IRC
(https://webchat.freenode.net/?channels=gluster-meeting  )
- date: every Tuesday
- time: 12:00 UTC
  (in your terminal, run: date -d "12:00 UTC")
- Agenda:https://github.com/gluster/glusterfs/wiki/Bug-Triage-Meeting

Currently the following items are listed:
* Roll Call
* Status of last weeks action items
* Group Triage
 - http://bit.ly/gluster-bugs-to-triage
* Open Floor

The last two topics have space for additions. If you have a suitable bug
or topic to discuss, please add it to the agenda.

Appreciate your participation.


Thanks,
Saravana

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users