On 10/27/2016 12:06 AM, Gandalf Corvotempesta wrote:
2016-10-26 23:38 GMT+02:00 Joe Julian :
Just do the reliability calculations and engineer a storage system to meet
(exceed) your obligations within the available budget.
http://www.eventhelix.com/realtimemantra/faulthandling/system_reliability
2016-10-26 23:38 GMT+02:00 Joe Julian :
> Just do the reliability calculations and engineer a storage system to meet
> (exceed) your obligations within the available budget.
> http://www.eventhelix.com/realtimemantra/faulthandling/system_reliability_availability.htm
>
This is good to evaluate reli
2016-10-26 23:38 GMT+02:00 Joe Julian :
> Quickly = MTTR is within tolerances to continue to meet SLA. It's just math.
Obviously yes. But in the real world, you can have the best SLAs in
the world, but if you loose data, you loose
customers.
> As for a dedicated heal network, split-horizon dns ha
2016-10-27 0:14 GMT+02:00 Joe Julian :
> For now I can say that gluster performs better and has a much better
> worst-case resolution. If everything else goes to hell, I have disks with
> files on them that I can recover on a laptop if I have to.
Totally agree.
> Of course when you ask the Inkta
On 10/26/2016 03:42 PM, Lindsay Mathieson wrote:
On 27/10/2016 8:14 AM, Joe Julian wrote:
To be fair, though, I can't blame ceph. We had a cascading hardware
failure with those storage trays. Even still, if it had been gluster
- I would have had files on disks.
Ouch :(
In that regard how
On 27/10/2016 8:14 AM, Joe Julian wrote:
To be fair, though, I can't blame ceph. We had a cascading hardware
failure with those storage trays. Even still, if it had been gluster -
I would have had files on disks.
Ouch :(
In that regard how do you view sharding? why not as simple as pulling
On 10/26/2016 02:54 PM, Lindsay Mathieson wrote:
Maybe a controversial question (and hopefully not trolling), but any
particularly reason you choose gluster over ceph for these larger
setups Joe?
For myself, gluster is much easier to manage and provides better
performance on my small non-ente
Maybe a controversial question (and hopefully not trolling), but any
particularly reason you choose gluster over ceph for these larger setups
Joe?
For myself, gluster is much easier to manage and provides better
performance on my small non-enterprise setup, plus it plays nice with zfs.
But I
On 10/26/2016 02:12 PM, Gandalf Corvotempesta wrote:
2016-10-26 23:07 GMT+02:00 Joe Julian :
And yes, they can fail, but 20TB is small enough to heal pretty quickly.
20TB small enough to build quickly? On which network? Gluster doesn't
have a dedicated cluster network, if the cluster is being
On 10/26/2016 02:13 PM, Gandalf Corvotempesta wrote:
2016-10-26 23:09 GMT+02:00 Joe Julian :
Open Compute WiWynn Knox trays. I don't recommend them but they are pretty.
https://goo.gl/photos/tmkRE58xKKaWKdL96
What are you hosting on that huge cluster?
10GB network I suppose.
Nothing, anymore. W
2016-10-26 23:09 GMT+02:00 Joe Julian :
> Open Compute WiWynn Knox trays. I don't recommend them but they are pretty.
> https://goo.gl/photos/tmkRE58xKKaWKdL96
What are you hosting on that huge cluster?
10GB network I suppose.
___
Gluster-users mailing l
2016-10-26 23:07 GMT+02:00 Joe Julian :
> And yes, they can fail, but 20TB is small enough to heal pretty quickly.
20TB small enough to build quickly? On which network? Gluster doesn't
have a dedicated cluster network, if the cluster is being hevily
accessed, the healing will slow down everything
On 10/26/2016 02:06 PM, Gandalf Corvotempesta wrote:
2016-10-26 23:04 GMT+02:00 Joe Julian :
I just add enough disks to saturate (and I don't like zfs, personally)
per-brick. So with 30 disks on a server, I typically do 5-disk raid-0 and
create 6 bricks per server.
30 disks per server? which ch
On 10/26/2016 02:06 PM, Gandalf Corvotempesta wrote:
2016-10-26 23:04 GMT+02:00 Joe Julian :
I just add enough disks to saturate (and I don't like zfs, personally)
per-brick. So with 30 disks on a server, I typically do 5-disk raid-0 and
create 6 bricks per server.
30 disks per server? which ch
On 10/26/2016 02:04 PM, Joe Julian wrote:
On 10/26/2016 02:02 PM, Gandalf Corvotempesta wrote:
2016-10-26 22:59 GMT+02:00 Joe Julian :
Personally, I prefer raid0 bricks just to get the throughput to
saturate my
network, then I use replicate to meet my availability requirements
(typically rep
2016-10-26 23:04 GMT+02:00 Joe Julian :
> I just add enough disks to saturate (and I don't like zfs, personally)
> per-brick. So with 30 disks on a server, I typically do 5-disk raid-0 and
> create 6 bricks per server.
30 disks per server? which chassis are you using?
Why don't you like ZFS? I adm
On 27/10/2016 6:59 AM, Joe Julian wrote:
Personally, I prefer raid0 bricks just to get the throughput to
saturate my network, then I use replicate to meet my availability
requirements (typically replica 3).
My network is the limiting factor already :( Only 1G * 3 Bond. Cheap and
nasty D-Link
On 10/26/2016 02:02 PM, Gandalf Corvotempesta wrote:
2016-10-26 22:59 GMT+02:00 Joe Julian :
Personally, I prefer raid0 bricks just to get the throughput to saturate my
network, then I use replicate to meet my availability requirements
(typically replica 3).
Isn't the ZFS cache on SSD enough to
On 27/10/2016 6:58 AM, mabi wrote:
Sorry yes I meant vmstat, I was doing too much ionice/iostat today ;)
right now its averaging at 45000. Low load though.
--
Lindsay Mathieson
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.glu
2016-10-26 22:59 GMT+02:00 Joe Julian :
> Personally, I prefer raid0 bricks just to get the throughput to saturate my
> network, then I use replicate to meet my availability requirements
> (typically replica 3).
Isn't the ZFS cache on SSD enough to saturate the network?
I'll use replica 3, but I d
On 27/10/2016 6:56 AM, Gandalf Corvotempesta wrote:
Velociraptors: are still around ? I heard that were EOL a couple of years ago.
Legacy hardware :) I must admit they last really well.
--
Lindsay Mathieson
___
Gluster-users mailing list
Gluster-use
On 10/26/2016 01:56 PM, Gandalf Corvotempesta wrote:
2016-10-26 22:31 GMT+02:00 Lindsay Mathieson :
Yah, RAID10.
- Two nodes with 4 WD 3TB RED
I really hate RAID10.
Personally, I prefer raid0 bricks just to get the throughput to saturate
my network, then I use replicate to meet my availabil
Sorry yes I meant vmstat, I was doing too much ionice/iostat today ;)
Original Message
Subject: Re: [Gluster-users] Production cluster planning
Local Time: October 26, 2016 10:56 PM
UTC Time: October 26, 2016 8:56 PM
From: lindsay.mathie...@gmail.com
To: gluster-users
2016-10-26 22:31 GMT+02:00 Lindsay Mathieson :
> Yah, RAID10.
>
> - Two nodes with 4 WD 3TB RED
I really hate RAID10.
I'm evaluating 2 RAIZ2 on each gluster node (12 disks: 6+6 on each
RAIDZ2) or one huge RAIDZ3 with 12 disks.
The biggest drawback with RAIDZ is that is impossible to add disks on
a
On 27/10/2016 6:35 AM, mabi wrote:
I was wondering with your setup you mention, how high are your context
switches? I mean what is your typical average context switch and what
are your highest context switch peeks (as seen in iostat).
Wouldn't that be vmstat?
--
Lindsay Mathieson
___
On 27/10/2016 3:53 AM, Gandalf Corvotempesta wrote:
Are you using any ZFS RAID on your servers?
Yah, RAID10.
- Two nodes with 4 WD 3TB RED
- One node with 2 WD 3TB reds and 6 * 600MB SAS Velocirtprs
High Endurance High Speed SSD's for SLOG devices on each node. Std SSD's
don't cut it, I've
I was wondering with your setup you mention, how high are your context
switches? I mean what is your typical average context switch and what are your
highest context switch peeks (as seen in iostat).
Best,
M.
Original Message
Subject: Re: [Gluster-users] Production
2016-10-05 23:48 GMT+02:00 Lindsay Mathieson :
> Its enough? I also run 10 windows VM's per node.
>
>
> My servers typically run at 4-6% max ioload. They idle under 1%
Are you using any ZFS RAID on your servers?
___
Gluster-users mailing list
Gluster-use
On 6/10/2016 6:37 AM, Gandalf Corvotempesta wrote:
Only 8GB ? Why ?
Its enough? I also run 10 windows VM's per node.
My servers typically run at 4-6% max ioload. They idle under 1%
--
Lindsay Mathieson
___
Gluster-users mailing list
Gluster-users@
2016-10-05 22:35 GMT+02:00 Lindsay Mathieson :
> 64Gb RAM in each server, 8GB reversed for ZFS.
Only 8GB ? Why ?
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
On 6/10/2016 6:20 AM, Gandalf Corvotempesta wrote:
> L2ARC depends on your workload. For me its not useful - VM hosting
on a sharded volume, never got better than 6% cache hits. The vast
majority of hits were via ARC (memory). ZFS seems to be really good at
that :)
>
How many ram do you hav
Il 05 ott 2016 10:12 PM, "Lindsay Mathieson"
ha scritto:
>
> L2ARC depends on your workload. For me its not useful - VM hosting on a
sharded volume, never got better than 6% cache hits. The vast majority of
hits were via ARC (memory). ZFS seems to be really good at that :)
>
How many ram do you h
On 6/10/2016 4:29 AM, Gandalf Corvotempesta wrote:
> I was thinking about creating one or more raidz2 to use as bricks,
with 2 ssd. One small partition on these ssd would be used as a
mirrored SLOG and the other 2 would be used as standalone arc cache.
will this worth the use of SSD or would
On September 30, 2016 1:46:31 PM GMT+02:00, Gandalf Corvotempesta
wrote:
>
>As suggestion for gluster developers: if ZFS is considered stable it
>could
>be used as default (replacing xfs) and many features that zfs already
>has
>could be removed from gluster (like bitrot) keeping gluster small
On Wed, Oct 5, 2016 at 2:14 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> 2016-10-05 20:50 GMT+02:00 David Gossage :
> > The mirrored slog will be useful. Depending on what you put on the pool
> > l2arc may not get used much. I removed mine as it got such a low hit
> rate
2016-10-05 20:50 GMT+02:00 David Gossage :
> The mirrored slog will be useful. Depending on what you put on the pool
> l2arc may not get used much. I removed mine as it got such a low hit rate
> serving VM's.
I'll use shards. The most accessed shard isn't cached in L2ARC?
I'll also other pools
On Wed, Oct 5, 2016 at 1:29 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> Il 30 set 2016 1:46 PM, "Gandalf Corvotempesta" <
> gandalf.corvotempe...@gmail.com> ha scritto:
>
> > I was thinking about creating one or more raidz2 to use as bricks, with
> 2 ssd. One small partit
Il 30 set 2016 1:46 PM, "Gandalf Corvotempesta" <
gandalf.corvotempe...@gmail.com> ha scritto:
> I was thinking about creating one or more raidz2 to use as bricks, with 2
ssd. One small partition on these ssd would be used as a mirrored SLOG and
the other 2 would be used as standalone arc cache. w
On 1/10/2016 4:15 AM, mabi wrote:
The data will not be in "any" state as you mention or please define
what you mean by "any". In the worst case you will just loose 5
seconds of data that's all as far as I understand.
By "Any" state I mean *Any*, you have no way of predicting how much data
w
Sorry the link is missing in my previous post:
https://groups.google.com/a/zfsonlinux.org/d/msg/zfs-discuss/OI5dchl7d_8/vLRMZgJGYUoJ
Original Message
Subject: Re: [Gluster-users] Production cluster planning
Local Time: September 30, 2016 8:15 PM
UTC Time: September 30
it all boils down to this specific
Original Message ----
Subject: Re: [Gluster-users] Production cluster planning
Local Time: September 30, 2016 12:41 PM
UTC Time: September 30, 2016 10:41 AM
From: lindsay.mathie...@gmail.com
To: mabi , Gluster Users
On 29/09/2016 4:32 AM, mabi w
2016-09-30 12:41 GMT+02:00 Lindsay Mathieson :
> Your missing what he said - *ZFS* will not be corrupted but the data written
> could be in any state, in this case the gluster filesystem data and meta
> data. To have one ndoe in a cluster out of sync with out the cluster knowing
> would be very bad
On 29/09/2016 4:32 AM, mabi wrote:
hat's not correct. There is no risk of corruption using
"sync=disabled". In the worst case you just end up with old data but
no corruption. See the following comment from a master of ZFS (Aaron
Toponce):
https://pthree.org/2013/01/25/glusterfs-linked-list-to
Il 30 set 2016 11:35, "mabi" ha scritto:
>
> That's not correct. There is no risk of corruption using "sync=disabled".
In the worst case you just end up with old data but no corruption. See the
following comment from a master of ZFS (Aaron Toponce):
>
>
https://pthree.org/2013/01/25/glusterfs-link
---
Subject: Re: [Gluster-users] Production cluster planning
Local Time: September 26, 2016 11:08 PM
UTC Time: September 26, 2016 9:08 PM
From: lindsay.mathie...@gmail.com
To: gluster-users@gluster.org
On 27/09/2016 4:13 AM, mabi wrote:
> I would also say do not forget to set "sync=disabled"
2016-09-26 23:08 GMT+02:00 Lindsay Mathieson :
> I wouldn't be doing that - very high risk of gluster corruption in the event
> of power loss or server crash. Up to 5 seconds of writes could be lost that
> way.
>
> If writes aren't fast enough I'd add a SSD partition for slog. Preferably a
> data c
On 27/09/2016 4:13 AM, mabi wrote:
I would also say do not forget to set "sync=disabled".
I wouldn't be doing that - very high risk of gluster corruption in the
event of power loss or server crash. Up to 5 seconds of writes could be
lost that way.
If writes aren't fast enough I'd add a SSD
I would also say do not forget to set "sync=disabled".
Original Message
Subject: Re: [Gluster-users] Production cluster planning
Local Time: September 26, 2016 7:59 PM
UTC Time: September 26, 2016 5:59 PM
From: jlawre...@squaretrade.com
To: Lindsay Mathieson
> On Sep 26, 2016, at 4:26 AM, Lindsay Mathieson
> wrote:
>
> On 26/09/2016 8:18 PM, Gandalf Corvotempesta wrote:
>> No one ?
>> And what about gluster on ZFS? Is that fully supported ?
>
> I certainly hope so because I'm running a Replica 3 production cluster on ZFS
> :)
Same here- in my ex
Hi,
Red Hat only supports XFS for some reason:
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Installation_Guide/sect-Prerequisites1.html
--
Dmitry Glushenok
Jet Infosystems
> 26 сент. 2016 г., в 14:26, Lindsay Mathieson
> написал(а):
>
> On 26/09/2016 8:18 PM, Gandal
On 26/09/2016 8:18 PM, Gandalf Corvotempesta wrote:
No one ?
And what about gluster on ZFS? Is that fully supported ?
I certainly hope so because I'm running a Replica 3 production cluster
on ZFS :)
--
Lindsay Mathieson
___
Gluster-users mailing l
2016-09-26 12:22 GMT+02:00 David Gossage :
> I don't know that they test on zfs, but when I had zfs related issues with
> some changes they were very accommodating in assisting the testing and bug
> fix when I opened a bugzilla report.
I'm asking this because I would like to use some features from
On Mon, Sep 26, 2016 at 5:18 AM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> 2016-09-21 1:04 GMT+02:00 Pranith Kumar Karampuri :
> > I am not sure about this one, I was waiting for someone else to respond
> on
> > this point.
>
> No one ?
> And what about gluster on ZFS? Is t
2016-09-21 1:04 GMT+02:00 Pranith Kumar Karampuri :
> I am not sure about this one, I was waiting for someone else to respond on
> this point.
No one ?
And what about gluster on ZFS? Is that fully supported ?
___
Gluster-users mailing list
Gluster-users@
On Tue, Sep 20, 2016 at 5:44 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> 2016-09-20 12:06 GMT+02:00 Pranith Kumar Karampuri :
> > Replica 3 is more costly compared to Arbiter. If you are comfortable with
> > replica-3 that is the best option.
>
> And what about raid and s
2016-09-20 12:06 GMT+02:00 Pranith Kumar Karampuri :
> Replica 3 is more costly compared to Arbiter. If you are comfortable with
> replica-3 that is the best option.
And what about raid and ssd tiering?
Which is the most common and stable configuration available?
__
On Tue, Sep 20, 2016 at 3:31 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> 2016-09-20 11:57 GMT+02:00 Pranith Kumar Karampuri :
> > I would suggest to use separate volumes for this, because the
> optimizations
> > for VM image store on the volume would slow things down for
2016-09-20 11:57 GMT+02:00 Pranith Kumar Karampuri :
> I would suggest to use separate volumes for this, because the optimizations
> for VM image store on the volume would slow things down for webservers kind
> of workload IMHO. Most of the perf xlators should be disabled in VM image
> store usecas
On Tue, Sep 20, 2016 at 3:24 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> 2016-09-20 10:28 GMT+02:00 Pranith Kumar Karampuri :
> > Are you willing to take VM snapshots at the right times?
>
> Yes, sure.
>
> In addition, the same cluster would also be used to store files sh
2016-09-20 10:28 GMT+02:00 Pranith Kumar Karampuri :
> Are you willing to take VM snapshots at the right times?
Yes, sure.
In addition, the same cluster would also be used to store files shared
between webservers, like the uploaded images and so on.
> Please don't use EC for Random write workloa
On Fri, Sep 16, 2016 at 10:23 PM, Gandalf Corvotempesta <
gandalf.corvotempe...@gmail.com> wrote:
> Next year i'll start with our first production cluster.
> I'll put on that many VMs images (XenServer, ProxMox, ...).
>
> Currently I have 3 SuperMicro 6028R-E1CR12T to be used as storage nodes.
>
>
No one?
Il 16 set 2016 18:53, "Gandalf Corvotempesta" <
gandalf.corvotempe...@gmail.com> ha scritto:
> Next year i'll start with our first production cluster.
> I'll put on that many VMs images (XenServer, ProxMox, ...).
>
> Currently I have 3 SuperMicro 6028R-E1CR12T to be used as storage nodes.
62 matches
Mail list logo