On 04/29/13 20:28, Anand Avati wrote:
On Mon, Apr 29, 2013 at 9:19 AM, Heath Skarlupka
<[email protected] <mailto:[email protected]>>
wrote:
Gluster-Users,
We currently have a 30 node Gluster Distributed-Replicate 15 x 2
filesystem. Each node has a ~20TB xfs filesystem mounted to /data
and the bricks live on /data/brick. We have been very happy with
this setup, but are now collecting more data that doesn't need to
be replicated because it can be easily regenerated. Most of the
data lives on our replicated volume and is starting to waste
space. My plan was to create a second directory under the /data
partition called /data/non_replicated_brick on each of the 30
nodes and start up a second Gluster filesystem. This would allow
me to dynamically size the replicated and non_replicated space
based on our current needs.
I'm a bit worried about going forward with this because I haven't
seen many users talk about putting two gluster bricks on the same
underlying filesystem. I've gotten passed the technical hurdle
and know that it is technically possible, but I'm worried about
corner cases and issues that might crop up when we add more bricks
and need to rebalance both gluster volumes at once. Does anybody
have any insight in what the caveats of doing this are or are
there any users putting multiple bricks on a single filesystem in
the 50-100 node size range. Thank you all for your insights and help!
This is a very common use case and should work fine. In the future we
are exploring better integration with dm-thinp so that each brick has
its own XFS filesystem on a thin provisioned logical volume. But for
now you can create a second volume on the same XFS filesystems.
Avati
There is an issue when replicated bricks fill unevenly. The
non-replicated volume will cause uneven filling of bricks as seen in the
replicated volume.
I am not sure how ENOSPC is handled asymmetrically, but if the fuller
brick happens to be down during a write that would be causing ENOSPC,
you won't get the error and replication will fail, when the self-heal
kicks in.
--
Mr. Flibble
King of the Potato People
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
