Hi All,
First, some background:
I have been running a small (4 compute nodes) xen server cluster
backed by both a small ceph (4 other nodes with a total of 18x 1-spindle
osd's) and small gluster cluster (2 nodes each with a 14 spindle RAID
array). I started with gluster 3-4 years ago, at
Wanted to check if anyone has a ceph cluster which has mixed vendor servers
both with same disk size i.e. 8TB but different count i.e. Example 10 OSD
servers from Dell with 60 Disk per server and other 10 OSD servers from HP with
26 Disk per server.
If so does that change any performance dynami
On Thu, Jun 8, 2017 at 11:31 PM, Alejandro Comisario
wrote:
> Hi Brad.
> Taking into consideration the unlikely posibility that someone
> realizes what the problem is in this specific case, that would be
> higly apreciated.
>
> I presume that having jewel, if you can somehow remediate this, will
>
On Thu, 8 Jun 2017 07:06:04 -0400 Alfredo Deza wrote:
> On Thu, Jun 8, 2017 at 3:38 AM, Christian Balzer wrote:
> > On Thu, 8 Jun 2017 17:03:15 +1000 Brad Hubbard wrote:
> >
> >> On Thu, Jun 8, 2017 at 3:47 PM, Christian Balzer wrote:
> >> > On Thu, 8 Jun 2017 15:29:05 +1000 Brad Hubbard wro
I can reproduce this.
The key is to look at debug logging on the primary.
2017-06-09 09:30:14.776355 7f9cf26a4700 20
/home/brad/working/src/ceph3/src/cls/lock/cls_lock.cc:247: lock_op
2017-06-09 09:30:14.776359 7f9cf26a4700 20
/home/brad/working/src/ceph3/src/cls/lock/cls_lock.cc:162: requested
This has come up quite a few times before, but since I was only working with
RBD before I didn't pay too close attention to the conversation. I'm looking
for the best way to handle existing clusters that have buckets with a large
number of objects (>20 million) in them. The cluster I'm doing test
We have an internal management service that works at a higher layer
upstream on top of multiple Ceph clusters. It needs a way to
differentiate and connect separately to each of those clusters.
Presently making that distinction is relatively easy since we create
those connections based on /etc/conf/
Hi Sage,
We do use cluster names, we do not use ceph-deploy or ceph-ansible so in
the short term it is not an issue. We have scripts that call cli
commands with the --cluster XX parameter, would that still work ? What
time frame do you have in mind for removing this ?
Cheers /Maged
On 2017-06
Hi Sage,
We did at one time run multiple clusters on our OSD nodes and RGW
nodes (with Jewel). We accomplished this by putting code in our
puppet-ceph module that would create additional systemd units with
appropriate CLUSTER=name environment settings for clusters not named
ceph. IE, if the modu
Hi Sage,
We need named clusters on the client side. RBD or CephFS clients, or
monitoring/admin machines all need to be able to access several clusters.
Internally, each cluster is indeed called "ceph", but the clients use
distinct names to differentiate their configs/keyrings.
Cheers, Dan
On J
On Thu, 8 Jun 2017, Bassam Tabbara wrote:
> Thanks Sage.
>
> > At CDM yesterday we talked about removing the ability to name your ceph
> > clusters.
>
> Just to be clear, it would still be possible to run multiple ceph
> clusters on the same nodes, right?
Yes, but you'd need to either (1) use
Thanks Sage.
> At CDM yesterday we talked about removing the ability to name your ceph
> clusters.
Just to be clear, it would still be possible to run multiple ceph clusters on
the same nodes, right?
___
ceph-users mailing list
ceph-users@lists.ce
At CDM yesterday we talked about removing the ability to name your ceph
clusters. There are a number of hurtles that make it difficult to fully
get rid of this functionality, not the least of which is that some
(many?) deployed clusters make use of it. We decided that the most we can
do at th
Thank jake, can you confirm are you testing this in which ceph version -
the out of memory you noticed. There is already a memory leak issue
reported in kraken v11.2.0 . which addressed in this tracker ..
http://tracker.ceph.com/issues/18924 ..
#ceph -v
Ok so you are mounting/mapping ceph as a r
Sorry I didn't get to reply until now. The thing is I believe I *do*
have a lifecycle configured on at least one bucket. As noted in that
issue, I get an error returned when trying to set the lifecycle, but it
does appear to get stored:
% aws --endpoint-url https://xxx.xxx.xxx.xxx s3api \
ge
On Thu, Jun 8, 2017 at 3:33 PM, McFarland, Bruce
wrote:
> John,
>
> Thanks for your answers. I have a clarification on my questions see below
> inline.
>
> Bruce
>
>
>
> From: John Spray
> Date: Thursday, June 8, 2017 at 1:45 AM
> To: "McFarland, Bruce"
> Cc: "ceph-users@lists.ceph.com"
> Subje
Hello,
David Turner wrote:
: How long have you waited?
About a day.
: I don't do much with rados objects directly. I usually use RBDs and
: cephfs. If you just need to clean things up, you can delete the pool and
: recreate it since it looks like it's testing. However this is
How long have you waited? Watchers of objects in ceph time out after a
while and you should be able to delete it. I'm talking around the range of
30 minutes, so it's likely this isn't the problem if you've been wrestling
with it long enough to write in about.
I don't do much with rados objects di
I did stumble across Samsung PM1725/a in both AIC and 2.5” U.2 form factor.
AIC starts at 1.6T and goes up to 6.4T, while 2.5” goes from 800G up to 6.4T.
The thing that caught my eye with this model is the x8 lanes in AIC, and the
5DWPD over 5 years.
No idea on how available it is, or how it co
Hello,
I have created a RADOS striped object using
$ dd someargs | rados --pool testpool --striper put testfile -
and interrupted it in the middle of writing. Now I cannot remove this object:
$ rados --pool testpool --striper rm testfile
error removing testpool>testfile: (16) Device or
Hi Mark / Jayaram,
After running the cluster last night, I noticed lots of
"Out Of Memory" errors in /var/log/messages, many of these correlate to
dead OSD's. If this is the problem, this might now be another case of
the high memory use issues reported in Kraken.
e.g. my script logs:
Thu 8 Jun 08
John,
Thanks for your answers. I have a clarification on my questions see below
inline.
Bruce
From: John Spray
Date: Thursday, June 8, 2017 at 1:45 AM
To: "McFarland, Bruce"
Cc: "ceph-users@lists.ceph.com"
Subject: Re: [ceph-users] CephFS Snapshot questions
On Wed, Jun 7, 2017 at 11:46 PM, Mc
Whether or not 2x replica is possible has little to do with the technology
and EVERYTHING to do with your use case. How redundant is your hardware
for instance? If you have the best drives in the world that will never
fail after constant use over 100 years but you don't have redundant
power,
Bluestore will make 2x Replica’s “safer” to use in theory. Until Bluestore is
in use in the wild, I don’t think anyone can give any guarantees.
From: i...@witeq.com [mailto:i...@witeq.com]
Sent: 08 June 2017 14:32
To: nick
Cc: Vy Nguyen Tan ; ceph-users
Subject: Re: [ceph-users] 2x replic
Hi Brad.
Taking into consideration the unlikely posibility that someone
realizes what the problem is in this specific case, that would be
higly apreciated.
I presume that having jewel, if you can somehow remediate this, will
be something that i will not be able to have on this deploy right?
best.
I'm thinking to delay this project until Luminous release to have Bluestore
support.
So are you telling me that checksum capability will be present in Bluestore and
therefore considering using NVMe with 2x replica for production data will be
possibile?
From: "nick"
To: "Vy Nguyen Tan" ,
Hello Mark,
As this issue noticed while giving write via librados {C API} only , the
same can't be reproduce with rados user space utility.
Ref;- http://docs.ceph.com/docs/master/rados/api/librados/
Jack, I guess you also creating load via librados.
Thanks
Jayaram
On Thu, Jun 8, 2017 at 5:46 PM
There are two main concerns with using 2x replicas, recovery speed and coming
across inconsistent objects.
With spinning disks their size to access speed means recovery can take a long
time and increases the chance that additional failures may happen during the
recovery process. NVME will re
Hi,
I think that the replica 2x on HDD/SSD are the same. You should read quote
from Wido bellow:
""Hi,
As a Ceph consultant I get numerous calls throughout the year to help people
with getting their broken Ceph clusters back online.
The causes of downtime vary vastly, but one of the biggest ca
Hi Jayaram,
Thanks for creating a tracker entry! Any chance you could add a note
about how you are generating the 200MB/s client workload? I've not seen
this problem in the lab, but any details you could give that would help
us reproduce the problem would be much appreciated!
Mark
On 06/08
Hello Mark,
Raised tracker for the issue -- http://tracker.ceph.com/issues/20222
Jake can you share the restart_OSD_and_log-this.sh script
Thanks
Jayaram
On Wed, Jun 7, 2017 at 9:40 PM, Jake Grimmett wrote:
> Hi Mark & List,
>
> Unfortunately, even when using yesterdays master version of cep
On Thu, Jun 8, 2017 at 3:38 AM, Christian Balzer wrote:
> On Thu, 8 Jun 2017 17:03:15 +1000 Brad Hubbard wrote:
>
>> On Thu, Jun 8, 2017 at 3:47 PM, Christian Balzer wrote:
>> > On Thu, 8 Jun 2017 15:29:05 +1000 Brad Hubbard wrote:
>> >
>> >> On Thu, Jun 8, 2017 at 3:10 PM, Christian Balzer wrot
Hi all,
i'm going to build an all-flash ceph cluster, looking around the existing
documentation i see lots of guides and and use case scenarios from various
vendor testing Ceph with replica 2x.
Now, i'm an old school Ceph user, I always considered 2x replica really
dangerous for production d
On Wed, Jun 7, 2017 at 11:46 PM, McFarland, Bruce
wrote:
> I have a couple of CephFS snapshot questions
>
> - Is there any functionality similar to rbd clone/flatten such that
> the snapshot can be made writable? Or is that as simple as copying the
> .snap/ to another cluster?
No, there
Looking at that anandtech comparison it seems the Micron usually is
worse than the P3700.
This week I asked for a few nodes with P3700 400G and got an answer as
they're end of sale, and the supplier wouldn't be able to get it
anywhere in the world. Has anyone got a good replacement for these?
The
On Thu, 8 Jun 2017 17:03:15 +1000 Brad Hubbard wrote:
> On Thu, Jun 8, 2017 at 3:47 PM, Christian Balzer wrote:
> > On Thu, 8 Jun 2017 15:29:05 +1000 Brad Hubbard wrote:
> >
> >> On Thu, Jun 8, 2017 at 3:10 PM, Christian Balzer wrote:
> >> > On Thu, 8 Jun 2017 14:21:43 +1000 Brad Hubbard wro
On Thu, Jun 8, 2017 at 3:47 PM, Christian Balzer wrote:
> On Thu, 8 Jun 2017 15:29:05 +1000 Brad Hubbard wrote:
>
>> On Thu, Jun 8, 2017 at 3:10 PM, Christian Balzer wrote:
>> > On Thu, 8 Jun 2017 14:21:43 +1000 Brad Hubbard wrote:
>> >
>> >> On Thu, Jun 8, 2017 at 1:06 PM, Christian Balzer wrot
37 matches
Mail list logo