On Fri, Jan 25, 2019 at 9:40 AM Martin Palma wrote:
>
> > Do you see them repeating every 30 seconds?
>
> yes:
>
> Jan 25 09:34:37 sdccgw01 kernel: [6306813.737615] libceph: mon4
> 10.8.55.203:6789 session lost, hunting for new mon
> Jan 25 09:34:37 sdccgw01 kernel: [6306813.737620] libceph: mon3
Hi All,
Starting February, Docubetter meetings will be held twice a month, on
the second and fourth Wednesday of each month. We will alternate
meeting times to ensure that all time zones have the opportunity to
participate.
Second Wednesday: 12:30 ET (starting February 13)
Fourth Wednesday:
Hi Frank,
you might want to use ceph-kvstore-tool, e.g.
ceph-kvstore-tool bluestore-kv dump
Thanks,
Igor
On 1/25/2019 10:49 PM, F Ritchie wrote:
Hi all,
Is there a way to dump the contents of block.db to a text file?
I am not trying to fix a problem just curious and want to poke around.
I figured it out. I had to dig through the ceph-deploy code to get it but
here it is. Apparently, one of the upgrades create the keys in ceph but the
bootstrap keyrings were never created.
ceph auth get client.bootstrap-osd >
/var/lib/ceph/bootstrap-osd/ceph.keyring
On Thu, Jan 24, 2019 at 4:56
Hi all,
Is there a way to dump the contents of block.db to a text file?
I am not trying to fix a problem just curious and want to poke around.
thx
Frank
___
ceph-users mailing list
ceph-users@lists.ceph.com
It has been resolved.
It seems that it in fact only removes the objects which you list with
the list_missing
option.
On Tue, Jan 22, 2019 at 9:42 AM Mathijs van Veluw <
mathijs.van.ve...@gmail.com> wrote:
> Hello.
> I have a question about `ceph pg {pg.num} mark_unfound_lost delete`.
> Will
also, here the result of "perf diff 1mslatency.perfdata 3mslatency.perfdata"
http://odisoweb1.odiso.net/perf_diff_ok_vs_bad.txt
- Mail original -
De: "aderumier"
À: "Sage Weil"
Cc: "ceph-users" , "ceph-devel"
Envoyé: Vendredi 25 Janvier 2019 17:32:02
Objet: Re: [ceph-users] ceph
Hi again,
I was able to perf it today,
before restart, commit latency was between 3-5ms
after restart at 17:11, latency is around 1ms
http://odisoweb1.odiso.net/osd3_latency_3ms_vs_1ms.png
here some perf reports:
with 3ms latency:
-
perf report by caller:
I've also seen this effect a few times since Mimic (never happened in
Luminous). It always resolved itself but the HEALTH_ERROR can be
confusing to users.
Paul
--
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
On 24/01/2019, Marc Roos wrote:
>
>
> This should do it sort of.
>
> {
> "Id": "Policy1548367105316",
> "Version": "2012-10-17",
> "Statement": [
> {
> "Sid": "Stmt1548367099807",
> "Effect": "Allow",
> "Action": "s3:ListBucket",
> "Principal": { "AWS":
Your caps are invalid -- you are missing the access-spec on the first
OSD cap clause. See [1] for more information for how to properly
format caps.
What version of Ceph are you using? If I remember correctly, since
Luminous it will at least throw an error when you give it invalid
caps.
Thanks.
This procedure works very well.
Am 25.01.2019 um 14:24 schrieb Janne Johansson:
> Den fre 25 jan. 2019 kl 09:52 skrev cmonty14 <74cmo...@gmail.com>:
>> Hi,
>> I have identified a major issue with my cluster setup consisting of 3 nodes:
>> all monitors are connected to cluster network.
That doesn't appear to be an error -- that's just stating that it
found a dead client that was holding the exclusice-lock, so it broke
the dead client's lock on the image (by blacklisting the client).
On Fri, Jan 25, 2019 at 5:09 AM ST Wong (ITSC) wrote:
>
> Oops, while I can map and mount the
The "rbdmap" systemd unit file should take care of it [1].
[1] https://github.com/ceph/ceph/blob/master/systemd/rbdmap.service.in#L4
On Fri, Jan 25, 2019 at 3:00 AM Gao, Wenjun wrote:
>
> Thanks, what’s the configuration you mentioned?
>
>
>
> --
>
> Thanks,
>
> Wenjun
>
>
>
> From: Gregory
Den fre 25 jan. 2019 kl 09:52 skrev cmonty14 <74cmo...@gmail.com>:
>
> Hi,
> I have identified a major issue with my cluster setup consisting of 3 nodes:
> all monitors are connected to cluster network.
>
> Question:
> How can I modify the network configuration of mon?
>
> It's not working to
Update:
I have identified the root cause: user caps are not correct.
Errornous caps:
root@ld4257:/etc/ceph# ceph auth get client.gbsadm
exported keyring for client.gbsadm
[client.gbsadm]
key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg==
caps mon = "allow r"
caps osd = "allow
Hi,
unfortunately it's not working, yet.
I have modified user gbsadm:
root@ld4257:/etc/ceph# ceph auth get client.gbsadm
exported keyring for client.gbsadm
[client.gbsadm]
key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg==
caps mon = "allow r"
caps osd = "allow pool backup
Hi,
just a comment:
the RBD pool also contain management objects, e.g. the rbd_directory and
rbd_info objects. And AFAIK these objects storage the name->id mapping
for images.
This means in your case, looking up the name backup/gbs requires read
access to these objects in the backup
You can check all objects of that pool to see if your caps match:
rados -p backup ls | grep rbd_id
Zitat von Eugen Block :
caps osd = "allow pool backup object_prefix
rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix
rbd_header.18102d6b8b4567; allow rx pool backup object_prefix
caps osd = "allow pool backup object_prefix
rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix
rbd_header.18102d6b8b4567; allow rx pool backup object_prefix
rbd_id.rbd-image"
I think your caps are not entirely correct, the part "[...]
object_prefix rbd_id.rbd-image" should contain
Thanks.
Unfortunately this is still not working.
Here's the info of my image:
root@ld4257:/etc/ceph# rbd info backup/gbs
rbd image 'gbs':
size 500GiB in 128000 objects
order 22 (4MiB objects)
block_name_prefix: rbd_data.18102d6b8b4567
format: 2
features:
Hi,
I replied to your thread a couple of days ago, maybe you didn't notice:
Restricting user access is possible on rbd image level. You can grant
read/write access for one client and only read access for other
clients, you have to create different clients for that, see [1] for
more
Hi,
I can create a block device user with this command:
ceph auth get-or-create client.{ID} mon 'profile rbd' osd 'profile
{profile name} [pool={pool-name}][, profile ...]'
Question:
How can I create a user that has access only to a specific image
created in pool ?
If this is not possible this
Oops, while I can map and mount the filesystem, still found error as below,
while rebooting the client machine freezes and have to power reset her.
Jan 25 17:57:30 acapp1 kernel: XFS (rbd0): Mounting V5 Filesystem
Jan 25 17:57:30 acapp1 kernel: rbd: rbd0: client74700 seems dead, breaking lock
>>Can you capture a perf top or perf record to see where teh CPU time is
>>going on one of the OSDs wth a high latency?
Yes, sure. I'll do it next week and send result to the mailing list.
Thanks Sage !
- Mail original -
De: "Sage Weil"
À: "aderumier"
Cc: "ceph-users" , "ceph-devel"
Hi, It works. Thanks a lot.
/st
-Original Message-
From: Jason Dillaman
Sent: Tuesday, January 22, 2019 9:29 PM
To: ST Wong (ITSC)
Cc: Ilya Dryomov ; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] RBD client hangs
Your "mon" cap should be "profile rbd" instead of "allow r"
Can you capture a perf top or perf record to see where teh CPU time is
going on one of the OSDs wth a high latency?
Thanks!
sage
On Fri, 25 Jan 2019, Alexandre DERUMIER wrote:
>
> Hi,
>
> I have a strange behaviour of my osd, on multiple clusters,
>
> All cluster are running mimic
Hi,
I have a strange behaviour of my osd, on multiple clusters,
All cluster are running mimic 13.2.1,bluestore, with ssd or nvme drivers,
workload is rbd only, with qemu-kvm vms running with librbd + snapshot/rbd
export-diff/snapshotdelete each day for backup
When the osd are refreshly
Hi,
I have identified a major issue with my cluster setup consisting of 3 nodes:
all monitors are connected to cluster network.
Here's my /etc/ceph/ceph.conf:
[global]
auth client required = cephx
auth cluster required = cephx
auth service required = cephx
> Do you see them repeating every 30 seconds?
yes:
Jan 25 09:34:37 sdccgw01 kernel: [6306813.737615] libceph: mon4
10.8.55.203:6789 session lost, hunting for new mon
Jan 25 09:34:37 sdccgw01 kernel: [6306813.737620] libceph: mon3
10.8.55.202:6789 session lost, hunting for new mon
Jan 25 09:34:37
On Fri, Jan 25, 2019 at 8:37 AM Martin Palma wrote:
>
> Hi Ilya,
>
> thank you for the clarification. After setting the
> "osd_map_messages_max" to 10 the io errors and the MDS error
> "MDS_CLIENT_LATE_RELEASE" are gone.
>
> The messages of "mon session lost, hunting for new new mon" didn't go
>
Thanks, what’s the configuration you mentioned?
--
Thanks,
Wenjun
From: Gregory Farnum
Date: Friday, January 25, 2019 at 3:35 PM
To: "Gao, Wenjun"
Cc: "ceph-users@lists.ceph.com"
Subject: Re: [ceph-users] krbd reboot hung
Looks like your network deactivated before the rbd volume was
32 matches
Mail list logo
