Re: [ceph-users] Monitoring bluestore compression ratio

Finally, I've founded the command: ceph daemon osd.1 perf dump | grep bluestore And there you have compressed data Regards, Rafał Wądołowski http://cloudferro.com/ On 04.12.2017 14:17, Rafał Wądołowski wrote: Hi, Is there any command or tool to show

Re: [ceph-users] Luminous, RGW bucket resharding

Thanks! Is there anything in the bug tracker about the resharding issues that I can check, just to follow progress? Regards, Andreas On 4 December 2017 at 18:57, Orit Wasserman wrote: > Hi Andreas, > > On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder >

[ceph-users] 答复: Question about BUG #11332

Thanks for your reply, greg:-) Monitor processes its requests in the main dispatch loop, however, the "PAXOS COMMIT" transaction is executed by another thread MonitorDBStore::io_work, so I think it could be possible that they run concurrently. On the other hand, although the commit transaction

Re: [ceph-users] Replaced a disk, first time. Quick question

I've seen that before (over 100%) but I forget the cause. At any rate, the way I replace disks is to first set the osd weight to 0, wait for data to rebalance, then down / out the osd. I don't think ceph does any reads from a disk once you've marked it out so hopefully there are other copies.

Re: [ceph-users] injecting args output misleading

On Tue, Dec 5, 2017 at 6:12 AM, Brady Deetz wrote: > I'm not sure if this is a bug where ceph incorrectly reports to the user or > if this is just a matter of misleading language. Thought I might bring it up > in any case. > > I under stand that "may require restart" is fairly

[ceph-users] tcmu-runner failing during image creation

I thought I was good to go with tcmu-runner on Kernel 4.14, but I guess not? Any thoughts on the output below? 2017-12-04 17:44:09,631ERROR [rbd-target-api:665:_disk()] - LUN alloc problem - Could not set LIO device attribute cmd_time_out/qfull_time_out for device: iscsi-primary.primary00.

Re: [ceph-users] Adding multiple OSD

Thank you for detailed explanation! Got one another doubt, This is the total space available in the cluster : TOTAL : 23490G Use : 10170G Avail : 13320G But ecpool shows max avail as just 3 TB. What am I missing ? == $ ceph df GLOBAL: SIZE AVAIL RAW USED %RAW

Re: [ceph-users] Adding multiple OSD

Thank you for detailed explanation! Got one another doubt, This is the total space available in the cluster : TOTAL 23490G Use 10170G Avail : 13320G But ecpool shows max avail as just 3 TB. Karun Josy On Tue, Dec 5, 2017 at 1:06 AM, David Turner wrote: > No, I

Re: [ceph-users] Question about BUG #11332

On Thu, Nov 23, 2017 at 1:55 AM 许雪寒 wrote: > Hi, everyone. > > We also encountered this problem: http://tracker.ceph.com/issues/11332. > And we found that this seems to be caused by the lack of mutual exclusion > between applying "trim" and handling subscriptions. Since >

Re: [ceph-users] injecting args output misleading

On Mon, Dec 4, 2017 at 12:12 PM Brady Deetz wrote: > I'm not sure if this is a bug where ceph incorrectly reports to the user > or if this is just a matter of misleading language. Thought I might bring > it up in any case. > > I under stand that "may require restart" is fairly

[ceph-users] injecting args output misleading

I'm not sure if this is a bug where ceph incorrectly reports to the user or if this is just a matter of misleading language. Thought I might bring it up in any case. I under stand that "may require restart" is fairly direct in its ambiguity, but this probably shouldn't be ambiguous without a good

Re: [ceph-users] HELP with some basics please

Yep, you are correct, thanks! On 12/04/2017 07:31 PM, David Turner wrote: "The journals can only be moved back by a complete rebuild of that osd as to my knowledge." I'm assuming that since this is a cluster that he's inherited and that it's configured like this that it's probably not

Re: [ceph-users] HELP with some basics please

On 04.12.2017 19:18, tim taler wrote: In size=2 losing any 2 discs on different hosts would probably cause data to be unavailable / lost, as the pg copys are randomly distribbuted across the osds. Chances are, that you can find a pg which's acting group is the two failed osd (you lost all your

Re: [ceph-users] Adding multiple OSD

No, I would only add disks to 1 failure domain at a time. So in your situation where you're adding 2 more disks to each node, I would recommend adding the 2 disks into 1 node at a time. Your failure domain is the crush-failure-domain=host. So you can lose a host and only lose 1 copy of the

Re: [ceph-users] Adding multiple OSD

Thanks for your reply! I am using erasure coded profile with k=5, m=3 settings $ ceph osd erasure-code-profile get profile5by3 crush-device-class= crush-failure-domain=host crush-root=default jerasure-per-chunk-alignment=false k=5 m=3 plugin=jerasure technique=reed_sol_van w=8 Cluster has 8

[ceph-users] luminous 12.2.2 traceback (ceph fs status)

Hi, I just upgrade a ceph cluster from version 12.2.0 (rc) to 12.2.2 (stable), and i'm getting a traceback while trying to run: *# ceph fs status* Error EINVAL: Traceback (most recent call last): File "/usr/lib/ceph/mgr/status/module.py", line 301, in handle_command return

Re: [ceph-users] Adding multiple OSD

Depending on how well you burn-in/test your new disks, I like to only add 1 failure domain of disks at a time in case you have bad disks that you're adding. If you are confident that your disks aren't likely to fail during the backfilling, then you can go with more. I just added 8 servers (16

Re: [ceph-users] HELP with some basics please

"The journals can only be moved back by a complete rebuild of that osd as to my knowledge." I'm assuming that since this is a cluster that he's inherited and that it's configured like this that it's probably not running luminous or bluestore OSDs. Again more information needed about your cluster

[ceph-users] Any way to get around selinux-policy-base dependency

Hi all, I would like to upgrade to the latest Luminous release but found that it requires the absolute latest selinux-policy-base. We aren't using selinux, so was wondering if there is a way around this dependency requirement? [carf-ceph-osd15][WARNIN] Error: Package:

Re: [ceph-users] HELP with some basics please

> In size=2 losing any 2 discs on different hosts would probably cause data to > be unavailable / lost, as the pg copys are randomly distribbuted across the > osds. Chances are, that you can find a pg which's acting group is the two > failed osd (you lost all your replicas) okay I see, getting

Re: [ceph-users] Luminous, RGW bucket resharding

Hi Andreas, On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder wrote: > Hello, > With release 12.2.2 dynamic resharding bucket index has been disabled > when running a multisite environment > (http://tracker.ceph.com/issues/21725). Does this mean that resharding >

[ceph-users] Adding multiple OSD

Hi, Is it recommended to add OSD disks one by one or can I add couple of disks at a time ? Current cluster size is about 4 TB. Karun ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] HELP with some basics please

Hi, I would not rip out the discs, but I would reweight the osd to 0, wait for the cluster to reconfigure, and when it is done, you can remove the disc / raid pair without ever going down to 1 copy only. The jornals can only be moved back by a complete rebiuld of that osd as to my

Re: [ceph-users] HELP with some basics please

Flushing a journal, and creating a new journal device before turning the OSD on is viable and simple enough to do. Moving a raid0 while the new host doesn't have the same controller wouldn't be recommended for obvious reasons. That would change my recommendation for how to distribute the OSDs,

Re: [ceph-users] Replaced a disk, first time. Quick question

19446/16764 objects degraded (115.999%) <-- I noticed that number seems odd I don't think that's normal! 40795/16764 objects degraded (243.349%) <-- Now I’m really concerned. I'd recommend providing more info, Ceph version, bluestore or filestore, crushmap etc. Hi, thanks for the reply.

Re: [ceph-users] HELP with some basics please

thnx a lot again, makes sense to me. We have all journals of the HDD-OSDs on partitions on an extra SSD-raid1 (each OSD got it's own journal partition on that raid1) but as I understand they could be moved back to the OSD, at least for the time of the restructuring. What makes my tommy turn

Re: [ceph-users] HELP with some basics please

Hi, On 12/04/2017 12:12 PM, tim taler wrote: Hi, thnx a lot for the quick response and for laying out some of the issues I'm also new, but I'll try to help. IMHO most of the pros here would be quite worried about this cluster if it is production: thought so ;-/ -A prod ceph cluster

Re: [ceph-users] Replaced a disk, first time. Quick question

On Mon, Dec 4, 2017 at 4:39 PM, Drew Weaver wrote: > Howdy, > > > > I replaced a disk today because it was marked as Predicted failure. These > were the steps I took > > > > ceph osd out osd17 > > ceph -w #waited for it to get done > > systemctl stop ceph-osd@osd17 > >

[ceph-users] Replaced a disk, first time. Quick question

Howdy, I replaced a disk today because it was marked as Predicted failure. These were the steps I took ceph osd out osd17 ceph -w #waited for it to get done systemctl stop ceph-osd@osd17 ceph osd purge osd17 --yes-i-really-mean-it umount /var/lib/ceph/osd/ceph-osdX I noticed that after I ran

Re: [ceph-users] dropping trusty

On Mon, Dec 4, 2017 at 11:48 PM, David Galloway wrote: > > On 12/04/2017 01:12 AM, kefu chai wrote: >> On Fri, Dec 1, 2017 at 1:55 AM, David Galloway wrote: >>> On 11/30/2017 12:21 PM, Sage Weil wrote: We're talking about dropping trusty support for

Re: [ceph-users] dropping trusty

On 12/04/2017 01:12 AM, kefu chai wrote: > On Fri, Dec 1, 2017 at 1:55 AM, David Galloway wrote: >> On 11/30/2017 12:21 PM, Sage Weil wrote: >>> We're talking about dropping trusty support for mimic due to the old >>> compiler (incomplete C++11), hassle of using an updated

Re: [ceph-users] HELP with some basics please

Your current node configuration cannot do size=3 for any pools. You only have 2 hosts with HDDs and 2 hosts with SSDs in each root. You cannot put 3 copies of data for an HDD pool on 3 separate nodes when you only have 2 nodes with HDDs... In this configuration, size=2 is putting a copy of the

Re: [ceph-users] Luminous 12.2.2 rpm's not signed?

Total size: 51 M Is this ok [y/d/N]: y Downloading packages: Package ceph-common-12.2.2-0.el7.x86_64.rpm is not signed http://tracker.ceph.com/issues/22311 ___ ceph-users mailing list ceph-users@lists.ceph.com

[ceph-users] Luminous 12.2.2 rpm's not signed?

Total size: 51 M Is this ok [y/d/N]: y Downloading packages: Package ceph-common-12.2.2-0.el7.x86_64.rpm is not signed -Original Message- From: Rafał Wądołowski [mailto:rwadolow...@cloudferro.com] Sent: maandag 4 december 2017 14:18 To: ceph-users@lists.ceph.com Subject:

Re: [ceph-users] [Docs] s/ceph-disk/ceph-volume/g ?

On Mon, Dec 4, 2017 at 3:34 AM, Yoann Moulin wrote: > Hello, > > By the fact ceph-disk is now deprecated, that would be great to update > documentation to have also processes with ceph-volume. > > for example : > > add-or-rm-osds => >

[ceph-users] Monitoring bluestore compression ratio

Hi, Is there any command or tool to show effectiveness of bluestore compression? I see the difference (in ceph osd df tree), while uploading a object to ceph, but maybe there are more friendly method to do it. -- Regards, Rafał Wądołowski ___

Re: [ceph-users] ceph all-nvme mysql performance tuning

I got error on this:  sysbench --test=/usr/share/sysbench/tests/include/oltp_legacy/parallel_prepare.lua --mysql-host=127.0.0.1 --mysql-port=33033 --mysql-user=sysbench --mysql-password=password --mysql-db=sysbench --mysql-table-engine=innodb --db-driver=mysql --oltp_tables_count=10

Re: [ceph-users] osd/bluestore: Get block.db usage

> Op 4 december 2017 om 13:10 schreef Hans van den Bogert > : > > > Hi all, > > Is there a way to get the current usage of the bluestore's block.db? > I'd really like to monitor this as we have a relatively high number of > objects per OSD. > Yes, using 'perf dump':

[ceph-users] osd/bluestore: Get block.db usage

Hi all, Is there a way to get the current usage of the bluestore's block.db? I'd really like to monitor this as we have a relatively high number of objects per OSD. A second question related to the above, are there mechanisms to influence which objects' metadata gets spilled once the block.db is

Re: [ceph-users] ceph all-nvme mysql performance tuning

Could anyone run the tests? and share some results.. Thanks in advance, Best, *German* 2017-11-30 14:25 GMT-03:00 German Anders : > That's correct, IPoIB for the backend (already configured the irq > affinity), and 10GbE on the frontend. I would love to try rdma but

Re: [ceph-users] Another OSD broken today. How can I recover it?

On 04. des. 2017 10:22, Gonzalo Aguilar Delgado wrote: Hello, Things are going worse every day. ceph -w     cluster 9028f4da-0d77-462b-be9b-dbdf7fa57771 health HEALTH_ERR     1 pgs are stuck inactive for more than 300 seconds     8 pgs inconsistent     1 pgs

Re: [ceph-users] HELP with some basics please

Hi, thnx a lot for the quick response and for laying out some of the issues > I'm also new, but I'll try to help. IMHO most of the pros here would be quite > worried about this cluster if it is production: thought so ;-/ > -A prod ceph cluster should not be run with size=2 min_size=1,

Re: [ceph-users] Ceph+RBD+ISCSI = ESXI issue

Hi Nigel, On Fri, 1 Dec 2017 13:32:43 +, nigel davies wrote: > Ceph version 10.2.5 > > i have had an Ceph cluster going for a few months, with iscsi servers that > are linked to Ceph by RBD. > > All of an sudden i am starting the ESXI server will louse the isscsi data > store (disk space

Re: [ceph-users] Increasing mon_pg_warn_max_per_osd in v12.2.2

> the option is now called 'mon_max_pg_per_osd'. > > this was originally slated for v12.2.1 where it was erroneously > mentioned in the release notes[1] despite note being part of the > release (I remember asking for updated/fixed release notes after 12.2.1, > seems like that never happened?).

Re: [ceph-users] Increasing mon_pg_warn_max_per_osd in v12.2.2

On Mon, Dec 04, 2017 at 11:21:42AM +0100, SOLTECSIS - Victor Rodriguez Cortes wrote: > > > Why are you OK with this? A high amount of PGs can cause serious peering > > issues. OSDs might eat up a lot of memory and CPU after a reboot or such. > > > > Wido > > Mainly because there was no warning

Re: [ceph-users] Increasing mon_pg_warn_max_per_osd in v12.2.2

> Why are you OK with this? A high amount of PGs can cause serious peering > issues. OSDs might eat up a lot of memory and CPU after a reboot or such. > > Wido Mainly because there was no warning at all in v12.2.1 and it just appeared after upgrading to v12.2.2. Besides,its not a "too high"

Re: [ceph-users] Increasing mon_pg_warn_max_per_osd in v12.2.2

> Op 4 december 2017 om 10:59 schreef SOLTECSIS - Victor Rodriguez Cortes > : > > > Hello, > > I have upgraded from v12.2.1 to v12.2.2 and now a warning shows using > "ceph status": > > --- > # ceph status > cluster: > id: > health: HEALTH_WARN >

[ceph-users] Increasing mon_pg_warn_max_per_osd in v12.2.2

Hello, I have upgraded from v12.2.1 to v12.2.2 and now a warning shows using "ceph status": --- # ceph status   cluster:     id:     health: HEALTH_WARN     too many PGs per OSD (208 > max 200) --- I'm ok with the amount of PGs, so I'm trying to increase the max PGs. I've tried

Re: [ceph-users] HELP with some basics please

Hi, I'm also new, but I'll try to help. IMHO most of the pros here would be quite worried about this cluster if it is production: -A prod ceph cluster should not be run with size=2 min_size=1, because: --In case of a down'ed osd / host the cluster could have problems determining which data

[ceph-users] Luminous, RGW bucket resharding

Hello, With release 12.2.2 dynamic resharding bucket index has been disabled when running a multisite environment (http://tracker.ceph.com/issues/21725). Does this mean that resharding of bucket indexes shouldn't be done at all, manually, while running multisite as there's a risk of corruption?

Re: [ceph-users] Another OSD broken today. How can I recover it?

Hello, Things are going worse every day. ceph -w     cluster 9028f4da-0d77-462b-be9b-dbdf7fa57771 health HEALTH_ERR     1 pgs are stuck inactive for more than 300 seconds     8 pgs inconsistent     1 pgs repair     1 pgs stale     1 pgs stuck stale   

[ceph-users] HELP with some basics please

Hi I'm new to ceph but have to honor to look after a cluster that I haven't set up by myself. Rushing to the ceph docs and having a first glimpse on our cluster I start worrying about our setup, so I need some advice and guidance here. The set up is: 3 machines, each running a ceph-monitor. all

[ceph-users] [Docs] s/ceph-disk/ceph-volume/g ?

Hello, By the fact ceph-disk is now deprecated, that would be great to update documentation to have also processes with ceph-volume. for example : add-or-rm-osds => http://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/ bluestore-migration =>