Re: [ceph-users] Ceph-Deploy error on 15/71 stage

2018-09-04 Thread Jones de Andrade
Hi Eugen. Just tried everything again here by removing the /sda4 partitions and letting it so that either salt-run proposal-populate or salt-run state.orch ceph.stage.configure could try to find the free space on the partitions to work with: unsuccessfully again. :( Just to make things clear:

[ceph-users] How to setup Ceph OSD auto boot up on node reboot

2018-09-04 Thread Pardhiv Karri
Hi, I created a ceph cluster manually (not using ceph-deploy). When I reboot the node the osd's doesn't come backup because the OS doesn't know that it need to bring up the OSD. I am running this on Ubuntu 1604. Is there a standardized way to initiate ceph osd start on node reboot? "sudo start

Re: [ceph-users] CephFS small files overhead

2018-09-04 Thread Sergey Malinin
You need to re-deploy OSDs for bluestore_min_alloc_size to take effect. > On 4.09.2018, at 18:31, andrew w goussakovski wrote: > > Hello > > We are trying to use cephfs as storage for web graphics, such as > thumbnails and so on. > Is there any way to reduse overhead on storage? On test

Re: [ceph-users] CephFS small files overhead

2018-09-04 Thread Paul Emmerich
You could probably cut the overhead in half with the inline data feature: http://docs.ceph.com/docs/master/cephfs/experimental-features/#inline-data However, that is an experimental feature. CephFS is unfortunately not very good at storing lots of small files in a storage-efficient manner :(

[ceph-users] v12.2.8 Luminous released

2018-09-04 Thread Abhishek Lekshmanan
We're glad to announce the next point release in the Luminous v12.2.X stable release series. This release contains a range of bugfixes and stability improvements across all the components of ceph. For detailed release notes with links to tracker issues and pull requests, refer to the blog post at

Re: [ceph-users] Luminous RGW errors at start

2018-09-04 Thread Robert Stanford
This was the issue (could not create the pool, because it would have exceeded the new (luminous) limitation on pgs /osd. On Tue, Sep 4, 2018 at 10:35 AM David Turner wrote: > I was confused what could be causing this until Janne's email. I think > they're correct that the cluster is

Re: [ceph-users] Luminous new OSD being over filled

2018-09-04 Thread David Turner
Instead of manually weighting the OSDs, you can use the mgr module to slowly add the OSDs and balance your cluster at the same time. I believe you can control the module by telling it a maximum percent of misplaced objects, or other similar metrics, to control adding in the OSD, while also

Re: [ceph-users] Luminous RGW errors at start

2018-09-04 Thread David Turner
I was confused what could be causing this until Janne's email. I think they're correct that the cluster is preventing pool creation due to too many PGs per OSD. Double check how many PGs you have in each pool and what your defaults are for that. On Mon, Sep 3, 2018 at 7:19 AM Janne Johansson

Re: [ceph-users] Ceph Luminous - journal setting

2018-09-04 Thread David Turner
Are you planning on using bluestore or filestore? The settings for filestore haven't changed. If you're planning to use bluestore there is a lot of documentation in the ceph docs as well as a wide history of questions like this on the ML. On Mon, Sep 3, 2018 at 5:24 AM M Ranga Swami Reddy

[ceph-users] CephFS small files overhead

2018-09-04 Thread andrew w goussakovski
Hello We are trying to use cephfs as storage for web graphics, such as thumbnails and so on. Is there any way to reduse overhead on storage? On test cluster we have 1 fs, 2 pools (meta and data) with replica size = 2     objects: 1.02 M objects, 1.1 GiB     usage:   144 GiB used, 27 GiB / 172

Re: [ceph-users] data_extra_pool for RGW Luminous still needed?

2018-09-04 Thread Casey Bodley
On 09/03/2018 10:07 PM, Nhat Ngo wrote: Hi all, I am new to Ceph and we are setting up a new RadosGW and Ceph storage cluster on Luminous. We are using only EC for our `buckets.data` pool at the moment. However, I just read the Red Hat Ceph object Gateway for Production article and

Re: [ceph-users] Degraded data redundancy: NUM pgs undersized

2018-09-04 Thread Jörg Kastning
Hello Lothar, Thanks for your reply. Am 04.09.2018 um 11:20 schrieb Lothar Gesslein: By pure chance 15 pgs are now actually replicated to all 3 osds, so they have enough copies (clean). But the placement is "wrong", it would like to move the data to different osds (remapped) if possible.

Re: [ceph-users] "no valid command found" when running "ceph-deploy osd create"

2018-09-04 Thread Alfredo Deza
On Sun, Sep 2, 2018 at 3:01 PM, David Wahler wrote: > On Sun, Sep 2, 2018 at 1:31 PM Alfredo Deza wrote: >> >> On Sun, Sep 2, 2018 at 12:00 PM, David Wahler wrote: >> > Ah, ceph-volume.log pointed out the actual problem: >> > >> > RuntimeError: Cannot use device (/dev/storage/bluestore). A

Re: [ceph-users] SSD OSDs crashing after upgrade to 12.2.7

2018-09-04 Thread Alfredo Deza
On Tue, Sep 4, 2018 at 3:59 AM, Wolfgang Lendl wrote: > is downgrading from 12.2.7 to 12.2.5 an option? - I'm still suffering > from high frequent osd crashes. > my hopes are with 12.2.9 - but hope wasn't always my best strategy 12.2.8 just went out. I think that Adam or Radoslaw might have some

Re: [ceph-users] MDS does not always failover to hot standby on reboot

2018-09-04 Thread John Spray
It's mds_beacon_grace. Set that on the monitor to control the replacement of laggy MDS daemons, and usually also set it to the same value on the MDS daemon as it's used there for the daemon to hold off on certain tasks if it hasn't seen a mon beacon recently. John On Mon, Sep 3, 2018 at 9:26 AM

Re: [ceph-users] Degraded data redundancy: NUM pgs undersized

2018-09-04 Thread Lothar Gesslein
On 09/04/2018 09:47 AM, Jörg Kastning wrote: > My questions are: > >  1. What does active+undersized actually mean? I did not find anything > about it in the documentation on docs.ceph.com. http://docs.ceph.com/docs/master/rados/operations/pg-states/ active Ceph will process requests to the

[ceph-users] osd_journal_aio=false and performance

2018-09-04 Thread Rudenko Aleksandr
Hi, guys. I made a few tests and i see that performance is better if osd_journal_aio=false for LV-journals. Setup: 2 servers x 4 OSD (SATA HDD + journal on SSD LV) 12.2.5, filestore cluster: id: ce305aae-4c56-41ec-be54-529b05eb45ed health: HEALTH_OK services: mon: 2

Re: [ceph-users] SSD OSDs crashing after upgrade to 12.2.7

2018-09-04 Thread Wolfgang Lendl
is downgrading from 12.2.7 to 12.2.5 an option? - I'm still suffering from high frequent osd crashes. my hopes are with 12.2.9 - but hope wasn't always my best strategy br wolfgang On 2018-08-30 19:18, Alfredo Deza wrote: > On Thu, Aug 30, 2018 at 5:24 AM, Wolfgang Lendl > wrote: >> Hi Alfredo,

[ceph-users] Degraded data redundancy: NUM pgs undersized

2018-09-04 Thread Jörg Kastning
Good morning folks, As a newbie to Ceph yesterday was the first time I've configured my CRUSH map, added a CRUSH rule and created my first pool using this rule. Since then I get the status HEALTH_WARN with the following output: ~~~ $ sudo ceph status cluster: id:

Re: [ceph-users] No announce for 12.2.8 / available in repositories

2018-09-04 Thread Linh Vu
We're going to reproduce this again in testing (12.2.8 drops right between our previous testing and going production) and compare it to 12.2.7. Will update with our findings soon. :) From: Dan van der Ster Sent: Tuesday, 4 September 2018 3:41:01 PM To: Linh Vu