ything has been good until this. No other
services seem to be suffering from this and it even appears that the
MDS works okay even with these messages. We would like to figure out
how to resolve this.
Thank you,
Robert LeBlanc
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904
s helped a couple of weeks ago when `ceph -s`
would cause 100% CPU and lock up the service much worse than this) and
didn't see the high CPU load. So I'm guessing it's triggered from some
external source.
I'm happy to provide more info, just let me know what would be helpful.
Than
On Thu, Apr 8, 2021 at 10:22 AM Robert LeBlanc wrote:
>
> I upgraded our Luminous cluster to Nautilus a couple of weeks ago and
> converted the last batch of FileStore OSDs to BlueStore about 36 hours ago.
> Yesterday our monitor cluster went nuts and started constantly calling
On Thu, Apr 8, 2021 at 11:24 AM Robert LeBlanc wrote:
>
> On Thu, Apr 8, 2021 at 10:22 AM Robert LeBlanc wrote:
> >
> > I upgraded our Luminous cluster to Nautilus a couple of weeks ago and
> > converted the last batch of FileStore OSDs to BlueStore about 36 hours
mon_lease = 30
mon_osd_cache_size = 20
mon_sync_max_payload_size = 4096
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu, Apr 8, 2021 at 1:11 PM Stefan Kooman wrote:
>
> On 4/8/21 6:22 PM, Robert LeBlanc wrote:
> > I upgraded our Lum
Good thought. The storage for the monitor data is a RAID-0 over three
NVMe devices. Watching iostat, they are completely idle, maybe 0.8% to
1.4% for a second every minute or so.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu, Apr 8, 2021
I'm attempting to deep scrub all the PGs to see if that helps clear up
some accounting issues, but that's going to take a really long time on
2PB of data.
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu, Apr 8, 2021 at 9:48
to fail over to it so I took it back out. I have run `ceph
osd-require-osd-release nautilus` after the upgrade of all the OSDs.
I'll go back and double check all the steps, but I think I got them
all.
Thank you,
Robert LeBlanc
___
ceph-users mail
The only step not yet taken was to move to straw2. That was the last
step we were going to do next.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Fri, Apr 9, 2021 at 10:41 AM Robert LeBlanc wrote:
>
> On Fri, Apr 9, 2021 at 9:25 AM
dex.php/s/OtHsBAYN9r5eSbU
You can look near the end of the log for the verbose logging. I'm not
sure what to look for in there, nothing sticks out to me. I did
disable cephx in the config file to see if that would help, but we
still have the 100% C
ught may help get `ceph df` working properly so we know how
much storage is actually being used.
We do have crons running to get metrics into graphite and some of them
do ceph commands, others are creating/writing/reading/deleting files
to get some file system performanc
On Fri, Apr 9, 2021 at 2:04 PM Dan van der Ster wrote:
>
> On Fri, Apr 9, 2021 at 9:37 PM Dan van der Ster wrote:
> >
> > On Fri, Apr 9, 2021 at 8:39 PM Robert LeBlanc wrote:
> > >
> > > On Fri, Apr 9, 2021 at 11:49 AM Dan van der Ster
> > >
elections. Thank you for helping us track down the
bad clients out of over 2,000 clients.
> > Maybe if that code path isn't needed in Nautilus it can be removed in
> > the next point release?
>
> I think there were other major changes in this area that might make
> such a back
past
dead), but a good chunk of machines are Ubuntu 14.04 not by choice.
Getting this upgrade done was sorely needed, but very risky at the
same time.
Thanks,
Robert LeBlanc
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an
hink the toolchain has evolved too much since then?
Thanks,
Robert LeBlanc
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
"6242'1003",
"last_deep_scrub_stamp": "2022-03-14 22:53:26.138458",
"last_clean_scrub_stamp": "2022-03-14 22:53:26.138458",
"log_size": 1003,
"ondisk_log_size": 1003,
again. I had exported the first PG from the up set, but didn't wind
up needing it so I didn't do it on the rest.
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Tue, Mar 15, 2022 at 10:26 AM Robert LeBlanc
wrote:
> We had a host
ERS
<https://docs.ceph.com/docs/nautilus/cephfs/kernel-features/#multiple-active-metadata-servers>
The feature has been supported since the Luminous release. It is
recommended to use Linux kernel clients >= 4.14 when there are multiple
active MDS.
Thank you,
Robert LeBlanc
---
evice_class: 'hdd'
- data: 'blk-09'
data_vg: 'ceph-blk-09'
db: 'db-09'
db_vg: 'ceph-db-00'
crush_device_class: 'hdd'
- data: 'blk-10'
data_vg: 'ceph-blk-10'
Yes, but they are just LVs, so you can not create them or delete them
easily so that it returns the space to the VG for something else.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Tue, Apr 28, 2020 at 6:55 PM Szabo, Istvan (Agoda
ple to show
the worst case.
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Wed, Apr 29, 2020 at 8:17 PM Szabo, Istvan (Agoda) <
istvan.sz...@agoda.com> wrote:
> Ok, so this is how it works with LVM.
>
>
>
> I was pla
Thanks guys. We are so close to the edge that we may just take that chance,
usually the only reason an active client has to reconnect is because we
have to bounce the MDS because it's overwhelmed.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62
hanks
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Fri, May 1, 2020 at 7:37 PM Robert LeBlanc wrote:
> Thanks guys. We are so close to the edge that we may just take that
> chance, usually the only reason an active client has to reconnect is
&
tubborn
clients release their caps?
Thanks,
Robert LeBlanc
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, May 4, 2020 at 7:51 PM Gregory Farnum wrote:
> On Sat, May 2, 2020 at 3:12 PM Robert LeBlanc
> wrote:
> >
> > If ther
On Wed, May 6, 2020 at 2:45 PM Patrick Donnelly wrote:
> On Wed, Mar 11, 2020 at 10:41 PM Robert LeBlanc
> wrote:
> >
> > This is the second time this happened in a couple of weeks. The MDS locks
> > up and the stand-by can't take over so the Montiors black list
thub.com/ceph/ceph/pull/34571 may help if "ceph daemon mds.a
> dump_mempools" shows buffer_anon uses lots of memory.
>
We struggle with cache management as well and I just thought it was due to
our really old kernel clients (I'm sure that doesn't help). I'
On Thu, May 7, 2020 at 9:41 AM Robert LeBlanc wrote:
> On Thu, May 7, 2020 at 6:22 AM Yan, Zheng wrote:
>
>> On Thu, May 7, 2020 at 1:27 AM Patrick Donnelly
>> wrote:
>> >
>> > Hello Robert,
>> >
>> > On Mon, Mar 9, 2020 at 7:55 PM Rober
on. I spent a
lot of time studying the issue to come up with WPQ and saw it work great
when I switched this cluster from PRIO to WPQ. I've also spent countless
hours studying how it's changed in Nautilus.
Thank you,
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4
We are using high and the people on the list that have also changed
have not seen the improvements that I would expect.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Wed, May 20, 2020 at 1:38 AM Dan van der Ster wrote:
>
> Hi
Adding the right dev list.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Wed, May 20, 2020 at 12:40 AM Robert LeBlanc wrote:
>
> We upgraded our Jewel cluster to Nautilus a few months ago and I've noticed
> that op behavi
VMe drives for RocksDB and then an NVMe pool for CephFS metadata. The
problem with some of these configurations is that the tools don't set
them up well, so I did the partitioning/LVM setup with a script I
wrote, then just had Ceph use the final partition/LV.
Hope that helps,
there. The LVM layer generally is pretty light and you
usually get more benefits than performance hit such relocating and
extending devices online. Sorry, I can't talk to how cephadm operates.
Robert LeBlanc
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3B
st as
the slowest disk), but could help with the VM with the SCSI resets.
Put this in your ceph.conf and restart your OSDs.
osd op queue = wpq
osd op queue cut off = high
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Tue, Aug 13, 2019 at 6:
when one drive was at a constant 100%.
Did you notice any improvement in performance or system utilization in
other ways?
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu, Aug 15, 2019 at 3:49 PM Rich Bade wrote:
> Unfortunately the sc
btrfs/zfs volumes that send/receive each other. I really don't think Ceph
is a good fit.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu, Aug 15, 2019 at 12:37 AM Lorenz Kiefner
wrote:
> Oh no, it's not that bad. It's
&
look.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Sat, Aug 17, 2019 at 10:26 AM Lorenz Kiefner
wrote:
> Hello again,
>
> all links are at least 10/50 mbit upstream/downstream, mostly 40/100 mbit,
> with some VMs at hosting compa
The WPQ scheduler may help your clients back off when things get busy.
Put this in your ceph.conf and restart your OSDs.
osd op queue = wpq
osd op queue cut off = high
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Fri, Aug 23, 2019 at 5:03
We have separate public and cluster networks, but on the same physical
switches with different VLANs.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Fri, Aug 23, 2019 at 1:29 PM wrote:
> Frank;
>
> Just out of curiosity; do
. The client latencies and
backfill pain that my co-workers experienced on a daily basis have been all
alleviated since moving to WPQ.
Honestly, WPQ may do what you need without having to try to configure QoS.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB
High should be the default with WPQ.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Aug 26, 2019 at 10:44 AM Paul Emmerich
wrote:
> WPQ has been the default queue for quite some time now (Luminous?).
>
> However, the defaul
If it is the default, then the documentation should be updated. [0]
[0]
https://docs.ceph.com/docs/master/rados/configuration/osd-config-ref/?highlight=wpq#operations
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Aug 26, 2019 at 1:22
more distributed instead of doing a reweight, move the
PGs which take a long time to just rinse and repeat over and over again.
Thanks,
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ce
nt the client realizes that the storage is 'slow' and starts throttling
the traffic and will then match the speed at which the HDDs can perform the
work. If you run the job for a long time, are you still seeing the trimming
errors, or just a steady rise in blocked IO on the OSDs without any
7; go down as far as
8+2 for instance and still be protected if 4 hosts in a chassis went down.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
On Thu, Sep 5, 2019 at 2:12 AM wrote:
> As far as I could tell, there wasn't anything in the logs (at the default
> log levels), other than the forwarded request to the mons...
>
> (I did read the documentation, which is why I was using the long form...
> it's the inconsistency which is problemat
ing through all 700+ osds reweighting them.
Thanks,
Robert LeBlanc
----
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Sep 2, 2019 at 10:29 AM Paul Emmerich
wrote:
> crushtool can simulate mappings for you, it's help/man page shou
ipermail/ceph-large-ceph.com/attachments/20170110/11d6382f/attachment.py
[1]
https://github.com/ceph/ceph/blob/cdb8df13cb7c7242f95ebbf4e404ec0006504a9f/src/osd/OSDMap.cc#L3403
[2]
https://github.com/ceph/ceph/blob/f7376c0754ec700a0063bf14d0293cf31ad77798/src/osd/OSDMap.h#L768-L770
----
Robert
en seconds by ordinary users.
> Potentially with damaging effects. I guess something similar could be
> achieved with a modified rogue client.
>
> I would expect that a storage cluster should have basic self-defence
> mechanisms that prevent this kind of overload or DOS attack by throttl
We are very interested in the RGW Passthrough mentioned for Octupus. What's
the status and how can we help? We want to connect with AWS S3.
Thank you,
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62
high
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Sep 16, 2019 at 11:34 PM Thomas <74cmo...@gmail.com> wrote:
> Hi,
>
> I have defined pool hdd which is exclusively used by virtual disks of
> multiple KVMs / LXCs.
> Yest
, so probably a single copy (or two copies on different
tapes at the same time) when the object is created.
Bonus points for being near-line.
Thanks,
Robert LeBlanc
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
_
d all the objects that were PUT within a time frame, then read the
objects off the cluster and write them to tape. Maybe that's not as
easy as I'm thinking either.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
ited for WAL (I highly recommend
that for any HDD install) for each OSD, then carve out the rest as an
SSD pool that you can put the CephFS metadata pool on. I don't think
you would have a good experience with cache tier at that size.
However, you know your access patterns far bette
On Fri, Sep 20, 2019 at 5:41 AM Amudhan P wrote:
> I have already set "mon osd memory target to 1Gb" and I have set max-backfill
> from 1 to 8.
Reducing the number of backfills should reduce the amount of memory,
especially for EC pools.
--------
Robert LeBlanc
PGP
On Mon, Sep 23, 2019 at 10:51 AM Shawn A Kwang wrote:
>
> On 9/23/19 9:38 AM, Robert LeBlanc wrote:
> > On Wed, Sep 18, 2019 at 11:47 AM Shawn A Kwang wrote:
> >>
> >> We are planning our ceph architecture and I have a question:
> >>
> >> How shou
stripe size or object size would help, but
they you may have issues in other areas.
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send a
. That my
unblock things and allow things to start recovering.
Then you need to fix the too_full messages by reweighting some of the
full OSDs, deleting unneeded data or adding more OSDs.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
__
removed?
If you are writing objects while the PGs are undersized (host/osds
down), then it will have to sync those writes to the OSDs that were
down. This is the number of degraded objects.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
POSIX. If a client requests an exclusive write lock
than no one else will be able to get a write lock. This does not mean
that other clients will automatically receive a read-only lock, it
depends on the client if it will fall back to read-only or fail if it
can't get a write lock.
----
urnal,
but not sure if that will help in this situation.
The other issue I'm seeing is that some IO just gets stuck when the
OSDs are getting marked down and coming back through the cluster.
Thanks,
Robert LeBlanc
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E
On Wed, Oct 2, 2019 at 3:41 PM Paul Emmerich wrote:
>
> On Wed, Oct 2, 2019 at 10:56 PM Robert LeBlanc wrote:
> > Is there a way to have leveldb compact more frequently or cause it to
> > come up for air more frequently and respond to heartbeats and process
> > some
You have 4 OSDs that are near_full, and the errors seem to be pointed
to pg_create, possibly from a backfill. Ceph will stop backfills to
near_full osds. I'm guessing that is the cause of your blocked IO. Try
reweighting the full OSDs down to move PGs off them.
Robert LeBlan
ideas how to mitigate this? It seems that IOs are being scheduled
on a thread and if they get unlucky enough to be scheduled behind a
big IO, they are just stuck, in this case HAProxy could kick out the
backend before the IO is returned and it has to re-request it.
Thank you,
Robe
many RGW setups, we
> default to 8192 and 800MB
>
> Sometimes "ms async op threads" and "ms async max op threads" might
> help as well (we adjust them by default, but for other reasons)
Thanks, I'll look into these options and see if they help.
-
the PG replicatas off the HDDs.
Setting the following in /etc/ceph/ceph.conf on the OSDs and restarting the
OSDs before backfilling will reduce the impact of the backfills.
osd op queue = wpq
osd op queue cut off = high
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB
de it to a Luminous based one?
>
Only thing I can think that means is no upmap support, but you don't need
upmap. As long as you have the CLASS column, you can do the device classes.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
__
tand what is going on here and what should be the best course of
action.
Thank you,
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
On Tue, Oct 15, 2019 at 8:05 AM Robert LeBlanc wrote:
>
> On Mon, Oct 14, 2019 at 2:58 PM Paul Emmerich wrote:
> >
> > Could the 4 GB GET limit saturate the connection from rgw to Ceph?
> > Simple to test: just rate-limit the health check GET
>
> I don't think
On Wed, Oct 16, 2019 at 2:50 PM Paul Emmerich wrote:
>
> On Wed, Oct 16, 2019 at 11:23 PM Robert LeBlanc wrote:
> >
> > On Tue, Oct 15, 2019 at 8:05 AM Robert LeBlanc wrote:
> > >
> > > On Mon, Oct 14, 2019 at 2:58 PM Paul Emmerich
> > > wr
On Thu, Oct 17, 2019 at 2:50 AM Paul Emmerich wrote:
>
> On Thu, Oct 17, 2019 at 12:17 AM Robert LeBlanc wrote:
> >
> > On Wed, Oct 16, 2019 at 2:50 PM Paul Emmerich
> > wrote:
> > >
> > > On Wed, Oct 16, 2019 at 11:23 PM Robert LeBlanc
> > &g
512
If I match the two, do you think it would help prevent small IO from
being blocked by larger IO?
I'm also happy to look into the code to suggest improvements if you
can give me some quick points into the code to start will help.
Robert LeBlanc
PGP Fingerpri
On Thu, Oct 17, 2019 at 11:46 AM Casey Bodley wrote:
>
>
> On 10/17/19 12:59 PM, Robert LeBlanc wrote:
> > On Thu, Oct 17, 2019 at 9:22 AM Casey Bodley wrote:
> >
> >> With respect to this issue, civetweb and beast should behave the same.
> >> Both fronte
ads in the thread pool than max
concurrent requests so that there is a buffer of idle threads rather
than IOs being scheduled behind one another.
--------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-u
pull/27898. It
> extends the get_bucket_info() call to take optional_yield, and passes
> one in where available, using null_yield to mark the synchronous cases
> where one isn't available.
I'll work to get familiar with the code base and see if I can submit
some PRs to help out
version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be)
nautilus (stable)": 754,
"ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba)
nautilus (stable)": 22
}
}
I restarted one of the monitors and it dropped out of the list only
showing 2 blocked ops, but then showed up again a littl
On Fri, Nov 1, 2019 at 6:10 PM Robert LeBlanc wrote:
>
> We had an OSD host with 13 OSDs fail today and we have a weird blocked
> OP message that I can't understand. There are no OSDs with blocked
> ops, just `mon` (multiple times), and some of the rgw instances.
>
bout promoting
when there are X hits in Y time?
Thanks
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
t will cause peering again. All
these peerings could cause other OSDs to miss their heartbeats and the
problem only gets worse as it compounds. Server failure events should be
fairly infrequent that 100 seconds is a good compromise. You are able to
adjust the timeout, but I highly recommend you don&
t work, we have time
to test it and remove the caching if it doesn't work.
Thank you for your insights and I'm interested to hear what you think given
the additional info.
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
The second id is to overlay device types in the crush map. In the olden
days, we would define two seperate roots, one for hdd and another for SSD
(each host would have a duplicate entry with a slightly different name and
different id), then have a crush rule use the different root for the type
of s
actually gotten to the
implementation phase of this idea.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
rsion.
Is there a way to move OSDs from leveldb to rocksdb without trashing and
rebuilding all the osds?
[0] https://docs.ceph.com/docs/master/releases/jewel/
Thanks,
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62
let that run for a few
days on one host before converting the others.
I really hope this helps with the heavy write/delete load kicking out OSDs.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-
dmin reshard list
2020-03-02 05:31:40.875642 7fdb983f7e40 0 failed reading realm info: ret
-1 (1) Operation not permitted
Any nudge in the right direction would be helpful. I manually sharded the
indexes, but I'd really like to have it done automatically from now on so I
don't have
"data_pool": "default.rgw.buckets.data",
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0,
"compression": ""
}
}
],
"metadata_heap": "",
"tier_config": [],
"realm_id": ""
}
```
Also the command would not work without the `--rgw-zone default` option,
otherwise I'd get:
```
$ radosgw-admin zone get
unable to initialize zone: (2) No such file or directory
```
Thank you,
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
I don't know if it is that specifically, but they are all are running the
latest version of Luminous and I set the cluster to only allow Luminous
OSDs in. All services have been upgraded to Luminous. Do I need to run a
command to activate the cls_rgw API?
Robert LeBlan
a pg dump, it looks like the epoch is passed that.
$ ceph pg map 3.756
osdmap e234953 pg 3.756 (3.756) -> up [113,180,115] acting [113,180,115]
Last time, it seemed to just recover after about an hour all by it's self.
Any way to speed this up?
Thank you,
Robert LeBlanc
--
the MDS got way too behind on segments ~14,000 from some naughty clients
and caused the journal to explode and the MDS to eventually just not
respond to the monitors.
Thank you,
Robert LeBlanc
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Thu
d\.(\d+)$'
Thanks,
Robert LeBlanc
----
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
We are already gathering the Ceph admin socket stats with the Diamond
plugin and sending that to graphite, so I guess I just need to look through
that to find what I'm looking for.
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Fri, M
90 matches
Mail list logo
