Hello Yuri,
On the RGW side I would very much like to get this [1] patch in that release
that is already merged in reef [2] and pacific [3].
Perhaps Casey can approve and merge that so you can bring it into your
testing.
Thanks!
[1] https://github.com/ceph/ceph/pull/53414
[2]
Thank you for your response, Igor.
Currently debug_rocksdb is set to 4/5:
# ceph config get osd debug_rocksdb
4/5
This setting seems to be default. Is my understanding correct that you're
suggesting setting it to 3/5 or even 0/5? Would setting it to 0/5 have any
negative effects on the cluster?
Hi Eugen,
warnings continue to spam cluster log.Actually for the whole picture of the
issue please see:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/VDL56J75FG5LO4ZECIWWGGBW4ULPZUIP/
I was thinking about the following options:
1, restart problematic nodes: 24,32,34,36: need to
Hi Yuri (resend it because I forgot to add ccs to MLs.)
2023年10月5日(木) 5:57 Yuri Weinstein :
Hello
We are getting very close to the next Quincy point release 17.2.7
Here is the list of must-have PRs https://pad.ceph.com/p/quincy_17.2.7_prs
We will start the release testing/review/approval
On Tue, Oct 03, 2023 at 06:10:17PM +0200, Matthias Ferdinand wrote:
> On Sun, Oct 01, 2023 at 12:00:58PM +0200, Peter Goron wrote:
> > Hi Matthias,
> >
> > One possible way to achieve your need is to set a quota on number of
> > buckets at user level (see
> >
Hello
We are getting very close to the next Quincy point release 17.2.7
Here is the list of must-have PRs https://pad.ceph.com/p/quincy_17.2.7_prs
We will start the release testing/review/approval process as soon as
all PRs from this list are merged.
If you see something missing please speak up
Hi Everyone,
I've been trying to get S3 Select working on our system and whenever I
send a query I get the following in the Payload (Result 200 from RGW):
# aws --endpoint-url http://cephtest1 s3api select-object-content
--bucket test1 --expression-type SQL --input-serialization '{"CSV":
Hi Zakhar,
do reduce rocksdb logging verbosity you might want to set debug_rocksdb
to 3 (or 0).
I presume it produces a significant part of the logging traffic.
Thanks,
Igor
On 04/10/2023 20:51, Zakhar Kirpichenko wrote:
Any input from anyone, please?
On Tue, 19 Sept 2023 at 09:01,
Any input from anyone, please?
On Tue, 19 Sept 2023 at 09:01, Zakhar Kirpichenko wrote:
> Hi,
>
> Our Ceph 16.2.x cluster managed by cephadm is logging a lot of very
> detailed messages, Ceph logs alone on hosts with monitors and several OSDs
> has already eaten through 50% of the endurance of
Hi folks,
I am aware that dynamic resharding isn't supported before Reef with multisite.
However, does manual resharding work? It doesn't seem to be so, either. First
of all, running "bucket reshard" has to be in the master zone. But if the
objects of that bucket isn't in the master zone,
> Tried a negative number ("--max-buckets=-1"), but that had no effect at
> all (not even an error message).
must have mistyped the command; trying again with "-max-buckets=-1", it
shows the wanted effect: user cannot create any bucket.
So, an effective and elegant method indeed :-)
Matthias
On Tue, Oct 03, 2023 at 06:10:17PM +0200, Matthias Ferdinand wrote:
> On Sun, Oct 01, 2023 at 12:00:58PM +0200, Peter Goron wrote:
> > Hi Matthias,
> >
> > One possible way to achieve your need is to set a quota on number of
> > buckets at user level (see
> >
Hi Ceph users and developers,
We are gearing up for the next User + Developer Monthly Meeting, happening
October 19th at 10am EST.
If you are interested in being a guest speaker, you are invited to submit a
focus topic to this Google form:
On Wed, Oct 4, 2023 at 7:19 PM Kushagr Gupta
wrote:
>
> Hi Milind,
>
> Thank you for your swift response.
>
> >>How many hours did you wait after the "start time" and decide to restart
> >>mgr ?
> We waited for ~3 days before restarting the mgr-service.
The only thing I can think of is a stale
Also found what the 2nd problem was:
When there are pools using the default replicated_ruleset while there are
multiple rulesets with differenct device classes, the autoscaler does not
produce any output.
Should I open a bug for that?
Am Mi., 4. Okt. 2023 um 14:36 Uhr schrieb Boris Behrens :
>
Hello Eugen,
yes, we followed the documentation and everything worked fine. The cache
is gone.
Removing the pool worked well. Everything is clean.
The PGs are empty active+clean.
Possible solutions:
1.
ceph pg {pg-id} mark_unfound_lost delete
I do not think this is the right way since it
Hi Milind,
Thank you for your swift response.
>>How many hours did you wait after the "start time" and decide to restart
mgr ?
We waited for ~3 days before restarting the mgr-service.
There was one more instance where we waited for 2 hours and then re-started
and in the third hour the schedule
Hi everybody,
I tried to reshard a bucket belonging to the tenant "test-tenant", but got an
"No such file or directory" error.
$ radosgw-admin reshard add --bucket test-tenant/test-bucket --num-shards 40
$ radosgw-admin reshard process
2023-10-04T12:12:52.470+0200 7f654237afc0 0
Hi Team,Milind
*Ceph-version:* Quincy, Reef
*OS:* Almalinux 8
*Issue:* snap_schedule works after 1 hour of schedule
*Description:*
We are currently working in a 3-node ceph cluster.
We are currently exploring the scheduled snapshot capability of the
ceph-mgr module.
To enable/configure
Found the bug for the TOO_MANY_PGS: https://tracker.ceph.com/issues/62986
But I am still not sure, why I don't have any output on that one cluster.
Am Mi., 4. Okt. 2023 um 14:08 Uhr schrieb Boris Behrens :
> Hi,
> I've just upgraded to our object storages to the latest pacific version
>
On Wed, Oct 4, 2023 at 3:40 PM Kushagr Gupta
wrote:
>
> Hi Team,Milind
>
> Ceph-version: Quincy, Reef
> OS: Almalinux 8
>
> Issue: snap_schedule works after 1 hour of schedule
>
> Description:
>
> We are currently working in a 3-node ceph cluster.
> We are currently exploring the scheduled
Hi,
I've just upgraded to our object storages to the latest pacific version
(16.2.14) and the autscaler is acting weird.
On one cluster it just shows nothing:
~# ceph osd pool autoscale-status
~#
On the other clusters it shows this when it is set to warn:
~# ceph health detail
...
[WRN]
Hi,
I just did this successfully on a test Reef cluster (no multi-site)
$ radosgw-admin object rewrite --bucket=bucket1 --object="myfile.txt"
where "--object" is the object name. The epoch and the tag have been
updated, so I guess it worked.
But I also got a segfault on a Octopus test
Hi,
I suspect the auth_allow_insecure_global_id_reclaim config option. If
you really need this to work you can set
$ ceph config set mon auth_allow_insecure_global_id_reclaim true
and the client should be able to connect. You will get a warning though:
mon is allowing insecure global_id
Hi,
is this still an issue? If so, I would try to either evict the client
via admin socket:
ceph tell mds.5 client evict [...] --- Evict client
session(s) based on a filter
alternatively locally on the MDS:
cephadm enter mds.
ceph daemon mds. client evict
or restart the MDS which
Hi,
we have often seen strange behavior and also interesting pg targets from
pg_autoscaler in the last years.
That's why we disable it globally.
The commands:
ceph osd reweight-by-utilization
ceph osd test-reweight-by-utilization
are from the time before the upmap balancer was introduced and
Hi,
you could change the target_max_misplaced_ratio to 1, the balancer has
a default 5% ratio of misplaced objects, see [1] for more information:
ceph config get mgr target_max_misplaced_ratio
0.05
[1] https://docs.ceph.com/en/latest/rados/operations/balancer/#throttling
Zitat von
Hi all,
I would like to follow up on this, it turns out that overwriting the
file doesn't actually hang, but is just super slow, like several
minutes. The process is busy in a syscall reading large amounts of what
I'm assuming is filesystem metadata until the operation finally completes.
Hi,
just for clarity, you're actually talking about the cache tier as
described in the docs [1]? And you followed the steps until 'ceph osd
tier remove cold-storage hot-storage' successfully? And the pool has
been really deleted successfully ('ceph osd pool ls detail')?
[1]
Hi,Please take a look at the following thread: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/PWHG6QJ6N2TJEYD2U4AXJAJ23CRPJG4E/#7ZMBM23GXYFIGY52ZWJDY5NUSYSDSYL6In short, the value for "osd_mclock_cost_per_byte_usec_hdd" isn't correct. With the release of 17.2.7 this option will be
Did you apply the changes to the containers.conf file on all hosts?
The MGR daemon is issuing the cephadm commands on the remote hosts, so
it would need that as well. That setup works for me quite well for
years now. What distro is your host running on? We mostly use openSUSE
or SLES, but
To help complete the recovery, you can temporarily try disabling scrub and
deep scrub
operations by running:
ceph osd set noscrub
ceph osd set nodeep-scrub
This should help speed up the recovery process. Once the recovery is done,
you
can unset the above scrub flags and revert the mClock profile
Hi Venky,
thanks for help on this. Will change to multimds with subtree pinning.
For the moment, it needs to get the segments list items go by loop of
expiring -> expired -> trimmed. It is observed that each problematic mds
has a few expiring segment stuck in the road of trimming. the segment
33 matches
Mail list logo
