Hello, I have an object multisite setup. I'm on squid 19.2.3 (upgraded, not a greenfield deployment - both clusters are on the same builds), so I upgraded to notification_v2 as well. However, while the primary zone converted the notifications and topics to v2 easily, the secondary zone seems to have issues with this (topics and notifications are not synced to the secondary, but the rados objects are synced, shown via rados -p site2.rgw.log -N notif ls. The radosgw-admin topic list command also gives back the v1 format of the json output on the secondary zone, while having the v2 json output on the primary). What's (almost) certain is, that one of the clusters (the primary) wants to use v2, and the other the v1 notification style, and mixing these two causes issues (topics randomly breaking, etc). Also, since 19.2.3, in theory, if I create a topic on the secondary zone, the operation should be forwarded to the primary zone. While this happens, the topic creation still fails with an 503 error, while the topic still gets created. The rgw on the secondary site after this half-failed operation also spits out logs like: rgw debug 2025-11-26T11:51:14.241+0000 7ffa54948640 1 req 12442569609481506669 0.000000000s sns:pubsub_topic_create ERROR: topic not found rgw debug 2025-11-26T11:51:14.254+0000 7ffa2e8fc640 1 req 12442569609481506669 0.013000983s sns:pubsub_topic_create WARNING: topic migration in process. please try again later I tried to create another multisite setup (same clusters, different realm), and there the notification v2 scheme works without an issue. I think if I would delete the secondary zone and re-create it (including the pools), it would fix the issue, but since that operation has a relatively large blast radius (especially if something goes wrong, but there are some services that use only the secondary zone, so they should be redirected beforehand), I'm a bit hesitant to do it (before the christmas holidays). I also tried searching for these error messages on google, with no luck. I also tried searching for these error messages in the RGW source code, and still no luck. I can find them in the compiled RGW binary at least.
Any help would be greatly appreciated! I would be also very happy if I wouldn't have to nuke the secondary zone. Redards, György Lázár _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
