Hello, If you're seeing this error, it means that there are clients in your cluster that do not support Reef features, which `pg-upmap-primary` is. To get rid of the errors, you have to change the balancer back to upmap mode, then remove the pg-upmap-primary mappings:
ceph balancer mode upmap ceph osd rm-pg-upmap-primary-all This behavior is expected when older clients try to connect to the cluster with newer (in this case, Reef) features. I suspect that you enabled upmap-read mode before mounting the filesystem, since enabling upmap-read mode requires you to set "require-min-compat-client=reef", which is basically a gate to prevent older clients from connecting to a cluster with Reef features. Can you confirm that this was the order of operations? Also, please confirm that running the above commands gets rid of the errors. Thanks, Laura On Mon, Aug 11, 2025 at 2:38 PM Eugen Block <ebl...@nde.ag> wrote: > I haven't tried the upmap-read balancer myself, but have you removed > the upmap entries before trying to revert? > > ceph osd rm-pg-upmap-primary-all > > > https://docs.ceph.com/en/reef/rados/operations/read-balancer/#removing-pg-upmap-primary-mappings > > Zitat von quag...@bol.com.br: > > > I enabled the upmap-read balancemant, but in my environment, it is no > > longer possible to mount a ceph filesystem. I tried to revert to the old > > settings, but I get the following message in logs: > > > > [Mon Aug 11 11:42:05 2025] libceph: mon2 (1)192.168.193.153:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:05 2025] libceph: mon2 (1)192.168.193.153:6789 missing > > required protocol features > > [Mon Aug 11 11:42:06 2025] libceph: mon2 (1)192.168.193.153:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:06 2025] libceph: mon2 (1)192.168.193.153:6789 missing > > required protocol features > > [Mon Aug 11 11:42:06 2025] libceph: mon0 (1)192.168.193.151:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:06 2025] libceph: mon0 (1)192.168.193.151:6789 missing > > required protocol features > > [Mon Aug 11 11:42:06 2025] libceph: mon2 (1)192.168.193.153:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:06 2025] libceph: mon2 (1)192.168.193.153:6789 missing > > required protocol features > > [Mon Aug 11 11:42:07 2025] ceph: No mds server is up or the cluster is > > laggy > > [Mon Aug 11 11:42:07 2025] ceph: No mds server is up or the cluster is > > laggy > > [Mon Aug 11 11:42:07 2025] ceph: No mds server is up or the cluster is > > laggy > > [Mon Aug 11 11:42:07 2025] libceph: mon2 (1)192.168.193.153:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:07 2025] libceph: mon2 (1)192.168.193.153:6789 missing > > required protocol features > > [Mon Aug 11 11:42:08 2025] libceph: mon1 (1)192.168.193.152:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:08 2025] libceph: mon1 (1)192.168.193.152:6789 missing > > required protocol features > > [Mon Aug 11 11:42:08 2025] libceph: mon1 (1)192.168.193.152:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:08 2025] libceph: mon1 (1)192.168.193.152:6789 missing > > required protocol features > > [Mon Aug 11 11:42:09 2025] libceph: mon1 (1)192.168.193.152:6789 feature > > set mismatch, my 2f018fb87aa4aafe < server's 2f018fb8faa4aafe, missing > > 80000000 > > [Mon Aug 11 11:42:09 2025] libceph: mon1 (1)192.168.193.152:6789 missing > > required protocol features > > > > I can't mount any more ceph filesystems. The ceph processes are up. > > > > # uname -a > > Linux k02 5.15.0-141-generic #151-Ubuntu SMP Sun May 18 21:35:19 UTC 2025 > > x86_64 x86_64 x86_64 GNU/Linux > > > > I've already tried to revert to the old settings, but when I try to do > so, > > I get the following alert: > > # ceph osd set-require-min-compat-client luminous > > Error EPERM: osdmap current utilizes features that require reef; cannot > set > > require_min_compat_client below that to luminous > > > > Could you help? Enviado através do Bol Mail[1] > > > > > > Links: > > ------ > > [1] https://produtos.uol.com.br/bol > > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflo...@ibm.com | lflo...@redhat.com <lflo...@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
