Hi Yuri, On Thu, Feb 12, 2026 at 9:30 PM Yuri Weinstein via ceph-users <[email protected]> wrote: > > Still seeking approvals for: > > (smoke - FAILED, was reviewed and approved by Laura and Patrick, see Slack > thread for details) > rados - FAILED, Afreen, Travis? ( Laura approved for core) > > rgw - FAILED, Adam E approved? > > (rbd, krbd - FAILED, Ilya approved) > fs - FAILED, Venky approved?
I wasn't able to finish reviewing the run and teuthology results are unavailable right now due to the lab outage. > orch - FAILED, Adam K approved? > > upgrade/pacific-x (reef) - FAILED Laura, approved? > upgrade/quincy-x (reef) - FAILED Laura, approved? > upgrade/reef-p2p - FAILED Laura, approved? > > (powercycle - FAILED Brad, approved) > ceph-volume - Approved Guillaume? > > On Wed, Feb 11, 2026 at 12:10 PM Laura Flores via Dev <[email protected]> wrote: > > > The rados suite is approved: > > https://tracker.ceph.com/projects/rados/wiki/REEF#reef-v1828-httpstrackercephcomissues73906note-1 > > > > On Tue, Feb 10, 2026 at 6:33 PM Brad Hubbard via Dev <[email protected]> wrote: > > > >> Powercycle approved. > >> > >> On Tue, Feb 10, 2026 at 10:35 AM Brad Hubbard via Dev <[email protected]> > >> wrote: > >> > >>> > >>> > >>> On Fri, Feb 6, 2026 at 12:36 PM Brad Hubbard <[email protected]> > >>> wrote: > >>> > >>>> > >>>> > >>>> On Fri, Feb 6, 2026 at 8:33 AM Brad Hubbard via ceph-users < > >>>> [email protected]> wrote: > >>>> > >>>>> On Fri, Feb 6, 2026 at 7:59 AM Yuri Weinstein via ceph-users < > >>>>> [email protected]> wrote: > >>>>> > >>>>> > We are continuing on this release. > >>>>> > > >>>>> > > >>>>> > powercycle - FAILED Brad, approved? > >>>>> > > >>>>> > >>>>> Unfortunately, I don't see any logs anywhere where I have access to > >>>>> them so > >>>>> I have no way of checking this? > >>>>> > >>>> > >>>> OK, I have the latest logs. The three failing jobs appear to have all > >>>> timed out after running for about six hours. At this time I don't know > >>>> why > >>>> but I continue to investigate. > >>>> > >>> > >>> One point of commonality of the three tests that failed is that they are > >>> all running tests using ceph fuse. I'll reach out to the FS guys to see if > >>> they can help me get to the bottom of these failures. > >>> > >>> > >>>> > >>>> > >>>>> > >>>>> > >>>>> > ceph-volume - Approved Guillaume? > >>>>> > > >>>>> > Patrick was reviewing some tests, please see his comments in the same > >>>>> > tracker. > >>>>> > TIA > >>>>> > > >>>>> > > >>>>> > On Sat, Dec 13, 2025 at 1:19 AM Ilya Dryomov via ceph-users < > >>>>> > [email protected]> wrote: > >>>>> > > >>>>> > > On Mon, Dec 8, 2025 at 11:31 PM Yuri Weinstein via Dev < > >>>>> [email protected]> > >>>>> > > wrote: > >>>>> > > > > >>>>> > > > This test branch was built loner then 2.5 weeks ago > >>>>> > > > I have to rebuild it first and then will schedule krbd > >>>>> > > > >>>>> > > krbd approved based on > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > >>>>> https://pulpito.ceph.com/yuriw-2025-12-10_21:26:56-krbd-reef-release-testing-default-smithi/ > >>>>> > > > >>>>> > > > >>>>> > > >>>>> https://pulpito.ceph.com/yuriw-2025-12-12_18:20:32-krbd-reef-release-testing-default-smithi/ > >>>>> > > > >>>>> > > Thanks, > >>>>> > > > >>>>> > > Ilya > >>>>> > > > >>>>> > > > > >>>>> > > > On Mon, Dec 8, 2025 at 1:39 PM Ilya Dryomov via ceph-users < > >>>>> > > [email protected]> wrote: > >>>>> > > >> > >>>>> > > >> On Mon, Dec 8, 2025 at 8:50 PM Ilya Dryomov <[email protected] > >>>>> > > >>>>> > wrote: > >>>>> > > >> > > >>>>> > > >> > On Mon, Dec 8, 2025 at 5:00 PM Venky Shankar via ceph-users > >>>>> > > >> > <[email protected]> wrote: > >>>>> > > >> > > > >>>>> > > >> > > Hi Yuri, > >>>>> > > >> > > > >>>>> > > >> > > On Fri, Nov 21, 2025 at 4:34 AM Yuri Weinstein < > >>>>> > [email protected]> > >>>>> > > wrote: > >>>>> > > >> > > > > >>>>> > > >> > > > Note: this is an attempt to test and publish post EOL > >>>>> reef > >>>>> > point > >>>>> > > release > >>>>> > > >> > > > > >>>>> > > >> > > > Details of this release are summarized here: > >>>>> > > >> > > > > >>>>> > > >> > > > https://tracker.ceph.com/issues/73906#note-1 > >>>>> > > >> > > > Release Notes - TBD > >>>>> > > >> > > > LRC upgrade - N/A > >>>>> > > >> > > > > >>>>> > > >> > > > distro => jammy focal centos9 windows bookworm > >>>>> > > >> > > > > >>>>> > > >> > > > Seeking approvals/reviews for: > >>>>> > > >> > > > > >>>>> > > >> > > > smoke - Laura approved? > >>>>> > > >> > > > > >>>>> > > >> > > > rados - Radek, Laura approved? Travis? Nizamudeen? Adam > >>>>> King > >>>>> > > approved? > >>>>> > > >> > > > (rados is also running on build 2 == build 1 + > >>>>> > > >> > > > https://github.com/ceph/ceph/pull/66056 per Igor's > >>>>> request) > >>>>> > > >> > > > > >>>>> > > >> > > > rgw - Adam E approved? > >>>>> > > >> > > > > >>>>> > > >> > > > fs - Venky approved? > >>>>> > > >> > > > >>>>> > > >> > > There are lot many failures in the fs suite resembling > >>>>> > > >> > > > >>>>> > > >> > > Error mounting /var/lib/ceph: mount: /var/lib/ceph: > >>>>> wrong > >>>>> > fs > >>>>> > > >> > > type, bad option ... > >>>>> > > >> > > > >>>>> > > >> > > which is an issue with the testing kernel. Adding Alex who > >>>>> was > >>>>> > > looking > >>>>> > > >> > > into this issue. > >>>>> > > >> > > > >>>>> > > >> > > As far as the run validation is concerned, it's risky to > >>>>> ignore > >>>>> > the > >>>>> > > >> > > large number of failures. We have two options: > >>>>> > > >> > > > >>>>> > > >> > > - fix the testing kernel issue and rerun > >>>>> > > >> > > - rerun with `for-linus` branch (with `-k for-linus` in the > >>>>> > > >> > > teuthology-suite schedule command) > >>>>> > > >> > > > >>>>> > > >> > > The latter would be using an older kernel branch, but I > >>>>> think it > >>>>> > > >> > > should suffice for this case. > >>>>> > > >> > > >>>>> > > >> > Hi Venky, > >>>>> > > >> > > >>>>> > > >> > With your report of the fs suite being affected as well, I > >>>>> carved > >>>>> > out > >>>>> > > >> > some time to take a look. > >>>>> > > >> > > >>>>> > > >> > I think it's happening because XFS maintainers disabled > >>>>> support for > >>>>> > V4 > >>>>> > > >> > filesystems that was deprecated 5 years ago -- as of 6.18 > >>>>> kernel > >>>>> > it's > >>>>> > > >> > no longer compiled in by default. The proper fix would be to > >>>>> create > >>>>> > > an > >>>>> > > >> > image with/move to V5 on-disk format; I have opened [1] to > >>>>> try to > >>>>> > > >> > achieve that. We can also re-enable V4 on-disk format in our > >>>>> kernel > >>>>> > > >> > builds as a workaround. > >>>>> > > >> > > >>>>> > > >> > [1] https://github.com/ceph/ceph-cm-ansible/pull/799 > >>>>> > > >> > >>>>> > > >> This should be fixed now. Yuri, please schedule a rerun for > >>>>> krbd. > >>>>> > > >> > >>>>> > > >> Thanks, > >>>>> > > >> > >>>>> > > >> Ilya > >>>>> > > >> _______________________________________________ > >>>>> > > >> ceph-users mailing list -- [email protected] > >>>>> > > >> To unsubscribe send an email to [email protected] > >>>>> > > > > >>>>> > > > _______________________________________________ > >>>>> > > > Dev mailing list -- [email protected] > >>>>> > > > To unsubscribe send an email to [email protected] > >>>>> > > _______________________________________________ > >>>>> > > ceph-users mailing list -- [email protected] > >>>>> > > To unsubscribe send an email to [email protected] > >>>>> > > > >>>>> > _______________________________________________ > >>>>> > ceph-users mailing list -- [email protected] > >>>>> > To unsubscribe send an email to [email protected] > >>>>> > > >>>>> > >>>>> > >>>>> -- > >>>>> Cheers, > >>>>> Brad > >>>>> _______________________________________________ > >>>>> ceph-users mailing list -- [email protected] > >>>>> To unsubscribe send an email to [email protected] > >>>>> > >>>> > >>>> > >>>> -- > >>>> Cheers, > >>>> Brad > >>>> > >>> > >>> > >>> -- > >>> Cheers, > >>> Brad > >>> _______________________________________________ > >>> Dev mailing list -- [email protected] > >>> To unsubscribe send an email to [email protected] > >>> > >> > >> > >> -- > >> Cheers, > >> Brad > >> _______________________________________________ > >> Dev mailing list -- [email protected] > >> To unsubscribe send an email to [email protected] > >> > > > > > > -- > > > > Laura Flores > > > > She/Her/Hers > > > > Software Engineer, Ceph Storage <https://ceph.io> > > > > Chicago, IL > > > > [email protected] | [email protected] <[email protected]> > > M: +17087388804 > > > > > > _______________________________________________ > > Dev mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] -- Cheers, Venky _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
