Powercycle approved. On Tue, Feb 10, 2026 at 10:35 AM Brad Hubbard via Dev <[email protected]> wrote:
> > > On Fri, Feb 6, 2026 at 12:36 PM Brad Hubbard <[email protected]> wrote: > >> >> >> On Fri, Feb 6, 2026 at 8:33 AM Brad Hubbard via ceph-users < >> [email protected]> wrote: >> >>> On Fri, Feb 6, 2026 at 7:59 AM Yuri Weinstein via ceph-users < >>> [email protected]> wrote: >>> >>> > We are continuing on this release. >>> > >>> > >>> > powercycle - FAILED Brad, approved? >>> > >>> >>> Unfortunately, I don't see any logs anywhere where I have access to them >>> so >>> I have no way of checking this? >>> >> >> OK, I have the latest logs. The three failing jobs appear to have all >> timed out after running for about six hours. At this time I don't know why >> but I continue to investigate. >> > > One point of commonality of the three tests that failed is that they are > all running tests using ceph fuse. I'll reach out to the FS guys to see if > they can help me get to the bottom of these failures. > > >> >> >>> >>> >>> > ceph-volume - Approved Guillaume? >>> > >>> > Patrick was reviewing some tests, please see his comments in the same >>> > tracker. >>> > TIA >>> > >>> > >>> > On Sat, Dec 13, 2025 at 1:19 AM Ilya Dryomov via ceph-users < >>> > [email protected]> wrote: >>> > >>> > > On Mon, Dec 8, 2025 at 11:31 PM Yuri Weinstein via Dev <[email protected]> >>> > > wrote: >>> > > > >>> > > > This test branch was built loner then 2.5 weeks ago >>> > > > I have to rebuild it first and then will schedule krbd >>> > > >>> > > krbd approved based on >>> > > >>> > > >>> > > >>> > >>> https://pulpito.ceph.com/yuriw-2025-12-10_21:26:56-krbd-reef-release-testing-default-smithi/ >>> > > >>> > > >>> > >>> https://pulpito.ceph.com/yuriw-2025-12-12_18:20:32-krbd-reef-release-testing-default-smithi/ >>> > > >>> > > Thanks, >>> > > >>> > > Ilya >>> > > >>> > > > >>> > > > On Mon, Dec 8, 2025 at 1:39 PM Ilya Dryomov via ceph-users < >>> > > [email protected]> wrote: >>> > > >> >>> > > >> On Mon, Dec 8, 2025 at 8:50 PM Ilya Dryomov <[email protected]> >>> > wrote: >>> > > >> > >>> > > >> > On Mon, Dec 8, 2025 at 5:00 PM Venky Shankar via ceph-users >>> > > >> > <[email protected]> wrote: >>> > > >> > > >>> > > >> > > Hi Yuri, >>> > > >> > > >>> > > >> > > On Fri, Nov 21, 2025 at 4:34 AM Yuri Weinstein < >>> > [email protected]> >>> > > wrote: >>> > > >> > > > >>> > > >> > > > Note: this is an attempt to test and publish post EOL reef >>> > point >>> > > release >>> > > >> > > > >>> > > >> > > > Details of this release are summarized here: >>> > > >> > > > >>> > > >> > > > https://tracker.ceph.com/issues/73906#note-1 >>> > > >> > > > Release Notes - TBD >>> > > >> > > > LRC upgrade - N/A >>> > > >> > > > >>> > > >> > > > distro => jammy focal centos9 windows bookworm >>> > > >> > > > >>> > > >> > > > Seeking approvals/reviews for: >>> > > >> > > > >>> > > >> > > > smoke - Laura approved? >>> > > >> > > > >>> > > >> > > > rados - Radek, Laura approved? Travis? Nizamudeen? Adam >>> King >>> > > approved? >>> > > >> > > > (rados is also running on build 2 == build 1 + >>> > > >> > > > https://github.com/ceph/ceph/pull/66056 per Igor's request) >>> > > >> > > > >>> > > >> > > > rgw - Adam E approved? >>> > > >> > > > >>> > > >> > > > fs - Venky approved? >>> > > >> > > >>> > > >> > > There are lot many failures in the fs suite resembling >>> > > >> > > >>> > > >> > > Error mounting /var/lib/ceph: mount: /var/lib/ceph: >>> wrong >>> > fs >>> > > >> > > type, bad option ... >>> > > >> > > >>> > > >> > > which is an issue with the testing kernel. Adding Alex who was >>> > > looking >>> > > >> > > into this issue. >>> > > >> > > >>> > > >> > > As far as the run validation is concerned, it's risky to >>> ignore >>> > the >>> > > >> > > large number of failures. We have two options: >>> > > >> > > >>> > > >> > > - fix the testing kernel issue and rerun >>> > > >> > > - rerun with `for-linus` branch (with `-k for-linus` in the >>> > > >> > > teuthology-suite schedule command) >>> > > >> > > >>> > > >> > > The latter would be using an older kernel branch, but I think >>> it >>> > > >> > > should suffice for this case. >>> > > >> > >>> > > >> > Hi Venky, >>> > > >> > >>> > > >> > With your report of the fs suite being affected as well, I >>> carved >>> > out >>> > > >> > some time to take a look. >>> > > >> > >>> > > >> > I think it's happening because XFS maintainers disabled support >>> for >>> > V4 >>> > > >> > filesystems that was deprecated 5 years ago -- as of 6.18 kernel >>> > it's >>> > > >> > no longer compiled in by default. The proper fix would be to >>> create >>> > > an >>> > > >> > image with/move to V5 on-disk format; I have opened [1] to try >>> to >>> > > >> > achieve that. We can also re-enable V4 on-disk format in our >>> kernel >>> > > >> > builds as a workaround. >>> > > >> > >>> > > >> > [1] https://github.com/ceph/ceph-cm-ansible/pull/799 >>> > > >> >>> > > >> This should be fixed now. Yuri, please schedule a rerun for krbd. >>> > > >> >>> > > >> Thanks, >>> > > >> >>> > > >> Ilya >>> > > >> _______________________________________________ >>> > > >> ceph-users mailing list -- [email protected] >>> > > >> To unsubscribe send an email to [email protected] >>> > > > >>> > > > _______________________________________________ >>> > > > Dev mailing list -- [email protected] >>> > > > To unsubscribe send an email to [email protected] >>> > > _______________________________________________ >>> > > ceph-users mailing list -- [email protected] >>> > > To unsubscribe send an email to [email protected] >>> > > >>> > _______________________________________________ >>> > ceph-users mailing list -- [email protected] >>> > To unsubscribe send an email to [email protected] >>> > >>> >>> >>> -- >>> Cheers, >>> Brad >>> _______________________________________________ >>> ceph-users mailing list -- [email protected] >>> To unsubscribe send an email to [email protected] >>> >> >> >> -- >> Cheers, >> Brad >> > > > -- > Cheers, > Brad > _______________________________________________ > Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > -- Cheers, Brad _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
