[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
The regression that this patch fixes wasn't introduced (or backported to) Luminous by upstream. So this doesn't affect Bionic (confirmed by checking Ubuntu's latest Bionic source too). ** Changed in: ceph (Ubuntu Bionic) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Hi Pon, if you still need Bionic SRU for this one can you attach a debdiff for bionic. Thanks. ** Changed in: ceph (Ubuntu Bionic) Status: In Progress => New ** Changed in: cloud-archive Assignee: Ponnuvel Palaniyappan (pponnuvel) => (unassigned) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
This bug was fixed in the package ceph - 15.2.8-0ubuntu0.20.04.1~cloud0 --- ceph (15.2.8-0ubuntu0.20.04.1~cloud0) bionic-ussuri; urgency=medium . * New upstream release for the Ubuntu Cloud Archive. . ceph (15.2.8-0ubuntu0.20.04.1) focal; urgency=medium . [ Chris MacNaughton ] * New upstream point release (LP: #1912355): - d/rules,cephadm.install,librgw-dev.install,librgw2.install: Drop files no longer included in point release. * d/p/disable-log-slow-requests.patch: Remove logging every slow request details to monitors LP: #1909162). . [ Ponnuvel Palaniyappan ] * d/p/bug1911900-fix-scrub-blocking-balancer.patch: Prevent scrub from stopping balancer (LP: #1911900) ** Changed in: cloud-archive/ussuri Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
This bug was fixed in the package ceph - 15.2.8-0ubuntu0.20.04.1 --- ceph (15.2.8-0ubuntu0.20.04.1) focal; urgency=medium [ Chris MacNaughton ] * New upstream point release (LP: #1912355): - d/rules,cephadm.install,librgw-dev.install,librgw2.install: Drop files no longer included in point release. * d/p/disable-log-slow-requests.patch: Remove logging every slow request details to monitors LP: #1909162). [ Ponnuvel Palaniyappan ] * d/p/bug1911900-fix-scrub-blocking-balancer.patch: Prevent scrub from stopping balancer (LP: #1911900) -- Ponnuvel Palaniyappan Thu, 04 Feb 2021 11:28:51 + ** Changed in: ceph (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
This bug was fixed in the package ceph - 15.2.8-0ubuntu0.20.10.1 --- ceph (15.2.8-0ubuntu0.20.10.1) groovy; urgency=medium [ Chris MacNaughton ] * New upstream point release (LP: #1912355): - d/cephadm.install, d/librgw-dev.install, d/librgw2.install: Upstream point release removes files that were being installed. - d/rules: Remove installation of /etc/sudoers.d/cephadm as it is removed upstream. * d/p/disable-log-slow-requests.patch: Remove logging every slow request details to monitors LP: #1909162). [ Ponnuvel Palaniyappan ] * d/p/bug1911900-fix-scrub-blocking-balancer.patch: Prevent scrub from stopping balancer (LP: #1911900) -- Ponnuvel Palaniyappan Thu, 04 Feb 2021 11:18:13 + ** Changed in: ceph (Ubuntu Groovy) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Same steps followed for installing & test groovy ceph packages (15.2.8-0ubuntu0.20.10.1) ** Attachment added: "sru_groovy.txt" https://bugs.launchpad.net/cloud-archive/+bug/1911900/+attachment/5466340/+files/sru_groovy.txt ** Tags removed: verification-needed-groovy ** Tags added: verification-done-groovy ** Tags removed: verification-needed ** Tags added: verification-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
This is fix released in hirsute via 16.1.0 ** Changed in: ceph (Ubuntu Hirsute) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
@Ponnuvel, by any chance would you be able to verify this on groovy? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Ussuri packages are installed and verified (15.2.8-0ubuntu0.20.04.1~cloud0). In tests for focal and ussuri, rados bench was used to drive some I/O and randomly chosen PGs were repeated deeb-scrubbed in a loop (ceph pg deep-scrub ) to keep scrubbing process run at all times. ** Attachment added: "sru_ceph15.2.8_ussuri.txt" https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1911900/+attachment/5466042/+files/sru_ceph15.2.8_ussuri.txt ** Tags removed: verification-ussuri-needed ** Tags added: verification-ussuri-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
focal packages are installed & verified for ceph 15.2.8. Attached the steps used. ** Attachment added: "sru_focal_ceph15.2.8.txt" https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1911900/+attachment/5466037/+files/sru_focal_ceph15.2.8.txt ** Tags removed: verification-needed-focal ** Tags added: verification-done-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Hello Ponnuvel, or anyone else affected, Accepted ceph into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/15.2.8-0ubuntu0.20.10.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-groovy. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: ceph (Ubuntu Groovy) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-groovy ** Changed in: ceph (Ubuntu Focal) Status: Triaged => Fix Committed ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Thanks very much Ponnuvel. I've pushed these changes to the groovy and focal package branches. The builds are failing for unrelated 15.2.8 point release updates and I've asked Chris to take a look at those. Once the package gets uploaded to groovy/focal we'll need to subscribe ubuntu-sru to this bug. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Thanks, Corey, for catching the unintended change in the previous debdiff. I've corrected it and attached a new debdiff here. ** Patch added: "new-focal-octopus.debdiff" https://bugs.launchpad.net/cloud-archive/+bug/1911900/+attachment/5458823/+files/new-focal-octopus.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Description changed: [Impact] When scrubs are in progress, balancer stop due to the bug [0]. And shows: calc_pg_upmaps abort due to max <= 0 in the logs. Typically when deep-scrub is done in maintenance windows and can take few hours. If balancing is paused for the duration, it can affect client I/O performance later when balacing starts happening after deep-scrub is done. This bug was introduced in Octopus. We need to backport upstream bug [0] to just Octopus. It's been fixed in upstream master branch [1]. [Test Case] In an Octopus Ceph cluster that has some data (large enough to be able to notice balancing), take down one or more OSDs to introduce "unbalanced" objects. Make sure Ceph balancer module is enabled and active (which should be the default case in Octopus). Perform some I/O so that data goes to the rest of the OSDs. Then start deep-scrubbing and re-add the previously taken down so balancing start to happen. [Regression potential] Low potential. This is a bug fix of a previously correct code. If anything goes wrong, the balancer module might not function properly and thus leaving the cluster unbalanced and potentially requiring manual balancing. [Other Info] It's been accepted upstream and backported to Octopus. Ref [0] and [1]. - + [0] https://tracker.ceph.com/issues/48309 [1] https://github.com/ceph/ceph/pull/38337 + + + For Hirsute, James is working on a snapshot of Pacific and that should include the fix for this. ** Changed in: ceph (Ubuntu Hirsute) Status: New => Triaged ** Changed in: ceph (Ubuntu Groovy) Status: New => Triaged ** Changed in: ceph (Ubuntu Focal) Status: New => Triaged ** Changed in: cloud-archive/ussuri Status: New => Triaged ** Changed in: cloud-archive Status: In Progress => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
The attachment "focal-octopus.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
Attaching debdiff for Focal/Octopus. ** Changed in: ceph (Ubuntu Bionic) Status: New => In Progress ** Patch added: "focal-octopus.debdiff" https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1911900/+attachment/5458789/+files/focal-octopus.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Changed in: cloud-archive Importance: Undecided => Medium ** Changed in: cloud-archive/ussuri Importance: Undecided => Medium ** Changed in: ceph (Ubuntu Bionic) Importance: Undecided => Medium ** Changed in: ceph (Ubuntu Focal) Importance: Undecided => Medium ** Changed in: ceph (Ubuntu Groovy) Importance: Undecided => Medium ** Changed in: ceph (Ubuntu Hirsute) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Also affects: ceph (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Hirsute) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Tags added: sts-sru-needed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Also affects: cloud-archive/ussuri Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** No longer affects: ceph (Ubuntu Hirsute) ** No longer affects: ceph (Ubuntu Groovy) ** No longer affects: ceph (Ubuntu Focal) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1911900] Re: [SRU] Active scrub blocks upmap balancer
** Also affects: ceph (Ubuntu) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: ceph (Ubuntu Hirsute) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1911900 Title: [SRU] Active scrub blocks upmap balancer To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1911900/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
