+1 (non binding) to merge. Disclaimer: I worked on this feature (implementation and code review)
Zita Stephen O'Donnell via dev <[email protected]> ezt írta (időpont: 2026. okt. 5., H, 11:05): > Summit, > > I think your questions are covered in the design from this part > > https://github.com/apache/ozone/pull/9664/changes#diff-93aadee36203b3e50605117504fae31353edba5844c084e8169fde778c1c1a1cR239 > > Datanodes will continue to be available for writes during finalize. In > short they will operate at the lowest version in the pipeline and any > blocks currently being written will be written / committed at the earlier > version. > > Stephen. > > > On Mon, Oct 5, 2026 at 4:13 AM Sumit Agrawal via dev <[email protected] > > > wrote: > > > +1 for merge. > > > > Pipeline can exist in mixed version of DN (old and new). Design doc do > not > > provide much clarity about the behavior: > > - Weather write is supported in this case? allocating new blocks and > > pipeline > > - How about pipeline / block already assigned for write? Client will not > be > > aware of changes. > > > > Since SCM is already finalied and DN finalization is in progress (which > can > > take 1-2 HB for few set of DNs where SCM will know). So there may be a > > delay and DN upgrade may also take time. > > > > - How about DN supporting read and write operation when finalization is > in > > progress? is it blocking or request will be rejected by DNs ? > > > > > > On Thu, Oct 1, 2026 at 7:46 PM Andrey Yarovoy via dev < > > [email protected]> > > wrote: > > > > > +1 for merge. > > > > > > On Thu, Oct 1, 2026 at 10:01 AM Uma Maheswara Rao Gangumalla < > > > [email protected]> wrote: > > > > > > > Great work. Thank you Ethan and all others for working on this. > > > > > > > > +1 for the merge. > > > > > > > > Regards, > > > > Uma > > > > > > > > On Mon, Sep 28, 2026 at 4:02 PM Ethan Rose <[email protected]> wrote: > > > > > > > > > Hi Ozone devs, > > > > > > > > > > This is a vote thread to merge the zero downtime upgrade feature > > > branch ( > > > > > HDDS-14496-zdu) into master, which will enable zero downtime > upgrades > > > > from > > > > > the first release containing this feature to all future releases. > > > Please > > > > > familiarize yourself with the following documents when voting. All > > docs > > > > PRs > > > > > remain open so we can address questions and comments as they arise > > > during > > > > > the voting process. > > > > > > > > > > - Design document <https://github.com/apache/ozone/pull/9664> > > > > > - Branch merge checklist < > > > > https://github.com/apache/ozone-site/pull/562 > > > > > > > > > > > - User Docs <https://github.com/apache/ozone-site/pull/545> > > > > > - Developer docs <https://github.com/apache/ozone-site/pull/560 > >: > > > > This > > > > > guide covers how to handle compatibility for zero downtime > upgrade > > > and > > > > > should be understood by all developers and reviewers. > > > > > > > > > > Summary of changes: New Requirements for Developers > > > > > > > > > > *Once this branch is merged, all further commits must be ZDU > > compatible > > > > as > > > > > outlined in the developer guide*. > > > > > > > > > > The new versioning framework created on this branch provides tools > to > > > > > safely incorporate incompatible changes, but it does not > > automatically > > > > > resolve them. That remains the responsibility of developers and > > > > reviewers. > > > > > Improved Developer Experience > > > > > > > > > > Each component now uses a single ComponentVersion to track all > > > > incompatible > > > > > changes across disk and network as outlined in the design doc. > > > Developers > > > > > no longer need to reason about whether their incompatible change > > > > requires a > > > > > LayoutFeature, ComponentVersion, or both. As part of this change, > the > > > > > internal upgrade framework was rewritten and exposes a simpler API > to > > > > > developers. This includes strongly typed component versions (with > > > integer > > > > > conversion deferred until serialization), and an isSupportedBy > method > > > to > > > > > handle all version comparisons. > > > > > Improved Admin Experience > > > > > > > > > > To ensure finalization proceeds in the correct order as outlined in > > the > > > > > design document, admins no longer have to finalize OM and SCM > > > > separately. A > > > > > single ozone admin upgrade finalize command triggers asynchronous > > > > > finalization throughout the cluster in the defined order. A single > > > status > > > > > endpoint can be queried by ozone admin upgrade status, and clients > > can > > > > > trigger and block on finalization with one ozone admin upgrade > > finalize > > > > > --wait command, which handles polling of the status endpoint by the > > > > client > > > > > automatically and is idempotent. > > > > > > > > > > Additionally, a Grafana dashboard has been added to provide a heads > > up > > > > view > > > > > of all components during an upgrade. It includes filtering to zoom > in > > > on > > > > a > > > > > particular area and aggregates across Datanodes to handle large > > > clusters. > > > > > Removed Prepare For Upgrade > > > > > > > > > > The "prepare for upgrade" command which put OMs in a read-only mode > > > > before > > > > > an upgrade is no longer required. The CLI has been left as a no-op > > for > > > > > compatibility with older upgrade scripts. See the developer guide > > > linked > > > > > above for instructions to handle incompatible changes to OM write > > > > requests. > > > > > Work In Progress > > > > > > > > > > There is some ongoing work we will continue in parallel with the > > merge > > > > vote > > > > > and finish before the branch is merged: > > > > > > > > > > - We are currently merging the latest master branch into the > > feature > > > > > branch, resolving conflicts, and running it through CI. The > final > > > hash > > > > > for > > > > > merge will be shared here when ready. > > > > > - The ZDU design doc will be updated based on the latest copilot > > > > review > > > > > and other minor deviations identified from the resulting > > > > implementation. > > > > > - A PR to add a missed admin check on the ozone admin upgrade > > status > > > > > command is in flight: > https://github.com/apache/ozone/pull/11354 > > > > > > > > > > Future Work > > > > > > > > > > As mentioned in the merge checklist, the current OM request > > versioning > > > > > framework on master was left intact on the ZDU branch. However, the > > > > opt-in > > > > > annotation based approach does not suit the new ZDU requirements > > where > > > > > every new request needs to be versioned. Dev work has started on a > > new > > > > > framework, but the change is large and was deliberately saved for > > > master > > > > > after the branch merge so that it can be reviewed by a wider > > audience. > > > > > > > > > > Additionally, we will be investigating static analysis and AI > skills > > to > > > > > flag potentially incompatible changes during code reviews and the > > > release > > > > > process. > > > > > ------------------------------ > > > > > > > > > > Thanks to Stephen, Zita, and Roland who also worked on the > > development > > > of > > > > > this feature and to everyone who shared inputs on the design. > > > > > > > > > > We will leave the vote thread open for at least 7 days, although > more > > > > time > > > > > may be required for developers to familiarize themselves with the > new > > > ZDU > > > > > requirements. > > > > > > > > > > - Ethan > > > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Andrey. > > > > > > > > > -- > > *Sumit Agrawal* | Senior Staff Engineer > > cloudera.com <https://www.cloudera.com> > > [image: Cloudera] <https://www.cloudera.com/> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > Cloudera > > on LinkedIn] <https://www.linkedin.com/company/cloudera> > > ------------------------------ > > >
