Hey all, I reviewed the entirety of the backport PR. It's almost all net-new code, which matches identically with what was already reviewed and approved in the original implementation. There are some minor changes to HFileOutputFormat2 and WALPlayer (and related tests).
Of the changes to those 2 files, there was only 1 incompatible change -- a change to the directory structure of the output path for HFileOutputFormat2. This change is necessary to differentiate output files for tables in different namespaces. However, in discussing with Mallikarjun we are going to wrap this change in a config to preserve backwards compatibility. In addition to use by backup/restore, the config would provide users of HFileOutputFormat2 to have a path towards upgrading to 3.0 in the future without breaking their downstream jobs which depend on the current structure. Let me know if there are any other concerns. Otherwise once the compatibility fix is in place and I give it a final review, I will merge it to branch-2 (with a release note that it is experimental) and update our refguide. On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell <apurt...@apache.org> wrote: > Agreed that the open tasks are not essential before considering a backport > for (near term) release. We have often released backported features from > the main branch in new minors with documentation -- release notes and > updates to the online book, typically -- describing them as "experimental", > until something causes the community to reconsider that designation. I > assume this would happen in this case too? > > On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault > <bbeaudrea...@hubspot.com.invalid> wrote: > > > Hi again all, > > > > We have a relatively full featured backup solution in master branch. It > > looks like the original development had intended to be included in > branch-2 > > [1], but did not make the deadline for 2.0.0 release and was removed [2]. > > Later the idea of backporting was forgotten, potentially with some of the > > main devs moving onto other projects. > > > > In the interim, one company, Flipkart (Mallikarjun works there), took it > > upon themselves to backport the feature to their own fork. They've been > > running that backport in production for some time now. Mallikarjun has > been > > trying to contribute some improvements, but has lacked committer support. > > > > At my company, we're considering redesigning our backup/restore solution > > which has been relatively static since originally built back in 2014 and > is > > showing its age. While investigating options, I reached out to > Mallikarjun > > and he was graciously willing to provide a backport PR [3]. The backport > > applied cleanly with small conflicts in one file. > > > > There were a few blockers listed in the original thread in [1] and from > > what I can tell, they are all done. There is a remaining "Phase 4" > umbrella > > [4] with all of the issues looking like nice-to-haves. Most could just be > > tackled based on community interest. > > > > I think a big reason why there is no committer support and relatively > > little uptake on this feature is because it has for years been stuck on > > master, when pretty much everyone runs a 2.x release. So no one is using > it > > or has the ability to test it out, outside flipkart who backported it > > themselves. > > > > We are currently evaluating the backport PR in our own fork to see if the > > solution as-is can be a good foundation for what we're trying to > > accomplish. If the evaluation works out and we decide to move forward, > I'd > > like to work with Mallikarjun to get his backport committed. > > > > Before doing that, I wanted to check if any other devs have concerns > about > > this initiative given the size of the patch and history of the project. > Let > > me know what you think. > > > > [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn > <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn> > > [2] https://issues.apache.org/jira/browse/HBASE-19407 > <https://issues.apache.org/jira/browse/HBASE-19407> > > [3] https://github.com/apache/hbase/pull/4770 > <https://github.com/apache/hbase/pull/4770> > > [4] https://issues.apache.org/jira/browse/HBASE-17362 > <https://issues.apache.org/jira/browse/HBASE-17362> > > > > > -- > Best regards, > Andrew > > Unrest, ignorance distilled, nihilistic imbeciles - > It's what we’ve earned > Welcome, apocalypse, what’s taken you so long? > Bring us the fitting end that we’ve been counting on > - A23, Welcome, Apocalypse >
