[jira] [Created] (HBASE-27586) Bump up commons-codec to 1.15
Rajeshbabu Chintaguntla created HBASE-27586:
---
Summary: Bump up commons-codec to 1.15
Key: HBASE-27586
URL: https://issues.apache.org/jira/browse/HBASE-27586
Project: HBase
Issue Type: Bug
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.3
commons-codec 1.15 has proper fix of few CVEs which may not effect in HBase but
better to upgrade to ensure compliance.
Ex: ** While [a
fix|https://github.com/apache/commons-codec/commit/48b615756d1d770091ea3322eefc08011ee8b113]
was earlier made to {{commons-codec:commons-codec}} version 1.13, it was later
found out to be incomplete. A [complete
fix|https://github.com/apache/commons-codec/pull/29] exists in version 1.14 and
that is the version users should upgrade to.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27585) Bump up jruby to 9.3.9.0
Rajeshbabu Chintaguntla created HBASE-27585: --- Summary: Bump up jruby to 9.3.9.0 Key: HBASE-27585 URL: https://issues.apache.org/jira/browse/HBASE-27585 Project: HBase Issue Type: Bug Components: security Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.3 Bump up Jruby to 9.3.9.0 to ensure compliance which has multiple CVEs fixed related to openssl,snakeyaml etc. * rdoc has been updated to 6.3.3 to fix all known CVEs. ([#7396|https://github.com/jruby/jruby/issues/7396], [#7404|https://github.com/jruby/jruby/issues/7404]) * rexml has been updated to 3.2.5 to fix all known CVEs. ([#7395|https://github.com/jruby/jruby/issues/7395], [#7405|https://github.com/jruby/jruby/issues/7405]) * jruby-openssl has been updated to 0.14.0 to fix weak HMAC key hashing in bouncycastle, which itself is updated to 1.71. ([#7335|https://github.com/jruby/jruby/issues/7335], [#7385|https://github.com/jruby/jruby/issues/7385], [#7399|https://github.com/jruby/jruby/issues/7399]) * psych has been updated to 3.3.4 to fix CVE-2022-38752 in the SnakeYAML library, which itself is updated to 1.33. ([#7386|https://github.com/jruby/jruby/issues/7386], [#7388|https://github.com/jruby/jruby/issues/7388], [#7400|https://github.com/jruby/jruby/issues/7400]) * rubygems has been updated to 3.2.33 and bundler updated to 2.2.33 to address CVE-2021-43809. ([#7397|https://github.com/jruby/jruby/issues/7397], [#7401|https://github.com/jruby/jruby/issues/7401]) -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] Time to release 2.5.3 and 2.4.16?
Fyi I’m executing the release build via do-release-docker.sh, ran till the mvn site and Java doc generation. But it failed with 20:05:46 [ERROR] javadoc: error - java.lang.NullPointerException , and few error/warning for @result tag as well as @link not found for the generated proto class BlockingInterface. I’m still working on it, if possible, please avoid to push commits to branch-2.5 Thanks, Stephen On Sun, Jan 22, 2023 at 1:29 PM Andrew Purtell wrote: > Sounds good Stephen, please ping me as you need assistance. > > On Sat, Jan 21, 2023 at 9:32 PM Tak Lon (Stephen) Wu > wrote: > > > Thanks Andrew and Bryan, all the blockers for branch-2.5 should have > > been resolved. > > > > I'm running the compatibility check again, and will continue the > > release tasks tomorrow or Monday morning (PST). > > > > -Stephen > > > > On Fri, Jan 20, 2023 at 11:16 AM Bryan Beaudreault > > wrote: > > > > > > You probably saw, but just to close the loop here -- HBASE-27579 got > > > resolved, so no blocker there anymore. Thanks all! > > > > > > On Fri, Jan 20, 2023 at 12:30 PM Andrew Purtell > > wrote: > > > > > > > I think HBASE-27539 should be reverted from the releasing branches. > > Keeping > > > > it in branch-2 and master would be fine, we do frequently change LP > > > > interface details in minor releases. It is the compatibility break > in a > > > > patch release with a real affected downstream (Phoenix) that is a > > problem. > > > > As RM I would go ahead and do that, I suggest the same to you. > > > > > > > > On Thu, Jan 19, 2023 at 10:25 PM Tak Lon (Stephen) Wu < > > tak...@apache.org> > > > > wrote: > > > > > > > > > Thanks Andrew for sharing the normal flow and helped me off this > > > > > thread. I have tried steps 1 and 2, and found the following JIRAs > may > > > > > need a few days before closing for 2.5.3 RC. > > > > > > > > > > HBASE-27579 under review > > > > > HBASE-27539 broke the patch version backward compatibility with > > > > > StoreFileReader, pending confirmation if we can revert > > > > > HBASE-27578 should be merging soon > > > > > > > > > > -Stephen > > > > > > > > > > On Thu, Jan 19, 2023 at 8:34 AM Andrew Purtell < > > andrew.purt...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > Sure. We are not rushing. > > > > > > > > > > > > Welcome to the RM club. You have commit privileges so all steps > > > > outlined > > > > > below should be achievable. Let me briefly outline the steps for > > making a > > > > > release. We can deal with details privately by email. > > > > > > > > > > > > 1. Run the compatibility checker. Verify no changes from prior > > release > > > > > that are not allowed. Exceptions are fine if you as RM are willing > to > > > > > defend them when the PMC has questions. If there are problems that > > need > > > > > fixing, fix them. > > > > > > > > > > > > 2. Clean up fix versions in JIRA. All issues targeting the > release > > eg > > > > > 2.5.3 should be resolved with that fix version in the set if a > > commit was > > > > > made, and if no commit but still resolved the fix version should be > > > > removed > > > > > from the set. If the issue is not resolved the fix version should > be > > > > > removed and replaced with one for the next eg 2.5.3 -> 2.5.4. When > > you > > > > are > > > > > done a report for the fix version of your release candidate should > > show > > > > all > > > > > issues resolved. > > > > > > > > > > > > 3. Run the create-release script. It will handle the production > and > > > > > staging of artifacts end to end. This includes the production of > > release > > > > > notes derived from JIRA. This includes tagging in git on the > branch. > > The > > > > > script will ask for your Apache committer credentials at the start. > > > > > > > > > > > > 4. Take the artifacts produced by create-release and sanity check > > them. > > > > > Check sums and signatures. Unpack and check included contents. > > Launch a > > > > > minicluster, load some data with PE or LTT. The same actions you’d > > take > > > > if > > > > > voting on someone else’s release candidate. Assuming no issue, > > proceed. > > > > > > > > > > > > 5. Send the vote mail produced by create-release to get the vote > > > > started. > > > > > > > > > > > > 6. Respond to comments on the vote thread as required. > > > > > > > > > > > > 7. Close the vote once sufficient votes have been received. Give > > it a > > > > > week at least. If it seems like voting is slow to complete gently > > prod > > > > the > > > > > PMC on dev@. > > > > > > > > > > > > 8. Create and push a release tag in git. Move the release > candidate > > > > from > > > > > staging to release in svn. > > > > > > > > > > > > 9. Edit download.xml and trigger a new site build. Wait for your > > > > changes > > > > > to go live. > > > > > > > > > > > > 10. Send out a release announcement email. > > > > > > > > > > > > > On Jan 18, 2023, at 8:49 PM, Tak Lon (Stephen) Wu < > > tak...@apache.org > > > > > > > > >
Re: [DISCUSS] Backporting hbase-backup module to branch-2
This has landed in branch-2. Your assumption on the configuration toggle is correct: we should expect no changes to the output of these jobs (and thus no downstream impact in other jobs). The new directory structure is toggled via hbase.hfileoutputformat.tablename.namespace.inclusive, which I've documented in the release note. Thank you both for your guidance here, and big thanks to Mallikarjun for his hard work on the backport and fixes. We've filed two follow-ons: https://issues.apache.org/jira/browse/HBASE-27584 - document the availability of experimental support as of 2.6.0 https://issues.apache.org/jira/browse/HBASE-27582 - errorprone cleanup in hbase-backup, which we noticed during the backport Members of my team have also pushed some improvements to backups in master recently, which I'll also backport shortly. Mallikarjun has some long open PRs for backups enhancements, which I hope to take a look at soon. On Wed, Jan 18, 2023 at 1:35 AM Andrew Purtell wrote: > I assume supporting either directory structure with a configuration toggle > means existing code can coexist (in other jobs) with the backup feature, > and so it will be fine to port this into a code line that will release a > new minor. Sounds like a plan. +1 > > > On Tue, Jan 17, 2023 at 5:21 PM Bryan Beaudreault > wrote: > > > Hey all, > > > > I reviewed the entirety of the backport PR. It's almost all net-new code, > > which matches identically with what was already reviewed and approved in > > the original implementation. There are some minor changes to > > HFileOutputFormat2 and WALPlayer (and related tests). > > > > Of the changes to those 2 files, there was only 1 incompatible change -- > a > > change to the directory structure of the output path for > > HFileOutputFormat2. This change is necessary to differentiate output > files > > for tables in different namespaces. However, in discussing with > Mallikarjun > > we are going to wrap this change in a config to preserve backwards > > compatibility. In addition to use by backup/restore, the config would > > provide users of HFileOutputFormat2 to have a path towards upgrading to > 3.0 > > in the future without breaking their downstream jobs which depend on the > > current structure. > > > > Let me know if there are any other concerns. Otherwise once the > > compatibility fix is in place and I give it a final review, I will merge > it > > to branch-2 (with a release note that it is experimental) and update our > > refguide. > > > > On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell > wrote: > > > > > Agreed that the open tasks are not essential before considering a > > backport > > > for (near term) release. We have often released backported features > from > > > the main branch in new minors with documentation -- release notes and > > > updates to the online book, typically -- describing them as > > "experimental", > > > until something causes the community to reconsider that designation. I > > > assume this would happen in this case too? > > > > > > On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault > > > wrote: > > > > > > > Hi again all, > > > > > > > > We have a relatively full featured backup solution in master branch. > It > > > > looks like the original development had intended to be included in > > > branch-2 > > > > [1], but did not make the deadline for 2.0.0 release and was removed > > [2]. > > > > Later the idea of backporting was forgotten, potentially with some of > > the > > > > main devs moving onto other projects. > > > > > > > > In the interim, one company, Flipkart (Mallikarjun works there), took > > it > > > > upon themselves to backport the feature to their own fork. They've > been > > > > running that backport in production for some time now. Mallikarjun > has > > > been > > > > trying to contribute some improvements, but has lacked committer > > support. > > > > > > > > At my company, we're considering redesigning our backup/restore > > solution > > > > which has been relatively static since originally built back in 2014 > > and > > > is > > > > showing its age. While investigating options, I reached out to > > > Mallikarjun > > > > and he was graciously willing to provide a backport PR [3]. The > > backport > > > > applied cleanly with small conflicts in one file. > > > > > > > > There were a few blockers listed in the original thread in [1] and > from > > > > what I can tell, they are all done. There is a remaining "Phase 4" > > > umbrella > > > > [4] with all of the issues looking like nice-to-haves. Most could > just > > be > > > > tackled based on community interest. > > > > > > > > I think a big reason why there is no committer support and relatively > > > > little uptake on this feature is because it has for years been stuck > on > > > > master, when pretty much everyone runs a 2.x release. So no one is > > using > > > it > > > > or has the ability to test it out, outside flipkart who backported it > > > > themselves. > > > > > > > > We are currently
[jira] [Created] (HBASE-27584) Document availability of experimental Backup/Restore feature as of release 2.6.0
Bryan Beaudreault created HBASE-27584: - Summary: Document availability of experimental Backup/Restore feature as of release 2.6.0 Key: HBASE-27584 URL: https://issues.apache.org/jira/browse/HBASE-27584 Project: HBase Issue Type: Task Reporter: Bryan Beaudreault -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27238) Backport Backup/Restore to 2.x
[ https://issues.apache.org/jira/browse/HBASE-27238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault resolved HBASE-27238. --- Fix Version/s: 2.6.0 Release Note: Provides EXPERIMENTAL support for native backup/restore functionality to HBASE 2.x releases, as originally documented in https://hbase.apache.org/book.html#backuprestore. In addition to all of the changes there, introduces a new config to ensure backwards compatibility. Users of MultiTableHFileOutputFormat who wish to disambiguate output files for tables in different namespaces may set hbase.hfileoutputformat.tablename.namespace.inclusive to true. This is enabled by default in backup and restore related jobs, and will be the default behavior in 3.0 release. Also adds a new wal.multi.tables.support config to WALPlayer which enables parsing WALs into bulkloadable hfiles for multiple tables. This also is defaulted to true for restores. Resolution: Fixed > Backport Backup/Restore to 2.x > -- > > Key: HBASE-27238 > URL: https://issues.apache.org/jira/browse/HBASE-27238 > Project: HBase > Issue Type: New Feature > Components: backport, backuprestore >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 2.6.0 > > > Backport backup/restore to 2.x branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[NOTICE] Code signing key for taklwu@a.o appended to KEYS
I have appended key 0921555AE935AD54: "Tak Lon (Stephen) Wu (CODE SIGNING KEY) " to KEYS. -- Best regards, Andrew Unrest, ignorance distilled, nihilistic imbeciles - It's what we’ve earned Welcome, apocalypse, what’s taken you so long? Bring us the fitting end that we’ve been counting on - A23, Welcome, Apocalypse
