[
https://issues.apache.org/jira/browse/HBASE-30250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-30250:
----------------------------------------
Description:
Design document:
[[link]|https://docs.google.com/document/d/1i3xJn-fBJELD-1KG06HzViWBhH8ZC_eh2zRdo4VPBmU/edit?tab=t.0]
HBase ships an interactive operator shell implemented as a JRuby IRB. Operators
rely on this shell for creating and altering tables, putting and scanning data
typically for investigative or exploratory reasons, snapshotting, moving
regions, managing replication peers, granting permissions, configuring quotas,
manipulating region server groups, and many less common operations.
This issue proposes a replacement shell, named simply {_}newshell{_}, that
delivers the same operator command set, preserves the existing input syntax for
single one-shot commands for those UNIX shell scripts and runbooks that depend
on it, but is implemented entirely in Java using only libraries that are
already present on the project’s compilation classpath. Newshell is delivered
as a new Maven module, hbase-newshell, and is launched through a new subcommand
of the existing bin/hbase script invoked as ./bin/hbase newshell.
The replacement ultimately serves the goals of removing the project’s
dependency on JRuby and on the jruby-complete uber jar. It does so in a way
that allows the existing JRuby shell and newshell to coexist for some period,
so that operators and downstream distributions can migrate at their own pace.
jruby-complete contains, in addition to the JRuby interpreter itself, a full
Ruby standard library, the entire RubyGems infrastructure, a curated set of
bundled gems including rake, rdoc, irb, bcrypt, the OpenSSL gem, the protobuf
gem, and assorted developer tooling, and a substantial number of Java-side
libraries that the JRuby runtime needs for its own internal use, including
joni, jcodings, jffi, jnr-ffi and its native bridges, bytelist, jzlib, the ASM
bytecode toolkit, the OSGi core API, the Jansi console library which
transitively pulls HawtJNI, the SnakeYAML parser, and the BouncyCastle
cryptographic suite. Every one of these components accretes its own stream of
published vulnerabilities over time. Because they are physically bundled into
jruby-complete, every advisory against any of them shows up on HBase’s software
bill of materials as soon as a downstream consumer runs a vulnerability scanner
against the binary distribution, regardless of whether the shell exercises the
affected code path.
The current pinned version of jruby-complete in HBase’s 2.5 and 2.6 releasing
lines is version 9.3.15.0, which is the final release of the JRuby 9.3 line.
JRuby upstream marked the 9.3 line end-of-life when it shipped 9.3.15.0 in June
2024, with no further backports planned. The most consequential consequence of
that policy is that CVE-2025-46551, a hostname verification bypass in the
jruby-openssl gem disclosed in May 2025 and assigned moderate severity, affects
every 9.3.x release and is fixed only on the 9.4 and 10.0 series. An earlier
attempt to onboard a 9.4 release (HBASE-27921) was abandoned. A later attempt
against 9.4.8.0 (HBASE-28250 and HBASE-28968) succeeded for the trunk and the
2.7 line only by carrying a monkey-patched copy of upstream’s irb gem to
compensate for incompatible API changes. None of that work is backportable to
the 2.5 and 2.6 lines that everyone runs in production. (Although we could
decide to move forward with a compatibility break.)
Recent examples of CVE issues in 9.3 include the cluster of REXML
denial-of-service advisories from 2024 covering CVE-2024-35176, CVE-2024-39908,
CVE-2024-41123, CVE-2024-41946, CVE-2024-43398, and CVE-2024-49761; the RDoc
deserialization advisory CVE-2024-27281; the regex engine out-of-bounds read
CVE-2024-27282; the StringIO buffer over-read CVE-2024-27280; ReDoS findings in
URI and Time captured by CVE-2023-36617, CVE-2023-28755, and CVE-2023-28756;
the Net::IMAP memory-allocation denial-of-service CVE-2025-43857; an ERB CVE
published as CVE-2026-41316; older but still scanner-flagged WEBrick
request-smuggling and HTTP-splitting advisories CVE-2020-25613, CVE-2019-16254,
and CVE-2019-16201; the CGI cookie prefix advisory CVE-2021-41819; and the Date
ReDoS CVE-2021-41817. To these must be added the SnakeYAML issues, including
the critical severity CVE-2022-1471 deserialization advisory. The BouncyCastle
versions bundled by the JRuby runtime carry their own thread of advisories,
including CVE-2024-29857, CVE-2024-30171, and CVE-2024-30172. The 2.x line of
jline that the JRuby runtime depends on has itself been abandoned upstream and
is vulnerable to CVE-2013-2035. The RubyGems infrastructure has a cluster of
seven distinct advisories, including the critical CVE-2018-1000076. Removing
the JRuby dependency removes all of these bundled software vulnerabilities,
including those of JRuby itself.
Beyond the question of upgrading to a version of jruby-complete that does not
carry a long list of software vulnerabilities, is the fundamental need for an
IRB in the first place. This proposal describes JRuby-free alternatives that
may suffice instead (a new native command format, supporting also a JRuby-alike
syntax) or improve integration (a proper CLI integration).
was:
Design document:
[[link]|https://docs.google.com/document/d/1i3xJn-fBJELD-1KG06HzViWBhH8ZC_eh2zRdo4VPBmU/edit?tab=t.0]]
HBase ships an interactive operator shell implemented as a JRuby IRB. Operators
rely on this shell for creating and altering tables, putting and scanning data
typically for investigative or exploratory reasons, snapshotting, moving
regions, managing replication peers, granting permissions, configuring quotas,
manipulating region server groups, and many less common operations.
This issue proposes a replacement shell, named simply {_}newshell{_}, that
delivers the same operator command set, preserves the existing input syntax for
single one-shot commands for those UNIX shell scripts and runbooks that depend
on it, but is implemented entirely in Java using only libraries that are
already present on the project’s compilation classpath. Newshell is delivered
as a new Maven module, hbase-newshell, and is launched through a new subcommand
of the existing bin/hbase script invoked as ./bin/hbase newshell.
The replacement ultimately serves the goals of removing the project’s
dependency on JRuby and on the jruby-complete uber jar. It does so in a way
that allows the existing JRuby shell and newshell to coexist for some period,
so that operators and downstream distributions can migrate at their own pace.
jruby-complete contains, in addition to the JRuby interpreter itself, a full
Ruby standard library, the entire RubyGems infrastructure, a curated set of
bundled gems including rake, rdoc, irb, bcrypt, the OpenSSL gem, the protobuf
gem, and assorted developer tooling, and a substantial number of Java-side
libraries that the JRuby runtime needs for its own internal use, including
joni, jcodings, jffi, jnr-ffi and its native bridges, bytelist, jzlib, the ASM
bytecode toolkit, the OSGi core API, the Jansi console library which
transitively pulls HawtJNI, the SnakeYAML parser, and the BouncyCastle
cryptographic suite. Every one of these components accretes its own stream of
published vulnerabilities over time. Because they are physically bundled into
jruby-complete, every advisory against any of them shows up on HBase’s software
bill of materials as soon as a downstream consumer runs a vulnerability scanner
against the binary distribution, regardless of whether the shell exercises the
affected code path.
The current pinned version of jruby-complete in HBase’s 2.5 and 2.6 releasing
lines is version 9.3.15.0, which is the final release of the JRuby 9.3 line.
JRuby upstream marked the 9.3 line end-of-life when it shipped 9.3.15.0 in June
2024, with no further backports planned. The most consequential consequence of
that policy is that CVE-2025-46551, a hostname verification bypass in the
jruby-openssl gem disclosed in May 2025 and assigned moderate severity, affects
every 9.3.x release and is fixed only on the 9.4 and 10.0 series. An earlier
attempt to onboard a 9.4 release (HBASE-27921) was abandoned. A later attempt
against 9.4.8.0 (HBASE-28250 and HBASE-28968) succeeded for the trunk and the
2.7 line only by carrying a monkey-patched copy of upstream’s irb gem to
compensate for incompatible API changes. None of that work is backportable to
the 2.5 and 2.6 lines that everyone runs in production. (Although we could
decide to move forward with a compatibility break.)
Recent examples of CVE issues in 9.3 include the cluster of REXML
denial-of-service advisories from 2024 covering CVE-2024-35176, CVE-2024-39908,
CVE-2024-41123, CVE-2024-41946, CVE-2024-43398, and CVE-2024-49761; the RDoc
deserialization advisory CVE-2024-27281; the regex engine out-of-bounds read
CVE-2024-27282; the StringIO buffer over-read CVE-2024-27280; ReDoS findings in
URI and Time captured by CVE-2023-36617, CVE-2023-28755, and CVE-2023-28756;
the Net::IMAP memory-allocation denial-of-service CVE-2025-43857; an ERB CVE
published as CVE-2026-41316; older but still scanner-flagged WEBrick
request-smuggling and HTTP-splitting advisories CVE-2020-25613, CVE-2019-16254,
and CVE-2019-16201; the CGI cookie prefix advisory CVE-2021-41819; and the Date
ReDoS CVE-2021-41817. To these must be added the SnakeYAML issues, including
the critical severity CVE-2022-1471 deserialization advisory. The BouncyCastle
versions bundled by the JRuby runtime carry their own thread of advisories,
including CVE-2024-29857, CVE-2024-30171, and CVE-2024-30172. The 2.x line of
jline that the JRuby runtime depends on has itself been abandoned upstream and
is vulnerable to CVE-2013-2035. The RubyGems infrastructure has a cluster of
seven distinct advisories, including the critical CVE-2018-1000076. Removing
the JRuby dependency removes all of these bundled software vulnerabilities,
including those of JRuby itself.
Beyond the question of upgrading to a version of jruby-complete that does not
carry a long list of software vulnerabilities, is the fundamental need for an
IRB in the first place. This proposal describes JRuby-free alternatives that
may suffice instead (a new native command format, supporting also a JRuby-alike
syntax) or improve integration (a proper CLI integration).
> Newshell: An Alternative Interactive and CLI Shell
> --------------------------------------------------
>
> Key: HBASE-30250
> URL: https://issues.apache.org/jira/browse/HBASE-30250
> Project: HBase
> Issue Type: New Feature
> Components: security, shell
> Reporter: Andrew Kyle Purtell
> Priority: Major
>
> Design document:
> [[link]|https://docs.google.com/document/d/1i3xJn-fBJELD-1KG06HzViWBhH8ZC_eh2zRdo4VPBmU/edit?tab=t.0]
> HBase ships an interactive operator shell implemented as a JRuby IRB.
> Operators rely on this shell for creating and altering tables, putting and
> scanning data typically for investigative or exploratory reasons,
> snapshotting, moving regions, managing replication peers, granting
> permissions, configuring quotas, manipulating region server groups, and many
> less common operations.
> This issue proposes a replacement shell, named simply {_}newshell{_}, that
> delivers the same operator command set, preserves the existing input syntax
> for single one-shot commands for those UNIX shell scripts and runbooks that
> depend on it, but is implemented entirely in Java using only libraries that
> are already present on the project’s compilation classpath. Newshell is
> delivered as a new Maven module, hbase-newshell, and is launched through a
> new subcommand of the existing bin/hbase script invoked as ./bin/hbase
> newshell.
> The replacement ultimately serves the goals of removing the project’s
> dependency on JRuby and on the jruby-complete uber jar. It does so in a way
> that allows the existing JRuby shell and newshell to coexist for some period,
> so that operators and downstream distributions can migrate at their own pace.
> jruby-complete contains, in addition to the JRuby interpreter itself, a full
> Ruby standard library, the entire RubyGems infrastructure, a curated set of
> bundled gems including rake, rdoc, irb, bcrypt, the OpenSSL gem, the protobuf
> gem, and assorted developer tooling, and a substantial number of Java-side
> libraries that the JRuby runtime needs for its own internal use, including
> joni, jcodings, jffi, jnr-ffi and its native bridges, bytelist, jzlib, the
> ASM bytecode toolkit, the OSGi core API, the Jansi console library which
> transitively pulls HawtJNI, the SnakeYAML parser, and the BouncyCastle
> cryptographic suite. Every one of these components accretes its own stream of
> published vulnerabilities over time. Because they are physically bundled into
> jruby-complete, every advisory against any of them shows up on HBase’s
> software bill of materials as soon as a downstream consumer runs a
> vulnerability scanner against the binary distribution, regardless of whether
> the shell exercises the affected code path.
> The current pinned version of jruby-complete in HBase’s 2.5 and 2.6 releasing
> lines is version 9.3.15.0, which is the final release of the JRuby 9.3 line.
> JRuby upstream marked the 9.3 line end-of-life when it shipped 9.3.15.0 in
> June 2024, with no further backports planned. The most consequential
> consequence of that policy is that CVE-2025-46551, a hostname verification
> bypass in the jruby-openssl gem disclosed in May 2025 and assigned moderate
> severity, affects every 9.3.x release and is fixed only on the 9.4 and 10.0
> series. An earlier attempt to onboard a 9.4 release (HBASE-27921) was
> abandoned. A later attempt against 9.4.8.0 (HBASE-28250 and HBASE-28968)
> succeeded for the trunk and the 2.7 line only by carrying a monkey-patched
> copy of upstream’s irb gem to compensate for incompatible API changes. None
> of that work is backportable to the 2.5 and 2.6 lines that everyone runs in
> production. (Although we could decide to move forward with a compatibility
> break.)
> Recent examples of CVE issues in 9.3 include the cluster of REXML
> denial-of-service advisories from 2024 covering CVE-2024-35176,
> CVE-2024-39908, CVE-2024-41123, CVE-2024-41946, CVE-2024-43398, and
> CVE-2024-49761; the RDoc deserialization advisory CVE-2024-27281; the regex
> engine out-of-bounds read CVE-2024-27282; the StringIO buffer over-read
> CVE-2024-27280; ReDoS findings in URI and Time captured by CVE-2023-36617,
> CVE-2023-28755, and CVE-2023-28756; the Net::IMAP memory-allocation
> denial-of-service CVE-2025-43857; an ERB CVE published as CVE-2026-41316;
> older but still scanner-flagged WEBrick request-smuggling and HTTP-splitting
> advisories CVE-2020-25613, CVE-2019-16254, and CVE-2019-16201; the CGI cookie
> prefix advisory CVE-2021-41819; and the Date ReDoS CVE-2021-41817. To these
> must be added the SnakeYAML issues, including the critical severity
> CVE-2022-1471 deserialization advisory. The BouncyCastle versions bundled by
> the JRuby runtime carry their own thread of advisories, including
> CVE-2024-29857, CVE-2024-30171, and CVE-2024-30172. The 2.x line of jline
> that the JRuby runtime depends on has itself been abandoned upstream and is
> vulnerable to CVE-2013-2035. The RubyGems infrastructure has a cluster of
> seven distinct advisories, including the critical CVE-2018-1000076. Removing
> the JRuby dependency removes all of these bundled software vulnerabilities,
> including those of JRuby itself.
> Beyond the question of upgrading to a version of jruby-complete that does not
> carry a long list of software vulnerabilities, is the fundamental need for an
> IRB in the first place. This proposal describes JRuby-free alternatives that
> may suffice instead (a new native command format, supporting also a
> JRuby-alike syntax) or improve integration (a proper CLI integration).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
