[
https://issues.apache.org/jira/browse/HBASE-30250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118919#comment-18118919
]
Aman Poonia edited comment on HBASE-30250 at 9/24/26 6:18 PM:
--------------------------------------------------------------
Hi [~nihaljain], thanks for surfacing this -- great to see independent interest
in a JRuby-free CLI.
For context: {{hbase-newshell}} is the name we have been using for the
Java-based shell replacement module this JIRA proposes (per the design doc) --
a new Maven module launched via {{bin/hbase newshell}}, implemented against the
public HBase client API with no JRuby dependency. There is no public branch/PR
yet, but I am planning to push this work up as a PR soon so you (and anyone
else interested) can review/build on it directly. In the meantime, here is
where things stand and how we have approached the compatibility question, in
case it is useful before you re-look at your PoC:
Progress update: most of the shell command surface is ported already (over 90
of 184 {{hbase-shell}} Ruby commands have Java equivalents in
{{hbase-newshell}}, covering ddl, namespace, dml, and large chunks of
tools/replication/snapshots/security/rsgroup). The {{bin/hbase}} JRuby-script
dispatch is also being replaced: a new {{.jsh}}/{{.java}} single-file-source
dispatch mode (JShell script mode / JEP 330) lets the standalone {{bin/*.rb}}
utility scripts (region movers, drain helpers, etc.) get rewritten against the
public HBase API with no JRuby runtime involved at all.
On approach and syntax compatibility, since that is the core design tension:
{{hbase-newshell}} keeps the existing Ruby-shell _line grammar_ as the REPL
contract -- same one-liner syntax, same {{-n}}/piped-stdin one-shot semantics
that existing ops scripts and runbooks depend on. That is a hard
backward-compat requirement, not a nice-to-have, since a lot of real-world
tooling shells single commands into {{hbase shell -n}} today and cannot be
expected to migrate syntax.
That said, "Ruby-compatible" and "better CLI ergonomics" are not mutually
exclusive, and it looks like we have converged on some similar ground
independently. {{hbase-newshell}} already has a few machine-facing additions
layered onto the existing seams, rather than adopting picocli as the REPL core
or introducing a second, incompatible grammar:
* Structured output formats ({{JsonFormatter}}/{{CsvFormatter}} behind the
existing {{Formatter}} interface, opt-in via {{--output}}/{{-o}}, including
NDJSON streaming for {{scan}})
* An exit-code taxonomy (client/server/auth/user-abort mapped to distinct
codes) instead of collapsing every failure to exit 1
* Session-level flags ({{--verbose}}, {{--quiet}}, {{--yes}}) and explicit
destructive-batch confirmation semantics for commands like
{{disable_all}}/{{drop_all}} under non-interactive execution
So the new syntax is additive machine-facing polish layered on top of the same
interactive grammar, not a replacement for it. We deliberately left "make a
flag-based CLI the primary grammar" out of scope for this JIRA -- command
classes sit behind a stable {{ShellAdmin}}/{{ShellTable}} facade specifically
so a future thin CLI frontend could reuse the same ops layer later without
depending on newshell REPL parser.
Given where things stand, happy to have you pick up one of the still-unported
groups (quotas, security, visibility labels, or rsgroup are the largest
remaining chunks) rather than re-deriving the module from scratch -- the
command/registry/formatter scaffolding is already in place to build on. Let me
know if that is useful or if you would rather explore the CLI-ergonomics angle
further as a follow-on.
was (Author: mnpoonia):
Hi [~nihaljain], thanks for surfacing this -- great to see independent interest
in a JRuby-free CLI.
For context: {{hbase-newshell}} is the name we have been using for the
Java-based shell replacement module this JIRA proposes (per the design doc) --
a new Maven module launched via {{bin/hbase newshell}}, implemented against the
public HBase client API with no JRuby dependency. There is no public branch/PR
yet, but I am planning to push this work up as a PR soon so you (and anyone
else interested) can review/build on it directly. In the meantime, here is
where things stand and how we have approached the compatibility question, in
case it is useful before you re-look at your PoC:
Progress update: most of the shell command surface is ported already (over 90
of 184 {{hbase-shell}} Ruby commands have Java equivalents in
{{hbase-newshell}}, covering ddl, namespace, dml, and large chunks of
tools/replication/snapshots/security/rsgroup). The {{bin/hbase}} JRuby-script
dispatch is also being replaced: a new {{.jsh}}/{{.java}} single-file-source
dispatch mode (JShell script mode / JEP 330) lets the standalone {{bin/*.rb}}
utility scripts (region movers, drain helpers, etc.) get rewritten against the
public HBase API with no JRuby runtime involved at all.
On approach and syntax compatibility, since that is the core design tension:
{{hbase-newshell}} keeps the existing Ruby-shell _line grammar_ as the REPL
contract -- same one-liner syntax, same {{-n}}/piped-stdin one-shot semantics
that existing ops scripts and runbooks depend on. That is a hard
backward-compat requirement, not a nice-to-have, since a lot of real-world
tooling shells single commands into {{hbase shell -n}} today and cannot be
expected to migrate syntax.
That said, we did not treat "Ruby-compatible" and "better CLI ergonomics" as
mutually exclusive. We looked at your PoC/spec and pulled several of its ideas
into newshell existing seams rather than adopting picocli as the REPL core or
introducing a second, incompatible grammar:
* Structured output formats ({{JsonFormatter}}/{{CsvFormatter}} behind the
existing {{Formatter}} interface, opt-in via {{--output}}/{{-o}}, including
NDJSON streaming for {{scan}})
* An exit-code taxonomy (client/server/auth/user-abort mapped to distinct
codes) instead of collapsing every failure to exit 1
* Session-level flags ({{--verbose}}, {{--quiet}}, {{--yes}}) and explicit
destructive-batch confirmation semantics for commands like
{{disable_all}}/{{drop_all}} under non-interactive execution
So the new syntax is additive machine-facing polish layered on top of the same
interactive grammar, not a replacement for it. We deliberately left "make a
flag-based CLI the primary grammar" out of scope for this JIRA -- command
classes sit behind a stable {{ShellAdmin}}/{{ShellTable}} facade specifically
so a future thin CLI frontend (possibly informed by your PoC) could reuse the
same ops layer later without depending on newshell REPL parser.
Given where things stand, happy to have you pick up one of the still-unported
groups (quotas, security, visibility labels, or rsgroup are the largest
remaining chunks) rather than re-deriving the module from scratch -- the
command/registry/formatter scaffolding is already in place to build on. Let me
know if that is useful or if you would rather explore the CLI-ergonomics angle
further as a follow-on.
> Newshell: An Alternative Interactive and CLI Shell
> --------------------------------------------------
>
> Key: HBASE-30250
> URL: https://issues.apache.org/jira/browse/HBASE-30250
> Project: HBase
> Issue Type: New Feature
> Components: security, shell
> Reporter: Andrew Kyle Purtell
> Priority: Major
>
> Design document:
> [[link]|https://docs.google.com/document/d/1i3xJn-fBJELD-1KG06HzViWBhH8ZC_eh2zRdo4VPBmU/edit?tab=t.0]
> HBase ships an interactive operator shell implemented as a JRuby IRB.
> Operators rely on this shell for creating and altering tables, putting and
> scanning data typically for investigative or exploratory reasons,
> snapshotting, moving regions, managing replication peers, granting
> permissions, configuring quotas, manipulating region server groups, and many
> less common operations.
> This issue proposes a replacement shell, named simply {_}newshell{_}, that
> delivers the same operator command set, preserves the existing input syntax
> for single one-shot commands for those UNIX shell scripts and runbooks that
> depend on it, but is implemented entirely in Java using only libraries that
> are already present on the project’s compilation classpath. Newshell is
> delivered as a new Maven module, hbase-newshell, and is launched through a
> new subcommand of the existing bin/hbase script invoked as ./bin/hbase
> newshell.
> The replacement ultimately serves the goals of removing the project’s
> dependency on JRuby and on the jruby-complete uber jar. It does so in a way
> that allows the existing JRuby shell and newshell to coexist for some period,
> so that operators and downstream distributions can migrate at their own pace.
> jruby-complete contains, in addition to the JRuby interpreter itself, a full
> Ruby standard library, the entire RubyGems infrastructure, a curated set of
> bundled gems including rake, rdoc, irb, bcrypt, the OpenSSL gem, the protobuf
> gem, and assorted developer tooling, and a substantial number of Java-side
> libraries that the JRuby runtime needs for its own internal use, including
> joni, jcodings, jffi, jnr-ffi and its native bridges, bytelist, jzlib, the
> ASM bytecode toolkit, the OSGi core API, the Jansi console library which
> transitively pulls HawtJNI, the SnakeYAML parser, and the BouncyCastle
> cryptographic suite. Every one of these components accretes its own stream of
> published vulnerabilities over time. Because they are physically bundled into
> jruby-complete, every advisory against any of them shows up on HBase’s
> software bill of materials as soon as a downstream consumer runs a
> vulnerability scanner against the binary distribution, regardless of whether
> the shell exercises the affected code path.
> The current pinned version of jruby-complete in HBase’s 2.5 and 2.6 releasing
> lines is version 9.3.15.0, which is the final release of the JRuby 9.3 line.
> JRuby upstream marked the 9.3 line end-of-life when it shipped 9.3.15.0 in
> June 2024, with no further backports planned. The most consequential
> consequence of that policy is that CVE-2025-46551, a hostname verification
> bypass in the jruby-openssl gem disclosed in May 2025 and assigned moderate
> severity, affects every 9.3.x release and is fixed only on the 9.4 and 10.0
> series. An earlier attempt to onboard a 9.4 release (HBASE-27921) was
> abandoned. A later attempt against 9.4.8.0 (HBASE-28250 and HBASE-28968)
> succeeded for the trunk and the 2.7 line only by carrying a monkey-patched
> copy of upstream’s irb gem to compensate for incompatible API changes. None
> of that work is backportable to the 2.5 and 2.6 lines that everyone runs in
> production. (Although we could decide to move forward with a compatibility
> break.)
> Recent examples of CVE issues in 9.3 include the cluster of REXML
> denial-of-service advisories from 2024 covering CVE-2024-35176,
> CVE-2024-39908, CVE-2024-41123, CVE-2024-41946, CVE-2024-43398, and
> CVE-2024-49761; the RDoc deserialization advisory CVE-2024-27281; the regex
> engine out-of-bounds read CVE-2024-27282; the StringIO buffer over-read
> CVE-2024-27280; ReDoS findings in URI and Time captured by CVE-2023-36617,
> CVE-2023-28755, and CVE-2023-28756; the Net::IMAP memory-allocation
> denial-of-service CVE-2025-43857; an ERB CVE published as CVE-2026-41316;
> older but still scanner-flagged WEBrick request-smuggling and HTTP-splitting
> advisories CVE-2020-25613, CVE-2019-16254, and CVE-2019-16201; the CGI cookie
> prefix advisory CVE-2021-41819; and the Date ReDoS CVE-2021-41817. To these
> must be added the SnakeYAML issues, including the critical severity
> CVE-2022-1471 deserialization advisory. The BouncyCastle versions bundled by
> the JRuby runtime carry their own thread of advisories, including
> CVE-2024-29857, CVE-2024-30171, and CVE-2024-30172. The 2.x line of jline
> that the JRuby runtime depends on has itself been abandoned upstream and is
> vulnerable to CVE-2013-2035. The RubyGems infrastructure has a cluster of
> seven distinct advisories, including the critical CVE-2018-1000076. Removing
> the JRuby dependency removes all of these bundled software vulnerabilities,
> including those of JRuby itself.
> Beyond the question of upgrading to a version of jruby-complete that does not
> carry a long list of software vulnerabilities, is the fundamental need for an
> IRB in the first place. This proposal describes JRuby-free alternatives that
> may suffice instead (a new native command format, supporting also a
> JRuby-alike syntax) or improve integration (a proper CLI integration).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
