This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 7eff07d HBASE-11686 Shell code should create a binding / irb
workspace instead of polluting the root namespace (#2141)
7eff07d is described below
commit 7eff07d6bfc74080addb7b2ab4076a9e75d3175c
Author: Elliot <[email protected]>
AuthorDate: Mon Jul 27 23:56:09 2020 -0400
HBASE-11686 Shell code should create a binding / irb workspace instead of
polluting the root namespace (#2141)
* HBASE-11686 Shell code should create a binding / irb workspace instead of
polluting the root namespace
- Refactor Shell.export_commands to define commands using ruby lambdas.
Additionally, this change stores a reference to shell_inst in scope so
that
we no longer need to assume the existance of the variable @shell.
- Add logic to Shell class for constructing an IRB workspace with its own
binding and non-global receiver. This workspace is loaded with all HBase
and
IRB commands.
- Create new method on Shell for evaluating input from an arbitrary IO
instance
within the created IRB workspace. This is based on work by Hsieh that was
previously in bin/hirb.rb. This method is more generic and more testable.
This single pattern can be used for both executing files and reading from
stdin, therefore reducing complexity.
- Move special 'help' and 'tools' command definitions to shell.rb. These
commands are tightly linked with an instance of the shell, so it is
easiest
to have them defined together.
- Remove all global includes of HBaseConstants from ruby test files. Before
this change, tests were loading these constants into the top level, which
could cause tests to pass that should really fail.
- Try to reduce the number of places that constants are included. I think
it's
best to reference each ruby constant's full name, but where that would
cause
a big diff I instead moved the include to the innermost Module or Class.
- Update docs and comments
- Remove unneccessary includes
- Add shell --top-level-cmds compatibility flag. Since this PR removes all
the
HBase symbols from the top-level receiver (ie. main Object), it is
possible
(albeit unlikely) that this will break operator scripts. This flag will
export all the commands at the top-level like the shell previously did.
* HBASE-11686 Light refactoring with added unit tests
- Fixes some constants references by admin test 2
- Install HBase commands as singleton methods on recevier instances so that
multiple receivers may exist.
- Rename new flag from --top-level-cmds to --top-level-defs to be more
semantically accurate.
- Create new helper method Shell::Shell#export_all to install @hbase,
@shell,
constants, and all hbase commands to a target receiver. As a result, the
HBaseReceiver became very simple and could be moved to shell.rb.
- Add unit tests for Shell::Shell#eval_io and Shell::Shell#export_all
- Add @hbase and @shell to hbase-shell IRB workspace
- Fix robocop issues within patch
* Typo s/is/if/
---
bin/hirb.rb | 87 ++++----------
hbase-shell/src/main/ruby/hbase/admin.rb | 3 +-
hbase-shell/src/main/ruby/hbase/quotas.rb | 3 +
hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb | 8 +-
hbase-shell/src/main/ruby/hbase/security.rb | 2 -
hbase-shell/src/main/ruby/hbase/taskmonitor.rb | 2 -
hbase-shell/src/main/ruby/hbase_constants.rb | 4 +-
hbase-shell/src/main/ruby/shell.rb | 130 +++++++++++++++++++--
.../src/main/ruby/shell/commands/clone_snapshot.rb | 2 +-
.../src/main/ruby/shell/commands/describe.rb | 2 +-
.../main/ruby/shell/commands/describe_namespace.rb | 2 +-
.../src/main/ruby/shell/commands/list_regions.rb | 17 ++-
.../main/ruby/shell/commands/restore_snapshot.rb | 2 +-
.../src/main/ruby/shell/commands/set_quota.rb | 29 +++--
hbase-shell/src/test/ruby/hbase/admin2_test.rb | 7 +-
hbase-shell/src/test/ruby/hbase/admin_test.rb | 16 ++-
.../ruby/hbase/list_regions_test_no_cluster.rb | 3 +-
hbase-shell/src/test/ruby/hbase/quotas_test.rb | 4 +-
.../src/test/ruby/hbase/quotas_test_no_cluster.rb | 3 +-
.../src/test/ruby/hbase/replication_admin_test.rb | 2 +-
.../src/test/ruby/hbase/security_admin_test.rb | 3 +-
hbase-shell/src/test/ruby/hbase/table_test.rb | 6 +-
.../test/ruby/hbase/test_connection_no_cluster.rb | 3 +-
.../ruby/hbase/visibility_labels_admin_test.rb | 3 +-
hbase-shell/src/test/ruby/shell/converter_test.rb | 3 +-
.../src/test/ruby/shell/list_procedures_test.rb | 3 +-
hbase-shell/src/test/ruby/shell/shell_test.rb | 58 +++++++++
hbase-shell/src/test/ruby/test_helper.rb | 2 +-
28 files changed, 271 insertions(+), 138 deletions(-)
diff --git a/bin/hirb.rb b/bin/hirb.rb
index fcbec78..5b502f4 100644
--- a/bin/hirb.rb
+++ b/bin/hirb.rb
@@ -58,6 +58,8 @@ Usage: shell [OPTIONS] [SCRIPTFILE [ARGUMENTS]]
-h | --help This help.
-n | --noninteractive Do not run within an IRB session and exit with
non-zero
status on first error.
+ --top-level-defs Compatibility flag to export HBase shell commands onto
+ Ruby's main object
-Dkey=value Pass hbase-*.xml Configuration overrides. For
example, to
use an alternate zookeeper ensemble, pass:
-Dhbase.zookeeper.quorum=zookeeper.example.org
@@ -81,6 +83,7 @@ script2run = nil
log_level = org.apache.log4j.Level::ERROR
@shell_debug = false
interactive = true
+top_level_definitions = false
_configuration = nil
D_ARG = '-D'.freeze
while (arg = ARGV.shift)
@@ -108,6 +111,8 @@ while (arg = ARGV.shift)
warn '[INFO] the -r | --return-values option is ignored. we always behave
'\
'as though it was given.'
found.push(arg)
+ elsif arg == '--top-level-defs'
+ top_level_definitions = true
else
# Presume it a script. Save it off for running later below
# after we've set up some environment.
@@ -143,21 +148,10 @@ require 'shell/formatter'
@shell = Shell::Shell.new(@hbase, interactive)
@shell.debug = @shell_debug
-# Add commands to this namespace
-# TODO avoid polluting main namespace by using a binding
[email protected]_commands(self)
-
-# Add help command
-def help(command = nil)
- @shell.help(command)
-end
-
-# Backwards compatibility method
-def tools
- @shell.help_group('tools')
-end
-
-# Debugging method
+##
+# Toggle shell debugging
+#
+# @return [Boolean] true if debug is turned on after updating the flag
def debug
if @shell_debug
@shell_debug = false
@@ -173,26 +167,34 @@ def debug
debug?
end
+##
+# Print whether debug is on or off
def debug?
puts "Debug mode is #{@shell_debug ? 'ON' : 'OFF'}\n\n"
nil
end
-# Include hbase constants
-include HBaseConstants
+# For backwards compatibility, this will export all the HBase shell commands,
constants, and
+# instance variables (@hbase and @shell) onto Ruby's top-level receiver object
known as "main".
[email protected]_all(self) if top_level_definitions
# If script2run, try running it. If we're in interactive mode, will go on to
run the shell unless
# script calls 'exit' or 'exit 0' or 'exit errcode'.
-load(script2run) if script2run
[email protected]_io(File.new(script2run)) if script2run
+
+# If we are not running interactively, evaluate standard input
[email protected]_io(STDIN) unless interactive
if interactive
# Output a banner message that tells users where to go for help
@shell.print_banner
require 'irb'
+ require 'irb/ext/change-ws'
require 'irb/hirb'
module IRB
+ # Override of the default IRB.start
def self.start(ap_path = nil)
$0 = File.basename(ap_path, '.rb') if ap_path
@@ -207,7 +209,12 @@ if interactive
HIRB.new
end
+ shl = TOPLEVEL_BINDING.receiver.instance_variable_get :'@shell'
+ hirb.context.change_workspace shl.get_workspace
+
@CONF[:IRB_RC].call(hirb.context) if @CONF[:IRB_RC]
+ # Storing our current HBase IRB Context as the main context is
imperative for several reasons,
+ # including auto-completion.
@CONF[:MAIN_CONTEXT] = hirb.context
catch(:IRB_EXIT) do
@@ -217,48 +224,4 @@ if interactive
end
IRB.start
-else
- begin
- # Noninteractive mode: if there is input on stdin, do a simple REPL.
- # XXX Note that this purposefully uses STDIN and not Kernel.gets
- # in order to maintain compatibility with previous behavior where
- # a user could pass in script2run and then still pipe commands on
- # stdin.
- require 'irb/ruby-lex'
- require 'irb/workspace'
- workspace = IRB::WorkSpace.new(binding)
- scanner = RubyLex.new
-
- # RubyLex claims to take an IO but really wants an InputMethod
- module IOExtensions
- def encoding
- external_encoding
- end
- end
- IO.include IOExtensions
-
- scanner.set_input(STDIN)
- scanner.each_top_level_statement do |statement, linenum|
- puts(workspace.evaluate(nil, statement, 'stdin', linenum))
- end
- # XXX We're catching Exception on purpose, because we want to include
- # unwrapped java exceptions, syntax errors, eval failures, etc.
- rescue Exception => exception
- message = exception.to_s
- # exception unwrapping in shell means we'll have to handle Java exceptions
- # as a special case in order to format them properly.
- if exception.is_a? java.lang.Exception
- warn 'java exception'
- message = exception.get_message
- end
- # Include the 'ERROR' string to try to make transition easier for scripts
that
- # may have already been relying on grepping output.
- puts "ERROR #{exception.class}: #{message}"
- if $fullBacktrace
- # re-raising the will include a backtrace and exit.
- raise exception
- else
- exit 1
- end
- end
end
diff --git a/hbase-shell/src/main/ruby/hbase/admin.rb
b/hbase-shell/src/main/ruby/hbase/admin.rb
index 6bc09f1..807b5a1 100644
--- a/hbase-shell/src/main/ruby/hbase/admin.rb
+++ b/hbase-shell/src/main/ruby/hbase/admin.rb
@@ -27,6 +27,7 @@ java_import org.apache.hadoop.hbase.ServerName
java_import org.apache.hadoop.hbase.TableName
java_import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder
java_import org.apache.hadoop.hbase.client.TableDescriptorBuilder
+java_import org.apache.hadoop.hbase.HConstants
# Wrapper for org.apache.hadoop.hbase.client.HBaseAdmin
@@ -1011,7 +1012,7 @@ module Hbase
cfdb.setTimeToLive(arg.delete(ColumnFamilyDescriptorBuilder::TTL)) if
arg.include?(ColumnFamilyDescriptorBuilder::TTL)
cfdb.setDataBlockEncoding(org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::DATA_BLOCK_ENCODING)))
if arg.include?(ColumnFamilyDescriptorBuilder::DATA_BLOCK_ENCODING)
cfdb.setBlocksize(JInteger.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::BLOCKSIZE)))
if arg.include?(ColumnFamilyDescriptorBuilder::BLOCKSIZE)
-
cfdb.setMaxVersions(JInteger.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::VERSIONS)))
if arg.include?(ColumnFamilyDescriptorBuilder::VERSIONS)
+ cfdb.setMaxVersions(JInteger.valueOf(arg.delete(HConstants::VERSIONS)))
if arg.include?(HConstants::VERSIONS)
cfdb.setMinVersions(JInteger.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::MIN_VERSIONS)))
if arg.include?(ColumnFamilyDescriptorBuilder::MIN_VERSIONS)
cfdb.setKeepDeletedCells(org.apache.hadoop.hbase.KeepDeletedCells.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::KEEP_DELETED_CELLS).to_s.upcase))
if arg.include?(ColumnFamilyDescriptorBuilder::KEEP_DELETED_CELLS)
cfdb.setCompressTags(JBoolean.valueOf(arg.delete(ColumnFamilyDescriptorBuilder::COMPRESS_TAGS)))
if arg.include?(ColumnFamilyDescriptorBuilder::COMPRESS_TAGS)
diff --git a/hbase-shell/src/main/ruby/hbase/quotas.rb
b/hbase-shell/src/main/ruby/hbase/quotas.rb
index 62c54c9..75757c1 100644
--- a/hbase-shell/src/main/ruby/hbase/quotas.rb
+++ b/hbase-shell/src/main/ruby/hbase/quotas.rb
@@ -61,6 +61,9 @@ end
module Hbase
# rubocop:disable Metrics/ClassLength
class QuotasAdmin
+ include HBaseConstants
+ include HBaseQuotasConstants
+
def initialize(admin)
@admin = admin
end
diff --git a/hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb
b/hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb
index 2027c77..dfffb56 100644
--- a/hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb
+++ b/hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb
@@ -23,8 +23,6 @@ java_import org.apache.hadoop.hbase.util.Pair
module Hbase
class RSGroupAdmin
- include HBaseConstants
-
def initialize(connection)
@connection = connection
@admin = @connection.getAdmin
@@ -213,11 +211,11 @@ module Hbase
unless arg.is_a?(Hash)
raise(ArgumentError, "#{arg.class} of #{arg.inspect} is not of Hash
type")
end
- method = arg[METHOD]
+ method = arg[::HBaseConstants::METHOD]
if method == 'unset'
- configuration.remove(arg[NAME])
+ configuration.remove(arg[::HBaseConstants::NAME])
elsif method == 'set'
- arg.delete(METHOD)
+ arg.delete(::HBaseConstants::METHOD)
for k, v in arg
v = v.to_s unless v.nil?
configuration.put(k, v)
diff --git a/hbase-shell/src/main/ruby/hbase/security.rb
b/hbase-shell/src/main/ruby/hbase/security.rb
index e95f094..652459a 100644
--- a/hbase-shell/src/main/ruby/hbase/security.rb
+++ b/hbase-shell/src/main/ruby/hbase/security.rb
@@ -22,8 +22,6 @@ include Java
module Hbase
class SecurityAdmin
- include HBaseConstants
-
def initialize(admin)
@admin = admin
@connection = @admin.getConnection
diff --git a/hbase-shell/src/main/ruby/hbase/taskmonitor.rb
b/hbase-shell/src/main/ruby/hbase/taskmonitor.rb
index 8509b5d..02c8432 100644
--- a/hbase-shell/src/main/ruby/hbase/taskmonitor.rb
+++ b/hbase-shell/src/main/ruby/hbase/taskmonitor.rb
@@ -27,8 +27,6 @@ end
module Hbase
class TaskMonitor
- include HBaseConstants
-
#---------------------------------------------------------------------------------------------
# Represents information reported by a server on a single MonitoredTask
class Task
diff --git a/hbase-shell/src/main/ruby/hbase_constants.rb
b/hbase-shell/src/main/ruby/hbase_constants.rb
index 9d16aa6..cd21e78 100644
--- a/hbase-shell/src/main/ruby/hbase_constants.rb
+++ b/hbase-shell/src/main/ruby/hbase_constants.rb
@@ -116,7 +116,7 @@ module HBaseConstants
promote_constants(org.apache.hadoop.hbase.client.TableDescriptorBuilder.constants)
end
-# Include classes definition
+# Ensure that hbase class definitions are imported
require 'hbase/hbase'
require 'hbase/admin'
require 'hbase/taskmonitor'
@@ -126,5 +126,3 @@ require 'hbase/replication_admin'
require 'hbase/security'
require 'hbase/visibility_labels'
require 'hbase/rsgroup_admin'
-
-include HBaseQuotasConstants
diff --git a/hbase-shell/src/main/ruby/shell.rb
b/hbase-shell/src/main/ruby/shell.rb
index 92e52fa..1217ca6 100644
--- a/hbase-shell/src/main/ruby/shell.rb
+++ b/hbase-shell/src/main/ruby/shell.rb
@@ -16,6 +16,29 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+require 'irb'
+require 'irb/workspace'
+
+#
+# Simple class to act as the main receiver for an IRB Workspace (and its
respective ruby Binding)
+# in our HBase shell. This will hold all the commands we want in our shell.
+#
+class HBaseReceiver < Object
+ def get_binding
+ binding
+ end
+end
+
+##
+# HBaseIOExtensions is a module to be "mixed-in" (ie. included) to Ruby's IO
class. It is required
+# if you want to use RubyLex with an IO object. RubyLex claims to take an IO
but really wants an
+# InputMethod.
+module HBaseIOExtensions
+ def encoding
+ external_encoding
+ end
+end
+
# Shell commands module
module Shell
@@ -115,20 +138,48 @@ module Shell
@rsgroup_admin ||= hbase.rsgroup_admin
end
- def export_commands(where)
+ ##
+ # Create singleton methods on the target receiver object for all the
loaded commands
+ #
+ # Therefore, export_commands will create "class methods" if passed a
Module/Class and if passed
+ # an instance the methods will not exist on any other instances of the
instantiated class.
+ def export_commands(target)
+ # We need to store a reference to this Shell instance in the scope of
this method so that it
+ # can be accessed later in the scope of the target object.
+ shell_inst = self
+ # Define each method as a lambda. We need to use a lambda (rather than a
Proc or block) for
+ # its properties: preservation of local variables and return
::Shell.commands.keys.each do |cmd|
- # here where is the IRB namespace
- # this method just adds the call to the specified command
- # which just references back to 'this' shell object
- # a decently extensible way to add commands
- where.send :instance_eval, <<-EOF
- def #{cmd}(*args)
- ret = @shell.command('#{cmd}', *args)
- puts
- return ret
- end
- EOF
+ target.send :define_singleton_method, cmd.to_sym, lambda { |*args|
+ ret = shell_inst.command(cmd.to_s, *args)
+ puts
+ ret
+ }
end
+ # Export help method
+ target.send :define_singleton_method, :help, lambda { |command = nil|
+ shell_inst.help(command)
+ nil
+ }
+ # Export tools method for backwards compatibility
+ target.send :define_singleton_method, :tools, lambda {
+ shell_inst.help_group('tools')
+ nil
+ }
+ end
+
+ # Export HBase commands, constants, and variables to target receiver
+ def export_all(target)
+ raise ArgumentError, 'target should not be a module' if target.is_a?
Module
+
+ # add constants to class of target
+ target.class.include ::HBaseConstants
+ target.class.include ::HBaseQuotasConstants
+ # add instance variables @hbase and @shell for backwards compatibility
+ target.instance_variable_set :'@hbase', @hbase
+ target.instance_variable_set :'@shell', self
+ # add commands to target
+ export_commands(target)
end
def command_instance(command)
@@ -238,6 +289,61 @@ The HBase shell is the (J)Ruby IRB with the above
HBase-specific commands added.
For more on the HBase Shell, see http://hbase.apache.org/book.html
HERE
end
+
+ @irb_workspace = nil
+ ##
+ # Returns an IRB Workspace for this shell instance with all the IRB and
HBase commands installed
+ def get_workspace
+ return @irb_workspace unless @irb_workspace.nil?
+
+ hbase_receiver = HBaseReceiver.new
+ # Install all the hbase commands, constants, and instance variables
@shell and @hbase. This
+ # must be BEFORE the irb commands are installed so that our help command
is not overwritten.
+ export_all(hbase_receiver)
+ # install all the IRB commands onto our receiver
+ IRB::ExtendCommandBundle.extend_object(hbase_receiver)
+ ::IRB::WorkSpace.new(hbase_receiver.get_binding)
+ end
+
+ ##
+ # Read from an instance of Ruby's IO class and evaluate each line within
the shell's workspace
+ #
+ # Unlike Ruby's require or load, this method allows us to execute code
with a custom binding. In
+ # this case, we are using the binding constructed with all the HBase shell
constants and
+ # methods.
+ def eval_io(io)
+ require 'irb/ruby-lex'
+ # Mixing HBaseIOExtensions into IO allows us to pass IO objects to
RubyLex.
+ IO.include HBaseIOExtensions
+
+ workspace = get_workspace
+ scanner = RubyLex.new
+ scanner.set_input(io)
+
+ begin
+ scanner.each_top_level_statement do |statement, linenum|
+ puts(workspace.evaluate(nil, statement, 'stdin', linenum))
+ end
+ rescue Exception => e
+ message = e.to_s
+ # exception unwrapping in shell means we'll have to handle Java
exceptions
+ # as a special case in order to format them properly.
+ if e.is_a? java.lang.Exception
+ warn 'java exception'
+ message = e.get_message
+ end
+ # Include the 'ERROR' string to try to make transition easier for
scripts that
+ # may have already been relying on grepping output.
+ puts "ERROR #{e.class}: #{message}"
+ if $fullBacktrace
+ # re-raising the will include a backtrace and exit.
+ raise e
+ else
+ exit 1
+ end
+ end
+ nil
+ end
end
# rubocop:enable Metrics/ClassLength
end
diff --git a/hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb
b/hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb
index 8f0b35b..abc97591 100644
--- a/hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb
@@ -38,7 +38,7 @@ EOF
def command(snapshot_name, table, args = {})
raise(ArgumentError, 'Arguments should be a Hash') unless
args.is_a?(Hash)
- restore_acl = args.delete(RESTORE_ACL) || false
+ restore_acl = args.delete(::HBaseConstants::RESTORE_ACL) || false
admin.clone_snapshot(snapshot_name, table, restore_acl)
end
diff --git a/hbase-shell/src/main/ruby/shell/commands/describe.rb
b/hbase-shell/src/main/ruby/shell/commands/describe.rb
index 0553755..c32b77d 100644
--- a/hbase-shell/src/main/ruby/shell/commands/describe.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/describe.rb
@@ -48,7 +48,7 @@ EOF
# No QUOTAS if hbase:meta table
puts
formatter.header(%w[QUOTAS])
- count = quotas_admin.list_quotas(TABLE => table.to_s) do |_, quota|
+ count = quotas_admin.list_quotas(::HBaseConstants::TABLE =>
table.to_s) do |_, quota|
formatter.row([quota])
end
formatter.footer(count)
diff --git a/hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb
b/hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb
index fd14c45..a929887 100644
--- a/hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb
@@ -36,7 +36,7 @@ EOF
ns = namespace.to_s
if admin.exists?(::HBaseQuotasConstants::QUOTA_TABLE_NAME.to_s)
puts formatter.header(%w[QUOTAS])
- count = quotas_admin.list_quotas(NAMESPACE => ns) do |_, quota|
+ count = quotas_admin.list_quotas(::HBaseConstants::NAMESPACE => ns)
do |_, quota|
formatter.row([quota])
end
formatter.footer(count)
diff --git a/hbase-shell/src/main/ruby/shell/commands/list_regions.rb
b/hbase-shell/src/main/ruby/shell/commands/list_regions.rb
index 262ab1e..e003039 100644
--- a/hbase-shell/src/main/ruby/shell/commands/list_regions.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/list_regions.rb
@@ -47,7 +47,7 @@ EOF
elsif !options.is_a? Hash
# When options isn't a hash, assume it's the server name
# and create the hash internally
- options = { SERVER_NAME => options }
+ options = { ::HBaseConstants::SERVER_NAME => options }
end
raise "Table #{table_name} must be enabled." unless
admin.enabled?(table_name)
@@ -85,7 +85,7 @@ EOF
hregion_locator_instance =
conn_instance.getRegionLocator(TableName.valueOf(table_name))
hregion_locator_list =
hregion_locator_instance.getAllRegionLocations.to_a
results = []
- desired_server_name = options[SERVER_NAME]
+ desired_server_name = options[::HBaseConstants::SERVER_NAME]
begin
# Filter out region servers which we don't want, default to all RS
@@ -93,11 +93,16 @@ EOF
# A locality threshold of "1.0" would be all regions (cannot have
greater than 1 locality)
# Regions which have a `dataLocality` less-than-or-equal to this
value are accepted
locality_threshold = 1.0
- if options.key? LOCALITY_THRESHOLD
- value = options[LOCALITY_THRESHOLD]
+ if options.key? ::HBaseConstants::LOCALITY_THRESHOLD
+ value = options[::HBaseConstants::LOCALITY_THRESHOLD]
# Value validation. Must be a Float, and must be between [0, 1.0]
- raise "#{LOCALITY_THRESHOLD} must be a float value" unless
value.is_a? Float
- raise "#{LOCALITY_THRESHOLD} must be between 0 and 1.0, inclusive"
unless valid_locality_threshold? value
+ unless value.is_a? Float
+ raise "#{::HBaseConstants::LOCALITY_THRESHOLD} must be a float
value"
+ end
+ unless valid_locality_threshold? value
+ raise "#{::HBaseConstants::LOCALITY_THRESHOLD} must be between 0
and 1.0, inclusive"
+ end
+
locality_threshold = value
end
diff --git a/hbase-shell/src/main/ruby/shell/commands/restore_snapshot.rb
b/hbase-shell/src/main/ruby/shell/commands/restore_snapshot.rb
index be6ee3c..56168fa 100644
--- a/hbase-shell/src/main/ruby/shell/commands/restore_snapshot.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/restore_snapshot.rb
@@ -37,7 +37,7 @@ EOF
def command(snapshot_name, args = {})
raise(ArgumentError, 'Arguments should be a Hash') unless
args.is_a?(Hash)
- restore_acl = args.delete(RESTORE_ACL) || false
+ restore_acl = args.delete(::HBaseConstants::RESTORE_ACL) || false
admin.restore_snapshot(snapshot_name, restore_acl)
end
end
diff --git a/hbase-shell/src/main/ruby/shell/commands/set_quota.rb
b/hbase-shell/src/main/ruby/shell/commands/set_quota.rb
index 7e8e563..5a5669a 100644
--- a/hbase-shell/src/main/ruby/shell/commands/set_quota.rb
+++ b/hbase-shell/src/main/ruby/shell/commands/set_quota.rb
@@ -123,31 +123,36 @@ EOF
end
def command(args = {})
- if args.key?(TYPE)
- qtype = args.delete(TYPE)
+ if args.key?(::HBaseConstants::TYPE)
+ qtype = args.delete(::HBaseConstants::TYPE)
case qtype
- when THROTTLE
- if args[LIMIT].eql? NONE
- args.delete(LIMIT)
+ when ::HBaseQuotasConstants::THROTTLE
+ if args[::HBaseConstants::LIMIT].eql? ::HBaseConstants::NONE
+ args.delete(::HBaseConstants::LIMIT)
quotas_admin.unthrottle(args)
else
quotas_admin.throttle(args)
end
- when SPACE
- if args[LIMIT].eql? NONE
- args.delete(LIMIT)
+ when ::HBaseQuotasConstants::SPACE
+ if args[::HBaseConstants::LIMIT].eql? ::HBaseConstants::NONE
+ args.delete(::HBaseConstants::LIMIT)
# Table/Namespace argument is verified in remove_space_limit
quotas_admin.remove_space_limit(args)
else
- raise(ArgumentError, 'Expected a LIMIT to be provided') unless
args.key?(LIMIT)
- raise(ArgumentError, 'Expected a POLICY to be provided') unless
args.key?(POLICY)
+ unless args.key?(::HBaseConstants::LIMIT)
+ raise(ArgumentError, 'Expected a LIMIT to be provided')
+ end
+ unless args.key?(::HBaseConstants::POLICY)
+ raise(ArgumentError, 'Expected a POLICY to be provided')
+ end
+
quotas_admin.limit_space(args)
end
else
raise 'Invalid TYPE argument. got ' + qtype
end
- elsif args.key?(GLOBAL_BYPASS)
- quotas_admin.set_global_bypass(args.delete(GLOBAL_BYPASS), args)
+ elsif args.key?(::HBaseQuotasConstants::GLOBAL_BYPASS)
+
quotas_admin.set_global_bypass(args.delete(::HBaseQuotasConstants::GLOBAL_BYPASS),
args)
else
raise 'Expected TYPE argument'
end
diff --git a/hbase-shell/src/test/ruby/hbase/admin2_test.rb
b/hbase-shell/src/test/ruby/hbase/admin2_test.rb
index 16233c0..e420c74 100644
--- a/hbase-shell/src/test/ruby/hbase/admin2_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/admin2_test.rb
@@ -23,12 +23,11 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
module Hbase
# Tests for the `status` shell command
class StatusTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
@@ -126,7 +125,7 @@ module Hbase
define_test "Snapshot should work when SKIP_FLUSH args" do
drop_test_snapshot()
- command(:snapshot, @test_name, @create_test_snapshot, {SKIP_FLUSH =>
true})
+ command(:snapshot, @test_name, @create_test_snapshot,
{::HBaseConstants::SKIP_FLUSH => true})
list = command(:list_snapshots, @create_test_snapshot)
assert_equal(1, list.size)
end
@@ -156,7 +155,7 @@ module Hbase
assert_match(/f1/, admin.describe(restore_table))
assert_match(/f2/, admin.describe(restore_table))
command(:snapshot, restore_table, @create_test_snapshot)
- command(:alter, restore_table, METHOD => 'delete', NAME => 'f1')
+ command(:alter, restore_table, ::HBaseConstants::METHOD => 'delete',
::HBaseConstants::NAME => 'f1')
assert_no_match(/f1/, admin.describe(restore_table))
assert_match(/f2/, admin.describe(restore_table))
drop_test_table(restore_table)
diff --git a/hbase-shell/src/test/ruby/hbase/admin_test.rb
b/hbase-shell/src/test/ruby/hbase/admin_test.rb
index 11798c3..374d9c1 100644
--- a/hbase-shell/src/test/ruby/hbase/admin_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/admin_test.rb
@@ -23,7 +23,6 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
module Hbase
class AdminHelpersTest < Test::Unit::TestCase
@@ -63,6 +62,8 @@ module Hbase
# rubocop:disable Metrics/ClassLength
class AdminMethodsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
+ include HBaseQuotasConstants
def setup
setup_hbase
@@ -550,9 +551,9 @@ module Hbase
ns = @create_test_name
command(:create_namespace, ns)
command(:set_quota,
- TYPE => SPACE,
+ TYPE => ::HBaseQuotasConstants::SPACE,
LIMIT => '1G',
- POLICY => NO_INSERTS,
+ POLICY => ::HBaseQuotasConstants::NO_INSERTS,
NAMESPACE => ns)
output = capture_stdout { command(:describe_namespace, ns) }
puts output
@@ -567,8 +568,8 @@ module Hbase
assert(output.include?('1 row(s)'))
command(:set_quota,
- TYPE => SPACE,
- LIMIT => NONE,
+ TYPE => ::HBaseQuotasConstants::SPACE,
+ LIMIT => ::HBaseConstants::NONE,
NAMESPACE => ns)
output = capture_stdout { command(:describe_namespace, ns) }
assert(output.include?('0 row(s)'))
@@ -675,6 +676,8 @@ module Hbase
# Simple administration methods tests
class AdminCloneTableSchemaTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
+
def setup
setup_hbase
# Create table test table name
@@ -758,6 +761,8 @@ module Hbase
# Simple administration methods tests
class AdminRegionTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
+
def setup
setup_hbase
# Create test table if it does not exist
@@ -846,6 +851,7 @@ module Hbase
# rubocop:disable Metrics/ClassLength
class AdminAlterTableTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/hbase/list_regions_test_no_cluster.rb
b/hbase-shell/src/test/ruby/hbase/list_regions_test_no_cluster.rb
index 73d744c..6be2597 100644
--- a/hbase-shell/src/test/ruby/hbase/list_regions_test_no_cluster.rb
+++ b/hbase-shell/src/test/ruby/hbase/list_regions_test_no_cluster.rb
@@ -18,8 +18,6 @@
require 'shell'
require 'hbase_constants'
-include HBaseConstants
-
java_import 'org.apache.hadoop.hbase.HRegionLocation'
java_import 'org.apache.hadoop.hbase.client.RegionInfoBuilder'
java_import 'org.apache.hadoop.hbase.ServerName'
@@ -27,6 +25,7 @@ java_import 'org.apache.hadoop.hbase.ServerName'
module Hbase
class NoClusterListRegionsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
define_test 'valid_locality_values' do
command = ::Shell::Commands::ListRegions.new(nil)
diff --git a/hbase-shell/src/test/ruby/hbase/quotas_test.rb
b/hbase-shell/src/test/ruby/hbase/quotas_test.rb
index 9fddb83..c4fca28 100644
--- a/hbase-shell/src/test/ruby/hbase/quotas_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/quotas_test.rb
@@ -23,12 +23,12 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
module Hbase
# rubocop:disable Metrics/ClassLength
class SpaceQuotasTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
+ include HBaseQuotasConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/hbase/quotas_test_no_cluster.rb
b/hbase-shell/src/test/ruby/hbase/quotas_test_no_cluster.rb
index 7de1225..79f7350 100644
--- a/hbase-shell/src/test/ruby/hbase/quotas_test_no_cluster.rb
+++ b/hbase-shell/src/test/ruby/hbase/quotas_test_no_cluster.rb
@@ -23,8 +23,6 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
java_import org.apache.hadoop.hbase.quotas.SpaceQuotaSnapshot
java_import org.apache.hadoop.hbase.quotas.SpaceViolationPolicy
java_import org.apache.hadoop.hbase.TableName
@@ -32,6 +30,7 @@ java_import org.apache.hadoop.hbase.TableName
module Hbase
class NoClusterSpaceQuotasTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
define_test '_parse_size accepts various forms of byte shorthand' do
qa = ::Hbase::QuotasAdmin.new(nil)
diff --git a/hbase-shell/src/test/ruby/hbase/replication_admin_test.rb
b/hbase-shell/src/test/ruby/hbase/replication_admin_test.rb
index f6f41d2..72fbe94 100644
--- a/hbase-shell/src/test/ruby/hbase/replication_admin_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/replication_admin_test.rb
@@ -22,7 +22,6 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
include Java
java_import org.apache.hadoop.hbase.replication.SyncReplicationState
@@ -30,6 +29,7 @@ java_import
org.apache.hadoop.hbase.replication.SyncReplicationState
module Hbase
class ReplicationAdminTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
@peer_id = '1'
diff --git a/hbase-shell/src/test/ruby/hbase/security_admin_test.rb
b/hbase-shell/src/test/ruby/hbase/security_admin_test.rb
index e1360c2..6e9a50c 100644
--- a/hbase-shell/src/test/ruby/hbase/security_admin_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/security_admin_test.rb
@@ -22,12 +22,11 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
module Hbase
# Simple secure administration methods tests
class SecureAdminMethodsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/hbase/table_test.rb
b/hbase-shell/src/test/ruby/hbase/table_test.rb
index 0eef017..20bcb50 100644
--- a/hbase-shell/src/test/ruby/hbase/table_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/table_test.rb
@@ -19,12 +19,12 @@
require 'hbase_constants'
-include HBaseConstants
-
module Hbase
# Constructor tests
class TableConstructorTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
+
def setup
setup_hbase
end
@@ -97,6 +97,7 @@ module Hbase
# Simple data management methods tests
class TableSimpleMethodsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
@@ -225,6 +226,7 @@ module Hbase
# rubocop:disable Metrics/ClassLength
class TableComplexMethodsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/hbase/test_connection_no_cluster.rb
b/hbase-shell/src/test/ruby/hbase/test_connection_no_cluster.rb
index d240ff8..3619376 100644
--- a/hbase-shell/src/test/ruby/hbase/test_connection_no_cluster.rb
+++ b/hbase-shell/src/test/ruby/hbase/test_connection_no_cluster.rb
@@ -23,11 +23,10 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
module Hbase
class NoClusterConnectionTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
puts "starting shell"
diff --git a/hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb
b/hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb
index b42290f..e69710d 100644
--- a/hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb
+++ b/hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb
@@ -22,12 +22,11 @@ require 'hbase_constants'
require 'hbase/hbase'
require 'hbase/table'
-include HBaseConstants
-
module Hbase
# Simple secure administration methods tests
class VisibilityLabelsAdminMethodsTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/shell/converter_test.rb
b/hbase-shell/src/test/ruby/shell/converter_test.rb
index 8b6079b..51e6740 100644
--- a/hbase-shell/src/test/ruby/shell/converter_test.rb
+++ b/hbase-shell/src/test/ruby/shell/converter_test.rb
@@ -17,11 +17,10 @@
require 'hbase_constants'
require 'shell'
-include HBaseConstants
-
module Hbase
class ConverterTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
non_ascii_text = '⻆⻇'
non_ascii_row = '⻄'
diff --git a/hbase-shell/src/test/ruby/shell/list_procedures_test.rb
b/hbase-shell/src/test/ruby/shell/list_procedures_test.rb
index 1cad23b..2bf5824 100644
--- a/hbase-shell/src/test/ruby/shell/list_procedures_test.rb
+++ b/hbase-shell/src/test/ruby/shell/list_procedures_test.rb
@@ -20,11 +20,10 @@
require 'hbase_constants'
require 'shell'
-include HBaseConstants
-
module Hbase
class ListProceduresTest < Test::Unit::TestCase
include TestHelpers
+ include HBaseConstants
def setup
setup_hbase
diff --git a/hbase-shell/src/test/ruby/shell/shell_test.rb
b/hbase-shell/src/test/ruby/shell/shell_test.rb
index e7f5b26..9fd28c0 100644
--- a/hbase-shell/src/test/ruby/shell/shell_test.rb
+++ b/hbase-shell/src/test/ruby/shell/shell_test.rb
@@ -61,6 +61,46 @@ class ShellTest < Test::Unit::TestCase
#-------------------------------------------------------------------------------
+ define_test 'Shell::Shell#export_all export commands, constants, and
variables' do
+ module FooM; end
+ class FooC; end
+ foo = FooC.new
+
+ # export_all should reject classes and modules as targets
+ assert_raise(ArgumentError) do
+ @shell.export_all(FooM)
+ end
+ assert_raise(ArgumentError) do
+ @shell.export_all(FooC)
+ end
+
+ # For potency, verify that none of the commands, variables or constants
exist before export
+ assert(!foo.respond_to?(:version))
+ assert(foo.instance_variable_get(:'@shell').nil?)
+ assert(foo.instance_variable_get(:'@hbase').nil?)
+ assert(!foo.class.const_defined?(:IN_MEMORY_COMPACTION)) # From
HBaseConstants
+ assert(!foo.class.const_defined?(:QUOTA_TABLE_NAME)) # From
HBaseQuotasConstants
+
+ @shell.export_all(foo)
+
+ # Now verify that all the commands, variables, and constants are installed
+ assert(foo.respond_to?(:version))
+ assert(foo.instance_variable_get(:'@shell') == @shell)
+ assert(foo.instance_variable_get(:'@hbase') == @hbase)
+ assert(foo.class.const_defined?(:IN_MEMORY_COMPACTION)) # From
HBaseConstants
+ assert(foo.class.const_defined?(:QUOTA_TABLE_NAME)) # From
HBaseQuotasConstants
+
+ # commands should not exist on the class of target
+ assert_raise(NameError) do
+ FooC.method :version
+ end
+ assert_raise(NameError) do
+ FooC.instance_method :version
+ end
+ end
+
+
#-------------------------------------------------------------------------------
+
define_test "Shell::Shell#command_instance should return a command class" do
assert_kind_of(Shell::Commands::Command,
@shell.command_instance('version'))
end
@@ -73,6 +113,24 @@ class ShellTest < Test::Unit::TestCase
#-----------------------------------------------------------------------------
+ define_test 'Shell::Shell#eval_io should evaluate IO' do
+ # check that at least one of the commands is present while evaluating
+ io = StringIO.new <<~EOF
+ puts (respond_to? :list)
+ EOF
+ output = capture_stdout { @shell.eval_io(io) }
+ assert_match(/true/, output)
+
+ # check that at least one of the HBase constants is present while
evaluating
+ io = StringIO.new <<~EOF
+ ROWPREFIXFILTER
+ EOF
+ output = capture_stdout { @shell.eval_io(io) }
+ assert_match(/"ROWPREFIXFILTER"/, output)
+ end
+
+
#-----------------------------------------------------------------------------
+
define_test 'Shell::Shell#print_banner should display Reference Guide link'
do
@shell.interactive = true
output = capture_stdout { @shell.print_banner }
diff --git a/hbase-shell/src/test/ruby/test_helper.rb
b/hbase-shell/src/test/ruby/test_helper.rb
index 78fba7a..26b1426 100644
--- a/hbase-shell/src/test/ruby/test_helper.rb
+++ b/hbase-shell/src/test/ruby/test_helper.rb
@@ -119,7 +119,7 @@ module Hbase
def create_test_table_with_region_replicas(name, num_of_replicas, splits)
# Create the table if needed
unless admin.exists?(name)
- command(:create, name, 'f1', { REGION_REPLICATION => num_of_replicas },
+ command(:create, name, 'f1', { ::HBaseConstants::REGION_REPLICATION =>
num_of_replicas },
splits)
end
