[
https://issues.apache.org/jira/browse/HADOOP-12857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189837#comment-15189837
]
Allen Wittenauer commented on HADOOP-12857:
-------------------------------------------
Manual run without unit tests:
| Vote | Subsystem | Runtime | Comment
============================================================================
| 0 | reexec | 0m 21s | Docker mode activated.
| 0 | shelldocs | 0m 4s | Shelldocs was not available.
| +1 | @author | 0m 0s | The patch does not contain any @author
| | | | tags.
| +1 | test4tests | 0m 0s | The patch appears to include 6 new or
| | | | modified test files.
| 0 | mvndep | 3m 48s | Maven dependency ordering for branch
| +1 | mvninstall | 11m 42s | trunk passed
| +1 | compile | 13m 16s | trunk passed
| +1 | mvnsite | 12m 55s | trunk passed
| +1 | mvneclipse | 2m 37s | trunk passed
| +1 | javadoc | 10m 23s | trunk passed
| 0 | mvndep | 0m 20s | Maven dependency ordering for patch
| +1 | mvninstall | 23m 27s | the patch passed
| +1 | compile | 11m 19s | the patch passed
| +1 | javac | 11m 19s | the patch passed
| +1 | mvnsite | 13m 4s | the patch passed
| +1 | mvneclipse | 0m 57s | the patch passed
| +1 | shellcheck | 0m 7s | The applied patch generated 0 new + 94
| | | | unchanged - 5 fixed = 94 total (was 99)
| +1 | whitespace | 0m 0s | Patch has no whitespace issues.
| +1 | xml | 0m 5s | The patch has no ill-formed XML file.
| +1 | javadoc | 10m 53s | the patch passed
| +1 | asflicense | 0m 24s | Patch does not generate ASF License
| | | | warnings.
| | | 116m 18s |
Manually running unit tests:
{code}
hadoop-common-project/hadoop-common$ mvn test -DskipTests -Pshelltest
[INFO] --- maven-antrun-plugin:1.7:run (common-test-bats-driver) @
hadoop-common ---
[INFO] Executing tasks
main:
[exec] Running bats -t hadoop_add_classpath.bats
[exec] 1..11
[exec] ok 1 hadoop_add_classpath (simple not exist)
[exec] ok 2 hadoop_add_classpath (simple wildcard not exist)
[exec] ok 3 hadoop_add_classpath (simple exist)
[exec] ok 4 hadoop_add_classpath (simple wildcard exist)
[exec] ok 5 hadoop_add_classpath (simple dupecheck)
[exec] ok 6 hadoop_add_classpath (default order)
[exec] ok 7 hadoop_add_classpath (after order)
[exec] ok 8 hadoop_add_classpath (before order)
[exec] ok 9 hadoop_add_classpath (simple dupecheck 2)
[exec] ok 10 hadoop_add_classpath (dupecheck 3)
[exec] ok 11 hadoop_add_classpath (complex ordering)
[exec] Running bats -t hadoop_add_colonpath.bats
[exec] 1..9
[exec] ok 1 hadoop_add_colonpath (simple not exist)
[exec] ok 2 hadoop_add_colonpath (simple exist)
[exec] ok 3 hadoop_add_colonpath (simple dupecheck)
[exec] ok 4 hadoop_add_colonpath (default order)
[exec] ok 5 hadoop_add_colonpath (after order)
[exec] ok 6 hadoop_add_colonpath (before order)
[exec] ok 7 hadoop_add_colonpath (simple dupecheck 2)
[exec] ok 8 hadoop_add_colonpath (dupecheck 3)
[exec] ok 9 hadoop_add_colonpath (complex ordering)
[exec] Running bats -t hadoop_add_common_to_classpath.bats
[exec] 1..3
[exec] ok 1 hadoop_add_common_to_classpath (negative)
[exec] ok 2 hadoop_add_common_to_classpath (positive)
[exec] ok 3 hadoop_add_common_to_classpath (build paths)
[exec] Running bats -t hadoop_add_javalibpath.bats
[exec] 1..9
[exec] ok 1 hadoop_add_javalibpath (simple not exist)
[exec] ok 2 hadoop_add_javalibpath (simple exist)
[exec] ok 3 hadoop_add_javalibpath (simple dupecheck)
[exec] ok 4 hadoop_add_javalibpath (default order)
[exec] ok 5 hadoop_add_javalibpath (after order)
[exec] ok 6 hadoop_add_javalibpath (before order)
[exec] ok 7 hadoop_add_javalibpath (simple dupecheck 2)
[exec] ok 8 hadoop_add_javalibpath (dupecheck 3)
[exec] ok 9 hadoop_add_javalibpath (complex ordering)
[exec] Running bats -t hadoop_add_ldlibpath.bats
[exec] 1..9
[exec] ok 1 hadoop_add_ldlibpath (simple not exist)
[exec] ok 2 hadoop_add_ldlibpath (simple exist)
[exec] ok 3 hadoop_add_ldlibpath (simple dupecheck)
[exec] ok 4 hadoop_add_ldlibpath (default order)
[exec] ok 5 hadoop_add_ldlibpath (after order)
[exec] ok 6 hadoop_add_ldlibpath (before order)
[exec] ok 7 hadoop_add_ldlibpath (simple dupecheck 2)
[exec] ok 8 hadoop_add_ldlibpath (dupecheck 3)
[exec] ok 9 hadoop_add_ldlibpath (complex ordering)
[exec] Running bats -t hadoop_add_param.bats
[exec] 1..4
[exec] ok 1 hadoop_add_param (positive 1)
[exec] ok 2 hadoop_add_param (negative)
[exec] ok 3 hadoop_add_param (positive 2)
[exec] ok 4 hadoop_add_param (positive 3)
[exec] Running bats -t hadoop_add_to_classpath_tools.bats
[exec] 1..3
[exec] ok 1 hadoop_classpath_tools (load)
[exec] ok 2 hadoop_classpath_tools (not exist)
[exec] ok 3 hadoop_classpath_tools (function)
[exec] Running bats -t hadoop_add_to_classpath_userpath.bats
[exec] 1..7
[exec] ok 1 hadoop_add_to_classpath_userpath (nothing)
[exec] ok 2 hadoop_add_to_classpath_userpath (none)
[exec] ok 3 hadoop_add_to_classpath_userpath (only)
[exec] ok 4 hadoop_add_to_classpath_userpath (classloader)
[exec] ok 5 hadoop_add_to_classpath_userpath (1+1 dupe)
[exec] ok 6 hadoop_add_to_classpath_userpath (3+2 after)
[exec] ok 7 hadoop_add_to_classpath_userpath (3+2 before)
[exec] Running bats -t hadoop_basic_init.bats
[exec] 1..3
[exec] ok 1 hadoop_basic_init (bad dir errors)
[exec] ok 2 hadoop_basic_init (no non-dir overrides)
[exec] ok 3 hadoop_basic_init (test non-dir overrides)
[exec] Running bats -t hadoop_bootstrap.bats
[exec] 1..2
[exec] ok 1 hadoop_bootstrap (no libexec)
[exec] ok 2 hadoop_bootstrap (libexec)
[exec] Running bats -t hadoop_confdir.bats
[exec] 1..9
[exec] ok 1 hadoop_find_confdir (default)
[exec] ok 2 hadoop_find_confdir (bw compat: conf)
[exec] ok 3 hadoop_find_confdir (etc/hadoop)
[exec] ok 4 hadoop_verify_confdir (negative)
[exec] ok 5 hadoop_verify_confdir (positive)
[exec] ok 6 hadoop_exec_hadoopenv (positive)
[exec] ok 7 hadoop_exec_hadoopenv (negative)
[exec] ok 8 hadoop_exec_userfuncs
[exec] ok 9 hadoop_exec_hadooprc
[exec] Running bats -t hadoop_deprecate_envvar.bats
[exec] 1..2
[exec] ok 1 hadoop_deprecate_envvar (replace)
[exec] ok 2 hadoop_deprecate_envvar (no replace)
[exec] Running bats -t hadoop_entry_tests.bats
[exec] 1..4
[exec] ok 1 hadoop_add_entry (positive 1)
[exec] ok 2 hadoop_add_entry (negative)
[exec] ok 3 hadoop_add_entry (positive 2)
[exec] ok 4 hadoop_add_entry (positive 3)
[exec] Running bats -t hadoop_finalize.bats
[exec] 1..11
[exec] ok 1 hadoop_finalize (shellprofiles)
[exec] ok 2 hadoop_finalize (classpath)
[exec] ok 3 hadoop_finalize (libpaths)
[exec] ok 4 hadoop_finalize (heap)
[exec] ok 5 hadoop_finalize (opts)
[exec] ok 6 hadoop_finalize (cygwin prefix)
[exec] ok 7 hadoop_finalize (cygwin conf dir)
[exec] ok 8 hadoop_finalize (cygwin common)
[exec] ok 9 hadoop_finalize (cygwin hdfs)
[exec] ok 10 hadoop_finalize (cygwin yarn)
[exec] ok 11 hadoop_finalize (cygwin mapred)
[exec] Running bats -t hadoop_finalize_catalina_opts.bats
[exec] 1..2
[exec] ok 1 hadoop_finalize_catalina_opts (raw)
[exec] ok 2 # skip (catalina commands not supported under cygwin yet)
hadoop_finalize_catalina_opts (cygwin)
[exec] Running bats -t hadoop_finalize_classpath.bats
[exec] 1..4
[exec] ok 1 hadoop_finalize_classpath (only conf dir)
[exec] ok 2 hadoop_finalize_classpath (before conf dir)
[exec] ok 3 hadoop_finalize_classpath (adds user)
[exec] ok 4 hadoop_finalize_classpath (calls cygwin)
[exec] Running bats -t hadoop_finalize_hadoop_heap.bats
[exec] 1..8
[exec] ok 1 hadoop_finalize_hadoop_heap (negative)
[exec] ok 2 hadoop_finalize_hadoop_heap (no unit max)
[exec] ok 3 hadoop_finalize_hadoop_heap (no unit old)
[exec] ok 4 hadoop_finalize_hadoop_heap (unit max)
[exec] ok 5 hadoop_finalize_hadoop_heap (unit old)
[exec] ok 6 hadoop_finalize_hadoop_heap (no unit min)
[exec] ok 7 hadoop_finalize_hadoop_heap (unit min)
[exec] ok 8 hadoop_finalize_hadoop_heap (dedupe)
[exec] Running bats -t hadoop_finalize_hadoop_opts.bats
[exec] 1..2
[exec] ok 1 hadoop_finalize_hadoop_opts (raw)
[exec] ok 2 hadoop_finalize_hadoop_opts (cygwin)
[exec] Running bats -t hadoop_finalize_libpaths.bats
[exec] 1..2
[exec] ok 1 hadoop_finalize_libpaths (negative)
[exec] ok 2 hadoop_finalize_libpaths (positive)
[exec] Running bats -t hadoop_java_setup.bats
[exec] 1..4
[exec] ok 1 hadoop_java_setup (negative not set)
[exec] ok 2 hadoop_java_setup (negative not a dir)
[exec] ok 3 hadoop_java_setup (negative not exec)
[exec] ok 4 hadoop_java_setup (positive)
[exec] Running bats -t hadoop_os_tricks.bats
[exec] 1..3
[exec] ok 1 hadoop_os_tricks (cygwin sets cygwin)
[exec] ok 2 hadoop_os_tricks (linux sets arena max)
[exec] ok 3 hadoop_os_tricks (osx sets java_home)
[exec] Running bats -t hadoop_rotate_log.bats
[exec] 1..4
[exec] ok 1 hadoop_rotate_log (defaults)
[exec] ok 2 hadoop_rotate_log (one archive log)
[exec] ok 3 hadoop_rotate_log (default five archive logs)
[exec] ok 4 hadoop_rotate_log (ten archive logs)
[exec] Running bats -t hadoop_shellprofile.bats
[exec] 1..9
[exec] ok 1 hadoop_import_shellprofiles (negative)
[exec] ok 2 hadoop_import_shellprofiles (libexec sh import)
[exec] ok 3 hadoop_import_shellprofiles (libexec conf sh import+override)
[exec] ok 4 hadoop_import_shellprofiles (libexec no cmd import)
[exec] ok 5 hadoop_import_shellprofiles (H_O_T)
[exec] ok 6 hadoop_add_profile+hadoop_shellprofiles_init
[exec] ok 7 hadoop_add_profile+hadoop_shellprofiles_classpath
[exec] ok 8 hadoop_add_profile+hadoop_shellprofiles_nativelib
[exec] ok 9 hadoop_add_profile+hadoop_shellprofiles_finalize
[exec] Running bats -t hadoop_slaves.bats
[exec] 1..3
[exec] ok 1 hadoop_populate_slaves_file (specific file)
[exec] ok 2 hadoop_populate_slaves_file (specific conf dir file)
[exec] ok 3 hadoop_populate_slaves_file (no file)
[exec] Running bats -t hadoop_ssh.bats
[exec] 1..7
[exec] ok 1 # skip (Not implemented) hadoop_actual_ssh
[exec] ok 2 # skip (Not implemented) hadoop_connect_to_hosts
[exec] ok 3 # skip (Not implemented) hadoop_connect_to_hosts_without_pdsh
[exec] ok 4 hadoop_common_slave_mode_execute (--slaves 1)
[exec] ok 5 hadoop_common_slave_mode_execute (--slaves 2)
[exec] ok 6 hadoop_common_slave_mode_execute (--hosts)
[exec] ok 7 hadoop_common_slave_mode_execute (--hostnames 2)
[exec] Running bats -t hadoop_stop_daemon.bats
[exec] 1..1
[exec] ok 1 hadoop_stop_daemon
[exec] Running bats -t hadoop_stop_secure_daemon.bats
[exec] 1..1
[exec] ok 1 hadoop_stop_secure_daemon
[exec] Running bats -t hadoop_translate_cygwin_path.bats
[exec] 1..3
[exec] ok 1 hadoop_translate_cygwin_path (negative)
[exec] ok 2 hadoop_translate_cygwin_path (positive)
[exec] ok 3 hadoop_translate_cygwin_path (path positive)
[exec] Running bats -t hadoop_validate_classname.bats
[exec] 1..2
[exec] ok 1 hadoop_validate_classname (negative)
[exec] ok 2 hadoop_validate_classname (positive)
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51.188s
[INFO] Finished at: Thu Mar 10 11:42:32 PST 2016
[INFO] Final Memory: 26M/84M
[INFO] ------------------------------------------------------------------------
{code}
> Rework hadoop-tools
> -------------------
>
> Key: HADOOP-12857
> URL: https://issues.apache.org/jira/browse/HADOOP-12857
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Allen Wittenauer
> Attachments: HADOOP-12857.00.patch, HADOOP-12857.01.patch,
> HADOOP-12857.02.patch
>
>
> As hadoop-tools grows bigger and bigger, it's becoming evident that having a
> single directory that gets sucked in is starting to become a big burden as
> the number of tools grows. Let's rework this to be smarter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
