Re: [VOTE] Merge feature branch HADOOP-12930

[Masatake Iwasaki]

Hi Allen,

I had a problem when I was testing HADOOP-12930 branch. (built by mvn 
package -Pdist)

Single argument is ok.::

  [iwasakims@centos7 HADOOP-12930]$ bin/hadoop hello foo
  foo


Error on multiple arguments.  ::

  [iwasakims@centos7 HADOOP-12930]$ bin/hadoop hello foo bar
  bin/hadoop: line 200: foo bar: syntax error in expression (error 
token is "bar")
  Error: Could not find or load main class

  [iwasakims@centos7 HADOOP-12930]$ bin/hadoop distcp /foo /bar
  bin/hadoop: line 200: /foo /bar: syntax error: operand expected 
(error token is "/foo /bar")
  Error: Could not find or load main class


The fix of HADOOP-13120 seems to be related.::

  diff --git a/hadoop-common-project/hadoop-common/src/main/bin/hadoop 
b/hadoop-common-project/hadoop-common/src/main/bin/hadoop
  index 61fdc2e..0f6982b 100755
  --- a/hadoop-common-project/hadoop-common/src/main/bin/hadoop
  +++ b/hadoop-common-project/hadoop-common/src/main/bin/hadoop
  @@ -197,6 +197,7 @@ shift
   HADOOP_SUBCMD_ARGS=("$@")

   if declare -f hadoop_subcommand_"${HADOOP_SUBCMD}" >/dev/null 2>&1; then
  +  hadoop_debug "Calling dynamically: 
hadoop_subcommand_${HADOOP_SUBCMD} ${HADOOP_SUBCMD_ARGS[$*]}"
     "hadoop_subcommand_${HADOOP_SUBCMD}" "${HADOOP_SUBCMD_ARGS[@]}"
   else
     hadoopcmd_case "${HADOOP_SUBCMD}" "${HADOOP_SUBCMD_ARGS[@]}"


Thanks,
Masatake Iwasaki


On 5/15/16 02:33, Allen Wittenauer wrote:
This vote closes in 2 days and the only response has been from a non-committer 
and one of the 137 other committers on the project…. it’d be great if some 
others could take a look.

Thanks!

On May 12, 2016, at 6:07 PM, Andrew Wang <andrew.w...@cloudera.com> wrote:

+1. I looked at the patches on the branch, wasn't too bad to review. As
Allen said, there's some code movement, assorted other nice doc and shell
fixups.

Found one extra typo, which I added to HADOOP-13129.

Best,
Andrew

On Wed, May 11, 2016 at 1:14 AM, Sean Busbey <bus...@cloudera.com> wrote:

+1 (non-binding)

reviewed everything, filed an additional subtask for a very trivial
typo in the docs. should be fine to make a full issue after close and
then fix.

tried merging locally, tried running through new shell tests (both
with and without bats installed), tried making an example custom
command (valid and malformed). everything looks great.

On Mon, May 9, 2016 at 1:26 PM, Allen Wittenauer <a...@apache.org> wrote:
        Hey gang!

        I’d like to call a vote to run for 7 days (ending May 16 at
13:30 PT) to merge the HADOOP-12930 feature branch into trunk. This branch
was developed exclusively by me as per the discussion two months ago as a
way to make what would be a rather large patch hopefully easier to review.
The vast majority of the branch is code movement in the same file,
additional license headers, maven assembly hooks for distribution, and
variable renames. Not a whole lot of new code, but a big diff file
none-the-less.
        This branch modifies the ‘hadoop’, ‘hdfs’, ‘mapred’, and ‘yarn’
commands to allow for subcommands to be added or modified at runtime.  This
allows for individual users or entire sites to tweak the execution
environment to suit their local needs.  For example, it has been a practice
for some locations to change the distcp jar out for a custom one.  Using
this functionality, it is possible that the ‘hadoop distcp’ command could
run the local version without overwriting the bundled jar and for existing
documentation (read: results from Internet searches) to work as written
without modification. This has the potential to be a huge win, especially
for:
                * advanced end users looking to supplement the Apache
Hadoop experience
                * operations teams that may be able to leverage existing
documentation without having to remain local “exception” docs
                * development groups wanting an easy way to trial
experimental features
        Additionally, this branch includes the following, related
changes:
                * Adds the first unit tests for the ‘hadoop’ command
                * Adds the infrastructure for hdfs script testing and
the first unit test for the ‘hdfs’ command
                * Modifies the hadoop-tools components to be dynamic
rather than hard coded
                * Renames the shell profiles for hdfs, mapred, and yarn
to be consistent with other bundled profiles, including the ones introduced
in this branch
        Documentation, including a ‘hello world’-style example, is in
the UnixShellGuide markdown file.  (Of course!)
         I am at ApacheCon this week if anyone wants to discuss in-depth.

        Thanks!

P.S.,

        There are still two open sub-tasks.  These are blocked by other
issues so that we may add unit testing to the shell code in those
respective areas.  I’ll covert to full issues after HADOOP-12930 is closed.

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



--
busbey

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org