Kazuho Fujii created HADOOP-12830:
-------------------------------------
Summary: Bash environment for quick command operations
Key: HADOOP-12830
URL: https://issues.apache.org/jira/browse/HADOOP-12830
Project: Hadoop Common
Issue Type: New Feature
Components: bin
Reporter: Kazuho Fujii
Assignee: Kazuho Fujii
Hadoop file system shell commands are slow. This issue is about building a
shell environment for quick command operations.
Previously an interactive shell is tried to build in HADOOP-6541. But, it seems
to be poor because users are used to powerful shells like bash. This issue is
not about creating a new shell, but just opening a new bash process. Therefore,
user can operate commands as before.
{code}
fjk@x240:~/hadoop-2.7.2$ ./bin/hadoop shell
fjk@x240 hadoop> hadoop fs -ls /
Found 2 items
-rw-r--r-- 3 fjk supergroup 0 2016-02-21 00:26 /file1
-rw-r--r-- 3 fjk supergroup 0 2016-02-21 00:26 /file2
{code}
The shell has a mini daemon process that is living until the shell is closed.
The hadoop fs command delegates the operation to the daemon. They communicate
with named pipes. The daemon conducts the operation and returns the result to
the command.
In this shell the hadoop fs commands operation becomes quick. In a local
environment, "hadoop fs -ls" command is about 100 times faster than the normal
command.
{code}
fjk@x240 hadoop> time hadoop fs -ls hdfs://localhost:8020/ > /dev/null
real 0m0.021s
user 0m0.003s
sys 0m0.011s
{code}
Using bash's function, commands and file names are automatically completed.
{code}
fjk@x240 hadoop> hadoop fs -ch<TAB><TAB>
-checksum -chgrp -chmod -chown
fjk@x240 hadoop> hadoop fs -ls /file<TAB><TAB>
/file1 /file2 /file3
{code}
Additionally, we can make equivalents with bash build-in commands, e.g., cd,
umask. In this shell, they can work because the daemon remembers the state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
