Hi all,
Recently, i have met a problem, and i can not solve it after some
efforts. So I wanna look for help here, and any help will be appreciated.
Thanks!
My case is depicted as below:
I want to execute the HiveQL command :
select transform(a.col) using '/home/pc/mypython.py' as (col string) from
tmp_table a where a.col2='01';
where the 'mypython.py' is a python script of mine.
I have built a environment of hadoop within the vmware machine on my single
node PC-home, and the command works well on this environment within only single
node.
I also have a cluster of three PC servers, including node A, B, and C.
Then, I store the '/home/pc/mypython.py' on node A.
However, every time I issue the command to the cluster, i am always going to
get the error information like this:
-------------------------------------------------------------------------------------------------------------------
Caused by: java.io.IOException: Cannot run program "/home/pc/mypython.py":
java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at
org.apache.hadoop.hive.ql.exec.ScriptOperator.processOp(ScriptOperator.java:279)
... 20 more
Caused by: java.io.IOException: java.io.IOException: error=2, No such file
or directory
at java.lang.UNIXProcess.(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 21 more
-------------------------------------------------------------------------------------------------------------------
By looking up the Job logs, these errors were reported by node B and node C. It
seems that the tasktracker B and C can not find the script.
On hive wiki, I didn't find any instruction on how to place the user script.
What should I do to place my script in proper place?
Thanks in advance for any reply!
2011-03-01
Jianhua Wang
