[
https://issues.apache.org/jira/browse/HADOOP-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518470
]
stack commented on HADOOP-1622:
-------------------------------
I tried running latest patch and failed with below:
{code}
durruti:~/Documents/checkouts/hadoop-commit stack$ more
build/test/TEST-org.apache.hadoop.util.TestJarUtils.txt
Testsuite: org.apache.hadoop.util.TestJarUtils
Tests run: 5, Failures: 4, Errors: 0, Time elapsed: 0.258 sec
------------- Standard Output ---------------
2007-08-08 08:00:01,272 INFO util.JarUtils (JarUtils.java:jarAll(311)) -
Adding file2:102 to test-out2.jar.
2007-08-08 08:00:01,276 INFO util.JarUtils (JarUtils.java:jarAll(229)) -
Adding dir1/dir1-1/dir1-2/file1:0 to test-out2.jar.
2007-08-08 08:00:01,279 INFO util.JarUtils (JarUtils.java:jarAll(229)) -
Adding dir2/dir2-1/file2:0 to test-out2.jar.
2007-08-08 08:00:01,280 INFO util.JarUtils (JarUtils.java:jarAll(229)) -
Adding dir3/file3:0 to test-out2.jar.
2007-08-08 08:00:01,281 INFO util.JarUtils (JarUtils.java:jarAll(257)) -
Adding file4.txt:0 to test-out2.jar.
------------- ---------------- ---------------
Testcase: testGetJarPath took 0.175 sec
FAILED
expected:<.../dir1-2/...> but was:<...\dir1-2\...>
junit.framework.ComparisonFailure: expected:<.../dir1-2/...> but
was:<...\dir1-2\...>
at
org.apache.hadoop.util.TestJarUtils.testGetJarPath(TestJarUtils.java:88)
Testcase: testJar took 0.022 sec
FAILED
null
junit.framework.AssertionFailedError
at org.apache.hadoop.util.TestJarUtils.testJar(TestJarUtils.java:111)
Testcase: testIsJarOrZip took 0.015 sec
Testcase: testJarAll took 0.024 sec
FAILED
null
junit.framework.AssertionFailedError
at org.apache.hadoop.util.TestJarUtils.testJarAll(TestJarUtils.java:164)
Testcase: testCopyJarContents took 0.018 sec
FAILED
null
junit.framework.AssertionFailedError
at
org.apache.hadoop.util.TestJarUtils.testCopyJarContents(TestJarUtils.java:218)
{code}
If you are going to make a new patch, here's a couple of other things you could
fix:
License missing on unit test.
Creating files in tests, other folks seem to do something like following to
keep them under configured test directory:
System.getProperty("test.build.data","."). You might want to do the same.
Next time, to save yourself a bit of typing, you could use the local file
system in hdfs to do the recursive delete of a directory (I'm guessing thats
why you individually remove each of the items in the teardown):
{code}
FileSystem fs = FileSystem.getLocal(new Configuration());
fs.delete(new Path(f.toString()));
{code}
In isJarOrZipFile, you could wrap your read of 5 bytes in a try/finally so the
close always happens. In fact there are a bunch of places that could do w/
try/finally blocks (Its not critical in the usual case. The job jar will just
error out w/o leaving hanging open files).
> Hadoop should provide a way to allow the user to specify jar file(s) the user
> job depends on
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-1622
> URL: https://issues.apache.org/jira/browse/HADOOP-1622
> Project: Hadoop
> Issue Type: Improvement
> Reporter: Runping Qi
> Attachments: multipleJobJars.patch, multipleJobResources.patch,
> multipleJobResources2.patch
>
>
> More likely than not, a user's job may depend on multiple jars.
> Right now, when submitting a job through bin/hadoop, there is no way for the
> user to specify that.
> A walk around for that is to re-package all the dependent jars into a new jar
> or put the dependent jar files in the lib dir of the new jar.
> This walk around causes unnecessary inconvenience to the user. Furthermore,
> if the user does not own the main function
> (like the case when the user uses Aggregate, or datajoin, streaming), the
> user has to re-package those system jar files too.
> It is much desired that hadoop provides a clean and simple way for the user
> to specify a list of dependent jar files at the time
> of job submission. Someting like:
> bin/hadoop .... --depending_jars j1.jar:j2.jar
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
