[
https://issues.apache.org/jira/browse/HADOOP-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601817#comment-13601817
]
Chris Nauroth commented on HADOOP-9387:
---------------------------------------
Hi Ivan,
Thanks for working on cleaning it up. This code has tripped me up in the past.
{code}
public String getFilesystem() throws IOException {
if (Shell.WINDOWS) {
this.filesystem = dirFile.getCanonicalPath().substring(0, 2);
return this.filesystem;
} else {
run();
return filesystem;
}
}
{code}
It appears that there is a pitfall if running on non-Windows. If the user of
this class calls {{getFilesystem}} before calling
{{getMount}}, then {{filesystem}} will be uninitialized. This is because on
non-Windows, we assign to the {{filesystem}} member variable inside
{{parseOutput}}, which gets called from {{getMount}}, but not
{{getFilesystem}}. Perhaps it's safer to move all of the parsing logic back
into {{parseExecResult}}.
{code}
switch(getOSType()) {
case OS_TYPE_AIX:
Long.parseLong(tokens.nextToken()); // capacity
Long.parseLong(tokens.nextToken()); // available
Integer.parseInt(tokens.nextToken()); // pct used
tokens.nextToken();
tokens.nextToken();
this.mount = tokens.nextToken();
break;
case OS_TYPE_WIN:
case OS_TYPE_SOLARIS:
case OS_TYPE_MAC:
case OS_TYPE_UNIX:
default:
Long.parseLong(tokens.nextToken()); // capacity
Long.parseLong(tokens.nextToken()); // used
Long.parseLong(tokens.nextToken()); // available
Integer.parseInt(tokens.nextToken()); // pct used
this.mount = tokens.nextToken();
break;
}
{code}
The patch removes the special handling for AIX, so I think this would cause a
regression if running on that platform. I've never used AIX, but what I infer
from the old code is that the output of df -k on Linux places mount in column
6, whereas on AIX it goes in column 7. Therefore, we need an extra call to
{{StringTokenizer#nextToken}} if running on AIX. Unfortunately, I don't have
access to an AIX machine to confirm. Perhaps this code could be simplified to
always look at the last column without platform checks and special cases, i.e.:
{code}
String[] columns = line.split();
this.mount = columns[columns.length - 1];
{code}
However, that would be dependent on AIX printing mount in the last column, and
without an AIX machine, I can't confirm.
I did confirm that the patch works for Windows, so maybe the simplest path
forward is to prepare a smaller patch just for fixing Windows, and then file a
follow-up jira for future refactoring work. That would give more time to track
down someone with access to AIX to help with testing.
> TestDFVariations fails on Windows after the merge
> -------------------------------------------------
>
> Key: HADOOP-9387
> URL: https://issues.apache.org/jira/browse/HADOOP-9387
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-9387.trunk.2.patch, HADOOP-9387.trunk.patch
>
>
> Test fails with the following errors:
> {code}
> Running org.apache.hadoop.fs.TestDFVariations
> Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.186 sec <<<
> FAILURE!
> testOSParsing(org.apache.hadoop.fs.TestDFVariations) Time elapsed: 109 sec
> <<< ERROR!
> java.io.IOException: Fewer lines of output than expected
> at org.apache.hadoop.fs.DF.parseOutput(DF.java:203)
> at org.apache.hadoop.fs.DF.getMount(DF.java:150)
> at
> org.apache.hadoop.fs.TestDFVariations.testOSParsing(TestDFVariations.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
> testGetMountCurrentDirectory(org.apache.hadoop.fs.TestDFVariations) Time
> elapsed: 1 sec <<< ERROR!
> java.io.IOException: Fewer lines of output than expected
> at org.apache.hadoop.fs.DF.parseOutput(DF.java:203)
> at org.apache.hadoop.fs.DF.getMount(DF.java:150)
> at
> org.apache.hadoop.fs.TestDFVariations.testGetMountCurrentDirectory(TestDFVariations.java:139)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
