[
https://issues.apache.org/jira/browse/HADOOP-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041776#comment-15041776
]
Chris Nauroth commented on HADOOP-12603:
----------------------------------------
bq. do we need to care about atime at all, given we explicitly recommend
turning atime off on filesystems used for mounting HDFS.
We also need to keep in mind that Hadoop code can trigger local file system
interactions on client machines, external to the Hadoop cluster itself. The
standard recommendation to disable atime is less relevant on these client
machines. In some environments, these machines might not be governed by the
same system administrators that operate the cluster nodes. It might not be
feasible for them to enforce a noatime policy everywhere.
One use case for setting atime on the local file system is {{hdfs dfs
-copyToLocal -p}}. The expected semantics are to retain the atime stored in
HDFS when the file lands on the local file system. Prior to HADOOP-12045, that
wouldn't have worked correctly.
bq. Or is it that hdfs itself does atimes, and we want filesystems to be
roughly consistent?
Yes, I do also think it's good to make the semantics consistent across our file
system implementations as much as possible, unless there are compelling
technical reasons that prevent it.
I admit that setting local file system atime through the Hadoop code probably
isn't a very critical thing. However, I see it as a nice-to-have based on the
copyToLocal use case and the consistency argument. It was enough to motivate
me to review and commit HADOOP-12045.
> TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir occasionally fail
> ------------------------------------------------------------------------
>
> Key: HADOOP-12603
> URL: https://issues.apache.org/jira/browse/HADOOP-12603
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Labels: solaris, windows
> Attachments: HADOOP-12603.001.patch, HADOOP-12603.002.patch
>
>
> I have observed this test failure a few times in the past. When it fails, the
> expected access time (of the file link) is always 1000 less than the actual
> access time.
> Error Message
> {noformat}
> expected:<1448478654000> but was:<1448478655000>
> {noformat}
> Stacktrace
> {noformat}
> java.lang.AssertionError: expected:<1448478654000> but was:<1448478655000>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.fs.SymlinkBaseTest.testSetTimesSymlinkToDir(SymlinkBaseTest.java:1391)
> at
> org.apache.hadoop.fs.TestSymlinkLocalFS.testSetTimesSymlinkToDir(TestSymlinkLocalFS.java:233)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
> Standard Output
> {noformat}
> 2015-11-25 19:10:55,231 WARN fs.FileUtil (FileUtil.java:symLink(813)) -
> Command 'ln -s
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test1/file
>
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test2/linkToFile'
> failed 1 with: ln: failed to create symbolic link
> '/testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test2/linkToFile':
> No such file or directory
> 2015-11-25 19:10:56,212 WARN fs.FileUtil (FileUtil.java:symLink(813)) -
> Command 'ln -s
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test1/file
>
> /testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test1/linkToFile'
> failed 1 with: ln: failed to create symbolic link
> '/testptch/hadoop/hadoop-common-project/hadoop-common/target/test/data/4/vae1ng5t75/test1/linkToFile':
> File exists
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
