[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102666#comment-15102666 ] Ravi Prakash commented on HADOOP-12689: --- Thanks for the heads up Vinod! I must have missed the notification of that branch-2.8 being cut. I've cherry-picked this patch into branch-2.8 as well. > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089708#comment-15089708 ] Steve Loughran commented on HADOOP-12689: - removing -1, as yes, there are tests now. I just didn't want those tests to get forgotten about. > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089792#comment-15089792 ] Ravi Prakash commented on HADOOP-12689: --- Thanks for your consideration Steve! I appreciate all the great work you did for the contract tests and your efforts to keep the S3 implementations stable. I'm testing and reviewing Matt's patch on HADOOP-12696. Let's continue there. > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090006#comment-15090006 ] Steve Loughran commented on HADOOP-12689: - w.r.t to HADOOP-10542, I though there were tests in the system. i shouldn't have committed it then —and we are all suffering now because of that > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089524#comment-15089524 ] Ravi Prakash commented on HADOOP-12689: --- Steve! Your -1 here is completely unwarranted. 1. I don't remember a resolution overriding the ability of committers to commit patches without tests. If all patches must contain tests, I'm happy to go about -1ing all the JIRAs which have been committed already without a patch. 2. I shouldn't have to point out the hypocrisy here. *YOU* yourself committed HADOOP-10542 without tests. I'd ask you to reconsider your -1. If you insist, I'll roll the patch back. As it were, Matt *IS* working on adding tests. So in this case "later" did NOT mean "never". Thanks > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[
https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087169#comment-15087169
]
Steve Loughran commented on HADOOP-12689:
-
-1
Ravi. please, no: not without new tests to show this problem is fixed. if the
last fix was a regression, then we need something more to show the regression
has gone away. That doesn't have to be a general purpose contract test,
something in in hadoop-tools/hadoop-aws package.
look at the jenkins
{code}
The patch doesn't appear to include any new or modified tests. Please justify
why no new tests are needed for this patch. Also please list what manual steps
were performed to verify this patch.
{code}
S3n and siblings are burning sore in the Hadoop codebase: undermaintained,
undertested and incredibly brittle to change.
If HADOOP-10542 did break things —and I trust your claim there— then it slipped
through the current s3 test suite. We need another test to make sure this
problem never comes back. It doesn't have to be a full contract test, something
in hadoop-aws will be enough. But saying "we can add a test later" isn't the
right tactic —we both know "later" means "never" in this context. We also need
all the existing s3 tests run to make sure this patch doesn't change anything
else, either.
Can we roll this back and do another iteration of the patch which does include
a test? As mandating the "patches include tests" policy is the only way we can
keep test coverage up, especially on something this brittle
sorry
> S3 filesystem operations stopped working correctly
> --
>
> Key: HADOOP-12689
> URL: https://issues.apache.org/jira/browse/HADOOP-12689
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
>Affects Versions: 2.7.0
>Reporter: Matthew Paduano
>Assignee: Matthew Paduano
> Labels: S3
> Fix For: 2.8.0
>
> Attachments: HADOOP-12689.01.patch
>
>
> HADOOP-10542 was resolved by replacing "return null;" with throwing
> IOException. This causes several S3 filesystem operations to fail (possibly
> more code is expecting that null return value; these are just the calls I
> noticed):
> S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException
> but instead IOException)
> FileSystem.exists() (which no longer returns false but instead raises
> IOException)
> S3FileSystem.create() (which no longer succeeds but instead raises
> IOException)
> Run command:
> hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/
> Resulting stack trace:
> 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test
> doesn't exist
> at
> org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170)
> at
> org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source)
> at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230)
> at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> changing the "raise IOE..." to "return null" fixes all of the above code
> sites and allows distcp to succeed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083702#comment-15083702 ] Ravi Prakash commented on HADOOP-12689: --- Thanks for all your work with the FileSystem contracts, test and documentation Steve! Much appreciated. Thanks for pointing to the documentation. I was not aware of it. We are running with this patch inside Altiscale. I tested it also on my local laptop against a bucket in the "us-west-1" region (with a simple command). Matt also pointed me to http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/testing.html although he couldn't find tests for S3 (for S3n and S3a they exist). As an aside, I don't see the manual test results for HADOOP-10542 in that JIRA either. Were they run? > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085150#comment-15085150 ] Ravi Prakash commented on HADOOP-12689: --- I've committed this change to trunk and branch-2. Thanks for your contribution Matt and for your review Steve! > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Fix For: 2.8.0 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085137#comment-15085137 ] Hudson commented on HADOOP-12689: - FAILURE: Integrated in Hadoop-trunk-Commit #9055 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9055/]) HADOOP-12689. S3 filesystem operations stopped working correctly (raviprak: rev 2d16f40dab291a29b3fc005221b12fd587615d4e) * hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java * hadoop-common-project/hadoop-common/CHANGES.txt > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082910#comment-15082910 ] Steve Loughran commented on HADOOP-12689: - If there's a regression that hasn't been caught, we need tests to verify that the regression is not only fixed, that it stays fixed. w.r.t applying/submitting patches against the object stores, it is critical that the tests are manually run with the credentials: https://wiki.apache.org/hadoop/HowToContribute#Submitting_patches_against_object_stores_such_as_Amazon_S3.2C_OpenStack_Swift_and_Microsoft_Azure [~raviprakash] : I'm afraid that applies to you too. Please don't commit this without the test runs and the results logged here. > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083793#comment-15083793 ] Ravi Prakash commented on HADOOP-12689: --- In light of the above, I am still on track to commit this patch EOD. Matt, if you'd like, please feel free to create a new JIRA for adding the contract tests for S3 (although I fear it may be going the way of the dinosaur). I just don't want that effort blocking this patch. [~steve_l] Please let me know if you still have objections. > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Affects Versions: 2.7.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano > Labels: S3 > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[
https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15083775#comment-15083775
]
Ravi Prakash commented on HADOOP-12689:
---
Here's my log of what I ran
{code}
[raviprak@ravi trunk]$ hdfs dfs -get s3:///
-get: Fatal internal error
java.lang.NullPointerException
at
org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveBlock(Jets3tFileSystemStore.java:248)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:255)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at com.sun.proxy.$Proxy4.retrieveBlock(Unknown Source)
at
org.apache.hadoop.fs.s3.S3InputStream.blockSeekTo(S3InputStream.java:170)
at org.apache.hadoop.fs.s3.S3InputStream.read(S3InputStream.java:129)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:88)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:122)
at
org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWithDestination.java:482)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:404)
at
org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:339)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:274)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:259)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:321)
at
org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:293)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:254)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:275)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:259)
at
org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:225)
at
org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:119)
at org.apache.hadoop.fs.shell.Command.run(Command.java:166)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:319)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:377)
{code}
After the patch
{code}
[raviprak@ravi trunk]$ hdfs dfs -get s3:///
get: Block missing from S3 store: block_-3447106738088433847
{code}
When the file is valid, I was successfully able to get the file
> S3 filesystem operations stopped working correctly
> --
>
> Key: HADOOP-12689
> URL: https://issues.apache.org/jira/browse/HADOOP-12689
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
>Affects Versions: 2.7.0
>Reporter: Matthew Paduano
>Assignee: Matthew Paduano
> Labels: S3
> Attachments: HADOOP-12689.01.patch
>
>
> HADOOP-10542 was resolved by replacing "return null;" with throwing
> IOException. This causes several S3 filesystem operations to fail (possibly
> more code is expecting that null return value; these are just the calls I
> noticed):
> S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException
> but instead IOException)
> FileSystem.exists() (which no longer returns false but instead raises
> IOException)
> S3FileSystem.create() (which no longer succeeds but instead raises
> IOException)
> Run command:
> hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/
> Resulting stack trace:
> 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test
> doesn't exist
> at
> org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170)
> at
> org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082024#comment-15082024 ] Matthew Paduano commented on HADOOP-12689: -- The attached patch reverts the one line edit by HADOOP-10542 and includes code to address the possible NPE if a block is requested but not found in the S3 store. This seems the easiest way to resolve the possible NPE from 10542 and not rewrite many of the S3 file system operations that expect null to be returned by get() when an object is missing (and/or a FileNotFoundExecption to be raised, not IOException) > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano >Priority: Minor > Fix For: 2.7.1 > > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[
https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082050#comment-15082050
]
Hadoop QA commented on HADOOP-12689:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 10s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.8.0_66.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 11s
{color} | {color:green} hadoop-aws in the patch passed with JDK v1.7.0_91.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 6s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12780416/HADOOP-12689.01.patch
|
| JIRA Issue | HADOOP-12689 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a6525f1c552e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
[jira] [Commented] (HADOOP-12689) S3 filesystem operations stopped working correctly
[ https://issues.apache.org/jira/browse/HADOOP-12689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082146#comment-15082146 ] Ravi Prakash commented on HADOOP-12689: --- Patch looks good to me. Thanks Matt. I'll commit by EOD tomorrow if there are no objections > S3 filesystem operations stopped working correctly > -- > > Key: HADOOP-12689 > URL: https://issues.apache.org/jira/browse/HADOOP-12689 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0 >Reporter: Matthew Paduano >Assignee: Matthew Paduano >Priority: Minor > Attachments: HADOOP-12689.01.patch > > > HADOOP-10542 was resolved by replacing "return null;" with throwing > IOException. This causes several S3 filesystem operations to fail (possibly > more code is expecting that null return value; these are just the calls I > noticed): > S3FileSystem.getFileStatus() (which no longer raises FileNotFoundException > but instead IOException) > FileSystem.exists() (which no longer returns false but instead raises > IOException) > S3FileSystem.create() (which no longer succeeds but instead raises > IOException) > Run command: > hadoop distcp hdfs://localhost:9000/test s3://xxx:y...@com.bar.foo/ > Resulting stack trace: > 2015-12-11 10:04:34,030 FATAL [IPC Server handler 6 on 44861] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1449826461866_0005_m_06_0 - exited : java.io.IOException: /test > doesn't exist > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.get(Jets3tFileSystemStore.java:170) > at > org.apache.hadoop.fs.s3.Jets3tFileSystemStore.retrieveINode(Jets3tFileSystemStore.java:221) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy17.retrieveINode(Unknown Source) > at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:340) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:230) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > changing the "raise IOE..." to "return null" fixes all of the above code > sites and allows distcp to succeed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
