[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734141#comment-16734141 ] Gabor Bota commented on HADOOP-15819: - Created HADOOP-16027 to add this to docs. > S3A integration

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733557#comment-16733557 ] Steve Loughran commented on HADOOP-15819: - I think we should write something down about

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733167#comment-16733167 ] Gabor Bota commented on HADOOP-15819: - We could extend the docs with something like "Do not add

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732172#comment-16732172 ] Sean Mackrory commented on HADOOP-15819: I wouldn't have thought so as I think you have to go

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731950#comment-16731950 ] Steve Loughran commented on HADOOP-15819: - quick follow up here: do we need to add anything to

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729701#comment-16729701 ] Adam Antal commented on HADOOP-15819: - Thanks [~mackrorysd] for the commit. The credit goes to

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729695#comment-16729695 ] Hudson commented on HADOOP-15819: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15665 (See

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729682#comment-16729682 ] Sean Mackrory commented on HADOOP-15819: Committed. Thank you for the excellent work here

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726305#comment-16726305 ] Aaron Fabbri commented on HADOOP-15819: --- Just catching up on this as I hit the bug again. Sounds

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725843#comment-16725843 ] Adam Antal commented on HADOOP-15819: - Indeed. I also saw that it is used elsewhere, so I would

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725544#comment-16725544 ] Sean Mackrory commented on HADOOP-15819: Ah - FS.key not being set is the piece I was missing.

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725168#comment-16725168 ] Adam Antal commented on HADOOP-15819: - Not exactly, to be more clear though I enumerate the events

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724301#comment-16724301 ] Sean Mackrory commented on HADOOP-15819: The only failure I see with this patch is the

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723105#comment-16723105 ] Adam Antal commented on HADOOP-15819: - Thanks, [~gabor.bota]. I uploaded patch 

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721551#comment-16721551 ] Gabor Bota commented on HADOOP-15819: - Thanks for the patch [~adam.antal]. Not just the call, but

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713791#comment-16713791 ] Adam Antal commented on HADOOP-15819: - I removed it. > S3A integration test failures: FileSystem

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712683#comment-16712683 ] Gabor Bota commented on HADOOP-15819: - Thanks [~adam.antal], could you remove the bad one? > S3A

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711966#comment-16711966 ] Adam Antal commented on HADOOP-15819: - I'm sorry, uploaded bad patch (reuploaded the right one

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711957#comment-16711957 ] Gabor Bota commented on HADOOP-15819: - Thanks [~adam.antal], I'll test it tomorrow up and

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711917#comment-16711917 ] Adam Antal commented on HADOOP-15819: - I'd also like to add that it worked for my disabled-cache

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711783#comment-16711783 ] Adam Antal commented on HADOOP-15819: - Hi everyone! I uploaded patch v1 with the fix.

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703729#comment-16703729 ] Adam Antal commented on HADOOP-15819: - Strangely I rerun the tests in trunk with no modification,

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703616#comment-16703616 ] Adam Antal commented on HADOOP-15819: - Also some minor remark on {{S3ACloseEnforcedFileSystem}}:

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703567#comment-16703567 ] Adam Antal commented on HADOOP-15819: - I investigated a bit further, and when the caching is

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700891#comment-16700891 ] Adam Antal commented on HADOOP-15819: - Also tests were run in eu-ireland (eu-west-1). > S3A

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700823#comment-16700823 ] Adam Antal commented on HADOOP-15819: - I looked through the affected tests, and maybe this

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700370#comment-16700370 ] Gabor Bota commented on HADOOP-15819: - fyi [~adam.antal] I've created an issue for the problems

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645064#comment-16645064 ] Steve Loughran commented on HADOOP-15819: - maybe ask the fs itself, e,g {code} if (fs!=null) {

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644973#comment-16644973 ] Gabor Bota commented on HADOOP-15819: - I have some bad news: I get these issues for tests where

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644940#comment-16644940 ] Steve Loughran commented on HADOOP-15819: - bq. The FS cache really feels inherently broken in

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644730#comment-16644730 ] Gabor Bota commented on HADOOP-15819: - I think we should focus on solving this case first, so it

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643906#comment-16643906 ] Sean Mackrory commented on HADOOP-15819: So I think it was mentioned in the other JIRA that

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643749#comment-16643749 ] Gabor Bota commented on HADOOP-15819: - Also note: I was getting interesting errors when I removed

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643742#comment-16643742 ] Gabor Bota commented on HADOOP-15819: - If I remove {{IOUtils.closeStream(getFileSystem());}} from

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638351#comment-16638351 ] Gabor Bota commented on HADOOP-15819: - So that's {{fs.s3a.impl.disable.cache}}

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638306#comment-16638306 ] Gabor Bota commented on HADOOP-15819: - >From that perspective, every test could use the same

[jira] [Commented] (HADOOP-15819) S3A integration test failures: FileSystem is closed! - without parallel test run

[ https://issues.apache.org/jira/browse/HADOOP-15819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638286#comment-16638286 ] Steve Loughran commented on HADOOP-15819: - I see. we could find those FS.close() calls and