[
https://issues.apache.org/jira/browse/HADOOP-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634181#comment-16634181
]
Gabor Bota commented on HADOOP-15796:
-------------------------------------
Thanks for looking in this [[email protected]]!
I understand that those test should not run if fs caching is off, but throwing
an exception like this is not the nicest nor the most informative way to handle
this.
You think that forcing fs caching on for those tests will not make the
{{java.io.IOException: s3a://TEST-BUCKET: FileSystem is closed!}} failure
happen for other tests?
> ClassCastException: S3AFileSystem cannot be cast to MockS3AFileSystem when fs
> caching is disabled
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-15796
> URL: https://issues.apache.org/jira/browse/HADOOP-15796
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.1
> Reporter: Gabor Bota
> Priority: Major
>
> Sometimes we get {{java.io.IOException: s3a://TEST-BUCKET: FileSystem is
> closed!}} when running tests, because of filesystem caching: an fs instance
> is closed from a test teardown while another test is still using it. (That
> should be fixed in another issue.)
> To avoid test errors introduced by closed fs I've disabled fs caching while
> testing. This caused the following issue in lots of test:
> {noformat}
> [ERROR]
> testPartitionsResolution(org.apache.hadoop.fs.s3a.commit.staging.TestStagingPartitionedFileListing)
> Time elapsed: 1.994 s <<< ERROR!
> java.lang.ClassCastException: org.apache.hadoop.fs.s3a.S3AFileSystem cannot
> be cast to org.apache.hadoop.fs.s3a.MockS3AFileSystem
> at
> org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase.lookupWrapperFS(StagingTestBase.java:145)
> at
> org.apache.hadoop.fs.s3a.commit.staging.StagingTestBase$JobCommitterTest.setupJob(StagingTestBase.java:294)
> {noformat}
> We should fix the casting during these tests.
> To switch of fs caching during a test (mvn verify) run, add the following to
> the config:
> {code:java}
> <property>
> <name>fs.s3a.impl.disable.cache</name>
> <value>true</value>
> </property>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
