[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-35383: ---------------------------------- Description: {code} Exception in thread "main" org.apache.hadoop.fs.s3a.commit.PathCommitException: `s3a://my-spark-bucket`: Filesystem does not have support for 'magic' committer enabled in configuration option fs.s3a.committer.magic.enabled at org.apache.hadoop.fs.s3a.commit.CommitUtils.verifyIsMagicCommitFS(CommitUtils.java:74) at org.apache.hadoop.fs.s3a.commit.CommitUtils.getS3AFileSystem(CommitUtils.java:109) {code} > Improve s3a magic committer support by inferring missing configs > ---------------------------------------------------------------- > > Key: SPARK-35383 > URL: https://issues.apache.org/jira/browse/SPARK-35383 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 3.2.0 > Reporter: Dongjoon Hyun > Priority: Major > > {code} > Exception in thread "main" > org.apache.hadoop.fs.s3a.commit.PathCommitException: `s3a://my-spark-bucket`: > Filesystem does not have support for 'magic' committer enabled in > configuration option fs.s3a.committer.magic.enabled > at > org.apache.hadoop.fs.s3a.commit.CommitUtils.verifyIsMagicCommitFS(CommitUtils.java:74) > at > org.apache.hadoop.fs.s3a.commit.CommitUtils.getS3AFileSystem(CommitUtils.java:109) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org