[
https://issues.apache.org/jira/browse/SPARK-38958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840748#comment-17840748
]
ASF GitHub Bot commented on SPARK-38958:
----------------------------------------
hadoop-yetus commented on PR #6550:
URL: https://github.com/apache/hadoop/pull/6550#issuecomment-2076861454
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 01s | | No case conflicting files
found. |
| +0 :ok: | spotbugs | 0m 00s | | spotbugs executables are not
available. |
| +0 :ok: | codespell | 0m 01s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 01s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 00s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 00s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 92m 11s | | trunk passed |
| +1 :green_heart: | compile | 5m 02s | | trunk passed |
| +1 :green_heart: | checkstyle | 4m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 03s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 146m 50s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 55s | | the patch passed |
| +1 :green_heart: | compile | 2m 16s | | the patch passed |
| +1 :green_heart: | javac | 2m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 00s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 02s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 159m 37s | | patch has no errors
when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 5m 25s | | The patch does not
generate ASF License warnings. |
| | | 421m 41s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/6550 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | MINGW64_NT-10.0-17763 691d1e3161c7 3.4.10-87d57229.x86_64
2024-02-14 20:17 UTC x86_64 Msys |
| Build tool | maven |
| Personality | /c/hadoop/dev-support/bin/hadoop.sh |
| git revision | trunk / c8168fd0bc45331bd8b55dd53b537bec4b05fba5 |
| Default Java | Azul Systems, Inc.-1.8.0_332-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6550/1/testReport/
|
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6550/1/console
|
| versions | git=2.44.0.windows.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Override S3 Client in Spark Write/Read calls
> --------------------------------------------
>
> Key: SPARK-38958
> URL: https://issues.apache.org/jira/browse/SPARK-38958
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 3.2.1
> Reporter: Hershal
> Priority: Major
> Labels: pull-request-available
>
> Hello,
> I have been working to use spark to read and write data to S3. Unfortunately,
> there are a few S3 headers that I need to add to my spark read/write calls.
> After much looking, I have not found a way to replace the S3 client that
> spark uses to make the read/write calls. I also have not found a
> configuration that allows me to pass in S3 headers. Here is an example of
> some common S3 request headers
> ([https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonRequestHeaders.html).]
> Does there already exist functionality to add S3 headers to spark read/write
> calls or pass in a custom client that would pass these headers on every
> read/write request? Appreciate the help and feedback
>
> Thanks,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
