Steve Loughran created HADOOP-19033:
---------------------------------------
Summary: S3A: disable checksum validation
Key: HADOOP-19033
URL: https://issues.apache.org/jira/browse/HADOOP-19033
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Steve Loughran
Assignee: Steve Loughran
AWS v2 sdk turns on client-side checksum validation; this kills performance
Given we are using TLS to download from AWS s3, there's implicit channel
checksumming going on on, that's along with the IPv4 TCP checksumming.
We don't need it, all it does is slow us down.
proposed: disable in DefaultS3ClientFactory
I don't want to add an option to enable it as it only complicates life (yet
another config option), but I am open to persuasion
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
