[
https://issues.apache.org/jira/browse/HADOOP-16050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746263#comment-16746263
]
Steve Loughran commented on HADOOP-16050:
-----------------------------------------
Is this actually in session setup (slow, random IO with ORC/Parquet triggered
unless you set fs.s3a.experimental.fadvise = random), or in actual wire
communications?
If its the wire, then we have some fun, because that's all happening in the AWS
SDK layer, which is beyond our direct control.
Ideally, they'd copy the Azure teams and use the wildfly JARs if on the CP,
and, if bundled in their shaded SDK, come for ~free. Otherwise it looks like
some configuration problem, which will need to be pushed out to every single
process across a cluster. Which is always a pain to do
> Support setting cipher suites for s3a file system
> -------------------------------------------------
>
> Key: HADOOP-16050
> URL: https://issues.apache.org/jira/browse/HADOOP-16050
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.1
> Reporter: Justin Uang
> Priority: Major
> Attachments: Screen Shot 2019-01-17 at 2.57.06 PM.png
>
>
> We have found that when running the S3AFileSystem, it picks GCM as the ssl
> cipher suite. Unfortunately this is well known to be slow on java 8:
> [https://stackoverflow.com/questions/25992131/slow-aes-gcm-encryption-and-decryption-with-java-8u20.]
>
> In practice we have seen that it can take well over 50% of our CPU time in
> spark workflows. We should add an option to set the list of cipher suites we
> would like to use. !Screen Shot 2019-01-17 at 2.57.06 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
