FIY, I tried using hadoop-aws.jar from Hadoop 2.9 release, but it required a lot of other jars to be replaced, both in hadoop-dependencies and from Druid's extensions/druid-hdfs-storage/ (e.g. aws-java-sdk used by hadoop's 2.9 is already newer than Druid 0.12.1's version, so I had to make both directories converge). Ultimately S3A started working, but the way to achieve it was quite messy.
Not sure if shading would work, I couldn't build the custom jar, because Hadoop 2.7 requires dependencies (protoc) that are no longer available on OS at my disposal. For now I settled with using S3N, but it forces provising AWS secret credentials in plain text within config / task spec, so it's a pain. [ Full content available at: https://github.com/apache/incubator-druid/issues/4841 ] This message was relayed via gitbox.apache.org for [email protected]
