[
https://issues.apache.org/jira/browse/NIFI-4565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271161#comment-16271161
]
ASF GitHub Bot commented on NIFI-4565:
--------------------------------------
Github user jfrazee commented on the issue:
https://github.com/apache/nifi/pull/2293
@baank Definitely thanks for the contrib, but unfortunately I don't think
this is the right solution/it supports S3 without any code changes.
PutParquet provides an "Additional Classpath Resources" property that you
can point at a directory and provide all the S3 dependencies. Here's what I
used:
```
aws-java-sdk-1.7.4.jar
hadoop-aws-2.7.3.jar
hadoop-common-2.7.3.jar
httpclient-4.5.3.jar
httpcore-4.4.4.jar
jackson-annotations-2.6.0.jar
jackson-core-2.6.1.jar
jackson-databind-2.6.1.jar
```
We take the same approach with PutHDFS for filesystems that aren't included
in the core Hadoop libs, so it seems to make sense to keep doing the same here.
> PutParquet doesn't support S3
> -----------------------------
>
> Key: NIFI-4565
> URL: https://issues.apache.org/jira/browse/NIFI-4565
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.4.0
> Reporter: Franco
> Assignee: Joey Frazee
>
> PutParquet doesn't support S3 targets due to the lack of the hadoop-aws
> dependency.
> To recreate it:
> 1) Modify core-site.xml to change fs.defaultName to one starting with s3://
> 2) Add a PutParquet
> 3) It will fail to run. The logs will show the missing hadoop-aws dependency.
> Simple fix is just to add hadoop-aws.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
