[
https://issues.apache.org/jira/browse/NIFI-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861879#comment-15861879
]
ASF GitHub Bot commented on NIFI-3449:
--------------------------------------
Github user gene-telligent commented on the issue:
https://github.com/apache/nifi/pull/1482
Hi @jvwing :
I've gone through and made the changes that you've suggested. Thank you so
much for the feedback!
For the biggest issue -- using expression language for Project ID as per
the request of @fmaritato -- I couldn't figure out an easy way of incorporating
that functionality. The general pattern of NiFi processors seems to be "create
a connector object when scheduled", and in the case of the GCP java libraries,
the connector client *requires* a Project ID to be set in it's configuration.
Changing it to be dynamic on a per-flowfile basis would require rebuilding the
client configuration / instantiating it for each flowfile which seems like a
pretty big red flag. If there are any suggestions there I'd like to hear them.
Everything else should be resolved (hopefully) so if you have further
suggestions or comments they would be welcome.
> Create Google Cloud Platform/Google Cloud Storage Processors
> ------------------------------------------------------------
>
> Key: NIFI-3449
> URL: https://issues.apache.org/jira/browse/NIFI-3449
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Gene Peters
> Labels: features
>
> Hi all,
> We had a need in our production deployments to interact with Google Cloud
> Storage. At the time, NIFI-2809 hadn't seen much movement, and after applying
> the patch I found that the configuration was too specific for my needs (it's
> hardcoded to use "Application Default" credentials, everything uploaded to
> GCS is uploaded with the "public" ACL, etc). So I created a series of
> Processors / Controller Services based off of the AWS NiFi library, and would
> like to contribute them.
> Features:
> * All credentialing is handled by a controller service, allowing multiple
> processors to use the same service / credentials
> * An Abstract processor is provided which forms the basis for all GCP related
> processors.
> * The standard Google Cloud Storage operations are supported, very similarly
> to the AWS S3 processors: ListGCSBucket, DeleteGCSObject, FetchGCSObject,
> PutGCSObject
> * Everything is documented and unit tested.
> * I've also provided integration tests, but they're disabled by default (as
> they require Google Cloud credentials). To run them, use the flag
> {{skipGCPIntegrationTests=false}}
> Todo:
> * The GCP Java library's ReadChannel objects implement the "restorable"
> interface, which allows for state saving / checkpointing. I'd really like to
> leverage this with the State support that NiFi provides, but it would require
> serializing / deserializing the object.
> I'm going to be submitting this as a pull request through GitHub.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
