[
https://issues.apache.org/jira/browse/NIFI-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-8611:
---------------------------
Fix Version/s: 1.14.0
> GCP BigQuery processors support using designate project resource for ingestion
> ------------------------------------------------------------------------------
>
> Key: NIFI-8611
> URL: https://issues.apache.org/jira/browse/NIFI-8611
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.11.4
> Reporter: Chih Han Yu
> Assignee: Joe Witt
> Priority: Major
> Labels: GCP, bigquery
> Fix For: 1.14.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For now, *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor
> can only assign a single project id for consuming resources and do ingestion.
> But in some business cases, the project providing resources and the project
> which be inserted are not always the same.
> src/main/java/org/apache/nifi/processors/gcp/AbstractGCPProcessor.java
>
> {code:java}
> ......
> public static final PropertyDescriptor PROJECT_ID = new PropertyDescriptor
> .Builder().name("gcp-project-id")
> .displayName("Project ID")
> .description("Google Cloud Project ID")
> .required(false)
>
> .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
> .build();
> ......{code}
>
> We've test a solution which is workable, which is, adding another property
> *DESIGNATE_PROJECT_ID* in *AbstractBigQueryProcessor*, it'll only impact
> *PutBigQueryBatch* processor and *PutBigQueryStreaming* processor.
> If user provides designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *DESIGNATE_PROJECT_ID* (defined in
> AbstractBigQueryProcessor).
> If user does {color:#ff0000}not{color} provide designate project id:
> * Use *PROJECT_ID* (defined in AbstractGCPProcessor) as resource consuming
> project.
> * Put data into *PROJECT_ID* (defined in AbstractGCPProcessor).
> Since we already implemented this solution in production environment, I'll
> submit a PR later for this improvement.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
