[
https://issues.apache.org/jira/browse/NIFI-11791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742514#comment-17742514
]
Marcio Sugar commented on NIFI-11791:
-------------------------------------
I agree. It might also be a good idea to check if PutBigQueryStreaming is in a
similar situation.
> PutBigQuery processor lacks functionality found in PutBigQueryBatch
> -------------------------------------------------------------------
>
> Key: NIFI-11791
> URL: https://issues.apache.org/jira/browse/NIFI-11791
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 2.0.0, 1.22.0
> Reporter: Marcio Sugar
> Priority: Major
>
> Before PutBigQuery, we had PutBigQueryBatch and PutBigQueryStream, both now
> deprecated. Not sure if PutBigQuery was designed to completely replace its
> older brothers, but it cannot do that yet because of some missing features.
> For example, we can't use PubBigQuery alone to create snapshot tables,
> something that was easy to do with PutBigQueryBatch.
> A snapshot table is a recent copy of a table from a database or a subset of
> rows/columns of a table. It is used to dynamically replicate data between
> distributed databases. Using PutBigQueryBatch, we can achieve that by setting
> the following properties:
> * Create Disposition = CREATE_IF_NEEDED
> * Write Disposition = WRITE_TRUNCATE
> I understand that PutBigQuery uses the newer [BigQuery Storage Write
> API|https://cloud.google.com/bigquery/docs/write-api], so adding the missing
> functionality might not be possible.
> But please note the older BigQuery (core) API (the one I believe
> PutBigQueryBatch uses) allows the user to submit jobs to load data into
> BigQuery in a very convenient way. That is something I'd like to see
> preserved in future versions of NiFi
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
