Pierre Villard created NIFI-14461:
-------------------------------------
Summary: PutBigQuery - re-use of the writer stream
Key: NIFI-14461
URL: https://issues.apache.org/jira/browse/NIFI-14461
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Pierre Villard
At the moment, PutBigQuery creates one WriterStream per flow file. This not
efficient and it would be better to have proper management of a map of
destination table / writer stream (because of expression language support) with
proper handling of the offset in order to reuse the same streams across many
flow files. That would also be an opportunity to support concurrent tasks for
better performances/
https://cloud.google.com/bigquery/docs/write-api-best-practices#connection_pool_management
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
