Abdelrahim Ahmad created NIFI-11449:
---------------------------------------
Summary: add autocommit property to control commit in
PutDatabaseRecord processor
Key: NIFI-11449
URL: https://issues.apache.org/jira/browse/NIFI-11449
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 1.21.0
Environment: Any Nifi Deployment
Reporter: Abdelrahim Ahmad
The issue is with the {{PutDatabaseRecord}} processor in Apache NiFi. When
using the processor with the Trino-JDBC-Driver to write to an Iceberg catalog,
it disables the autocommit feature. This leads to errors such as "{*}Catalog
only supports writes using autocommit: iceberg{*}".
To fix this issue, the autocommit feature needs to be added in the processor to
be enabled/disabled.
enabling auto-commit in the Nifi PutDatabaseRecord processor is important for
Deltalake, Iceberg, and Hudi as it ensures data consistency and integrity by
allowing atomic writes to be performed in the underlying database. This will
allow the process to be widely used with bigger range of databases.
P.S.: using PutSQL is not a good option at all due to the sensitivity of these
tables when dealing with small inserts.
Thanks and best regards
Abdelrahim Ahmad
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
