[ https://issues.apache.org/jira/browse/NIFI-11449 ]
Abdelrahim Ahmad deleted comment on NIFI-11449:
-----------------------------------------
was (Author: abdelrahimk):
Improving this process will allow Nifi to be the main tool to ingest data into
these new Technologies.
So we don't have to deal with another way to ingest data.
> add autocommit property to control commit in PutDatabaseRecord processor
> ------------------------------------------------------------------------
>
> Key: NIFI-11449
> URL: https://issues.apache.org/jira/browse/NIFI-11449
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.21.0
> Environment: Any Nifi Deployment
> Reporter: Abdelrahim Ahmad
> Priority: Blocker
> Labels: Trino, autocommit, database, iceberg, putdatabaserecord
>
> The issue is with the {{PutDatabaseRecord}} processor in Apache NiFi. When
> using the processor with the Trino-JDBC-Driver to write to an Iceberg
> catalog, it disables the autocommit feature. This leads to errors such as
> "{*}Catalog only supports writes using autocommit: iceberg{*}".
> To fix this issue, the autocommit feature needs to be added in the processor
> to be enabled/disabled.
> enabling auto-commit in the Nifi PutDatabaseRecord processor is important for
> Deltalake, Iceberg, and Hudi as it ensures data consistency and integrity by
> allowing atomic writes to be performed in the underlying database. This will
> allow the process to be widely used with bigger range of databases.
> P.S.: using PutSQL is not a good option at all due to the sensitivity of
> these tables when dealing with small inserts.
> Thanks and best regards
> Abdelrahim Ahmad
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
