Daniel Beicht created NIFI-16405:
------------------------------------
Summary: PutIcebergRecord fails due to missing transitive runtime
dependencies
Key: NIFI-16405
URL: https://issues.apache.org/jira/browse/NIFI-16405
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.12.0
Reporter: Daniel Beicht
Attachments: stacktrace.txt
Writing to an Apache Iceberg table in Databricks Unity Catalog fails in NiFi
{{2.12.0}} due to missing transitive runtime dependencies in the Iceberg shared
NAR.
The target table is stored in Azure Data Lake Storage Gen2 (ADLS). NiFi uses
{{RESTIcebergCatalog}} to connect to the Databricks Unity Catalog Iceberg REST
catalog and {{ADLSIcebergFileIOProvider}} to write Parquet files to ADLS.
After manually adding multiple dependencies to {{nifi-iceberg-shared-nar}} I
could successfully write to the iceberg-table.
*Observed behavior*
Starting from the unmodified NiFi {{2.12.0}} distribution, the processor fails
with the following sequence. Each row shows the single dependency added to the
previous custom shared NAR.
||Newly added JAR||Result||
|None|{{ClassNotFoundException: org.apache.hadoop.fs.Path}}|
|{{hadoop-common-3.5.0.jar}}|{{ClassNotFoundException:
com.ctc.wstx.io.InputBootstrapper}}|
|{{woodstox-core-7.2.2.jar}}|{{NoClassDefFoundError:
org/codehaus/stax2/XMLInputFactory2}}|
|{{stax2-api-4.3.0.jar}}|{{NoClassDefFoundError:
org/apache/commons/collections4/map/UnmodifiableMap}}|
|{{commons-collections4-4.6.0.jar}}|{{NoClassDefFoundError:
org/apache/hadoop/thirdparty/com/google/common/collect/Interners}}|
|{{hadoop-shaded-guava-1.5.0.jar}}|{{ClassNotFoundException:
com.sun.jna.Platform}}|
|{{jna-5.13.0.jar}}|Write succeeds|
You can find the specific stacktraces in the attachment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
