David Handermann created NIFI-15775:
---------------------------------------
Summary: Refactor Snowflake Ingest Integration without Ingest SDK
Key: NIFI-15775
URL: https://issues.apache.org/jira/browse/NIFI-15775
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: David Handermann
Assignee: David Handermann
The {{SnowflakeIngestManagerProviderService}} interface abstracts access to the
Snowflake {{SimpleIngestManager}} class, which is the primary integration point
for ingesting files to Snowflake Snowpipe services, and polling ingestion
history. The
[snowflake-ingest-sdk|https://central.sonatype.com/artifact/net.snowflake/snowflake-ingest-sdk]
library includes the {{SimpleIngestManager}} along with other classes that
support Snowpipe Streaming. The library implements the [Snowpipe Streaming
Classic
Architecture|https://docs.snowflake.com/en/user-guide/snowpipe-streaming/snowpipe-streaming-classic-overview],
which is planned for future deprecation.
With current Snowflake file ingestion not using Snowpipe Streaming features,
the Service interface and implementation should be refactored to work directly
with the [Snowpipe REST
API|https://docs.snowflake.com/en/user-guide/data-load-snowpipe-rest-apis].
This should remove the dependency on the {{snowflake-ingest-sdk}} library while
maintaining current Snowpipe file ingest capabilities.
Removing the {{snowflake-ingest-sdk}} requires changing the interface contract
of {{{}SnowflakeIngestManagerProviderService{}}}, removing the reference to the
{{SimpleIngestManager}} class. With usage limited to the
{{GetSnowflakeIngestStatus}} and {{StartSnowflakeIngest}} Processors, the
impact is localized to the {{{}nifi-snowflake-bundle{}}}, keeping all changes
within the bundle boundary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
