Hi everyone!

I'd like to start a discussion around adding support for Flink procedures in
Iceberg.

# Background

Apache Flink already supports CALL SQL statements for invoking procedures, and
similar procedural capabilities are also available in Apache Paimon.

At the moment, Iceberg supports many workflows through APIs and
engine-specific integrations, but not through Flink SQL procedures.

# Motivation

The main motivation is to improve Flink and Iceberg integration for
SQL operational
workflows.

Flink is often used together with Iceberg in streaming and real-time
environments,
and exposing Iceberg operations through Flink procedures could simplify
maintenance tasks for users already working in Flink SQL. We already use
similar functionality internally in several real-world scenarios, and I’ve
got permission to upstream this work. I think it could be useful for the
broader community as well.

# Proposal

Issue: Flink: Add Flink SQL procedures for Iceberg similarly to Apache
Paimon <https://github.com/apache/iceberg/issues/16314>
PR: Add Flink SQL procedures <https://github.com/apache/iceberg/pull/16315>

The current PR introduces an initial implementation for integrating
Flink procedures
into Iceberg. Other procedures will be added after further discussion.

The implementation is intentionally minimal and mainly focuses on:

   - procedure discovery
   - procedure execution through Flink CALL statements
   - integration with existing Flink procedure infrastructure

# Questions for the community

Q1. Would Flink procedure support be useful for your workloads?

Q2. Are there concerns around API or maintainability?

Thanks,
Stepan Stepanishchev (SteveStevenpoor <https://github.com/SteveStevenpoor>)

Reply via email to