Hi Muhammad, all, +1 to both ideas, and thanks for raising them.
I work on streaming pipelines that replicate MySQL tables into Apache Paimon with Flink CDC, and Flink SQL jobs then read those tables downstream. Upstream schema changes already reach the Paimon tables automatically. The Flink SQL jobs that read and write them, though, still need their DDL changed by hand and a redeploy each time. ADD COLUMN IF NOT EXISTS would let us keep our SQL scripts idempotent across restarts. An opt-in evolution setting for INSERT INTO ... SELECT * would remove most of that manual step. I'd be happy to review the FLIP when it's ready. Best regards, James Pizagno On Thu, 5 Nov 2026 at 14:17, M Junaid <[email protected]> wrote: > Hi Devs, > > I would like to get some thoughts around the addition of a couple of > extensions related to schema evolution inside Flink SQL. There are a couple > of pain points that I have observed with Flink SQL jobs. > > > 1. > The first challenge is addition of `ALTER TABLE <table> ADD COLUMN` to a > Flink job makes the job kind of non-idempotent. This means the job will run > fine for the first time but later it'll fail until the ALTER TABLE command > is removed. A broader idea is to support something like ALTER TABLE <table> > ADD [COLUMN] IF NOT EXISTS pattern. This pattern is pretty common with many > datastores like Postgres, Cassandra, Clickhouse etc. > 2. > The second challenge is supporting schema evolution with statements like > `INSERT INTO dst_table SELECT * from src_table`. From interface > perspectives, a user can enable automatic schema evolution by having a > table configuration that can be set like `SET 'table-evolve-scheme' = > 'true'`. Similar patterns are used by Snowflake and Databricks for doing > schema evolution. > > > Looking forward to hearing some feedback on the above from community. I do > have some details carved out and willing to share as a FLIP soon. > > Thanks, > Muhammad >
