Hi Péter, I think a SQL-only workflow is closer than it looks. a Spark user doesn't need Spark to parse the Flink SQL, they need to read it, and showing another engine's SQL as text is engine-agnostic. What's missing is presentation, not translation.
The workflow I'd propose: 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink` representation. 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all* representations with their dialect labels. The strings are already in the metadata, so this is purely a display convention, no spec change. This answers "how can the Spark user know the Flink SQL". 3. The user reads the Flink SQL and authors the Spark equivalent. 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark` representation and carries the `flink` one forward unchanged. Issuing that statement *is* the consistency assertion — the "user authority" you described, made concrete. The version summary already records engine name/version, so we get per-version provenance for free. 5. On read, each engine executes exactly its own dialect. The rule that makes this flag-free is, I think, just a crisper version of what Dan said: ALTER VIEW ... AS = update/add my engine's representation, preserve the rest CREATE OR REPLACE VIEW = redefine the view; only my representation remains Two intents, two existing syntaxes, no properties, no new grammar. When someone genuinely changes the view's meaning with CREATE OR REPLACE, the other representations are gone, the next read from another engine fails fast, and that engine's user re-adds their dialect via steps 2–4. The failure is the signal, no cross-engine coordination protocol needed. That only holds if engines actually fail fast, which is Alex's fallback point: today sqlFor() silently returns the "closest" representation, which is exactly how an engine ends up executing another engine's SQL. That leniency was defensible while appending a dialect was impossible; with this workflow it loses its justification. #17874 adds an opt-in view-dialect-strict to Flink as a first step, and ideally the error message itself teaches the workflow: "View v has no 'flink' representation (found: spark). Use SHOW CREATE VIEW to see the existing SQL and ALTER VIEW to add a flink representation." Talat On 2026/09/07 14:38:10 Péter Váry wrote: > Do I understand correctly that the expected behavior for the user if they > want to create a view with Flink and Spark representations is: > > 1. CREATE VIEW my_view AS SELECT ... - in Flink > 2. ALTER VIEW my_view AS SELECT ... - in Spark > > How can a Spark user know the Flink SQL the user used to create the view? > > I'm trying to put together a workflow for creating interoperable views and > using SQL doesn't seem viable, so as Ryan mentioned the only option is > Iceberg API for now. > > > Péter Váry <[email protected]> ezt írta (időpont: 2026. szept. > 5., Szo, 9:16): > > > Thanks Alex for bringing this up. > > > > We had a very similar discussion with Guo when he wanted to add a Flink > > representation to a view: https://github.com/apache/iceberg/issues/15296. > > Talat also has a fresh PR for this issue in Flink: > > https://github.com/apache/iceberg/pull/17874 > > > > My main concern is consistency. How can Flink be sure that the Flink SQL > > returns the same results as the already existing Spark SQL. > > In the end we accept the users authority to state that the Flink SQL is > > consistent with the existing Spark SQL. This means we accept that they are > > able to parse both, and make decisions. > > I do think we should allow a user to update one or more of the SQLs > > without needing extra flags. > > > > > > > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <[email protected]> > > wrote: > > > >> Thanks for starting the thread Alex, > >> > >> I agree with Dan, an engine should only update / add their own dialect > >> and not make guarantees about other engine dialects without passing through > >> the other engine's compiler. > >> I would recommend ALTER too, I know engines such as starrocks have came > >> up with grammar [1] to be more specific that one is adding / modifying the > >> dialect, given the dialect is of starrocks, that could be an option : > >> > >> > >> > >> > >> > >> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [, > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>* > >> > >> > >> [1] > >> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect > >> > >> Best, > >> Prashant Singh > >> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <[email protected]> wrote: > >> > >>> I feel like if we want to support updating/adding a representation for a > >>> specific engine, the right way is to allow 'ALTER VIEW' to update or add > >>> the current engine's representation. > >>> > >>> I don't think we can expect an engine to add or update other engines' > >>> representations because that would potentially require parsing a different > >>> SQL syntax than is native to the engine. > >>> > >>> For the example you gave where there's an existing representation in > >>> Trino, then using `ALTER VIEW . . .` would just set the spark > >>> representation (as opposed to a `REPLACE` or `CREATE` that would create a > >>> new representation with only the spake dialect). > >>> > >>> I don't like the idea of adding properties to toggle behavior though as > >>> that makes something declarative subject to side-effects of the > >>> configuration/session. > >>> > >>> -Dan > >>> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev < > >>> [email protected]> wrote: > >>> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW > >>>> VIEWS` (which just calls the underlying List Views API), so they know > >>>> that > >>>> they don't need to create a view as much as alter it. > >>>> > >>>> The Java implementation > >>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> > >>>> surfaces > >>>> the first representation when a matching one doesn't exist. This assumes > >>>> representations are more-or-less compatible between engines, which isn't > >>>> the case. > >>>> > >>>> I'm concerned about the user journey where a user attempts to query a > >>>> view and receives an error message stating they cannot query it due to a > >>>> wrong engine or invalid SQL syntax (because their engine is using the > >>>> wrong > >>>> representation). At this point, their only current recourse is to use the > >>>> REST endpoints directly to add their new representation. > >>>> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <[email protected]> wrote: > >>>> > >>>>> I think the intent was for this to happen through API integration. > >>>>> What is the user interaction that you're trying to achieve? Do you want > >>>>> each `CREATE VIEW` call to automatically append a new representation? > >>>>> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev < > >>>>> [email protected]> wrote: > >>>>> > >>>>>> Hello, > >>>>>> > >>>>>> The view spec allows creating a single view with multiple > >>>>>> engine-specific representations. This allows a single view to be > >>>>>> queried by > >>>>>> multiple engines, each using its own specific SQL dialect. > >>>>>> > >>>>>> In practice, this is difficult to achieve. Users can use the > >>>>>> `replace.drop-dialect.allowed` view property to allow the replacing of > >>>>>> an > >>>>>> existing representation. However, there's no way to append a > >>>>>> representation > >>>>>> (by creating a new ViewVersion with the existing representations and a > >>>>>> new > >>>>>> one). This means that a direct REST API call is the only way to create > >>>>>> a > >>>>>> View with multiple representations. > >>>>>> > >>>>>> I'd like to propose the creation of a new view property called > >>>>>> `replace.append-dialect.allowed` that allows users to append a new > >>>>>> representation to an existing view. > >>>>>> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930 > >>>>>> > >>>>>> Please take a look. I'd love to hear from the community whether there > >>>>>> is value in this use case. > >>>>>> > >>>>>> Thanks! > >>>>>> > >>>>>> -- Alex Stephen > >>>>>> > >>>>> >
