Thanks, Laurent, for bringing up spec "readiness" and, I guess, by extension backward compatibility concerns.
Regardless of how deep current spec is in Polaris, I believe it is important to have it written down as an artifact in the Polaris repo. I know we had a design doc at some point, but the project is defined by what is in the repository, plus discussion docs can quickly get out of sync with actual code. I believe I raised this point before. The API change merged under [1543] is not sufficient to inform users of Polaris about the Generic Tables feature. I tend to regard comments in Open API yaml files as similar to javadoc. They are good for developers working with that specific aspect of the system, but do not provide a holistic view. Now that [1543] is merged and adds some concrete specialization to Generic Tables API, I believe it is even more important to make a proper plain English spec for this feature before 1.0. [1543] https://github.com/apache/polaris/pull/1543 Cheers, Dmitri. On Wed, Jun 11, 2025 at 10:56 AM Laurent Goujon <laur...@dremio.com.invalid> wrote: > What I was trying to say is that i'm sure there's plenty of value for > spark, but in it's current state the value is little from a Polaris point > of view as an open catalog service? > > Of course we can follow-up on that but is the current spec still considered > wip or when 1.0 will be released, we would have to keep supporting it even > if we come up with something more comprehensive? > > On Wed, Jun 11, 2025, 00:22 Eric Maynard <eric.w.mayn...@gmail.com> wrote: > > > > I don't think there's a lot of value where the specification of a table > > format is left to the client > > Considering that you currently can use non-Iceberg tables in Polaris with > > the Spark client and it works end-to-end, I'd have a hard time agreeing > > that there is no value. > > > > But I think this discussion is maybe best moved to another thread. The > > incremental change to add a location may make sense for the existing > > generic table implementation, even if later we reach a consensus to rip > it > > out and replace it with something more "comprehensive". > > > > --EM > > >
