Hi Dmitri, Thanks a lot for the feedback!
I definitely agree that the current generic table support still has limitations and that more work is needed. However, removing the “beta” label doesn’t mean the feature is finalized. Instead, it signals to users that we are committed to maintaining and improving it over time. Therefore, I believe this will not block any future enhancements including extending it to address the use cases described in the Table Source proposal. As Adam pointed out, we already have users trying it out and requesting improvements. I believe this is a good time to remove the “beta” label to encourage broader adoption and welcome new contributors who can help advance the feature. Best Regards, Yun On Mon, Nov 17, 2025 at 3:28 PM Adam Christian < [email protected]> wrote: > I'm in favor of removing the "beta" label because it is starting to be used > by users. From the Slack feedback we have seen, there are users who > have started to try out this feature. While they are still running into > issues such as not having a Spark 4 plugin [1] and not having credential > vending [2], there is real user usage indicating that folks find this > beneficial. > > Now, I do agree with Dmitri that there are known limitations we need to > handle that are impeding users. The two referenced examples above are > obvious ones that need fixes to get wider adoption. I have been working > with Yun & others to be able to solve some of these issues (like this PR > [3]), but I agree that there may need to be more significant changes > including a REST API change. > > From my understanding, during our last conversation in the community about > the "Table Sources" proposal [4], we decided we were going to analyze the > Parquet file use case and see if Generic Tables can support this use case > or whether we need to bring some ideas from the Table Sources proposal into > Generic Tables. I believe that this approach is still valid and will > benefit our adopting users by identifying any enhancements with these new > use cases. > > What do y'all think? > > [1] - https://github.com/apache/polaris/issues/3021 > [2] - https://github.com/apache/polaris/issues/3020 > [3] - https://github.com/apache/polaris/pull/3026 > [4] - https://lists.apache.org/thread/652z1f1n2pgf3g2ow5y382wlrtnoqth0 > > Go community, > > Adam > > On Mon, Nov 17, 2025 at 12:23 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yun, > > > > We should indeed review the status of the Generic Tables API, so thanks > for > > starting this discussion! > > > > From my POV the key question is: how do we intend to proceed with known > > limitations [1]? > > > > I believe the Table Sources proposal [2] addresses some (if not all) of > > those limitations. It is certainly suitable for the same applications > that > > currently go through the Generic Tables API. > > > > I believe it would be wise to allow the Table Sources proposal to develop > > further to see if there are any synergies that can be leveraged with > > respect to Generic Tables. If we removed the "beta" label from the > Generic > > Tables API now, it would make it harder for users to benefit from those > > synergies later (due to virtually freezing the "spec"). > > > > At it stands now, the Generic Tables API is supported by Polaris, so > > existing clients can continue operating normally. > > > > [1] > > > > > https://github.com/apache/polaris/blob/f056e22f7f3a7c53e233bef1b88d204d6a8e4d79/site/content/in-dev/unreleased/generic-table.md?plain=1#L162-L169 > > > > [2] https://lists.apache.org/thread/9f5b75fy25l9yzrtnlzqg6yh1bqdyjbt > > > > Thanks, > > Dmitri. > > > > > > On Fri, Nov 14, 2025 at 12:33 PM yun zou <[email protected]> > > wrote: > > > > > Hi All, > > > > > > Generic Table has been available since Polaris 1.0 and has attracted > > > interest from several users. We also have ongoing improvement and > > extension > > > work in progress, including Hudi support, Parquet support, and > credential > > > vending. > > > > > > Given this progress, I believe it’s a good time to remove the “beta” > > label. > > > If there are no objections, we will remove the “beta” label from the > > > Generic Table in Polaris 1.3. > > > > > > Best Regards, > > > > > > Yun > > > > > >
