Another point: I'm pretty sure sooner or later users will want to move their data to some other location. As an option users may want to write new files into another location but keep old files in place.
Also: if the location is a URI, how do we deal with s3 vs. s3a for example? In Iceberg it is quite common for different engines to use different access tools, which often leads to different URI schemes. Cheers, Dmitri. On Wed, May 7, 2025 at 6:46 PM Eric Maynard <eric.w.mayn...@gmail.com> wrote: > All good questions Dmitri — I’m especially interested in the first one as > from what I understand Iceberg tables can have metadata and data at two > different paths that we need to vend credentials for. > > For iceberg tables, we just use special properties to track these > locations. I wonder if we couldn’t do the same for generic tables. > > On Wed, May 7, 2025 at 3:42 PM Dmitri Bourlatchkov <di...@apache.org> > wrote: > > > Hi Yun, > > > > Please clarify the meaning of the value of the new location attribute. > > > > - Is is one value or many? > > - Is it a URI? > > - Does it point to any particular file? > > - Is it a common prefix of all files within a table? > > - What happens when a value does not match these expectation? > > > > Thanks, > > Dmitri. > > > > On 2025/05/07 21:50:19 yun zou wrote: > > > Hi folks, > > > > > > I would like to propose to add an optional `location` field to > > > CreateGenricTable Request and LoadGenericTable response. > > > > > > The `location` is the location for the table, which is common to most > > table > > > formats including Iceberg, Delta, Hudi, csv, parquet etc. The location > > > information is critical for loading the table at engine side, having a > > > dedicated keyword could help improve the robustness for cross engine > > > sharing, instead of relying on the properties passed by the client > side. > > > > > > Furthermore, this information is also required to provide credential > > > vending capabilities later. > > > > > > Here is the PR for adding the spec: > > > https://github.com/apache/polaris/pull/1543 > > > > > > Looking forward to your reply and feedback! > > > > > > Best Regards, > > > Yun > > > > > >
