Hi Robin, Welcome aboard!
+1 on the plan. Here's my comments on the three questions: 1. Incremental approach and scope It makes total sense to me. I agree with Yufei on the order: location validation, then overlap checks, then credential vending. Overlap checks have to land before credential vending, otherwise, a user allowed to create a generic table could get credentials for a location that belongs to another table :) 2. Compatibility / configuration for part 1 Part 1 is a behavior change, and Spark users will see it. I believe Delta and Hudi users often set an explicit LOCATION, and ALLOW_UNSTRUCTURED_TABLE_LOCATION to false by default. So any generic table created outside its namespace location will now be rejected. I'm fine with reusing the existing Iceberg flags rather than adding a new one: generic and Iceberg tables stay consistent. However, I would: - clearly state the behavior change and name ALLOW_UNSTRUCTURED_TABLE_LOCATION as the setting that restores the previous behavior. - say clearly in the docs that validation happens only on create. Generic tables already stored with a now disallowed location can still be loaded. 3. Concerns from PR #4237 I think the most important one is Dmitri's point about OPTIMIZED_SUBLING_CHECK. A generic table stores its base location in the internal properties. As far as I can see, the location indexes used by the optimized check read the location from the regular properties, only for Iceberg tables/views and namespaces. So with the optimized check enabled, generic tables are invisible, and overlaps would go through. Part 2 has to index generic tables in all backends. We also have to think about existing realms: generic tables created before the change won't be indexed. This is the same caveat as the existing ALLOW_OPTIMIZED_SIBLIING_CHECK readiness check. A good think would be to run the overlap tests wiith and without OPTIMIZED_SIBLING_CHECK, and in both orders (Iceberg table then generic table, and generic table then Iceberg table). I think it would be better to reuse the existing overlap logic from LocalIcebergCatalog instead of copying it into CatalogUtils. Also, worth to document how Delta Uniform and Apache XTable keep Delta (or Hudi) and Iceberg metadata under the same base location. With ALLOW_TABLE_LOCATION_OVERLAP=false (the default), you can't register the same data as both an Iceberg table and generic table. I think that's the right default, but users should know :) Tables without a location keep their current behavior, totally fine for Part 1. Credential vending will need a location, though, so we should discuss whether a generic table should get a default location derived from its namespace, like Iceberg tables do. I will take a look on #5748. Regards JB On 2026-10-20 22:54 UTC Ruobing Wang wrote: > Hi everyone, > > I'm Robin, a relatively new contributor to Apache Polaris. I'm excited to > get more involved in the community and continue contributing! > > I'd like to share a proposed implementation plan for* Generic Table > location validation *and gather feedback from the broader Polaris community. > > Background and context: > This work builds on the original Generic Table Support proposal and Yun's > earlier implementation in PR #4237. > Original proposal (Issue #1048, including the design document): > https://github.com/apache/polaris/issues/1048 > Previous implementation (PR #4237): > https://github.com/apache/polaris/pull/4237 > > Many thanks to Yun and everyone involved in the original design, > implementation, and discussions! > > I'd like to approach the remaining work incrementally to keep each change > focused and easier to review. > > Proposed implementation plan > > Part 1 - Explicit base-location validation > - Validate non-null, non-empty base-location values against the catalog's > allowed locations and parent namespace rules, respecting the existing > configuration. > - Preserve current behavior for omitted, null, or empty locations, as well > as name conflicts. > - Keep location overlap checks out of scope for this initial change. > The implementation is available in PR #5748: > https://github.com/apache/polaris/pull/5748 > Local validation has passed, including focused tests, format compileAll, > :polaris-runtime-service:check, and REST API checks. (I noticed we recently > onboard a bot for code review, I will start addressing those comments!) > > Part 2 - Location overlap checks > - Address overlapping locations between Generic Tables. > - Address overlaps between Generic Tables and Iceberg tables. > - Respect the existing overlap configuration and optimized-check behavior. > > Follow-up - Credential vending > - Explore Generic Table credential vending as a separate follow-up, subject > to further community discussion after the location-validation work. > > > I'd particularly appreciate feedback on: > 1. Whether this incremental approach and the proposed scope boundaries make > sense. > 2. Whether there are compatibility or configuration concerns that Part 1 > should address. > 3. Whether there are important concerns from the earlier PR discussions > that I should prioritize. > > I really appreciate the guidance and patience from the community as I get > more familiar with the codebase. > I'm looking forward to learning from everyone and contributing more! > > Thanks, > Robin >
