Given that the Iceberg community is moving toward a catalog based approach, I think option 1 should work reasonably well. From a reader perspective, there should be no behavior change, since the metadata path is always resolved from the catalog, regardless of whether the underlying path includes a metadata directory or not. For writers, as long as we converge on the convention of not implicitly adding a metadata subdirectory, this also seems acceptable. This is already how tables behave today. We have not enforced a strict rule for table metadata paths in the spec, and different impls have been able to interoperate without issues.
Yufei On Fri, Jan 23, 2026 at 11:35 AM Gábor Kaszab <[email protected]> wrote: > Thank you All for your responses! > Let me summarize what we have so far to see what potential next steps we > can make. > > 1) Change 'location' property to not create a 'metadata/' subfolder > - Doesn't seem to be a great issue that already written tables have > 'metadata/' subfolder, while newer ones won't > - Easy to implement in Iceberg code > - Doing this will let us drop 'write.metadata.path' property > My concern: > - We only have control on Iceberg code, that means non-REST catalogs, > but different REST catalog implementations could still create paths the old > way. > - Is there a way we can enforce this? Adding to spec seems overkill. > > 2) De-deprecate 'write.metadata.path' > - Using this will let us to avoid having 'metadata/' folder in the > path > - Doc <https://iceberg.apache.org/docs/latest/view-configuration/> > says "Base location for metadata files". Seems ambiguous if this means that > we can have subfolders or not > - Seems common sense to not add 'metadata/' folder, but there seems > to be no strict guarantee that all REST catalog implementations follow this > > WDYT about next steps? Maybe go with 2) and be more explicit in the docs? > > Best Regards, > Gabor Kaszab > > Ryan Blue <[email protected]> ezt írta (időpont: 2026. jan. 16., P, 1:03): > >> I think that the "non-catalog option" means that it works when you're >> pointed at a particular metadata.json file. Commits always have to go >> through some catalog-based mechanism (now that Hadoop/FS commits are >> deprecated in the spec). >> >> On Thu, Jan 15, 2026 at 1:23 AM Péter Váry <[email protected]> >> wrote: >> >>> Hi Yufei, >>> >>> I think keeping the non-catalog option available makes sense, and adding >>> an extra check for this non-standard approach doesn’t seem like a big issue >>> to me. That said, the community consensus appears to be gradually shifting >>> away from the non-catalog approach. If you look at the REST planning mode >>> discussion [1], we’re even considering Iceberg tables without any metadata >>> stored on the underlying storage. It might be worth defining a consistent, >>> general approach for decisions like these. >>> >>> Thanks, >>> Peter >>> >>> [1] - [DISCUSS] REST: Scan Planning mode - >>> https://lists.apache.org/thread/z1g4y8b4ogdrn0jjtjlgg7yjgxdbzpvg >>> >>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 15., Cs, >>> 0:17): >>> >>>> Peter, >>>> Based on the discussion(sorry, I don't have a link to share) so far, >>>> the community conclusion is that we should always keep the non catalog >>>> option available. In other words, even catalogs are the recommended way to >>>> locate view metadata, non catalog based access remains a supported option. >>>> This gives us flexibility to support different deployment models. >>>> >>>> Yufei >>>> >>>> >>>> On Wed, Jan 14, 2026 at 2:20 AM Péter Váry <[email protected]> >>>> wrote: >>>> >>>>> > For example, when locating the view metadata.json, clients would >>>>> need to check two possible locations, "/location" and >>>>> "/location/metadata". >>>>> >>>>> Do I understand correctly that users are expected to use the Catalog >>>>> to locate metadata.json? In my opinion they should not rely on filenames >>>>> to >>>>> locate views.” >>>>> >>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 14., >>>>> Sze, 3:49): >>>>> >>>>>> Thanks Gabor for raising this. The alternative approach is appealing, >>>>>> as it avoids supporting the same behavior through multiple properties. My >>>>>> main concern is potential impact on existing use cases. For example, when >>>>>> locating the view metadata.json, clients would need to check two possible >>>>>> locations, "/location" and "/location/metadata". This may not be a >>>>>> significant issue, since the spec does not mandate the metadata suffix, >>>>>> but >>>>>> it does introduce incompatibility with the existing convention. It would >>>>>> be >>>>>> helpful for others to chime in and share view use cases that rely on the >>>>>> "metadata" suffix today, and POV on whether it's OK to adapt the change. >>>>>> >>>>>> Yufei >>>>>> >>>>>> >>>>>> On Tue, Jan 13, 2026 at 5:23 AM Gábor Kaszab <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hey Iceberg Community, >>>>>>> >>>>>>> *TLDR* >>>>>>> I'd like to de-deprecate the 'write.metadata.path' view property >>>>>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/ViewProperties.java> >>>>>>> and would like to see if there are any objections. >>>>>>> >>>>>>> *Details* >>>>>>> We have use-cases where it's necessary to configure tables and views >>>>>>> to have custom paths for metadata, in particular not to enforce adding >>>>>>> the >>>>>>> metadata/ folder under the hood. For both tables and views there is >>>>>>> 'write.metadata.path' property that could achieve this. However, for >>>>>>> views >>>>>>> this is deprecated now. >>>>>>> >>>>>>> After talking to Yufei, the reason for this property being >>>>>>> deprecated is that for views this is somewhat duplicated functionality >>>>>>> with >>>>>>> the 'location' property seemingly serving the same purpose. There are >>>>>>> some >>>>>>> differences though: >>>>>>> - 'location': This gives the root folder of the view and metadata >>>>>>> would be written under 'location'/metadata folder. >>>>>>> - 'write.metadata.path': This has higher precedence over >>>>>>> 'location' and metadata would be written directly into >>>>>>> 'write.metadata.path' without the metadata/ folder. >>>>>>> >>>>>>> An *alternative approach* could be to drop 'write.metadata.path' >>>>>>> and change the behaviour of 'location' to not create a metadata/ folder >>>>>>> under the hood. The view spec doesn't mention explicitly that metadata/ >>>>>>> folder is a requirement, however, it gives an example >>>>>>> <https://iceberg.apache.org/view-spec/#appendix-a-an-example> where >>>>>>> it is present and also mentions that it is there to follow the >>>>>>> structure of >>>>>>> the tables. So changing this would be confusing at least, but could be >>>>>>> considered a breaking change too. I wouldn't go this way. >>>>>>> >>>>>>> So long story short, I'd like to keep 'write.metadata.path' for >>>>>>> views despite the similar purpose as 'location', to avoid generating the >>>>>>> metadata/ folder. >>>>>>> >>>>>>> Any feedback is appreciated! >>>>>>> Gabor >>>>>>> >>>>>>
