I'm OK with that. Yufei
On Tue, Feb 3, 2026 at 5:27 AM Gábor Kaszab <[email protected]> wrote: > Thank you Everyone for the inputs and thanks Kevin for describing the "we > have a use case" part while I was offline! :) > > Using `write.metadata.path` makes sense to me too. Yufei, since you had > the preference for the other (location param based) approach, do you think > you can live with keeping `write.metadata.path`? > > Best Regards, > Gabor > > > Ryan Blue <[email protected]> ezt írta (időpont: 2026. jan. 29., Cs, 1:49): > >> Thanks, Kevin. If you need to customize the metadata path within the view >> location, then it makes more sense to me to use `write.metadata.path`. >> >> On Tue, Jan 27, 2026 at 4:39 PM Kevin Liu <[email protected]> wrote: >> >>> Thanks for starting this thread, and for all the great discussions. >>> >>> I can chime in on use cases from the Microsoft OneLake perspective. We >>> have a requirement to customize the metadata path of the view metadata json >>> file. We want the file path to have a specific naming convention that can >>> be an arbitrary subdirectory name. >>> >>> Right now, using the `location` parameter, `metadata/` is added >>> automatically as the subdirectory for the json file [1], ie >>> {location}/metadata/view1.metadata.json >>> We want to customize the name of the subdirectory with >>> `write.metadata.path` [2], so the view will be stored in >>> {location}/{write.metadata.path}/view1.metadata.json >>> >>> In summary, I prefer option 2 to give writers more control over the >>> exact path of the metadata json. And this also follows the convention for >>> creating an Iceberg table with the write.metadata.path property. >>> >>> Best, >>> Kevin Liu >>> >>> >>> [1] >>> https://github.com/apache/iceberg/blob/1bf44d9fed8b99dd73115ec31659266248cbb363/core/src/main/java/org/apache/iceberg/view/BaseViewOperations.java#L173 >>> [2] >>> https://github.com/apache/iceberg/blob/1bf44d9fed8b99dd73115ec31659266248cbb363/core/src/main/java/org/apache/iceberg/view/BaseViewOperations.java#L166-L169 >>> >>> >>> On Tue, Jan 27, 2026 at 2:43 PM Ryan Blue <[email protected]> wrote: >>> >>>> I think it is okay to go with option 1, but I don't see a justification >>>> for why a change here is needed. Why break with existing convention when it >>>> should not matter to users what location is used for metadata? >>>> >>>> I'd prefer option 2, to let people configure this if they choose. But I >>>> think the best solution is to do nothing and encourage people not to care >>>> about where metadata files end up. Looks like the justification is "we have >>>> use cases". Could you share what those are? >>>> >>>> On Mon, Jan 26, 2026 at 1:27 PM Yufei Gu <[email protected]> wrote: >>>> >>>>> Given that the Iceberg community is moving toward a catalog based >>>>> approach, I think option 1 should work reasonably well. From a reader >>>>> perspective, there should be no behavior change, since the metadata path >>>>> is >>>>> always resolved from the catalog, regardless of whether the underlying >>>>> path >>>>> includes a metadata directory or not. For writers, as long as we converge >>>>> on the convention of not implicitly adding a metadata subdirectory, this >>>>> also seems acceptable. This is already how tables behave today. We have >>>>> not >>>>> enforced a strict rule for table metadata paths in the spec, and different >>>>> impls have been able to interoperate without issues. >>>>> >>>>> Yufei >>>>> >>>>> >>>>> On Fri, Jan 23, 2026 at 11:35 AM Gábor Kaszab <[email protected]> >>>>> wrote: >>>>> >>>>>> Thank you All for your responses! >>>>>> Let me summarize what we have so far to see what potential next steps >>>>>> we can make. >>>>>> >>>>>> 1) Change 'location' property to not create a 'metadata/' subfolder >>>>>> - Doesn't seem to be a great issue that already written tables >>>>>> have 'metadata/' subfolder, while newer ones won't >>>>>> - Easy to implement in Iceberg code >>>>>> - Doing this will let us drop 'write.metadata.path' property >>>>>> My concern: >>>>>> - We only have control on Iceberg code, that means non-REST >>>>>> catalogs, but different REST catalog implementations could still create >>>>>> paths the old way. >>>>>> - Is there a way we can enforce this? Adding to spec seems >>>>>> overkill. >>>>>> >>>>>> 2) De-deprecate 'write.metadata.path' >>>>>> - Using this will let us to avoid having 'metadata/' folder in >>>>>> the path >>>>>> - Doc >>>>>> <https://iceberg.apache.org/docs/latest/view-configuration/> says >>>>>> "Base location for metadata files". Seems ambiguous if this means that we >>>>>> can have subfolders or not >>>>>> - Seems common sense to not add 'metadata/' folder, but there >>>>>> seems to be no strict guarantee that all REST catalog implementations >>>>>> follow this >>>>>> >>>>>> WDYT about next steps? Maybe go with 2) and be more explicit in the >>>>>> docs? >>>>>> >>>>>> Best Regards, >>>>>> Gabor Kaszab >>>>>> >>>>>> Ryan Blue <[email protected]> ezt írta (időpont: 2026. jan. 16., P, >>>>>> 1:03): >>>>>> >>>>>>> I think that the "non-catalog option" means that it works when >>>>>>> you're pointed at a particular metadata.json file. Commits always have >>>>>>> to >>>>>>> go through some catalog-based mechanism (now that Hadoop/FS commits are >>>>>>> deprecated in the spec). >>>>>>> >>>>>>> On Thu, Jan 15, 2026 at 1:23 AM Péter Váry < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Yufei, >>>>>>>> >>>>>>>> I think keeping the non-catalog option available makes sense, and >>>>>>>> adding an extra check for this non-standard approach doesn’t seem like >>>>>>>> a >>>>>>>> big issue to me. That said, the community consensus appears to be >>>>>>>> gradually >>>>>>>> shifting away from the non-catalog approach. If you look at the REST >>>>>>>> planning mode discussion [1], we’re even considering Iceberg tables >>>>>>>> without >>>>>>>> any metadata stored on the underlying storage. It might be worth >>>>>>>> defining a >>>>>>>> consistent, general approach for decisions like these. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Peter >>>>>>>> >>>>>>>> [1] - [DISCUSS] REST: Scan Planning mode - >>>>>>>> https://lists.apache.org/thread/z1g4y8b4ogdrn0jjtjlgg7yjgxdbzpvg >>>>>>>> >>>>>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 15., >>>>>>>> Cs, 0:17): >>>>>>>> >>>>>>>>> Peter, >>>>>>>>> Based on the discussion(sorry, I don't have a link to share) so >>>>>>>>> far, the community conclusion is that we should always keep the non >>>>>>>>> catalog >>>>>>>>> option available. In other words, even catalogs are the recommended >>>>>>>>> way to >>>>>>>>> locate view metadata, non catalog based access remains a supported >>>>>>>>> option. >>>>>>>>> This gives us flexibility to support different deployment models. >>>>>>>>> >>>>>>>>> Yufei >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jan 14, 2026 at 2:20 AM Péter Váry < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> > For example, when locating the view metadata.json, clients >>>>>>>>>> would need to check two possible locations, "/location" and >>>>>>>>>> "/location/metadata". >>>>>>>>>> >>>>>>>>>> Do I understand correctly that users are expected to use the >>>>>>>>>> Catalog to locate metadata.json? In my opinion they should not rely >>>>>>>>>> on >>>>>>>>>> filenames to locate views.” >>>>>>>>>> >>>>>>>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. >>>>>>>>>> 14., Sze, 3:49): >>>>>>>>>> >>>>>>>>>>> Thanks Gabor for raising this. The alternative approach is >>>>>>>>>>> appealing, as it avoids supporting the same behavior through >>>>>>>>>>> multiple >>>>>>>>>>> properties. My main concern is potential impact on existing use >>>>>>>>>>> cases. For >>>>>>>>>>> example, when locating the view metadata.json, clients would need >>>>>>>>>>> to check >>>>>>>>>>> two possible locations, "/location" and "/location/metadata". This >>>>>>>>>>> may not >>>>>>>>>>> be a significant issue, since the spec does not mandate the metadata >>>>>>>>>>> suffix, but it does introduce incompatibility with the existing >>>>>>>>>>> convention. >>>>>>>>>>> It would be helpful for others to chime in and share view use cases >>>>>>>>>>> that >>>>>>>>>>> rely on the "metadata" suffix today, and POV on whether it's OK to >>>>>>>>>>> adapt >>>>>>>>>>> the change. >>>>>>>>>>> >>>>>>>>>>> Yufei >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jan 13, 2026 at 5:23 AM Gábor Kaszab < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Iceberg Community, >>>>>>>>>>>> >>>>>>>>>>>> *TLDR* >>>>>>>>>>>> I'd like to de-deprecate the 'write.metadata.path' view >>>>>>>>>>>> property >>>>>>>>>>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/ViewProperties.java> >>>>>>>>>>>> and would like to see if there are any objections. >>>>>>>>>>>> >>>>>>>>>>>> *Details* >>>>>>>>>>>> We have use-cases where it's necessary to configure tables and >>>>>>>>>>>> views to have custom paths for metadata, in particular not to >>>>>>>>>>>> enforce >>>>>>>>>>>> adding the metadata/ folder under the hood. For both tables and >>>>>>>>>>>> views there >>>>>>>>>>>> is 'write.metadata.path' property that could achieve this. >>>>>>>>>>>> However, for >>>>>>>>>>>> views this is deprecated now. >>>>>>>>>>>> >>>>>>>>>>>> After talking to Yufei, the reason for this property being >>>>>>>>>>>> deprecated is that for views this is somewhat duplicated >>>>>>>>>>>> functionality with >>>>>>>>>>>> the 'location' property seemingly serving the same purpose. There >>>>>>>>>>>> are some >>>>>>>>>>>> differences though: >>>>>>>>>>>> - 'location': This gives the root folder of the view and >>>>>>>>>>>> metadata would be written under 'location'/metadata folder. >>>>>>>>>>>> - 'write.metadata.path': This has higher precedence over >>>>>>>>>>>> 'location' and metadata would be written directly into >>>>>>>>>>>> 'write.metadata.path' without the metadata/ folder. >>>>>>>>>>>> >>>>>>>>>>>> An *alternative approach* could be to drop >>>>>>>>>>>> 'write.metadata.path' and change the behaviour of 'location' to >>>>>>>>>>>> not create >>>>>>>>>>>> a metadata/ folder under the hood. The view spec doesn't mention >>>>>>>>>>>> explicitly >>>>>>>>>>>> that metadata/ folder is a requirement, however, it gives an >>>>>>>>>>>> example >>>>>>>>>>>> <https://iceberg.apache.org/view-spec/#appendix-a-an-example> >>>>>>>>>>>> where it is present and also mentions that it is there to follow >>>>>>>>>>>> the >>>>>>>>>>>> structure of the tables. So changing this would be confusing at >>>>>>>>>>>> least, but >>>>>>>>>>>> could be considered a breaking change too. I wouldn't go this way. >>>>>>>>>>>> >>>>>>>>>>>> So long story short, I'd like to keep 'write.metadata.path' for >>>>>>>>>>>> views despite the similar purpose as 'location', to avoid >>>>>>>>>>>> generating the >>>>>>>>>>>> metadata/ folder. >>>>>>>>>>>> >>>>>>>>>>>> Any feedback is appreciated! >>>>>>>>>>>> Gabor >>>>>>>>>>>> >>>>>>>>>>>
