It seems that Iceberg is choosing to keep Parquet only, rather than
Avro. Parquet format manifests may be benefited in projection reads.

>
> Hi devs,
>
> I'd like to discuss removing the `manifest.format` option and
> standardizing Paimon manifest files on Avro.
>
> The related PR is:
> https://github.com/apache/paimon/pull/9154
>
> The main reasons are:
>
> 1. Avro is the default and de facto manifest format.
> 2. PyPaimon currently supports Avro manifests only.
> 3. Allowing alternative formats introduces unnecessary compatibility
> risks across implementations.
> 4. We checked our managed production tables and found no usage of
> `manifest.format`.
>
> The proposal is to:
>
> - Remove `manifest.format` from `CoreOptions` and the documentation.
> - Always use Avro for manifest files.
> - Keep `manifest.compression` configurable.
>
> The compatibility concern is that tables whose manifests were
> actually written in another format, such as ORC, would no longer be
> readable after upgrading. We have not found such usage, but would
> like to confirm whether anyone in the community relies on non-Avro
> manifests.
>
> Please share your concerns or use cases if you depend on this option.
>
> Best,
> Junhao

#/******本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件!
 This e-mail and its attachments contain confidential information from XIAOMI, 
which is intended only for the person or entity whose address is listed above. 
Any use of the information contained herein in any way (including, but not 
limited to, total or partial disclosure, reproduction, or dissemination) by 
persons other than the intended recipient(s) is prohibited. If you receive this 
e-mail in error, please notify the sender by phone or email immediately and 
delete it!******/#

Reply via email to