Hey everyone, In Iceberg 1.7.0, the Parquet dependency has been reverted from 1.14.3 to 1.13.1 to exclude the bug that Cheng pointed out. This way the release isn't blocked by the Parquet release.
Fei, thanks for the PR. I noticed that this is not a bug fix bug but it adds a constructur. Typically in patch releases, only bug fixes are included. I couldn't find any guidance on the Parquet website, so not sure how rigid this is for Parquet. I would suggest discussing this in the Parquet sync tomorrow. Kind regards, Fokko Op di 5 nov 2024 om 04:47 schreef Wang, Yuming <yumw...@ebay.com.invalid>: > Can we include this patch https://github.com/apache/parquet-java/pull/3030 > . > > With this patch, after making some changes on the Spark side, the RPC > volume can be significantly reduced to improve the performance of the > entire cluster. > > > > *From: *Fokko Driesprong <fo...@apache.org> > *Date: *Monday, November 4, 2024 at 16:50 > *To: *dev@parquet.apache.org <dev@parquet.apache.org> > *Subject: *Re: [DISCUSS] Parquet 1.14.1 release > > External Email > > Hey Gang, > > 1.14.4 indeed. I've created a milestone > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fpull%2F3041&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380205509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=jx%2F%2FmJsLWUG12luShaYpz8OFNR3hd7JRPfiTPYh29U0%3D&reserved=0 > <https://github.com/apache/parquet-java/pull/3041>> so we can track what > to > backport and added #3038 to it as well. > > Kind regards, > Fokko > > Op ma 4 nov 2024 om 09:46 schreef Gang Wu <ust...@gmail.com>: > > > Hi Fokko, > > > > Thanks for bringing this up! > > > > I'd like to propose to include > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fpull%2F3038&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380231237%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=iOTBh5kbBwLDVagJrmoLEHzKqFaAtzS2%2FB3Z3VrZr1I%3D&reserved=0 > <https://github.com/apache/parquet-java/pull/3038> as well. > > > > BTW, do you mean 1.14.4 instead of 1.14.1? > > > > Best, > > Gang > > > > On Mon, Nov 4, 2024 at 4:42 PM Fokko Driesprong <fo...@apache.org> > wrote: > > > > > Hi everyone, > > > > > > Cheng found a concerning bug > > > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fissues%2F3040&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380248982%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=x3vQG9VjHsQx9OnjHNPKQVsnpWt5yV5I566wjp9P6GU%3D&reserved=0 > <https://github.com/apache/parquet-java/issues/3040>> in Parquet and > > > created > > > a fix immediately < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fissues%2F3041&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380271192%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=wd40IOsh%2Ftwo3jXxuJ%2BUiYvX8YWd6wQisW7SxYo2Ags%3D&reserved=0 > <https://github.com/apache/parquet-java/issues/3041>>. > > > We should run a patch-release (1.14.1) to get this out to the public. > Is > > > there anything else that we want to include? > > > > > > Kind regards, > > > Fokko > > > > > >
