Can we include this patch https://github.com/apache/parquet-java/pull/3030. With this patch, after making some changes on the Spark side, the RPC volume can be significantly reduced to improve the performance of the entire cluster. [cid:image001.png@01DB2F77.CCF2F5D0]
From: Fokko Driesprong <fo...@apache.org> Date: Monday, November 4, 2024 at 16:50 To: dev@parquet.apache.org <dev@parquet.apache.org> Subject: Re: [DISCUSS] Parquet 1.14.1 release External Email Hey Gang, 1.14.4 indeed. I've created a milestone <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fpull%2F3041&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380205509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=jx%2F%2FmJsLWUG12luShaYpz8OFNR3hd7JRPfiTPYh29U0%3D&reserved=0<https://github.com/apache/parquet-java/pull/3041>> so we can track what to backport and added #3038 to it as well. Kind regards, Fokko Op ma 4 nov 2024 om 09:46 schreef Gang Wu <ust...@gmail.com>: > Hi Fokko, > > Thanks for bringing this up! > > I'd like to propose to include > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fpull%2F3038&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380231237%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=iOTBh5kbBwLDVagJrmoLEHzKqFaAtzS2%2FB3Z3VrZr1I%3D&reserved=0<https://github.com/apache/parquet-java/pull/3038> > as well. > > BTW, do you mean 1.14.4 instead of 1.14.1? > > Best, > Gang > > On Mon, Nov 4, 2024 at 4:42 PM Fokko Driesprong <fo...@apache.org> wrote: > > > Hi everyone, > > > > Cheng found a concerning bug > > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fissues%2F3040&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380248982%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=x3vQG9VjHsQx9OnjHNPKQVsnpWt5yV5I566wjp9P6GU%3D&reserved=0<https://github.com/apache/parquet-java/issues/3040>> > > in Parquet and > > created > > a fix immediately > > <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-java%2Fissues%2F3041&data=05%7C02%7Cyumwang%40ebay.com%7C9cb7ff15a1e5495aec7508dcfcadbf19%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638663070380271192%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=wd40IOsh%2Ftwo3jXxuJ%2BUiYvX8YWd6wQisW7SxYo2Ags%3D&reserved=0<https://github.com/apache/parquet-java/issues/3041>>. > > We should run a patch-release (1.14.1) to get this out to the public. Is > > there anything else that we want to include? > > > > Kind regards, > > Fokko > > >
