advancedxy commented on issue #589: URL: https://github.com/apache/incubator-uniffle/issues/589#issuecomment-1437791484
> But it looks writer will report the every partition compressed size, which will cause incorrect optimization by AQE. Do you have a concrete case that AQE doesn't work as expected? For that case, is vanilla spark worked as expected? > You are right. But I think this is wrong, and spark also has similar problem. I believe you could file a jira to Spark and discuss the possibility of add an additional field of uncompressed size of shuffle data with the Spark community. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
