dongjoon-hyun commented on PR #43729: URL: https://github.com/apache/spark/pull/43729#issuecomment-1807251130
First of all, @maryannxue 's comment was about the original PR, not for specific this PR (branch-3.4). > First of all, the test does NOT repro for us. It's not written in a robust way (certain enough to trigger the bug). Second, this patch seems to prevent some legit coalescing from happening, which can cause perf issues. Second, she already agreed the original PR here. - https://github.com/apache/spark/pull/43435#issuecomment-1786266047 > Synced with @cloud-fan offline, (2) in the above suggestion wouldn't work. Let's go ahead with current fix. Third, given the situation on @maryannxue 's PR (#43760), it's not written for branch-3.4 at all. To me, it seems that we need to spend more times on her PR. In addition, I'd like to have the consistent status for Apache Spark 4.0/3.5.2/3.4.2. In other words, her patch and test case will land in the same way to `master/3.5/3.4` later. > https://github.com/apache/spark/pull/43760#discussion_r1389723200 Lastly, for now, there is no landed patch not only `branch-3.4` but also for `master` and `branch-3.5`. I'm going to merge @eejbyfeldt 's contribution first as the fix and will wait @maryannxue 's further contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
