peter-toth commented on PR #58895: URL: https://github.com/apache/spark/pull/58895#issuecomment-5912892579
@dongjoon-hyun thank you. Round 6 is answered in the threads, and the description is up to date. - The row-index gate can no longer be bypassed by case. The planner normalizes the conjuncts it offers to the relation's column names. - A task kill or a fatal error behind a key expression's error is rethrown as it is, not absorbed. - A decode error in a key page gives the row group up, so a corrupt key page fails where a plain read fails. The conf doc describes the `ignoreCorruptFiles` difference in general terms now. - The row-index route measured 2% to 8% slower than master, and the lazy coalescing was the cause. It builds its ranges up front again, and a same-machine A/B puts it level with master. - Everything else is fixed as you suggested, and the new behavior has tests that fail without it. One correction to round 5. Removing the 2-arg `initialize` override tripped MiMa, so the override is back and only calls `super`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
