anshulbaliga7 commented on PR #58167: URL: https://github.com/apache/spark/pull/58167#issuecomment-5891464606
Thanks @cloud-fan for the review, have addressed both in bc5ac83. - Temp-file cleanup: The write and rename now run inside an inner try; any exception after the temp file is created deletes it before rethrowing to the existing best-effort catch, which still logs the inactive-check warning. The delete is itself guarded so a cleanup failure is logged and never masks the original publication failure. The rename-false path reuses the same helper and successful publication is unchanged. Two new failure-injection cases cover this: an output stream that throws on the first write to the temp file, and a rename onto the final path that throws instead of returning false. All four cases assert the action completes, the WARN fires and neither `_num_partitions` nor its temp file remains and the directory recovers with the actual partitions. - Independent format pin: The corruption test now asserts the exact 13-byte version-1 payload for count 4 against a hardcoded literal (version, big-endian count, big-endian CRC32 of the first 5 bytes), so a coordinated writer and reader change without a version bump goes red. A test-side encoder, checked against that same literal, builds checksum-valid payloads with an unsupported version and with a negative count and both are asserted to hit the tolerated path with the WARN and full recovery. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
