rangareddy commented on issue #17397: URL: https://github.com/apache/hudi/issues/17397#issuecomment-5351245123
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9143). **Findings: still open and still worked around on `master`.** The workaround added by #12942 (`eb212c9dca87`, 2025-03-09) is still in place today, together with the TODO that points back at this ticket: ```yaml # azure-pipelines-20230430.yml (Job 1, "FT client/spark-client") # TODO(HUDI-9143): Investigate why Jacoco execution data file is corrupt options: $(MVN_OPTS_TEST) -Pfunctional-tests -Djacoco.agent.dest.filename=jacoco2.corrupt -pl hudi-client/hudi-spark-client ``` Because `merge_jacoco_exec_files.sh` globs `**/jacoco-agent/**/*.exec`, renaming the destination file to `jacoco2.corrupt` simply excludes it. **Net effect: `hudi-client/hudi-spark-client` functional-test coverage has been silently missing from the Azure merged coverage report since March 2025.** The root cause was never investigated. Leading hypothesis (not yet verified, offered as a starting point): before the workaround, the "UT client/spark-client" and "FT client/spark-client" steps both passed `-Djacoco.agent.dest.filename=jacoco2.exec` for the same module, so both Maven runs used the same `hudi-client/hudi-spark-client/target/jacoco-agent/jacoco2.exec` destination (`destFile` in the `unit-tests` and `functional-tests` profiles of the root `pom.xml`). The JaCoCo agent defaults to `append=true`, so the second run reopens the file written by the first; a forked/killed FT JVM that does not finish its dump leaves a truncated file, which matches the reported sizes exactly (`jacoco1.exec` 1126236 bytes at 20:32 vs `jacoco2.exec` 1880 bytes at 20:50) and matches the `EOFException` from `CompactDataInput.readBooleanArray`. The "merge succeeds with a 0-byte file" case in the description is the same collision landing on a clean truncation instead of a partial one. Counter-evidence worth checking before committing to that theory: the GitHub Actions path added later (`.github/workflows/bot.yml` "UT - hudi-client/hudi-spark-client" and "FT - hudi-client/hudi-spark-client", both writing the default `jacoco.exec` in the same target dir, merged by `scripts/jacoco/generate_merged_coverage_report.sh`) has the same filename collision and does not appear to hit the corruption, so timing/fork behaviour on the Azure agent is likely part of the story. Suggested next steps: give the FT step its own `jacoco.agent.dest.filename` (e.g. `jacoco2-ft.exec`) so the two runs stop sharing a destination, confirm the merge succeeds with the FT data included, and then remove the `jacoco2.corrupt` workaround and this TODO. Also worth deciding whether the Azure coverage merge is still needed at all now that Codecov reporting runs in GitHub Actions (#18230, #19138). Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
