rangareddy commented on issue #5540: URL: https://github.com/apache/hudi/issues/5540#issuecomment-5100997465
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: The only exception captured here is `HoodieException: Commit 20220509105215 failed and rolled-back !` from `DeltaSync.writeToSink`. That is a wrapper message emitted after the write is already being rolled back -- the actual cause sits earlier in the driver log and was never isolated on this thread. Requests for debug-level driver logs and the contents of `.hoodie` (2022-08-12, 2022-08-28, 2022-09-06) and for a reproducible case went unanswered, and there has been no activity since 2023-01-04. One thing worth recording for anyone who lands here from a search, since it is verifiable from the reported environment: the classpath in the report combines `hudi-utilities-bundle_2.12:0.8.0` and `hudi-spark3-bundle_2.12:0.8.0` with **Spark 3.2.1** and `spark-avro_2.12:3.1.1`. Hudi 0.8.0's `hudi-spark3-bundle` is built against Spark **3.0.0** (`pom.xml:112` at tag `release-0.8.0` sets `spark3.version` to `3.0.0`); Spark 3.2 was not supported until Hudi 0.10.0. So that is three different Spark lines on one classpath, which on its own is enough to produce unpredictable failures during the write. Note also that the shared logs and the shared code did not correspond (the registered schema in the logs had no `CDC_TS` field), as flagged on 2022-05-17. For reference, the co-reporter on this thread resolved their variant separately in #6348. The class in the stack trace no longer exists in its reported form either -- `HoodieDeltaStreamer`/`DeltaSync` are now `HoodieStreamer`/`StreamSync` in `hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/`, so `DeltaSync.java:492` does not map to current code. Closing as stale. If you are still hitting a rolled-back commit, please open a fresh issue against a supported Hudi/Spark pairing (Hudi 1.0.x / 1.1.x / 1.2.0 with a matching `hudi-sparkX.Y-bundle`) and include the full driver log at DEBUG, the write configs, and a `.hoodie` timeline listing -- with those the root cause is usually identifiable quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
