nsivabalan commented on issue #6308: URL: https://github.com/apache/hudi/issues/6308#issuecomment-1232285154
@eric9204 : just so you are aware of how conflict resolution happens w/in hudi. If two writers are writing to same file, hudi will fail one of them and will succeed the other. But if two writers are writing to two different files, both should succeed (assuming both are concurrent writers). For eg, lets say writer1 ingests 100 records in commit C1. and at time t10, wirter1 updates the same 100 records and ingests to hudi. wirter2 updates the same 100 records and ingests to hudi. Hudi may not be in a position to choose the winner. and so will succeed one and will fail the other. In this situations, you will see ConflictResolutionFailed error messages. But lets say writer1 ingests 100 records in commit C1 and writer2 ingests diff set of 100 records to a diff partition in commit C2. at time t10, wirter1 updates records as part of C1 and ingests to hudi. writer2 updates the records as part of C2 and ingests to hudi. Both write will succeed if it happens concurrently. bcoz, there is no overlap. Hope that clarifies why one of the writer failed in your case. you can get some additions details [here](https://hudi.apache.org/docs/concurrency_control). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
