bhasudha commented on code in PR #10087: URL: https://github.com/apache/hudi/pull/10087#discussion_r1393131984
########## website/docs/concurrency_control.md: ########## @@ -61,6 +68,18 @@ With multiple writers using OCC, these are the write guarantees to expect: - *INCREMENTAL PULL Guarantee*: Data consumption and checkpoints are NEVER out of order. If there are inflight commits (due to multi-writing), incremental queries will not expose the completed commits following the inflight commits. +#### Multi Writer Guarantees with Non-Blocking Concurrency Control Mode + +`NON_BLOCKING_CONCURRENCY_CONTROL`, offers the same set of guarantees as mentioned in the case of OCC but without +explicit locks for serializing the writes. Lock is only needed for writing the commit metadata to the Hudi timeline. The +completion time for the commits reflects the serialization order and file slicing is done based on completion time. +Multiple writers can operate on the table with non-blocking conflict resolution. The writers can write into the same +file group with the conflicts resolved automatically by the query reader and the compactor. The new concurrency mode is +currently available for preview in version 1.0.0-beta only with following caveats: + +- It is only supported for Flink writers currently. +- Conflict resolution is not supported yet between clustering and ingestion. It works for compaction and ingestion, and + we can see an example of that in the [Flink quickstart guide](/docs/next/flink-quick-start-guide#non-blocking-concurrency-control). Review Comment: Can we move the example of NBCC from quickstart to either https://hudi.apache.org/docs/hoodie_streaming_ingestion#flink-ingestion or to the https://hudi.apache.org/docs/writing_data#flink-sql-writer. Quickstart should be simple and focus only on CRUD operations for easy start. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
