mandnhdaiyudfaio opened a new issue, #7081: URL: https://github.com/apache/hudi/issues/7081
I need to use the concurrent write control function (occ lock) of hudi. Official documents（ https://hudi.apache.org/docs/concurrency_control ）The demo provided is as follows: inputDF.write.format("hudi") .options(getQuickstartWriteConfigs) .option(PRECOMBINE_FIELD_OPT_KEY, "ts") .option("hoodie.cleaner.policy.failed.writes", "LAZY") .option("hoodie.write.concurrency.mode", "optimistic_concurrency_control") .option("hoodie.write.lock.zookeeper.url", "zookeeper") .option("hoodie.write.lock.zookeeper.port", "2181") .option("hoodie.write.lock.zookeeper.lock_key", "test_table") .option("hoodie.write.lock.zookeeper.base_path", "/test") .option(RECORDKEY_FIELD_OPT_KEY, "uuid") .option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath") .option(TABLE_NAME, tableName) .mode(Overwrite) .save(basePath) According to the demo of the official website document, I will translate hoodie.cleaner.policy.failed.writes, hoodie.write.contract.mode, hoodie.write.lock.zookeeper.url, hoodie.write.lock.zookeeper.port, hoodie.write.lock.zookeeper.lock_ key、hoodie.write.lock.zookeeper. base_ The five parameters, path, are written into the options parameter. My demo is as follows: spark. sql("create table da (id int, ts bigint, actid int, day string) using hudi tblproperties(type = 'cow', primaryKey = 'id,actid', preCombineField = 'ts') options (hoodie.write.lock.zookeeper.url = 'node4,node6,node7', hoodie.write.lock.zookeeper.port = '2181', hoodie.write.lock.zookeeper.lock_key = 'da', hoodie.write.lock.zookeeper.base_path = '/hudi/lock/table', hoodie.write.concurrency.mode = 'optimistic_concurrency_control', hoodie.cleaner.policy.failed.writes = 'LAZY') partitioned by (day) location '/user/hudi/warehouse/ods_ dev/da'"); Question: After the da table is created successfully, the data can be written and read normally. However, I went to the zoo keeper and found that the zoo keeper failed to create the node:/hudi/lock/table. Even if the official website's demo writing method inputDF. write. format ("hudi")... save (basePath) is used directly, data can be successfully written to the hudi table, but it is also not found that zookeeper successfully creates nodes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
