soumilshah1995 opened a new issue, #351: URL: https://github.com/apache/incubator-xtable/issues/351
Opened this on Hudi Side and OneTable as I am not sure which is best place to open this ip HUDI GH https://github.com/apache/hudi/issues/10784 # Description As a user of Hudi Delta Streamer, I appreciate the concept of OneTable, but I find it cumbersome to run a separate job for transaction processing. I'm seeking guidance on whether there's a way to directly write data from Delta Streamer into OneTable format without the need for additional jobs. # Details I've explored the available documentation but haven't found a comprehensive guide on achieving this. I'm eager to experiment with this functionality and share my findings with the community through videos and blogs. # Questions Is there a method to configure Hudi Delta Streamer to write data directly into OneTable format? Could you provide guidance or point me to relevant documentation for implementing this functionality? Additionally, I'm looking for the hudi-extensions-0.1.0-SNAPSHOT-bundled JAR. Is this available on Maven repositories, and if so, under which coordinates? Spark Submit Configuration (Working with Hudi) ``` spark-submit \ --class org.apache.hudi.utilities.streamer.HoodieStreamer \ --packages 'org.apache.hudi:hudi-spark3.4-bundle_2.12:0.14.0'\ --properties-file spark-config.properties \ --master 'local[*]' \ --executor-memory 1g \ /Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/jar/hudi-utilities-slim-bundle_2.12-0.14.0.jar \ --table-type COPY_ON_WRITE \ --op UPSERT \ --source-limit 4000000 \ --source-ordering-field ts \ --source-class org.apache.hudi.utilities.sources.CsvDFSSource \ --target-base-path 'file:///Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/hudi/bronze_orders' \ --target-table bronze_orders \ --props hudi_tbl.props ``` Desired Changes I'm looking for guidance on what modifications are necessary in the above Spark Submit configuration to enable writing into OneTable format directly. References [Syncing to OneTable Format - Hudi Documentation](https://hudi.apache.org/docs/next/syncing_onetable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
