soumilshah1995 opened a new issue, #10784:
URL: https://github.com/apache/hudi/issues/10784
# Description
As a user of Hudi Delta Streamer, I appreciate the concept of OneTable, but
I find it cumbersome to run a separate job for transaction processing. I'm
seeking guidance on whether there's a way to directly write data from Delta
Streamer into OneTable format without the need for additional jobs.
# Details
I've explored the available documentation but haven't found a comprehensive
guide on achieving this. I'm eager to experiment with this functionality and
share my findings with the community through videos and blogs.
# Questions
Is there a method to configure Hudi Delta Streamer to write data directly
into OneTable format?
Could you provide guidance or point me to relevant documentation for
implementing this functionality?
Additionally, I'm looking for the hudi-extensions-0.1.0-SNAPSHOT-bundled
JAR. Is this available on Maven repositories, and if so, under which
coordinates?
Spark Submit Configuration (Working with Hudi)
```
spark-submit \
--class org.apache.hudi.utilities.streamer.HoodieStreamer \
--packages 'org.apache.hudi:hudi-spark3.4-bundle_2.12:0.14.0'\
--properties-file spark-config.properties \
--master 'local[*]' \
--executor-memory 1g \
/Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/jar/hudi-utilities-slim-bundle_2.12-0.14.0.jar
\
--table-type COPY_ON_WRITE \
--op UPSERT \
--source-limit 4000000 \
--source-ordering-field ts \
--source-class org.apache.hudi.utilities.sources.CsvDFSSource \
--target-base-path
'file:///Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/hudi/bronze_orders'
\
--target-table bronze_orders \
--props hudi_tbl.props
```
Desired Changes
I'm looking for guidance on what modifications are necessary in the above
Spark Submit configuration to enable writing into OneTable format directly.
References
[Syncing to OneTable Format - Hudi
Documentation](https://hudi.apache.org/docs/next/syncing_onetable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
