hudi-bot opened a new issue, #16346: URL: https://github.com/apache/hudi/issues/16346
[https://github.com/apache/hudi/issues/10233] ``` NOW=$(date '+%Y%m%dt%H%M%S') ${SPARK_HOME}/bin/spark-submit \ --jars ${path_prefix}/jars/${SPARK_V}/hudi-spark${SPARK_VERSION}-bundle_2.12-${HUDI_VERSION}.jar \ --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer \ ${path_prefix}/jars/${SPARK_V}/hudi-utilities-slim-bundle_2.12-${HUDI_VERSION}.jar \ --target-base-path ${path_prefix}/testcases/stocks/data/target/${NOW} \ --target-table stocks${NOW} \ --table-type COPY_ON_WRITE \ --base-file-format PARQUET \ --props ${path_prefix}/testcases/stocks/configs/hoodie.properties \ --source-class org.apache.hudi.utilities.sources.JsonDFSSource \ --schemaprovider-class org.apache.hudi.utilities.schema.FilebasedSchemaProvider \ --hoodie-conf hoodie.deltastreamer.schemaprovider.source.schema.file=${path_prefix}/testcases/stocks/data/schema_without_ts.avsc \ --hoodie-conf hoodie.deltastreamer.schemaprovider.target.schema.file=${path_prefix}/testcases/stocks/data/schema_without_ts.avsc \ --op UPSERT \ --spark-master yarn \ --hoodie-conf hoodie.deltastreamer.source.dfs.root=${path_prefix}/testcases/stocks/data/source_without_ts \ --hoodie-conf hoodie.datasource.write.partitionpath.field=date \ --hoodie-conf hoodie.datasource.write.keygenerator.type=SIMPLE \ --hoodie-conf hoodie.datasource.write.hive_style_partitioning=false \ --hoodie-conf hoodie.metadata.enable=true ``` ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-7211 - Type: Bug - Fix version(s): - 1.1.0 --- ## Comments 29/May/24 22:44;shivnarayan;For auto record key gen, you need to set operation type to "INSERT". Can you give that a try. ;;; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
