Shagish commented on issue #7577: URL: https://github.com/apache/hudi/issues/7577#issuecomment-1371377842
Hi @yihua, Thanks for your response and providing us with the solution. Here are the Hoodie configuration details which we have in our Prod environment. hoodie.datasource.hive_sync.table=up_customer_info_2 hoodie.table.name=up_customer_info_2 s3.hudi.base.path=s3://XXXXXXXXXXX/up_customer_info_2/table checkpointLocation=s3://XXXXXXXXXXX/up_customer_info_2/checkpoint records.error.path=s3://XXXXXXXXXXX/up_customer_info_2/error hoodie.datasource.write.recordkey.field=accountID hoodie.datasource.write.precombine.field=op_ts hoodie.datasource.hive_sync.enable=true hoodie.datasource.hive_sync.auto_create_database=false hoodie.datasource.hive_sync.database=integration hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.NonPartitionedExtractor hoodie.index.type=SIMPLE hoodie.datasource.write.operation=UPSERT hoodie.datasource.write.table.type=COPY_ON_WRITE hoodie.datasource.write.hive_style_partitioning=true hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.NonpartitionedKeyGenerator hoodie.upsert.shuffle.parallelism=20 hoodie.insert.shuffle.parallelism=20 hoodie.cleaner.commits.retained=5 hoodie.metrics.reporter.type=GRAPHITE hoodie.metrics.graphite.host=d010082212016.ds.dtvops.net hoodie.metrics.graphite.port=9109 hoodie.compact.inline=true hoodie.embed.timeline.server.reuse.enabled=true hoodie.write.markers.type=timeline_server_based hoodie.markers.timeline_server_based.batch.num_threads=20 hoodie.markers.timeline_server_based.batch.interval_ms=50 hoodie.metrics.graphite.metric.prefix=spark_streaming_hudi hoodie.metrics.on=true Please review and provide us with your comments Thanks, Shagish -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
