vaibhavk1992 commented on issue #13996:
URL: https://github.com/apache/hudi/issues/13996#issuecomment-3476652723
@DamianoErnesti-Anritsu I tried reproduce the issue mentioned above. I am
using spark datasource API and the below code
**Record level index+partition/column stats+Secondary index+inline
clustering**
`inserts.write.format("hudi")
.option("hoodie.datasource.write.partitionpath.field", "city")
.option("hoodie.table.name", tableName)
.option("hoodie.datasource.write.table.type", "MERGE_ON_READ")
# Partition Stats
.option("hoodie.metadata.enable", "true")
.option("hoodie.metadata.partition.stats.enable", "true")
.option("hoodie.metadata.column.stats.enable", "true")
.option("hoodie.metadata.column.stats.column.list", "rider,fare")
# Record Level Index
.option("hoodie.metadata.record.index.enable", "true")
.option("hoodie.datasource.write.recordkey.field", "uuid")
.option("hoodie.metadata.index.secondary.enable", "true")
.option("hoodie.datasource.write.secondarykey.column", "city")
.option("hoodie.parquet.small.file.limit", "0")
.option("hoodie.clustering.inline", "true")
.option("hoodie.clustering.inline.max.commits", "4")
.option("hoodie.clustering.plan.strategy.target.file.max.bytes",
"1073741824")
.option("hoodie.clustering.plan.strategy.small.file.limit", "629145600")
.option("hoodie.clustering.plan.strategy.sort.columns", "city")
.mode("overwrite")
.save(basePath)`
When I clean up using spark submit **I am able to see** the record index
directory. May be if you can share the yours sample code and commands I will
try again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
