Vsevolod3 commented on issue #8071: URL: https://github.com/apache/hudi/issues/8071#issuecomment-1587175785
Thanks for your response. Here are our properties for state-backend (from flink-conf on EMR), and as you can see, incremental checkpointing is enabled: ``` "state.backend": "rocksdb", "state.backend.incremental": "true", "state.checkpoint-storage": "filesystem", "state.checkpoints.dir": "s3://*****/flink-checkpoints", ``` Do you see any potential problems with these configs? In relation to using the bucket index, are these the correct properties to try? I will do some testing with the defaults, but if you have any suggestions for improving these, please let me know. ``` hoodie.index.type=BUCKET hoodie.bucket.index.hash.field=<some field, probably PK> hoodie.bucket.index.num.buckets=256 hoodie.index.bucket.engine=SIMPLE hoodie.bucket.index.num.buckets=4 ``` Thanks in advance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
