nsivabalan commented on issue #3324: URL: https://github.com/apache/hudi/issues/3324#issuecomment-894489584
Hey @conanxjp : sorry for the late turn around. Would recommend using MOR table as its write optimized. Especially for streaming use-cases, COW might not work out since it takes previous snapshot and merges incoming records and rewrites new version for data files. with MOR(Merge on Read) table, writes are just appended as log files and later merged during real time read. You can schedule compaction which will compact the base data files and log files to a single compacted base file (newer version of data file). Once you have similar data w/ MOR table type, we can investigate further for index configs and so on. [hoodie.datasource.write.table.type](https://hudi.apache.org/docs/configurations/#table_type_opt_key): MERGE_ON_READ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
