Reo-LEI edited a comment on issue #3118: URL: https://github.com/apache/iceberg/issues/3118#issuecomment-920668254
@jackye1995 Yes, my original intention was to resolve the rewrite of streaming cdc data. At first I am trying to speed up the delete compaction progress, but I found that all engines can benefit from this optimization. > Maybe the second compactor in streaming pipeline approach is not avoidable although a bit complex. +1, I think flink really need a compactor to apply/compaction delete file in streaming. > And just curious, it looks like the code is running in the same application as #2680 , is your benchmark based on the in memory map or this rock DB backed map? This benchmark is based on the in-memory map, I'm not apply this PR in test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
