flyrain commented on PR #4546: URL: https://github.com/apache/iceberg/pull/4546#issuecomment-1099734525
> One concern I was thinking in general even from #4456 , whether all this JSON parsing increase the commit times which can be sensitive for Streaming applications (I guess having that flag to control partition length is good in #4456 as user can set it to 0 if so, but it doesn't help these ones). At some point should measure the commit time perf for large tables. Or just have a follow up pr for a flag to disable all these stats persistence altogether if its a concern, as right now there's no way to opt-out, wdyt? Schema, snapshot summary, partition spec are more likely impact the commit perf. We may think of a general way to control that. I'd suggest to handle that in another PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
