mbutrovich commented on issue #391: URL: https://github.com/apache/datafusion-comet/issues/391#issuecomment-2411913476
I ran TPC-H locally and profiled the sole executor with 4 CPU cores allocated to it. One thing I noticed is that `update_comet_metric` is taking 3.2% of the time. Within `Native_executePlan` it accounts for ~7-8% of an individual worker's CPU time in Comet. I want to look at the granularity that these operations occur at, and see if we can coalesce metrics on the native side and maybe ship more at once to reduce the JNI overhead. I want to add more metrics to Comet to understand where we're spending time, but the overhead is going to add up.  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
