viirya commented on issue #20: URL: https://github.com/apache/arrow-datafusion-comet/issues/20#issuecomment-1945457105
Yes, that's correct. At the boundary between Comet operator and Spark operator, we need ColumnarToRow operator to convert from ColumnarBatch to Spark InternalRow which takes some time cost. Overall we expect that the gain obtained from native operators can be bigger than these cost. And with more native operator support, we can reduce such fallback and the cost of columnar to row conversion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
