viirya commented on code in PR #59582:
URL: https://github.com/apache/spark/pull/59582#discussion_r4461165967
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ColumnarArrowEvalPythonEvaluatorFactory.scala:
##########
@@ -155,11 +158,17 @@ private[python] class
ColumnarArrowEvalPythonEvaluatorFactory(
val inputColumnIndices = resolveColumnIndices(allInputs.toSeq)
- // Peek at first batch to check if Arrow-backed.
+ // Peek at the first batch to check whether its columns are Arrow vectors
+ // that may be held until the result of the batch comes back.
val peekIter = new PeekableIterator(inputIter)
val isArrow = peekIter.peek().exists { batch =>
Review Comment:
Fixed in cfce388a0f5: `isArrow` is now `canHoldInput`, and the comments of
paths 2 and 3 describe the new check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
