viirya commented on code in PR #1055:
URL: https://github.com/apache/datafusion-comet/pull/1055#discussion_r1828819842
##########
common/src/main/scala/org/apache/comet/vector/NativeUtil.scala:
##########
@@ -133,34 +135,35 @@ class NativeUtil {
// the Arrow arrays. For example, Iceberg column reader will skip deleted
rows internally in
// its `CometVector` implementation. The `ColumnarBatch` returned by the
reader will report
// logical number of rows which is less than actual number of rows due to
row deletion.
- numRows.headOption.getOrElse(batch.numRows())
+
+ builder.result()
}
/**
* Gets the next batch from native execution.
*
- * @param numOutputCols
- * The number of output columns
* @param func
* The function to call to get the next batch
* @return
* The number of row of the next batch, or None if there are no more
batches
*/
- def getNextBatch(
- numOutputCols: Int,
- func: (Array[Long], Array[Long]) => Long): Option[ColumnarBatch] = {
- val (arrays, schemas) = allocateArrowStructs(numOutputCols)
-
- val arrayAddrs = arrays.map(_.memoryAddress())
- val schemaAddrs = schemas.map(_.memoryAddress())
-
- val result = func(arrayAddrs, schemaAddrs)
Review Comment:
ditto. We should follow Arrow spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
