xuechendi commented on pull request #34396: URL: https://github.com/apache/spark/pull/34396#issuecomment-976099176
@BryanCutler and @HyukjinKwon, Please help me take a review, I updated this PR per our discussion last week Here is my last update I moved all arrow related conversions to ArrowConverters, so now I added or modified previous c onversions as below 6 functions toArrowRecordBatchIterartor : Iterator[InternalRow] -> Iterator[ArrowRecordBatch] fromArrowRecordBatchIterartor : Iterator[ArrowRecordBatch] -> Iterator[InternalRow] toBatchIterator Iterator[InteralRow] -> Iterator[Array[Byte]] (will internally call toArrowRecordBatchIterartor ) fromBatchIterator Iterator[Array[Byte]]-> Iterator[InteralRow] (will internally call fromArrowRecordBatchIterator) toColumnarBatchIterator Iterator[ArrowRecordBatch] -> Iterator[ColumnarBatch] arrow backend fromColumnarBatchIterator Iterator[ColumnarBatch] -> Iterator[ArrowRecordBatch] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
