caldempsey commented on code in PR #152:
URL: https://github.com/apache/spark-connect-go/pull/152#discussion_r2203504201
##########
spark/sql/dataframe.go:
##########
@@ -213,6 +213,7 @@ type DataFrame interface {
Take(ctx context.Context, limit int32) ([]types.Row, error)
// ToArrow returns the Arrow representation of the DataFrame.
ToArrow(ctx context.Context) (*arrow.Table, error)
+ ToLocalIterator(ctx context.Context) (types.RowIterator, error)
Review Comment:
Yeah, I'd prefer to rename this. I don't like the name `ToLocalIterator`
because it doesn't tell us anything about the underlying mechanism. When I
tested with Scala, I was surprised to find I can use ToLocalIterator in Scala
Spark Connect on streams of Arrow batches. `IterArrowStream` or `StreamRows`
would be so much better. Let me know what you prefer, I'm glad we're on the
same page here (for different reasons)!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
