timsaucer commented on code in PR #1241:
URL:
https://github.com/apache/datafusion-python/pull/1241#discussion_r2355771369
##########
python/datafusion/dataframe.py:
##########
@@ -370,6 +375,39 @@ def schema(self) -> pa.Schema:
Describing schema of the DataFrame
"""
return self.df.schema()
+
+ def to_batches(self) -> list[pa.RecordBatch]:
+ """Convert DataFrame to list of RecordBatches."""
+ return self.collect() # delegate to existing method
Review Comment:
I understand that `pyarrow` is a large dependency, but I also have a feeling
from the community that nearly everyone who is using `datafusion-python` to do
things like `to_batches` is using `pyarrow` in the next portions of their code.
I'm sure we can always find exceptions.
Now if there is a way we can remove this dependency and it doesn't break
existing workflows, that would be even better. I haven't made the time to sit
down and play with it, though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
