TomAugspurger commented on issue #5800: URL: https://github.com/apache/iceberg/issues/5800#issuecomment-1553478936
> Generally, should this be part of the Dask library instead of PyIceberg? I was wondering that too. I think either would be sensible, but I'd lean slightly towards putting the implementation in Dask (and pyiceberg could add a `.to_dask_dataframe` to mirror `to_pandas` if desired). That said, the current implementation does use a private `_file_to_table`. I think pyiceberg would need to add a public method that achieves that before Dask could use it. And given the similarity to `pyiceberg.io.pyarrow`, maybe we could start with some refactoring to add some methods used by both the `pyarrow` and `dask.dataframe` implementations? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
