mohamd764 commented on issue #1795: URL: https://github.com/apache/datafusion-python/issues/1795#issuecomment-6221917689
`print(df)` only shows a preview (the first 10 rows by default), so "Data truncated." doesn't mean the result is incomplete. I checked these options with datafusion 55.0.0 (Python 3.13, Linux) using your table: ```python # 1. print more rows (default num=20) df.show(num=100) # 2. raise the limit used by print(df) / repr from datafusion.dataframe_formatter import configure_formatter configure_formatter(max_rows=50) print(df) # all 11 rows, no "Data truncated." # 3. get the full result as data instead of a printout df.to_pandas() # needs pandas df.to_pylist() # list of dicts (11 rows here) df.collect() # list of pyarrow RecordBatches df.to_arrow_table() ``` `configure_formatter` applies to every DataFrame for the rest of the session. `reset_formatter()` from the same module restores the defaults. One thing to watch for: `print(df)` and `df.show()` run the query each time. If you want to look at the result more than once, convert it once (for example `pdf = df.to_pandas()`), or use `df.cache()` to keep it in memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
