d33bs opened a new issue, #44530: URL: https://github.com/apache/arrow/issues/44530
### Describe the enhancement requested Thanks for the great efforts surrounding this project and the surrounding ecosystem! I find that when I'm using large PyArrow Tables that I sometimes need to create data tests which compare multiple tables for equality. At the moment I use the `pa.Table.equals()` method to make these comparisons. The output from these comparisons during, for example, `pytest` testing can be very hard to read with large tables (hundreds of thousands of rows, thousands of columns) - it can be nearly impossible to find the differences unless you know exactly where to look. As a result, I find myself leaning on a conversion via `pa.Table.to_pandas()` and then leveraging `pandas.testing.assert_frame_equals()` to garner better output and an idea of where the data issues may lie. The conversion can be costly with larger datasets both in terms of time and accuracy (conversions and `assert_frame_equals` settings). It'd be a fantastic time saver to have a function or some sort of testing suite which is ready to use within PyArrow itself! Perhaps something like a `pa.testing.assert_table_equals()` which could help with repetitive tasks like this without the cost of conversion into another package's format (like Pandas). If something like this already exists within another side of the Arrow project it'd be great to share that capability within PyArrow somehow. Please let me know if I may clarify at all and thanks in advance for any thoughts! ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
