jorgecarleitao edited a comment on pull request #8880: URL: https://github.com/apache/arrow/pull/8880#issuecomment-742959778
@sweb , datafusion is built on top of the arrow crate, and relies on both the array types and the `compute` kernels. Different nodes in the logical plan require different `compute` kernels. For example, `cast`, `concat` and `take` for `sort`, hashing for group-by ops with the type (if applicable), `MutableArrayData` for the `join`. [here](https://github.com/apache/arrow/blob/master/rust/datafusion/tests/sql.rs#L1491) is an example of a test that does not check IO with CSV, only the execute in memory. I do not know if we have DecimalType on the files `ARROW_TEST_DATA`, but if yes, it is a matter of reading it in DataFusion and use it directly (and see where it breaks). Essentially, almost everything that has a `match datatype` needs to be covered for a complete implementation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
