westonpace opened a new pull request #11223: URL: https://github.com/apache/arrow/pull/11223
This creates a black box test for testing that the Arrow kernels conform to a desired spec. The 'spec' in this case is a list of function descriptors in a bespoke DSL but it will eventually be replaced with something like substrait. At the moment this is only using empty arrays to test whether a kernel signature is supported or not. It is not testing the correctness of the function itself. I'd like to eventually expand on these test cases to check return types, shapes other than arrays, and so on. At the moment the biggest issue is probably that these tests are kind of slow. It takes about 60 seconds to run on my server. I believe most of this time is spent in pytest. I think it could be sped up considerably by parameterizing it ourselves instead of using pytest.mark.parameterize. The downside is that it is then difficult to tell what percentage of expected kernels pass and how many were xfail'd out. I'm not sure if that is important or not. Maybe we can write our own pytest.mark.parameterize that is configurable for "fast mode". In the meantime, I'm open to ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
