WillAyd commented on PR #45272:
URL: https://github.com/apache/arrow/pull/45272#issuecomment-3027916509
OK I am still seeing some runtime performance after refactoring the
RangeDataEqualsImpl class to `compare_internal.h` and creating custom functors
that call that directly per element, although its still a good deal behind
primitive performance:
```python
>>> arr1 = pa.array([list("abc"), list("def"), list("xyz")] * 1_000_000)
>>> arr2 = pa.array([list("abc"), list("def"), list("xyzz")] * 1_000_000)
>>> %timeit pc.equal(arr1, arr2)
336 ms ± 1.19 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```
I wonder if the ArrayIterator returning a `shared_ptr<ArrayData>` is the
next bottleneck?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
