contentis commented on issue #50398: URL: https://github.com/apache/arrow/issues/50398#issuecomment-5353930421
> By the way, while I think this is desirable from a resource consumption perspective (CI, artifact storage), I will point out that it might make some operations slower at the conversion edge between Python objects and Arrow arrays. I think that is an acceptable tradeoff to make. I built #50409 and its merge base in release mode against the same Arrow C++ build. This was on Python 3.12.3 with NumPy 2.5.2 and pandas 3.0.5. I ran paired pyperf benchmarks in both orders on pinned CPU. - Python floats to Arrow: around 6–8% slower - Floats with pandas null handling: around 11% slower - Integers with pandas null handling: around 5–6% slower - Tuple to struct conversion: around 8% slower - NumPy Unicode to Arrow: around 4–8% slower The main sources appear to be per-element calls such as `PyFloat_AsDouble`, `PyType_GetFlags` and `PyTuple_GetItem` replacing direct field or macro access. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
