pitrou opened a new issue, #40166:
URL: https://github.com/apache/arrow/issues/40166
### Describe the enhancement requested
When reading the logs of a wheel build on Windows I noticed these lines:
```
lib.cpp
C:\Python312\Lib\site-packages\numpy\_core\include\numpy\ndarraytypes.h(1250,38):
warning C4200: nonstandard extension used: zero-sized array in struct/union
[C:\arrow\python\build\temp.win-amd64-cpython-312\lib.vcxproj]
C:\Python312\Lib\site-packages\numpy\_core\include\numpy\ndarraytypes.h(1250,38):
message : This member will be ignored by a defaulted constructor or copy/move
assignment operator
[C:\arrow\python\build\temp.win-amd64-cpython-312\lib.vcxproj]
C:\arrow\python\build\temp.win-amd64-cpython-312\lib.cpp(335145,17): warning
C4244: '=': conversion from 'Py_ssize_t' to 'long', possible loss of data
[C:\arrow\python\build\temp.win-amd64-cpython-312\lib.vcxproj]
C:\arrow\python\build\temp.win-amd64-cpython-312\lib.cpp(335645,17): warning
C4244: '=': conversion from 'Py_ssize_t' to 'long', possible loss of data
[C:\arrow\python\build\temp.win-amd64-cpython-312\lib.vcxproj]
C:\arrow\python\build\temp.win-amd64-cpython-312\lib.cpp(335855,17): warning
C4244: '=': conversion from 'Py_ssize_t' to 'long', possible loss of data
[C:\arrow\python\build\temp.win-amd64-cpython-312\lib.vcxproj]
```
Ignoring what the warnings say, what stands out is that the `lib.cpp`
generated by Cython has *at least* 335000 lines. This is huge and can obviously
lead to enormous compile times, especially if the RAM is not large enough for
the C++ compiler to hold the entire intermediate representation(s) in memory.
We should definitely try to split the `_lib` into smaller parts, in order to
alleviate this problem.
(it is also a Cython problem that so much C++ code is generated, but I'm not
sure we can fix that).
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
