zhengruifeng commented on code in PR #44546: URL: https://github.com/apache/spark/pull/44546#discussion_r1439989732
########## dev/py-tests/requirements-sql.txt: ########## @@ -0,0 +1,11 @@ +# PySpark dependencies for SQL tests + +numpy==1.26.2 Review Comment: @nchammas I just notice the previous discussion https://github.com/apache/spark/pull/27928. I personally prefer using `requirements.txt` files with pinned versions, one reason is that the dependency is actually cached in docker file, and I was confused about the version used in CI from time to time, e.g. we used the cached `RUN python3.9 -m pip install numpy pyarrow ...` before, and when pyarrow 13 released at 2023-8-23, I didn't know this release broke PySpark before the cached image was refreshed (at 2023-9-13). But I don't feel very strong about it and defer to @HyukjinKwon and @dongjoon-hyun on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
