paultiq commented on PR #50992: URL: https://github.com/apache/arrow/pull/50992#issuecomment-5414158272
FYI, I have this (wheels building with sccache) working on a [branch](https://github.com/paultiq/arrow/pull/2)... but the branch is a bit messy (I used OIDC permissions instead of akey/skey). In case it's helpful: a few things that might be helpful, see: - SCCACHE_BASEDIR is really important, otherwise the random build dir will pollute the sccache key (lots of misses) - Added the following in `python\CMakeLists.txt`. This is longer than it needs to be, since you just need to check for SCCACHE_BUCKET, I guess. ``` find_program(SCCACHE_FOUND sccache) if(NOT SCCACHE_FOUND AND DEFINED ENV{SCCACHE_PATH}) message(STATUS "Did not find sccache, using envvar fallback.") set(SCCACHE_FOUND $ENV{SCCACHE_PATH}) endif() # Only use sccache if a storage backend is configured if(SCCACHE_FOUND AND NOT CMAKE_C_COMPILER_LAUNCHER AND NOT CMAKE_CXX_COMPILER_LAUNCHER AND (DEFINED ENV{SCCACHE_AZURE_BLOB_CONTAINER} OR DEFINED ENV{SCCACHE_BUCKET} OR DEFINED ENV{SCCACHE_DIR} OR DEFINED ENV{SCCACHE_GCS_BUCKET} OR DEFINED ENV{SCCACHE_MEMCACHED} OR DEFINED ENV{SCCACHE_REDIS} )) message(STATUS "Using sccache: ${SCCACHE_FOUND}") set(CMAKE_C_COMPILER_LAUNCHER ${SCCACHE_FOUND}) set(CMAKE_CXX_COMPILER_LAUNCHER ${SCCACHE_FOUND}) endif() ``` And, other things I did: - explicitly disabled ARROW_USE_CCACHE - Uploaded the sccache logs, helpful for diagnosing cache misses (nearly impossible without) - Split the "build image" from "build wheel": this was just helpful to not pollute the build wheel time with the image build / docker pull. There's a few things where I used OIDC to be ignored: AWS_SESSION_TOKEN and AWS_ROLE_ARN can be ignored, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
