[
https://issues.apache.org/jira/browse/BEAM-8544?focusedWorklogId=337536&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-337536
]
ASF GitHub Bot logged work on BEAM-8544:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Nov/19 20:17
Start Date: 01/Nov/19 20:17
Worklog Time Spent: 10m
Work Description: aaltay commented on pull request #9966: [BEAM-8544] Use
ccache for compiling the Beam Python SDK.
URL: https://github.com/apache/beam/pull/9966#discussion_r341735449
##########
File path: sdks/python/container/Dockerfile
##########
@@ -43,9 +45,13 @@ RUN \
# Remove pip cache.
rm -rf /root/.cache/pip
+# Configure ccache prior to installing the SDK.
+RUN ln -s /usr/bin/ccache /usr/local/bin/gcc
+# These parameters are needed as pip compiles artifacts in random temporary
directories.
+RUN ccache --set-config=sloppiness=file_macro && ccache
--set-config=hash_dir=false
COPY target/apache-beam.tar.gz /opt/apache/beam/tars/
-RUN pip install /opt/apache/beam/tars/apache-beam.tar.gz[gcp] && \
+RUN pip install -v /opt/apache/beam/tars/apache-beam.tar.gz[gcp] && \
# Remove pip cache.
rm -rf /root/.cache/pip
Review comment:
Could we add a `pip freeze --all`? From the logs that will give us a
complete list of what exact versions are installed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 337536)
Time Spent: 40m (was: 0.5h)
> Install Beam SDK with ccache for faster re-install.
> ---------------------------------------------------
>
> Key: BEAM-8544
> URL: https://issues.apache.org/jira/browse/BEAM-8544
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-harness
> Reporter: Robert Bradshaw
> Assignee: Robert Bradshaw
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Re-compliling the C modules of the SDK takes 2-3 minutes. This adds to worker
> startup time whenever a custom SDK is being used (in particular, during
> development and testing). We can use ccache to re-use the old compile results
> when the Cython files have not changed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
