[
https://issues.apache.org/jira/browse/BEAM-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414734#comment-16414734
]
Ahmet Altay commented on BEAM-3933:
-----------------------------------
The error is:
I Command "python setup.py egg_info" failed with error code 1 in
/tmp/pip-rFQTNL-build/
I You are using pip version 9.0.1, however version 9.0.3 is available.
I You should consider upgrading via the 'pip install --upgrade pip' command.
I /usr/local/bin/pip failed with exit status 1
I Dataflow base path override: https://dataflow.googleapis.com/
I Dataflow base path override: https://dataflow.googleapis.com/
I Installing setup packages ...
I Executing: /usr/local/bin/pip install
/var/opt/google/dataflow/dataflow_python_sdk.tar[gcp]
I Created new fluentd log writer for: /var/log/dataflow/boot-json.log
I Processing /var/opt/google/dataflow/dataflow_python_sdk.tar
I Complete output from command python setup.py egg_info:
I Traceback (most recent call last):
I File "<string>", line 1, in <module>
I IOError: [Errno 2] No such file or directory:
'/tmp/pip-FtI71Q-build/setup.py'
This is happening because DataflowRunner renames the sdk chosen by the user (by
setting --sdk_location) argument to dataflow_python_sdk.tar. And renaming and
installing a wheel file fails as above. Related code
(https://github.com/apache/beam/blob/597e3f92bc8be692d5d8e8040b33ce0c77350fa2/sdks/python/apache_beam/runners/dataflow/internal/dependency.py#L417)
needs to be fixed to name files correctly.
Reducing the priority, since this only affects developing with wheel files. It
should be addressed as a task for adding support to wheel files.
cc: [~charleschen] [~robertwb]
> Beam 2.4 python wheel is not working on dataflow runners
> --------------------------------------------------------
>
> Key: BEAM-3933
> URL: https://issues.apache.org/jira/browse/BEAM-3933
> Project: Beam
> Issue Type: Bug
> Components: examples-python, sdk-py-harness
> Affects Versions: 2.4.0
> Reporter: yifan zou
> Assignee: Ahmet Altay
> Priority: Major
>
> Running beam examples on DataflowRunner against python wheel are failing due
> to:
> Error syncing pod aaa0e0f09729d923b17726c465ccfcaf
> ("dataflow-beamapp-jenkins-032603332-03252033-3d3b-harness-hgb3_default(aaa0e0f09729d923b17726c465ccfcaf)"),
> skipping: failed to "StartContainer" for "python" with CrashLoopBackOff:
> "Back-off 5m0s restarting failed container=python
> pod=dataflow-beamapp-jenkins-032603332-03252033-3d3b-harness-hgb3_default(aaa0e0f09729d923b17726c465ccfcaf)
> Failed to install packages: failed to install SDK: exit status 1
>
> Dataflow job:
> [https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-25_20_33_34-14871401506794256600?project=apache-beam-testing]
> Jenkins job:
> https://builds.apache.org/job/beam_PostRelease_Python_Candidate/79/console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
