morvenhuang created BEAM-6259:
---------------------------------
Summary: Portable Beam failed to run Python wordcount on Flink
Key: BEAM-6259
URL: https://issues.apache.org/jira/browse/BEAM-6259
Project: Beam
Issue Type: Bug
Components: examples-python
Affects Versions: 2.9.0, 2.8.0
Environment: CentOS Linux release 7.6.1810 (also tried Ubuntu 18.10)
flink-1.5.3 (standalone, without changing any conf, just download and untar,
then kick off by ./start-cluster.sh)
Apache Beam, release-2.9.0 branch (also tried 2.8.0 branch)
Docker version 18.09.0 (also tried the old 1.13.1)
java version "1.8.0_111"
Python 2.7.5
Reporter: morvenhuang
Assignee: Ahmet Altay
Recently I tried to run portable python wordcount example following exactly
what it says in the guide page
[https://beam.apache.org/roadmap/portability/|https://beam.apache.org/roadmap/portability/,],
but still no luck after several days trying.
*Here're my steps,*
1. ./gradlew :beam-sdks-python-container:docker (to build the sdk container, no
error)
2. ./gradlew :beam-runners-flink_2.11-job-server:installShadowDist (to build
jobservice jar, no error)
3. ./gradlew :beam-runners-flink_2.11-job-server:runShadow
-PflinkMasterUrl=localhost:8081 (to run jobservice, no error)
4. ./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099
(to submit wordcount job, error as mentioned below)
*Have seen 2 errors, sometime, it says,*
{quote}BeamIOError: src and dst files do not exist. src:
/tmp/beam-temp-py-wordcount-direct-56d037a2ffa611e88f9208002721555c/020e033b-ec4f-495f-bfa4-e49b59ef9d36.py-wordcount-direct,
dst: /tmp/py-wordcount-direct-00000-of-00001 with exceptions None [while
running 'write/Write/WriteImpl/FinalizeWrite'] with exceptions None
{quote}
*Or it says,*
{quote}RuntimeError: IOError: [Errno 2] No such file or directory:
'/tmp/beam-temp-py-wordcount-direct-7013276afc6311e8a71108002721555c/72786d7f-a2b4-461c-91fa-c7e04d55ea3a.py-wordcount-direct'
[while running 'write/Write/WriteImpl/WriteBundles']
{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
