Re: Beam/flink/kubernetes/minikube/wordcount example
I got hung up on that issue earlier this week. Was using Flink 1.7. V2.15 of Beam. Wasn't using Kubernetes. Then gave up, so don't have a solution :-/ I don't understand the job server enough, but think I was getting error when I did not have it running (I still don't understand portability enough, so might not be using this terminology correctly). On Wed, Sep 11, 2019 at 1:26 PM Matthew Patterson wrote: > Nope: dang, thanks. > > On 9/11/19, 3:49 PM, "Robert Bradshaw" wrote: > > CAUTION: This email originated from outside of the organization. Do > not click links or open attachments unless you recognize the sender and > know the content is safe. > > > Is your input on a file system accessible to the workers? (Including, > from within Docker, if the workers are running in docker.) > > On Wed, Sep 11, 2019 at 12:03 PM Matthew Patterson > wrote: > > > > Hi Beamers, > > > > > > > > I am running the `wordcount` example, but following example from > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbeam.apache.org%2Fdocumentation%2Frunners%2Fflink%2Fdata=02%7C01%7Cmpatterson%40aligntech.com%7Cda40636a76c64746508108d736f12833%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C1%7C637038281702213802sdata=mYqB38n4myNxK9Krvh8opjxK%2BK4kHbwfBLZcgmt3OF8%3Dreserved=0, > that is, I change the pipeline initialization as follows. > > > > > > > > ``` > > > > import apache_beam as beam > > > > from apache_beam.options.pipeline_options import PipelineOptions > > > > > > > > options = PipelineOptions(["--runner=FlinkRunner", > "--flink_version=1.8", "--flink_master_url=localhost:8081"]) > > > > with beam.Pipeline(options) as p: > > > > … > > > > ``` > > > > > > > > Running against my minikube cluster (), I get: > > > > > > > > “RuntimeError: Pipeline > BeamApp-mpatterson-0911164258-7ef8768c_71984a02-5036-421e-9754-b57dbc628d3f > failed in state FAILED: java.io.IOException: FAILED_TO_UNCOMPRESS(5) > > > > ” > > > > > > > > Any ideas? > > > > > > > > Thanks, > > > > Matt > > > > > > > > (minikube version: v1.3.1 > > > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631 > > > > > > > > bash-3.2$ kubectl version > > > > Client Version: version.Info{Major:"1", Minor:"14", > GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", > GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", > GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"} > > > > Server Version: version.Info{Major:"1", Minor:"15", > GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", > GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z", > GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} > > > > > > > > Flink job- and task-manager containers both built from flink:1.8 > > > > ) > > > > > > > > Full output > > > > >>> > > > > > > > > /anaconda3/envs/aws/bin/python > /Users/mpatterson/Library/Preferences/PyCharmCE2019.2/scratches/beam_me_up.py > > > > /Users/mpatterson/dev/beam/sdks/python/apache_beam/__init__.py:84: > UserWarning: Some syntactic constructs of Python 3 are not yet fully > supported by Apache Beam. > > > > 'Some syntactic constructs of Python 3 are not yet fully supported > by ' > > > > executable: /anaconda3/envs/aws/bin/python > > > > beam.__version__: 2.15.0 > > > > WARNING:root:Make sure that locally built Python SDK docker image > has Python 3.7 interpreter. See also: BEAM-7474. > > > > INFO:root:Using latest locally built Python SDK docker image: > mpatterson-docker-apache.bintray.io/beam/python3:latest. > > > > INFO:root: 0x11850b200> > > > > INFO:root: > > > > > WARNING:root:Downloading job server jar from > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepo.maven.apache.org%2Fmaven2%2Forg%2Fapache%2Fbeam%2Fbeam-runners-flink-1.8-job-server%2F2.15.0%2Fbeam-runners-flink-1.8-job-server-2.15.0.jardata=02%7C01%7Cmpatterson%40aligntech.com%7Cda40636a76c64746508108d736f12833%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C1%7C637038281702213802sdata=Vj3F0j39Ped9UBS0196wZA1CN9UT0ckdXfLolNbM94E%3Dreserved=0 > > > > [main] INFO > org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - > ArtifactStagingService started on localhost:57443 > > > > [main] INFO > org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - Java > ExpansionService started on localhost:57444 > > > > [main] INFO > org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - > JobService started on localhost:57439 > > > > [grpc-default-executor-0] ERROR >
Re: Beam/flink/kubernetes/minikube/wordcount example
Nope: dang, thanks. On 9/11/19, 3:49 PM, "Robert Bradshaw" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Is your input on a file system accessible to the workers? (Including, from within Docker, if the workers are running in docker.) On Wed, Sep 11, 2019 at 12:03 PM Matthew Patterson wrote: > > Hi Beamers, > > > > I am running the `wordcount` example, but following example from https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbeam.apache.org%2Fdocumentation%2Frunners%2Fflink%2Fdata=02%7C01%7Cmpatterson%40aligntech.com%7Cda40636a76c64746508108d736f12833%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C1%7C637038281702213802sdata=mYqB38n4myNxK9Krvh8opjxK%2BK4kHbwfBLZcgmt3OF8%3Dreserved=0, that is, I change the pipeline initialization as follows. > > > > ``` > > import apache_beam as beam > > from apache_beam.options.pipeline_options import PipelineOptions > > > > options = PipelineOptions(["--runner=FlinkRunner", "--flink_version=1.8", "--flink_master_url=localhost:8081"]) > > with beam.Pipeline(options) as p: > > … > > ``` > > > > Running against my minikube cluster (), I get: > > > > “RuntimeError: Pipeline BeamApp-mpatterson-0911164258-7ef8768c_71984a02-5036-421e-9754-b57dbc628d3f failed in state FAILED: java.io.IOException: FAILED_TO_UNCOMPRESS(5) > > ” > > > > Any ideas? > > > > Thanks, > > Matt > > > > (minikube version: v1.3.1 > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631 > > > > bash-3.2$ kubectl version > > Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"} > > Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} > > > > Flink job- and task-manager containers both built from flink:1.8 > > ) > > > > Full output > > >>> > > > > /anaconda3/envs/aws/bin/python /Users/mpatterson/Library/Preferences/PyCharmCE2019.2/scratches/beam_me_up.py > > /Users/mpatterson/dev/beam/sdks/python/apache_beam/__init__.py:84: UserWarning: Some syntactic constructs of Python 3 are not yet fully supported by Apache Beam. > > 'Some syntactic constructs of Python 3 are not yet fully supported by ' > > executable: /anaconda3/envs/aws/bin/python > > beam.__version__: 2.15.0 > > WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. See also: BEAM-7474. > > INFO:root:Using latest locally built Python SDK docker image: mpatterson-docker-apache.bintray.io/beam/python3:latest. > > INFO:root: > > INFO:root: > > WARNING:root:Downloading job server jar from https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepo.maven.apache.org%2Fmaven2%2Forg%2Fapache%2Fbeam%2Fbeam-runners-flink-1.8-job-server%2F2.15.0%2Fbeam-runners-flink-1.8-job-server-2.15.0.jardata=02%7C01%7Cmpatterson%40aligntech.com%7Cda40636a76c64746508108d736f12833%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C1%7C637038281702213802sdata=Vj3F0j39Ped9UBS0196wZA1CN9UT0ckdXfLolNbM94E%3Dreserved=0 > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - ArtifactStagingService started on localhost:57443 > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - Java ExpansionService started on localhost:57444 > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - JobService started on localhost:57439 > > [grpc-default-executor-0] ERROR org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService - Encountered Unexpected Exception for Invocation job_9443af24-5ef1-4a65-8952-67b4f13eb5cb > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusException: NOT_FOUND > > at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Status.asException(Status.java:541) > > at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getInvocation(InMemoryJobService.java:364) > > at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getStateStream(InMemoryJobService.java:285) > >
Re: Beam/flink/kubernetes/minikube/wordcount example
Is your input on a file system accessible to the workers? (Including, from within Docker, if the workers are running in docker.) On Wed, Sep 11, 2019 at 12:03 PM Matthew Patterson wrote: > > Hi Beamers, > > > > I am running the `wordcount` example, but following example from > https://beam.apache.org/documentation/runners/flink/, that is, I change the > pipeline initialization as follows. > > > > ``` > > import apache_beam as beam > > from apache_beam.options.pipeline_options import PipelineOptions > > > > options = PipelineOptions(["--runner=FlinkRunner", "--flink_version=1.8", > "--flink_master_url=localhost:8081"]) > > with beam.Pipeline(options) as p: > > … > > ``` > > > > Running against my minikube cluster (), I get: > > > > “RuntimeError: Pipeline > BeamApp-mpatterson-0911164258-7ef8768c_71984a02-5036-421e-9754-b57dbc628d3f > failed in state FAILED: java.io.IOException: FAILED_TO_UNCOMPRESS(5) > > ” > > > > Any ideas? > > > > Thanks, > > Matt > > > > (minikube version: v1.3.1 > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631 > > > > bash-3.2$ kubectl version > > Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", > GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", > BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", > Platform:"darwin/amd64"} > > Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", > GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", > BuildDate:"2019-08-05T09:15:22Z", GoVersion:"go1.12.5", Compiler:"gc", > Platform:"linux/amd64"} > > > > Flink job- and task-manager containers both built from flink:1.8 > > ) > > > > Full output > > >>> > > > > /anaconda3/envs/aws/bin/python > /Users/mpatterson/Library/Preferences/PyCharmCE2019.2/scratches/beam_me_up.py > > /Users/mpatterson/dev/beam/sdks/python/apache_beam/__init__.py:84: > UserWarning: Some syntactic constructs of Python 3 are not yet fully > supported by Apache Beam. > > 'Some syntactic constructs of Python 3 are not yet fully supported by ' > > executable: /anaconda3/envs/aws/bin/python > > beam.__version__: 2.15.0 > > WARNING:root:Make sure that locally built Python SDK docker image has Python > 3.7 interpreter. See also: BEAM-7474. > > INFO:root:Using latest locally built Python SDK docker image: > mpatterson-docker-apache.bintray.io/beam/python3:latest. > > INFO:root: > > > INFO:root: > > > WARNING:root:Downloading job server jar from > https://repo.maven.apache.org/maven2/org/apache/beam/beam-runners-flink-1.8-job-server/2.15.0/beam-runners-flink-1.8-job-server-2.15.0.jar > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver > - ArtifactStagingService started on localhost:57443 > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver > - Java ExpansionService started on localhost:57444 > > [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver > - JobService started on localhost:57439 > > [grpc-default-executor-0] ERROR > org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService - > Encountered Unexpected Exception for Invocation > job_9443af24-5ef1-4a65-8952-67b4f13eb5cb > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusException: NOT_FOUND > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Status.asException(Status.java:541) > > at > org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getInvocation(InMemoryJobService.java:364) > > at > org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getStateStream(InMemoryJobService.java:285) > > at > org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:896) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86) > > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:322) > > at >
Beam/flink/kubernetes/minikube/wordcount example
Hi Beamers, I am running the `wordcount` example, but following example from https://beam.apache.org/documentation/runners/flink/, that is, I change the pipeline initialization as follows. ``` import apache_beam as beam from apache_beam.options.pipeline_options import PipelineOptions options = PipelineOptions(["--runner=FlinkRunner", "--flink_version=1.8", "--flink_master_url=localhost:8081"]) with beam.Pipeline(options) as p: … ``` Running against my minikube cluster (), I get: “RuntimeError: Pipeline BeamApp-mpatterson-0911164258-7ef8768c_71984a02-5036-421e-9754-b57dbc628d3f failed in state FAILED: java.io.IOException: FAILED_TO_UNCOMPRESS(5) ” Any ideas? Thanks, Matt (minikube version: v1.3.1 commit: ca60a424ce69a4d79f502650199ca2b52f29e631 bash-3.2$ kubectl version Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Flink job- and task-manager containers both built from flink:1.8 ) Full output >>> /anaconda3/envs/aws/bin/python /Users/mpatterson/Library/Preferences/PyCharmCE2019.2/scratches/beam_me_up.py /Users/mpatterson/dev/beam/sdks/python/apache_beam/__init__.py:84: UserWarning: Some syntactic constructs of Python 3 are not yet fully supported by Apache Beam. 'Some syntactic constructs of Python 3 are not yet fully supported by ' executable: /anaconda3/envs/aws/bin/python beam.__version__: 2.15.0 WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter. See also: BEAM-7474. INFO:root:Using latest locally built Python SDK docker image: mpatterson-docker-apache.bintray.io/beam/python3:latest. INFO:root: INFO:root: WARNING:root:Downloading job server jar from https://repo.maven.apache.org/maven2/org/apache/beam/beam-runners-flink-1.8-job-server/2.15.0/beam-runners-flink-1.8-job-server-2.15.0.jar [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - ArtifactStagingService started on localhost:57443 [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - Java ExpansionService started on localhost:57444 [main] INFO org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver - JobService started on localhost:57439 [grpc-default-executor-0] ERROR org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService - Encountered Unexpected Exception for Invocation job_9443af24-5ef1-4a65-8952-67b4f13eb5cb org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusException: NOT_FOUND at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Status.asException(Status.java:541) at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getInvocation(InMemoryJobService.java:364) at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getStateStream(InMemoryJobService.java:285) at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:896) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:322) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:762) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at org.apache.beam.vendor.grpc.v1p21p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at