🤣 okay I’ll look again, I assumed it just crashed in a ball of flames!
On 17 September 2019 at 15:39:33, Benjamin Tan ([email protected]) wrote: I got this too! Did you manage to get any output? (I did) I reported this in another thread. This looks like a key error when StopWorker is called. As far as I know, it seems like the work has been processed. On Tue, Sep 17, 2019 at 10:31 PM Tom Barber <[email protected]> wrote: > Well my errors are different but still terminal: > > ERROR:grpc._server:Exception calling application: u'1-1' > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/grpc/_server.py", line 434, > in _call_behavior > response_or_iterator = behavior(argument, context) > File > "/usr/local/lib/python2.7/dist-packages/apache_beam-2.17.0.dev0-py2.7.egg/apache_beam/runners/worker/worker_pool_main.py", > line 126, in StopWorker > worker_process = > self._worker_processes.pop(stop_worker_request.worker_id) > KeyError: u'1-1' > ERROR:grpc._server:Exception calling application: u'2-1' > Traceback (most recent call last): > File "/usr/local/lib/python2.7/dist-packages/grpc/_server.py", line 434, > in _call_behavior > response_or_iterator = behavior(argument, context) > File > "/usr/local/lib/python2.7/dist-packages/apache_beam-2.17.0.dev0-py2.7.egg/apache_beam/runners/worker/worker_pool_main.py", > line 126, in StopWorker > worker_process = > self._worker_processes.pop(stop_worker_request.worker_id) > KeyError: u'2-1' > > > On 17 September 2019 at 14:46:12, Benjamin Tan ( > [email protected]) wrote: > > If it helps, I’m using Spark 2.4.4. The Apache Beam Python library on > master is 2.17.0-dev. > > > On 17 Sep 2019, at 9:39 PM, Tom Barber <[email protected]> wrote: > > Cool thanks Benjamin, I’ll give it a shot. > > Tom > > > On 17 September 2019 at 13:56:14, Benjamin Tan ( > [email protected]) wrote: > > > I encountered the exact same thing today. High five! Here’s how I managed > to make some progress: > > 1. Used the master branch > 2. Built and installed the Python SDK > > cd into the sdk library and python ./setup install > > I got some other errors but they didn’t seem to be show stoppers. > > On 17 Sep 2019, at 8:26 PM, Tom Barber <[email protected]> wrote: > > Hello folks, > > Day 3 of trying to get the basics going with Python & Spark 2.2.3. > > I’ve downgraded the spark version to 2.2.3 in the cradle build so that I > can run jobs against it. > > I’ve then written this: > > options = PipelineOptions(["--runner=PortableRunner", > "--job_endpoint=localhost:8099", "--environment_type=LOOPBACK"]) > > with beam.Pipeline(options=options) as p: > print("hello") > lines = p | 'Create' >> beam.Create(['hello', 'world']) > > Which in turn explodes with: > > Caused by: > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusRuntimeException: > UNIMPLEMENTED: Method not found! > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:235) > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:216) > at > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:141) > at > org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc$BeamFnExternalWorkerPoolBlockingStub.startWorker(BeamFnExternalWorkerPoolGrpc.java:226) > > When run against python 2 and > > AttributeError: module 'apache_beam.coders.coders' has no > attribute ‘VarIntCoder' > > When run with python 3.5. > > Should I just give up trying the portable runner at the moment or am I > doing something dumb/incompatible? I’m stuck on Spark 2.2.3 so upgrading > isn’t currently an option. > > Tom > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > All engagements are subject to Spicule Terms and Conditions of Business. > This email and its contents are intended solely for the individual to whom > it is addressed and may contain information that is confidential, > privileged or otherwise protected from disclosure, distributing or copying. > Any views or opinions presented in this email are solely those of the > author and do not necessarily represent those of Spicule Limited. The > company accepts no liability for any damage caused by any virus transmitted > by this email. If you have received this message in error, please notify us > immediately by reply email before deleting it from your system. Service of > legal notice cannot be effected on Spicule Limited by email. > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > All engagements are subject to Spicule Terms and Conditions of Business. > This email and its contents are intended solely for the individual to whom > it is addressed and may contain information that is confidential, > privileged or otherwise protected from disclosure, distributing or copying. > Any views or opinions presented in this email are solely those of the > author and do not necessarily represent those of Spicule Limited. The > company accepts no liability for any damage caused by any virus transmitted > by this email. If you have received this message in error, please notify us > immediately by reply email before deleting it from your system. Service of > legal notice cannot be effected on Spicule Limited by email. > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > All engagements are subject to Spicule Terms and Conditions of Business. > This email and its contents are intended solely for the individual to whom > it is addressed and may contain information that is confidential, > privileged or otherwise protected from disclosure, distributing or copying. > Any views or opinions presented in this email are solely those of the > author and do not necessarily represent those of Spicule Limited. The > company accepts no liability for any damage caused by any virus transmitted > by this email. If you have received this message in error, please notify us > immediately by reply email before deleting it from your system. Service of > legal notice cannot be effected on Spicule Limited by email. > -- Spicule Limited is registered in England & Wales. Company Number: 09954122. Registered office: First Floor, Telecom House, 125-135 Preston Road, Brighton, England, BN1 6AF. VAT No. 251478891. All engagements are subject to Spicule Terms and Conditions of Business. This email and its contents are intended solely for the individual to whom it is addressed and may contain information that is confidential, privileged or otherwise protected from disclosure, distributing or copying. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Spicule Limited. The company accepts no liability for any damage caused by any virus transmitted by this email. If you have received this message in error, please notify us immediately by reply email before deleting it from your system. Service of legal notice cannot be effected on Spicule Limited by email.
