Re: Beam/Python/Flink: Unable to deserialize UnboundedSource for PubSub source

2020-06-17 Thread Maximilian Michels
You are using a proprietary connector which only works on Dataflow. You will have to use io.external.gcp.pubsub.ReadFromPubsub. PubSub support is experimental from Python. -Max On 09.06.20 06:40, Pradip Thachile wrote: > Quick update: this test code works just fine on Dataflow as well as the > D

Re: Beam/Python/Flink: Unable to deserialize UnboundedSource for PubSub source

2020-06-09 Thread Pradip Thachile
Quick update: this test code works just fine on Dataflow as well as the DirectRunner. Looks like the FlinkRunner is problematic for some reason here. On 2020/06/08 20:11:13, Pradip Thachile wrote: > Hey folks, > > I posted this on the Flink user mailing list but didn't get any traction > the

Beam/Python/Flink: Unable to deserialize UnboundedSource for PubSub source

2020-06-08 Thread Pradip Thachile
Hey folks, I posted this on the Flink user mailing list but didn't get any traction there (potentially since this is Beam related?). I've got a Beam/Python pipeline that works on the DirectRunner and now am trying to run this on a local dev Flink cluster. Running this yields an error out the g