Hi Team,

I am writing to get a better understanding of an issue I am facing with the Apache Beam - python .

I am running a python dataflow pipeline which reads in data from big query then, collates the data in a single pandas data frame and then processes. I did not have any issues while testing using the Direct runner, but ran into some unexpected issues while using the Dataflow Runner. I have posted a question regarding the same at Stack overflow. I was advised to contact you guys for better insight into this.

I am adding the link for the question. I would really appreciate if someone could help me figure out this issue.

https://stackoverflow.com/questions/60437931/python-dataflow-dofn-class-function-finish-bundle-running-multiple-times-and-giv

 

 

 

 

Thanks and Regards

Taranbir Wraich

Cloud Analyst

P: 613-366-7881 ext 13X

P: 888-243-4619 ext 13X

E: [email protected]

 

Napkyn Inc. | 888.243.4619 | Twitter | LinkedIn | napkyn.com

This email, including any attachments, is for the sole use of the intended recipient and may contain confidential information. If you are not the intended recipient, please immediately notify us by reply email or by telephone, delete this email and destroy any copies. Thank you.

 


Napkyn Inc. | 888.243.4619 | Twitter | LinkedIn | napkyn.com

This email, including any attachments, is for the sole use of the intended recipient and may contain confidential information. If you are not the intended recipient, please immediately notify us by reply email or by telephone, delete this email and destroy any copies. Thank you.


Reply via email to