Re: Testing a pipeline with external transform

2023-04-24 Thread Sahith Nallapareddy via dev
Sounds good, let me try that thank you again! Thanks, Sahith On Mon, Apr 24, 2023 at 2:11 PM Chamikara Jayalath wrote: > > > On Mon, Apr 24, 2023 at 11:06 AM Sahith Nallapareddy > wrote: > >> Hello, >> >> Ah I missed that thank you! I am assuming for java jobs with java >> external transform,

Re: Testing a pipeline with external transform

2023-04-24 Thread Chamikara Jayalath via dev
On Mon, Apr 24, 2023 at 11:06 AM Sahith Nallapareddy wrote: > Hello, > > Ah I missed that thank you! I am assuming for java jobs with java external > transform, we run a Java Job Service? > No, the job service is tied to the runner not to the pipeline SDK. The Java quickstart guide gives instruc

Re: Testing a pipeline with external transform

2023-04-24 Thread Sahith Nallapareddy via dev
Hello, Ah I missed that thank you! I am assuming for java jobs with java external transform, we run a Java Job Service? Thanks, Sahith On Mon, Apr 24, 2023 at 2:01 PM Chamikara Jayalath wrote: > Have you looked into quickstart guides (Java [1], Python [2]) which also > gives instructions for

Re: Testing a pipeline with external transform

2023-04-24 Thread Chamikara Jayalath via dev
Have you looked into quickstart guides (Java [1], Python [2]) which also gives instructions for running using DirectRunner ? Thanks, Cham [1] https://beam.apache.org/documentation/sdks/java-multi-language-pipelines/ [2] https://beam.apache.org/documentation/sdks/python-multi-language-pipelines/

Testing a pipeline with external transform

2023-04-24 Thread Sahith Nallapareddy via dev
Hello, I was wondering the best way to test a pipeline with external transform. We were trying to use the DirectRunner but it seemed to hang on the external transform step. I was wondering if someone could point me to somewhere in the beam code where this is done or give some pointers on how to te