gt; Does your code work without the launcher? Better check this step by step
>> to figure out which part causes this error.
>>
>> On Sat, Feb 24, 2024 at 3:25 AM George Dekermenjian
>> wrote:
>>
>>> I have a python pipeline that uses the bigquery st
ote:
>
>> I have a python pipeline that uses the bigquery storage write method
>> (cross language with Java). I’m building launcher and worker docker images
>> and then launching the flex template. The launcher fails due to the
>> following runtime error in dataflow.
>>
&
Does your code work without the launcher? Better check this step by step to
figure out which part causes this error.
On Sat, Feb 24, 2024 at 3:25 AM George Dekermenjian
wrote:
> I have a python pipeline that uses the bigquery storage write method
> (cross language with Java). I’m bu
I have a python pipeline that uses the bigquery storage write method (cross
language with Java). I’m building launcher and worker docker images and
then launching the flex template. The launcher fails due to the following
runtime error in dataflow.
I’m using runner v2 and it is a streaming
+user
-- Forwarded message -
From: Chamikara Jayalath
Date: Wed, Oct 12, 2022 at 9:38 AM
Subject: Re: Cross Language
To: phani geeth
On Wed, Oct 12, 2022 at 12:25 AM phani geeth wrote:
> Hi Chamikara,
>
> Thanks for replying.
>
> My use case is to write
11:06, phani geeth wrote:
> >
> > Hi,
> >
> > Does the present cross Language functionality support creating custom
> Java transforms and calling from python in Dataflow runner.
> >
> > Use case: use existing Java IO as cross Language transform and call in
> python pipeline in Dataflow runner.
> >
> >
> > Regards
> > Phani Geeth
>
>
Python SDK.
[1] https://beam.apache.org/documentation/io/connectors/
—
Alexey
> On 11 Oct 2022, at 11:06, phani geeth wrote:
>
> Hi,
>
> Does the present cross Language functionality support creating custom Java
> transforms and calling from python in Dataflow runner.
>
Hi,
Does the present cross Language functionality support creating custom Java
transforms and calling from python in Dataflow runner.
Use case: use existing Java IO as cross Language transform and call in
python pipeline in Dataflow runner.
Regards
Phani Geeth
:
>
>
>
> On Wed, Aug 17, 2022 at 3:05 AM Yu Watanabe wrote:
>>
>> Hello.
>>
>> I am trying to write code for cross language transform for
>> ElasticsearchIO but having trouble with it.
>> I would appreciate it if I could get help.
>>
>> A
On Wed, Aug 17, 2022 at 3:05 AM Yu Watanabe wrote:
> Hello.
>
> I am trying to write code for cross language transform for
> ElasticsearchIO but having trouble with it.
> I would appreciate it if I could get help.
>
> As describe in doc and also referencing
unsubscribe
On Wed, Aug 17, 2022 at 12:05 PM Yu Watanabe wrote:
> Hello.
>
> I am trying to write code for cross language transform for
> ElasticsearchIO but having trouble with it.
> I would appreciate it if I could get help.
>
> As describe in doc and also referencing
Hello.
I am trying to write code for cross language transform for
ElasticsearchIO but having trouble with it.
I would appreciate it if I could get help.
As describe in doc and also referencing KafkaIO ,
https://beam.apache.org/documentation/programming-guide/#1311-creating-cross-language-java
On Fri, Jun 11, 2021 at 9:49 AM Reynaldo Baquerizo <
reynaldo.michel...@bairesdev.com> wrote:
>
> Hi,
>
> What’s the state of runners support for cross-language transforms?
> Is this roadmap up to date
> https://beam.apache.org/roadmap/connectors-multi-sdk ?
>
>
Hi,
What’s the state of runners support for cross-language transforms?
Is this roadmap up to date
https://beam.apache.org/roadmap/connectors-multi-sdk ?
I’m looking for documentation on how to run (java pipeline calling python
transforms) on Dataflow/Flink. Any reference?
—
Reynaldo
n case more folks wish to take a look before the PR is
> merged.
>
> PR is https://github.com/apache/beam/pull/13317
>
> Thanks,
> Cham
>
> On Thu, Nov 12, 2020 at 1:17 PM Chamikara Jayalath
> wrote:
>
>> Seems like a good place to promote this PR that adds docu
Great. Thanks.
On Mon, Jun 1, 2020 at 9:14 AM Alexey Romanenko
wrote:
> Yes, I tested it with the cross-language transform (Java pipeline with
> Python external transform).
>
> On 1 Jun 2020, at 17:49, Chamikara Jayalath wrote:
>
> To clarify, is the error resolved with
Yes, I tested it with the cross-language transform (Java pipeline with Python
external transform).
> On 1 Jun 2020, at 17:49, Chamikara Jayalath wrote:
>
> To clarify, is the error resolved with the cross-language transform as well ?
> If not please file a Jira.
>
> On Mon
To clarify, is the error resolved with the cross-language transform as well
? If not please file a Jira.
On Mon, Jun 1, 2020 at 8:24 AM Kyle Weaver wrote:
> > It would be useful to print out such errors with Error level log, I
> think.
>
> I agree, using environment_type=PROCE
>> https://github.com/apache/beam/blob/a5b2046b10bebc59c5bde41d4cb6498058fdada2/sdks/python/apache_beam/pipeline.py#L209
>> [2] https://jira.apache.org/jira/browse/BEAM-10151
>>
>> On Fri, May 29, 2020 at 10:05 AM Alexey Romanenko <
>> aromanenko@gmail.com> wr
py#L209>
>> [2] https://jira.apache.org/jira/browse/BEAM-10151
>> <https://jira.apache.org/jira/browse/BEAM-10151>
>> On Fri, May 29, 2020 at 10:05 AM Alexey Romanenko > <mailto:aromanenko@gmail.com>> wrote:
>> Yes, I did run only Java pipeline with Portabl
line.py#L209
> [2] https://jira.apache.org/jira/browse/BEAM-10151
>
> On Fri, May 29, 2020 at 10:05 AM Alexey Romanenko <
> aromanenko@gmail.com> wrote:
>
>> Yes, I did run only Java pipeline with Portable Runner and there is the
>> same error.
>>
>> Als
lexey Romanenko <mailto:aromanenko@gmail.com>> wrote:
> Yes, I did run only Java pipeline with Portable Runner and there is the same
> error.
>
> Also, I did the same (without cross-language component) against Beam 2.19 and
> 2.20.
> It works fine against Beam 2.19 (
ame (without cross-language component) against Beam 2.19
> and 2.20.
> It works fine against Beam 2.19 (as expected, since I tested it already
> before) and fails with kind the same error against Beam 2.20:
>
> 20/05/29 15:59:23 ERROR
> org.apache.beam.runners.fnexecution.jobsubmission
Yes, I did run only Java pipeline with Portable Runner and there is the same
error.
Also, I did the same (without cross-language component) against Beam 2.19 and
2.20.
It works fine against Beam 2.19 (as expected, since I tested it already before)
and fails with kind the same error against
Can you try removing the cross-language component(s) from the pipeline and
see if it still has the same error?
On Thu, May 28, 2020 at 4:15 PM Alexey Romanenko
wrote:
> For testing purposes, it’s just “Create.of(“Name1”, “Name2”, ...)"
>
> On 28 May 2020, at 19:29, Kyle Weaver w
For testing purposes, it’s just “Create.of(“Name1”, “Name2”, ...)"
> On 28 May 2020, at 19:29, Kyle Weaver wrote:
>
> What source are you using?
>
> On Thu, May 28, 2020 at 1:24 PM Alexey Romanenko <mailto:aromanenko@gmail.com>> wrote:
> Hello,
>
&
included in the release ?
>
> On Thu, May 28, 2020 at 10:29 AM Kyle Weaver wrote:
>
>> What source are you using?
>>
>> On Thu, May 28, 2020 at 1:24 PM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Hello,
>>>
&
> wrote:
>
>> Hello,
>>
>> I’m trying to run a Cross-Language pipeline (Beam 2.21, Java pipeline
>> with an external Python transform) with a PROCESS SDK Harness and Spark
>> Portable Runner but it fails.
>> To do that I have a running Spark Runner Job Serve
What source are you using?
On Thu, May 28, 2020 at 1:24 PM Alexey Romanenko
wrote:
> Hello,
>
> I’m trying to run a Cross-Language pipeline (Beam 2.21, Java pipeline with
> an external Python transform) with a PROCESS SDK Harness and Spark Portable
> Runner but it fails.
>
Hello,
I’m trying to run a Cross-Language pipeline (Beam 2.21, Java pipeline with an
external Python transform) with a PROCESS SDK Harness and Spark Portable Runner
but it fails.
To do that I have a running Spark Runner Job Server (Spark local) and
standalone Expansion Service (Python) which
30 matches
Mail list logo
