I'm getting class not found error while running the word count example on
Dataproc 2.1 with Beam 2.50.0. The class exists under the jar. Does
anyone know how to resolve this?
This is a list of dependency versions:
2.50.0
v2-rev20230520-2.0.0
2.0.0
32.1.2-jre
2.1
2.14.1
Reshuffle/redistribute being a transform has the benefit of allowing
existing runners that aren't updated to be aware of the new urns to rely on
an SDK side implementation, which may be more expensive than what the
runner is able to do with that awareness.
Aka: it gives purpose to the fallback
Is it really toggleable in Java? I imagine that if it's a toggle it'd be a
very sticky toggle since it'd be easy for PTransforms to accidentally rely
on it.
On Thu, Oct 5, 2023 at 12:33 PM Robert Bradshaw wrote:
> Huh. This used to be a hard error in Java, but I guess it's togglable
> with an
+1 (binding)
Thanks !
Regards
JB
On Tue, Oct 3, 2023 at 7:58 PM Kenneth Knowles wrote:
>
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.51.0, as
> follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific
Huh. This used to be a hard error in Java, but I guess it's togglable
with an option now. We should probably add the option to toggle Python too.
(Unclear what the default should be, but this probably ties into
re-thinking how pipeline update should work.)
On Thu, Oct 5, 2023 at 4:58 AM Joey Tran
+1 (binding) - reviewed doc updates.
On Thu, Oct 5, 2023 at 12:28 AM Jan Lukavský wrote:
> +1 (binding)
>
> Tested Java SDK with Flink Runner on own test-cases.
>
> Jan
> On 10/4/23 21:10, Bruno Volpato via dev wrote:
>
> +1 (non-binding).
>
> Tested with
Another perspective, ignoring runners custom implementations and non-Java
SDKs could be that the semantics are perfectly well defined: it is a
composite and its semantics are defined by its implementation in terms of
primitives. It is just that this expansion is not what we want so we should
not
I totally agree. I am motivated right now by the fact that it is already
used all over the place but with no consistent semantics. Maybe it is
simpler to focus on just making the minimal change, which would basically
be to update the expansion of the Reshuffle in the Java SDK.
Kenn
On Thu, Oct
Given that this is a hint, I'm not sure redistribute should be a PTransform
as opposed to some other way to hint to a runner.
I'm not sure of what the syntax of that would be, but a semantic no-op
transform that the runner may or may not do anything with is odd.
On Thu, Oct 5, 2023 at 11:30 AM
So a high level suggestion from Robert that I want to highlight as a
top-post:
Instead of focusing on just fixing the SDKs and runners Reshuffle, this
could be an opportunity to introduce Redistribute which was proposed in the
long-ago thread. The semantics are identical but it is more clear that
On Wed, Oct 4, 2023 at 7:45 PM Robert Burke wrote:
> LGTM.
>
> It looks the Go SDK already adheres to these semantics as well for the
> reference impl(well, reshuffle/redistribute_randomly, _by_key isn't
> implemented in the Go SDK, and only uses the existing unqualified reshuffle
> URN [0].
>
>
Congrats!
On Thu, Oct 5, 2023 at 4:07 AM Ismaël Mejía wrote:
> Congratulations Robert, well deserved ! long live go !
>
> On Wed, Oct 4, 2023 at 11:58 PM Chamikara Jayalath
> wrote:
>
>> Congrats Rebo!
>>
>> On Wed, Oct 4, 2023 at 1:42 AM Jan Lukavský wrote:
>>
>>> Congrats Robert!
>>> On
Makes sense that the requirement is the same, but is the label
auto-generation behavior the same? I modified the BeamJava
wordcount example[1] to do the regex filter twice in a row, and unlike the
BeamPython example I posted before, it just warns instead of throwing an
exception.
Tangentially, is
This is your daily summary of Beam's current high priority issues that may need
attention.
See https://beam.apache.org/contribute/issue-priorities for the meaning and
expectations around issue priorities.
Unassigned P1 Issues:
https://github.com/apache/beam/issues/28811 [Failing Test]:
Congratulations Alex, well deserved!
On Wed, Oct 4, 2023 at 11:59 PM Chamikara Jayalath
wrote:
> Congrats Alex!
>
> On Wed, Oct 4, 2023 at 1:43 AM Jan Lukavský wrote:
>
>> Congrats Alex!
>> On 10/4/23 10:29, Alexey Romanenko wrote:
>>
>> Congrats Alex, very well deserved!
>>
>> —
>> Alexey
>>
Congratulations Robert, well deserved ! long live go !
On Wed, Oct 4, 2023 at 11:58 PM Chamikara Jayalath
wrote:
> Congrats Rebo!
>
> On Wed, Oct 4, 2023 at 1:42 AM Jan Lukavský wrote:
>
>> Congrats Robert!
>> On 10/4/23 10:29, Alexey Romanenko wrote:
>>
>> Congrats Robert, very well deserved!
Congratulations Valentyn, well deserved !
On Wed, Oct 4, 2023 at 11:58 PM Chamikara Jayalath
wrote:
> Congrats Valentyn!
>
> On Wed, Oct 4, 2023 at 1:42 AM Jan Lukavský wrote:
>
>> Congrats Valentyn!
>> On 10/4/23 10:26, Alexey Romanenko wrote:
>>
>> Congrats Valentyn, very well deserved!
>>
+1 (binding)
Tested Java SDK with Flink Runner on own test-cases.
Jan
On 10/4/23 21:10, Bruno Volpato via dev wrote:
+1 (non-binding).
Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates
(Java SDK 11, Dataflow Runner using both legacy and v2).
Thanks Kenn!
On Wed, Oct
18 matches
Mail list logo
