Hi Tobiasz,
It seems the cross platform pipeline option string handling is only applied
to gradle tasks with "enableJavaPerformanceTesting" applied, and it's
currently not applied to direct runner tests.
I created this PR, do you mind reviewing it?
https://github.com/apache/beam/pull/13449
Thank
Hello all,
Any time I have the misfortune of creating a new beam branch, building a
subtarget (sdks/io/google-cloud-platform/.../pubsublite in my case) takes
O(30 mins) on my laptop. A lot of the steps seem to block on each other and
even the leaf rebuild can take minutes since all the GCP I/O tra
I'm not sure there's a reason to use generics here, since this class will
likely only ever be instantiated once. Have you tried using Object instead
of T?
On Wed, Nov 25, 2020 at 10:08 AM Sonam Ramchand <
sonam.ramch...@venturedive.com> wrote:
> Hi Devs,
> I am trying to implement Array_Agg(
> h
On Wed, Nov 25, 2020 at 10:25 AM Robert Bradshaw
wrote:
> On Wed, Nov 25, 2020 at 10:15 AM Robert Burke wrote:
> >
> > It sounds like we've come to the position that non-correctness affecting
> Ptransform Annotations are valuable at both leaf and composite levels, and
> don't remove the potentia
Created a PR without unit tests at https://github.com/apache/beam/pull/13434.
Please have a look.
Thanks,
Mirac
On 2020/11/25 18:50:19, Robert Burke wrote:
> Hmmm. Fair. I'm mostly concerned about the pathological case where we end
> up with a distinct Environment per transform, but there are
If any of these are suitable for at least some development. I propose we
merge them, and update them with fixes later. Rather than trying to get
things 100% working in the first PR.
Looks like this one was opened in early Sept, and never got merged. Which
is a pretty long time. Perhaps abandoned f
I agree this is a good idea. I remember my first experience with Beam
development - I ran through the steps at [1] and had `./gradlew check`
fail. I don't think I ever got it working before moving on and just running
more specific tasks.
It would be great if we had a reliable way for new contributo
Please follow https://issues.apache.org/jira/browse/BEAM-11360 instead.
Thanks,
Cham
On Mon, Nov 30, 2020 at 10:26 AM Steve Niemitz wrote:
> alright, thank you. Is BEAM-10507 the jira to watch for any progress on
> that?
>
> On Mon, Nov 30, 2020 at 12:55 PM Boyuan Zhang wrote:
>
>> Hi Steve,
alright, thank you. Is BEAM-10507 the jira to watch for any progress on
that?
On Mon, Nov 30, 2020 at 12:55 PM Boyuan Zhang wrote:
> Hi Steve,
>
> Unfortunately I don't think there is a workaround before we have the
> change that Cham mentions.
>
> On Mon, Nov 30, 2020 at 8:16 AM Steve Niemitz
In Splittable DoFn, the trySplit[1] API in RestrictionTracker is for
performing checkpointing, keeping primary as current restriction and
returning residuals. In the DoFn, you can do Splittable DoFn initiated
checkpoint by returning ProcessContinuation.resume()[2]. Beam programming
guide[3] also ta
Hi Steve,
Unfortunately I don't think there is a workaround before we have the change
that Cham mentions.
On Mon, Nov 30, 2020 at 8:16 AM Steve Niemitz wrote:
> I'm trying to write an xlang transform that uses Reshuffle internally, and
> ran into this as well. Is there any workaround to this f
On Wed, Nov 25, 2020 at 11:09 AM Alexey Romanenko
wrote:
> Great job, it should be very helpful for users!
>
> Just a minor note - it would be great to add an example of how to finally
> run a cross-language pipeline with Portable Runner since, iirc, it was
> supposed to pass some additional argu
Thanks Boyuan for the pointers.
If you or anyone else here have any recommendations about the two
approaches, i.e., implementing a connector for Beam using the embedded
version of Debezium or relying on Kafka (even for the single node case),
that would be great too.
Regards
-B
On Wed, Nov 25, 2
I'm trying to write an xlang transform that uses Reshuffle internally, and
ran into this as well. Is there any workaround to this for now (other than
removing the reshuffle), or do I just need to wait for what Chamikara
mentioned? I noticed the same issue was mentioned in the SnowflakeIO.Read
PR
High Priority Dependency Updates Of Beam Python SDK:
Dependency Name
Current Version
Latest Version
Release Date Of the Current Used Version
Release Date Of The Latest Release
JIRA Issue
dill
0.3.1.1
0.3.3
2019-10-07
Hi Haizhou,
I suppose these quotations mark are handled here:
https://github.com/apache/beam/blob/8cdb8075b7d28e84591b13fb01d0144d941a5ef2/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1518
I was struggling with them while doing GA workflows which runs on
linux/mac/wind
16 matches
Mail list logo
