Yes, since you don't care about the grouping, you could use any random key
that you want.
On Fri, Mar 27, 2020 at 2:45 PM Andrew Wylie wrote:
> To be able to invoke GroupByKey, does that mean I need to create a KV
> object for my Pub Sub messages that are received from the topic?
>
> On 27 Mar 2
To be able to invoke GroupByKey, does that mean I need to create a KV object
for my Pub Sub messages that are received from the topic?
> On 27 Mar 2020, at 18:30, Luke Cwik wrote:
>
>
> The trigger only applies to when the output of a GroupByKey is produced and
> won't put in the delay witho
Nice : D
On Fri, Mar 27, 2020 at 12:44 AM Alex Van Boxel wrote:
> That's nicely done! Congrats, going to share this immediately.
>
> And I actually didn't know where the name Beam came from, now I know :-)
>
> _/
> _/ Alex Van Boxel
>
>
> On Fri, Mar 27, 2020 at 4:32 AM Henry Suryawirawan <
> h
Hi,
Just wanted to follow up on an earlier e-mail on Dataflow failing pipeline
validation when using a splittable DoFn (both unbounded and bounded). The
triggering condition seems to be an upstream DoFn with a side input from
FileIO.match -> readMatches.
The same pipeline(s) run fin on the direct
So my 'input topics' are a real time feed and I also want to provide a delayed
feed of the same information, so my 'output topics' are for my delayed feed. I
was hoping to use Beam to do this in Dataflow.
> On 27 Mar 2020, at 18:30, Luke Cwik wrote:
>
> GroupByKey
The trigger only applies to when the output of a GroupByKey is produced and
won't put in the delay without one so as long as inputEvents is followed by
a GroupByKey you'll see the delay.
Why do you want the delay as there might be different solutions for the
problem you're trying to solve?
On Fri
Thanks Luke. I wasn’t sure if that was advised or even possible, but it seems
to be working well.
I would like to introduce a 5 minute delay between reading each message and
publishing it to the output topic. Is the correct way to do this in Apache
Beam, using triggers?
I am trying the approac
That's nicely done! Congrats, going to share this immediately.
And I actually didn't know where the name Beam came from, now I know :-)
_/
_/ Alex Van Boxel
On Fri, Mar 27, 2020 at 4:32 AM Henry Suryawirawan
wrote:
> Hello,
>
> Just would like to share that recently the Apache Beam Katas is
Thanks for sharing.
The info is very informative and entertaining.
Nicely done 👏
On Thu, Mar 26, 2020 at 8:32 PM Henry Suryawirawan
wrote:
> Hello,
>
> Just would like to share that recently the Apache Beam Katas is featured
> in the Google Cloud Level Up YouTube video (
> https://www.youtube.co
