Re: [VOTE] Beam Mascot animal choice: vote for as many as you want

[Augustin Lafanechere]

[ ] Beaver
[ ] Hedgehog
[ ] Lemur
[ ] Owl
[x] Salmon
[ ] Trout
[ ] Robot dinosaur
[ ] Firefly
[ ] Cuttlefish
[ ] Dumbo Octopus
[x ] Angler fish

> Le 20 nov. 2019 à 13:38, Maximilian Michels  a écrit :
> 
> [ ] Beaver
> [ ] Hedgehog
> [x] Lemur
> [ ] Owl
> [ ] Salmon
> [ ] Trout
> [ ] Robot dinosaur
> [x] Firefly
> [x] Cuttlefish
> [ ] Dumbo Octopus
> [x] Angler fish
> 
> On 20.11.19 08:18, Alex Van Boxel wrote:
>> [ ] Beaver
>> [ ] Hedgehog
>> [ ] Lemur
>> [ ] Owl
>> [ ] Salmon
>> [ ] Trout
>> [ ] Robot dinosaur
>> [ X] Firefly
>> [ ] Cuttlefish
>> [ ] Dumbo Octopus
>> [ X] Angler fish
>>  _/
>> _/ Alex Van Boxel
>> On Wed, Nov 20, 2019 at 3:57 AM Reza Rokni > > wrote:
>>[ ] Beaver
>>[ ] Hedgehog
>>[ ] Lemur
>>[ ] Owl
>>[X] Salmon
>>[ ] Trout
>>[ ] Robot dinosaur
>>[ ] Firefly
>>[ ] Cuttlefish
>>[X] Dumbo Octopus
>>[X] Angler fish
>>On Wed, 20 Nov 2019 at 10:43, Kenneth Knowles >> wrote:
>>Please cast your votes of approval [1] for animals you would
>>support as Beam mascot. The animal with the most approval will
>>be identified as the favorite.
>>*** Vote for as many as you like, using this checklist as a
>>template 
>>[ ] Beaver
>>[ ] Hedgehog
>>[ ] Lemur
>>[ ] Owl
>>[ ] Salmon
>>[ ] Trout
>>[ ] Robot dinosaur
>>[ ] Firefly
>>[ ] Cuttlefish
>>[ ] Dumbo Octopus
>>[ ] Angler fish
>>This vote will remain open for at least 72 hours.
>>Kenn
>>[1] See
>>https://en.wikipedia.org/wiki/Approval_voting#Description and
>>https://www.electionscience.org/library/approval-voting/
>>-- This email may be confidential and privileged. If you received this
>>communication by mistake, please don't forward it to anyone else,
>>please erase all copies and attachments, and please let me know that
>>it has gone to the wrong person.
>>The above terms reflect a potential business arrangement, are
>>provided solely as a basis for further discussion, and are not
>>intended to be and do not constitute a legally binding obligation.
>>No legally binding obligations will be created, implied, or inferred
>>until an agreement in final form is executed in writing by all
>>parties involved.

Stateful processing : @OnWindowExpiration DoFn annotation

[Augustin Lafanechere]

Hello dear Beam community,
I would like to write to you for a question about OnWindowExpiration annotation 
on DoFn.
Does anyone of you have a working snippet with this ?

I try to write a DoFn with a Batch RPC on window closure. It is a BigQuery call 
for a historical metric value updated by an external process. I want to execute 
this query and sum the results with my events buffered in a state. The 
OnWindowExpiration looks very practical to accomplish this.

It looks like the function annotated with @OnWindowExpiration is never call. My 
pipeline runs on Dataflow, perhaps its not a supported feature on this runner…

Here is a snippet of what I try to accomplish. It seems like the annotated 
functions is never called, the log line is never appearing. Am I missing 
something ?
I tried to replicate the logic found in this blog post 
 and pieces of 
information found in this PR. 


// The window definition used in the pipeline sets in a higher transform
// Window> w =
// Window.into(FixedWindows.of(Duration.standardMinutes(1L)))
// .withAllowedLateness(Duration.ZERO)
// .discardingFiredPanes();

public final class Enrich extends DoFn, KV> {

  @StateId("buffer")
  private final StateSpec>> bufferedEvents = 
StateSpecs.bag();

  @ProcessElement
  public void process(
  final ProcessContext context,
  final @StateId("buffer") BagState> bufferState) {
bufferState.add(context.element());
context.output(context.element());
  }

  @OnWindowExpiration
  public void onWindowExpiration(
  final @StateId("buffer") BagState> bufferState,
  final OutputReceiver> outputReceiver) {
LOG.info("The window expired");
for (KV enrichedEvent : 
enrichWithBigQuery(bufferState.read())) {
  outputReceiver.output(enrichedEvent);
}
  }
}


Thanks for your help,


Augustin



-- 
Chauffeur Privé devient kapten_ Plus d'informations ici