Re: PubSub MultiReader

2020-09-09 Thread Iñigo San Jose
I made the modification to fuse topics and subscriptions, I now think it may be cleaner. Attached a file with both versions. Let me know what you think :D Thanks On Wed, Sep 9, 2020 at 5:35 PM Iñigo San Jose wrote: > > Hi everyone! > > I have seen a very common use case i

PubSub MultiReader

2020-09-09 Thread Iñigo San Jose
Hi everyone! I have seen a very common use case in Beam which is pipelines that read from multiple PubSub topics or multiple subscriptions to end up flattening them. In general, this makes the pipeline harder to understand since not much context can be taken from it. I was thinking of adding a

[PROPOSAL] Mean Fn without_defaults()

2020-06-04 Thread Iñigo San Jose
rms/combiners.html#Mean> -- *•** Iñigo San Jose | josein...@google.coom* *•** Big Data *Technical Solutions Engineer T2 *• *Google Cloud Platform • *Google, Dublin* import logging import random import time import datetime import apache_beam as beam from apache_beam.io import ReadFromText from