Re: Streaming data from Pubsub to Spanner with Beam dataflow pipeline

[Kenneth Knowles]

Moving to u...@beam.apache.org, the best mailing list for questions like
this.

Yes, this kind of workload is a core use case for Beam. If you have a
problem, please write to this user list with details.

Kenn

On Wed, Oct 30, 2019 at 4:07 AM Taher Koitawala  wrote:

> Hi All,
>   My current use-case is to write data from Pubsub to Spanner
> using a streaming pipeline. I do see that Beam does have a SpannerIO to
> write.
>
>   However, pubsub being streaming and Spanner being RDBMS like, it
> would be helpful to you guys can tell me if this will be performant enough
> or not. If someone has already tried this out and can give me a few
> caveats, then that would be really awesome.
>
>
> Regards,
> Taher Koitawala
>

Streaming data from Pubsub to Spanner with Beam dataflow pipeline

[Taher Koitawala]

Hi All,
  My current use-case is to write data from Pubsub to Spanner using
a streaming pipeline. I do see that Beam does have a SpannerIO to write.

  However, pubsub being streaming and Spanner being RDBMS like, it
would be helpful to you guys can tell me if this will be performant enough
or not. If someone has already tried this out and can give me a few
caveats, then that would be really awesome.


Regards,
Taher Koitawala