Re: Using the Go Beam SDK

[Henning Rohde]

Hi Philip,

 Thanks for expressing interest in the Go SDK! The documentation is indeed
still incomplete (BEAM-3826) and the main design document is probably be
the best starting point right now:

 https://s.apache.org/beam-go-sdk-design-rfc

It also contains links to some of the better documented examples in the
repository.

We do plan to prototype Go streaming w/ PubSub on Dataflow in the near
future (BEAM-3854, BEAM-3856). Is that the setup you're mainly looking at
as well? For Datastore reading, Braden even has a PR in flight. So things
are moving along in that direction and there are definitely lots of
opportunities to contribute. The long-term IO story will likely be based on
some combination of Go SplittableDoFns and cross-language IO.

Please feel free to ask questions, open JIRAs or send PRs. Love to hear
your feedback!

Henning


On Thu, Mar 15, 2018 at 7:21 PM Philip Gianfortoni <8...@tokentransit.com>
wrote:

> Hi dev team,
>
> I am an engineer at Token Transit, a company working on a mobile ticketing
> solution for transit companies. Through Holden (who's working on Py3 w/Beam
> and doing some dev advocacy), we heard about the experimental newly merged
> WIP Go Beam SDK. We are extremely excited about exploring and contributing
> to this new offering from Beam, as almost our entire codebase is written in
> Go, and we are looking for a bulk data processing solution that is
> compatible.
>
> Are there any code pointers, examples, or other docs that someone could
> point me to about this new SDK? Holden pointed us to this repository
> , but it does seem
> that documentation is a bit light at the moment. Any extra examples or docs
> would be greatly appreciated.
>
> Another thing that we were wondering was whether and/or when a PubSub or
> Datastore API will be supported. We are looking at using PubSub to process
> our data incrementally, and in the long term, support for this feature is a
> requirement.
>
> Looking forward to trying out this SDK. Thank you so much!
>
> 8
>

Using the Go Beam SDK

[Philip Gianfortoni]

Hi dev team,

I am an engineer at Token Transit, a company working on a mobile ticketing
solution for transit companies. Through Holden (who's working on Py3 w/Beam
and doing some dev advocacy), we heard about the experimental newly merged
WIP Go Beam SDK. We are extremely excited about exploring and contributing
to this new offering from Beam, as almost our entire codebase is written in
Go, and we are looking for a bulk data processing solution that is
compatible.

Are there any code pointers, examples, or other docs that someone could
point me to about this new SDK? Holden pointed us to this repository
, but it does seem that
documentation is a bit light at the moment. Any extra examples or docs
would be greatly appreciated.

Another thing that we were wondering was whether and/or when a PubSub or
Datastore API will be supported. We are looking at using PubSub to process
our data incrementally, and in the long term, support for this feature is a
requirement.

Looking forward to trying out this SDK. Thank you so much!

8