[
https://issues.apache.org/jira/browse/BEAM-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599278#comment-16599278
]
Sean Patrick Hagen commented on BEAM-5253:
------------------------------------------
Here's the code:
{code:java}
package main
// imports and some consts
var data = []string{
"one",
"two",
"three",
}
func main() {
flag.Parse()
beam.Init()
ctx := context.Background()
log.Infof(ctx, "Publishing %v messages to: %v", len(data), topic)
defer pubsubx.CleanupTopic(ctx, project, topic)
sub, err := pubsubx.Publish(ctx, project, topic, data...)
if err != nil {
log.Fatal(ctx, err)
}
log.Infof(ctx, "Running streaming wordcap with subscription: %v", sub.ID())
p := beam.NewPipeline()
s := p.Root()
opts := &pubsubio.ReadOptions{
Subscription: sub.ID(),
}
log.Info(ctx, "creating pipeline")
col := pubsubio.Read(s, project, topic, opts)
str := beam.ParDo(s, stringx.FromBytes, col)
cap := beam.ParDo(s, strings.ToUpper, str)
textio.Write(s, output, cap)
if err := beamx.Run(context.Background(), p); err != nil {
spew.Dump(err)
log.Exitf(ctx, "failed to execute job: %v", err)
}
}
{code}
Ran it with this target from my Makefile:
{code:java}
events: events.go
go build events.go
run: events
GOOGLE_APPLICATION_CREDENTIALS=./account.json ./events \
--runner dataflow \
--project biba-backbot \
--temp_location gs://events/tmp \
--staging_location gs://event-stage/tmp \
--worker_harness_container_image=apache-docker-beam-snapshots-docker.bintry.io/beam/go:20180515
{code}
Job ID: 2018-08-31_13_56_32-11609701038568083994
> Go SDK PubSub example currently broken
> --------------------------------------
>
> Key: BEAM-5253
> URL: https://issues.apache.org/jira/browse/BEAM-5253
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Sean Patrick Hagen
> Assignee: Henning Rohde
> Priority: Critical
>
> The Go SDK contains an example for creating a streaming pipeline that reads
> from pubsub and outputs the messages. It can be found here:
> [https://github.com/apache/beam/blob/master/sdks/go/examples/streaming_wordcap/wordcap.go]
>
> This example is broken and does not work. It fails with the error "failed to
> execute job: translation failed: no root units" when I try to run it with the
> direct runner, and it just fails with "Internal Issue (8ed815a0a259018f):
> 65177287:8503 " when run in Google Dataflow.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
