[
https://issues.apache.org/jira/browse/BEAM-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347840#comment-17347840
]
James Matthews commented on BEAM-12158:
---------------------------------------
Nope, sorry, still not fixed. I did at least get an error that's slightly more
explicit this time, though:
{code:java}
panic: proto: file "v1.proto" is already registered
previously from:
"github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/v1"
currently from: "github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1"
{code}
I think what's happening is [https://github.com/golang/protobuf/issues/1122
|https://github.com/golang/protobuf/issues/1122]- which seems like a really
egregious bug in how protobuf is handling registration. Here, I suspect (but
have not verified) it's being tripped by something related to the gen.go files
running protoc on each file individually.
If I run the wordcount binary with GOLANG_PROTOBUF_REGISTRATION_CONFLICT=warn
(workaround from that issue), it seems to work fine.
Steps to reproduce (from an empty $GOPATH)
* mkdir -p $GOPATH/src/github.com/apache
* cd $GOPATH/src/github.com/apache
* git clone [https://github.com/apache/beam]
* GO111MODULE='off' go get ./beam/sdks/go/...
* GO111MODULE='off' go build ./beam/sdks/go/examples/wordcount
* ./wordcount
> "v1.proto" is already registered error for golang dataflow
> ----------------------------------------------------------
>
> Key: BEAM-12158
> URL: https://issues.apache.org/jira/browse/BEAM-12158
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Kazuki Nishiura
> Assignee: Robert Burke
> Priority: P2
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> I tried to run wordcount example in go
> [https://beam.apache.org/get-started/quickstart-go/]
> It worked fine in dev but when I try to run it on GCP dataflow, I got
> '"v1.proto" is already registered' error and job crashes.
>
> It seems proto definition here is using very generic package name "v1" which
> I suspect is a cause of issue
> https://github.com/apache/beam/blob/v2.28.0/sdks/go/pkg/beam/io/pubsubio/v1/v1.proto
>
> Here is log in GCP
> {code:java}
> "panic: proto: file "v1.proto" is already registered "
> "See
> https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
> "
> "goroutine 1 [running]: "
> "google.golang.org/protobuf/reflect/protoregistry.glob..func1(0x1683000,
> 0xc0005201c0, 0x164a080, 0xc0004657e0, 0xc0005201c0) "
> "
> /Users/kazuki/go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:54
> +0x24c "
> "google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc000068350,
> 0x1688e40, 0xc0005201c0, 0x0, 0x0) "
> "
> /Users/kazuki/go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:127
> +0xb6c "
> "google.golang.org/protobuf/internal/filedesc.Builder.Build(0x0, 0x0,
> 0xc000361e00, 0x12a, 0x200, 0x100000001, 0x0, 0x1654b00, 0xc00003c330,
> 0x1665e40, ...) "
> "
> /Users/kazuki/go/pkg/mod/google.golang.org/[email protected]/internal/filedesc/build.go:113
> +0x195 "
> "github.com/golang/protobuf/proto.RegisterFile(0x148e167, 0x8, 0x1fdc7a0,
> 0xe2, 0xe2) "
> "
> /Users/kazuki/go/pkg/mod/github.com/golang/[email protected]/proto/registry.go:48
> +0x13b "
> "github.com/apache/beam/sdks/go/pkg/beam/io/pubsubio/v1.init.1() "
> "
> /Users/kazuki/go/pkg/mod/github.com/apache/[email protected]+incompatible/sdks/go/pkg/beam/io/pubsubio/v1/v1.pb.go:115
> +0x5a "
> "2021/04/12 19:32:22 User program exited: exit status 2 "
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
