[
https://issues.apache.org/jira/browse/BEAM-14486?focusedWorklogId=772808&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-772808
]
ASF GitHub Bot logged work on BEAM-14486:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/May/22 12:32
Start Date: 20/May/22 12:32
Worklog Time Spent: 10m
Work Description: damccorm commented on code in PR #17709:
URL: https://github.com/apache/beam/pull/17709#discussion_r878092046
##########
sdks/go/pkg/beam/io/pubsubio/pubsubio.go:
##########
@@ -73,15 +80,38 @@ func Read(s beam.Scope, project, topic string, opts
*ReadOptions) beam.PCollecti
return out[0]
}
-func unmarshalMessageFn(raw []byte) (*pb.PubsubMessage, error) {
+func unmarshalMessageFn(raw []byte, emit func(*pb.PubsubMessage)) error {
Review Comment:
This emitter should be registered
##########
sdks/go/pkg/beam/io/pubsubio/pubsubio.go:
##########
@@ -73,15 +80,38 @@ func Read(s beam.Scope, project, topic string, opts
*ReadOptions) beam.PCollecti
return out[0]
}
-func unmarshalMessageFn(raw []byte) (*pb.PubsubMessage, error) {
+func unmarshalMessageFn(raw []byte, emit func(*pb.PubsubMessage)) error {
var msg pb.PubsubMessage
if err := proto.Unmarshal(raw, &msg); err != nil {
- return nil, err
+ return err
}
- return &msg, nil
+ emit(&msg)
+ return nil
+}
+
+func wrapInMessage(raw []byte, emit func(*pb.PubsubMessage)) {
+ emit(&pb.PubsubMessage{
+ Data: raw,
+ })
}
-// Write writes PubSubMessages or bytes to the given pubsub topic.
+func marshalMessageFn(in *pb.PubsubMessage, emit func([]byte)) error {
Review Comment:
Same comment, this emitter should be registered
Issue Time Tracking
-------------------
Worklog Id: (was: 772808)
Time Spent: 1h (was: 50m)
> Better document Pubsubio & match Python implementation.
> -------------------------------------------------------
>
> Key: BEAM-14486
> URL: https://issues.apache.org/jira/browse/BEAM-14486
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: P2
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The pubsub IO has long been marked experimental for the Go SDK.
> It also has implementation deviations from the Python SDK, in that writes are
> not always sending encoded PubsubMessages to the sink.
> These deviations should be corrected, and the usage made clearer.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
