[
https://issues.apache.org/jira/browse/BEAM-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Burke updated BEAM-5350:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Open)
Looks like this works as of the last pass I made to the examples. The
autocomplete example is still a stub though.
> Running autocomplete.go on dataflow fails
> -----------------------------------------
>
> Key: BEAM-5350
> URL: https://issues.apache.org/jira/browse/BEAM-5350
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Affects Versions: Not applicable
> Reporter: Tomas Roos
> Priority: P3
>
> I'm in the process as a external developer make sure that all examples are
> runnable on both direct and the dataflow runner as its crucial for people
> onboarding this project.
>
> I've visted the projects before and some are runnable, some probably where
> previously, and some are def not runnable.
>
> So I started top down today, in order to make autocomplete.go run on dataflow
> as well as the direct runner i changed the input in order to make it platform
> independent instead of pointing to a local file.The reading of the source
> from the public cloud storage went fine but it fails to run the top.Largest
> anonymous less function (ran on id: go-job-1-1536575613531078735) failed with
>
>
> {code:java}
> RESP: instruction_id: "-205" error: "Invalid bundle desc: decode: bad userfn:
> bad struct encoding: failed to decode data: decode: failed to find symbol
> main.main.func1: main.main.func1 not found. Use runtime.RegisterFunction in
> unit tests" register: < >
>
> {code}
>
> [https://github.com/apache/beam/blob/master/sdks/go/examples/complete/autocomplete/autocomplete.go#L63]
>
> So in order to fix this I introduced the local func called lessFn and
> registered in the init process. This though now instead when running
>
> {code:java}
>
> go run autocomplete.go --project fair-app-213019 --runner dataflow
> --staging_location=gs://fair-app-213019/staging-test2
> --worker_harness_container_image=apache-docker-beam-snapshots-docker.bintray.io/beam/go:20180515
> {code}
>
>
> fails with
>
> {code:java}
> 2018/09/10 13:37:10 Running autocomplete
> Unable to encode combiner for lifting: failed to encode custom coder: bad
> underlying type: bad field type: bad element: unencodable type: interface
> {}2018/09/10 13:37:10 Using running binary as worker binary:
> '/tmp/go-build157286122/b001/exe/autocomplete'
> 2018/09/10 13:37:10 Staging worker binary:
> /tmp/go-build157286122/b001/exe/autocomplete{code}
>
> And I know this is when invoking the top.Largest since I've removed the piece
> of code and then the job runs fine, could you please point me in the right
> direction why my local func is not encoable as a interface {} and I will of
> course happily send a PR when this is working on direct and the dataflow
> direct so I can move on to the other examples
>
> (All changes can be seen here)
> [https://github.com/apache/beam/compare/master...ptomasroos:make-autocomplete-dataflowable?expand=1]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
