[
https://issues.apache.org/jira/browse/BEAM-5421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619423#comment-16619423
]
Robert Burke commented on BEAM-5421:
------------------------------------
OK, this seems like a manifestation of treating strings as bytes, which is
filed as BEAM-3580
Unfortunately this doesn't have an easy fix at present. I'm working on a
slightly more general code generator, which could solve this for us, but it's
not quite there yet.
There's an alternative solution that involves fixing how the beam reflectx
package handles value conversion.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/util/reflectx/call.go#L92
Essentially, since the SDK currently treats strings as bytes since per the
language, they're the same. But when it comes to typing, they're still
different types and require a conversion.
Afterall,
myBytes := []uint8("These are my bytes")
text := string(myBytes)
fmt.Println(text)
works just fine, but we have the explicit conversion there, and that's what's
missing.
So the ugly short term work around for this string/bytes problem is to accept
[]bytes instead, and convert that to string before checking the length (as
len([]bytes)!= len(string)), depending on the runes in the string)
A simple more SDK centric approach would be to change the reflectx Func
approach to be a bit more aware of strings/vs bytes (but this would probably be
a huge runtime penalty without caching this awareness in advance).
A medium term fix is to wait for the code generator I'm working on, which
negates the issue by also avoiding reflection. (See all the shims in the
[Optimized|https://github.com/apache/beam/tree/master/sdks/go/pkg/beam/core/runtime/exec/optimized]
package, which unfortunately doesn't currently include predicates. If they did
that would be a solution as well).
The best approach would be to resolve that coder issue so that the reflect
approach works without extra issues.
> filter.Exclude is panicking
> ---------------------------
>
> Key: BEAM-5421
> URL: https://issues.apache.org/jira/browse/BEAM-5421
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Affects Versions: 2.6.0
> Environment: Mac, Linux
> Reporter: Vadym
> Assignee: Robert Burke
> Priority: Major
>
> {code:java}
> package main
> import (
> "context"
> "github.com/apache/beam/sdks/go/pkg/beam"
> "github.com/apache/beam/sdks/go/pkg/beam/io/textio"
> "github.com/apache/beam/sdks/go/pkg/beam/log"
> "github.com/apache/beam/sdks/go/pkg/beam/transforms/filter"
> "github.com/apache/beam/sdks/go/pkg/beam/x/beamx"
> )
> func islongWord(s string) bool {
> return len(s) < 3
> }
> func main() {
> beam.Init()
> ctx := context.Background()
> p, s := beam.NewPipelineWithRoot()
> words := beam.Create(s, "a", "b", "long", "alsolong")
> long := filter.Exclude(s, words, islongWord)
> textio.Write(s, "long.txt", long)
> if err := beamx.Run(ctx, p); err != nil {
> log.Exitf(ctx, "Failed to execute job: %v", err)
> }
> }
> {code}
> The code above fails with the error message:
> {{2018/09/18 15:47:23 Failed to execute job: panic: reflect: Call using
> []uint8 as type string goroutine 1 [running]:}}
>
> The code was taken from Exclude example in the documentation:
> https://godoc.org/github.com/apache/beam/sdks/go/pkg/beam/transforms/filter#Exclude
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
