lostluck commented on issue #29209: URL: https://github.com/apache/beam/issues/29209#issuecomment-1793023883
First comment: This might be an issue with Beam Go on Windows (without using WSL). To my knowledge, we just haven't done any beam Go development on windows, and thus haven't implemented support for handling it's specific behaviors. Basically, without registration, we do a magic lookup in the binary's DWARF symbol table for the function name. From your output, we don't appear to handle looking for the EXE at all. That would happen here: https://github.com/apache/beam/blob/03c811e296e96ae682f53b877e9bc4c2820b36e7/sdks/go/pkg/beam/core/runtime/symbols.go#L56 ----- Second: Essentially due to the current structure of the SDK, we have beam.Init() called as early as possible in the binary. The point of this is to reduce the set of code that is executed on workers, that is only used as part of pipeline construction. So any registrations and such must happen before that point. The simplest thing is we recommend it happens at package init time. One can have automatic construction time registration which largely means that the pipeline graph is constructed (and all that code is executed) on every worker, *or* one can allow for arbitrary construction time work. Trying to satisfy both becomes awkward because then there ends up being two different ways of doing things, which makes the SDK (even) harder to learn. That said, I'm skeptical that the concept of `beam.Init` was the right approach generally for Beam Go, especially with the larger focus on Declaritive beam via YAML and similar. It wouldn't be my first choice for a next version of Go SDK. (And indeed it's not, I have an increasingly complete research prototype Go SDK that's trying out the "don't register, just rebuild" approach... It's not yet ready for sharing though.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
