Hello Beam,
I've been taking great steps forward in having Kettle generate Beam
pipelines and they actually execute just find in unit testing in IntelliJ.
The problem starts when I collect all the libraries needed for Beam and the
Runners and throw them into the Kettle project as a plugin.
Caused by: java.lang.IllegalArgumentException: No filesystem found for
scheme gs
at
org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(FileSystems.java:456)
at org.apache.beam.sdk.io.FileSystems.matchNewResource(FileSystems.java:526)
at
org.apache.beam.sdk.io.FileBasedSink.convertToFileResourceIfPossible(FileBasedSink.java:213)
at org.apache.beam.sdk.io.TextIO$TypedWrite.to(TextIO.java:700)
at org.apache.beam.sdk.io.TextIO$Write.to(TextIO.java:1028)
at
org.kettle.beam.core.transform.BeamOutputTransform.expand(BeamOutputTransform.java:87)
... 32 more
This also happens for local file execution ("scheme file" in that case).
So the questions are: how is Beam bootstrapped? How does Beam determine
which libraries to use and what is the recommended way for packaging things
up properly?
The Beam plugin is running in a separate URLClassloader so I think
something is going awry there.
Thanks a lot for any answers or tips you might have!
Matt
---
Matt Casters <m <[email protected]>[email protected]>
Senior Solution Architect, Kettle Project Founder
