That was it Juan Carlos. Can't thank you enough. I now have generic Kettle transformations running on the Direct Runner, Dataflow and Spark.
Cheers, Matt --- Matt Casters <m <[email protected]>[email protected]> Senior Solution Architect, Kettle Project Founder Op di 29 jan. 2019 om 18:19 schreef Matt Casters <[email protected]>: > No you're right. I got so focused on getting > org.apache.hadoop.fs.FileSystem in order that I forgot about the other > files. Doh! > > Thanks for the tip! > --- > Matt Casters <m <[email protected]>[email protected]> > Senior Solution Architect, Kettle Project Founder > > > Op di 29 jan. 2019 om 16:41 schreef Juan Carlos Garcia < > [email protected]>: > >> Hi Matt, >> >> Are the META-INF/services files merged correctly on the fat-jar? >> >> >> >> On Tue, Jan 29, 2019 at 2:33 PM Matt Casters <[email protected]> >> wrote: >> >>> Hi Beam! >>> >>> After I have my pipeline created and the execution started on the Spark >>> master I get this strange nugget: >>> >>> java.lang.IllegalStateException: No TransformEvaluator registered for >>> BOUNDED transform Read(CompressedSource) >>> >>> I'm reading from a HDFS file (hdfs:///input/customers-noheader-1M.txt) >>> It's the exact same file and pipeline I've been testing on local files >>> in the direct runner and on Google Storage in DataFlow without a problem. >>> What could I have missed in the Spark case? >>> >>> Thanks again for any suggestions! >>> >>> Matt >>> --- >>> Matt Casters <m <[email protected]>[email protected]> >>> Senior Solution Architect, Kettle Project Founder >>> >>> >> >> -- >> >> JC >> >>
