Probably either on Flink, or the Python Portable runner at this juncture. On Tue, Dec 27, 2022, 8:40 PM Byron Ellis via dev <[email protected]> wrote:
> Hi all, > > I spent some more time adding things to my dbt-for-Beam clone ( > https://github.com/apache/beam/pull/24670) and actually made a fair > amount of progress, including starting to add in the profile support so I > can start to run it against real workloads (though at the moment only the > "test" connector is properly configured). More interestingly, though, is > adding in support for Python Dataframe external transforms... which expands > properly, but then (unsurprisingly) hangs if you try to actually run the > pipeline with Java's TestPipeline. > > I was wondering how people go about testing Java/Python hybrid pipelines > locally? The Java<->Python tests don't seem to actually execute a pipeline, > but I was hoping that maybe the direct runner could be set up properly to > do that? > > Best, > B >
