Choosing a coder for a class that contains a Row?

2019-07-22 Thread Pablo Estrada
Hello all, I am writing a utility to push data to PubSub. My data class looks something like so: == class MyData { String someId; Row someRow; Row someOtherRow; } == The schema for the Rows is not known a-priori. It is contained by the Row. I am then pushing this data to p

Re: Portability framework: multiple environments in one pipeline

2019-07-22 Thread Robert Bradshaw
Yes, for sure. Support for this is available in some runners (like the Python Universal Local Runner and Flink) and actively being added to others (e.g. Dataflow). There are still some rough edges however--one currently must run an expansion service to define a pipeline step in an alternative envir