Re: Thoughts on coder evolution

2023-05-04 Thread Jan Lukavský
On 5/3/23 19:57, Kenneth Knowles wrote: My big picture hot take: this is useful, but the problem we really need to solve is topology change, which will obsolete coder evolution. I think Beam model has a role in this. It isn't just a runner-specific thing. We need to ensure the model makes it p

Re: Thoughts on coder evolution

2023-05-03 Thread Kenneth Knowles
My big picture hot take: this is useful, but the problem we really need to solve is topology change, which will obsolete coder evolution. I think Beam model has a role in this. It isn't just a runner-specific thing. We need to ensure the model makes it possible/easy to replace one pipeline with an

Re: Thoughts on coder evolution

2023-05-03 Thread Byron Ellis via dev
I think I'm not understanding the use case here? Are we talking about encoding of data in motion (e.g. between stages of a streaming pipeline) or data at rest? (e.g. input formats and output formats) Or maybe something else? On Wed, May 3, 2023 at 6:58 AM Jan Lukavský wrote: > Hi, > > I'd like t