lostluck commented on PR #25558: URL: https://github.com/apache/beam/pull/25558#issuecomment-1436198176
R: @johannaojeling This is where things get interesting! This converts/prepares the Pipeline Proto representations to fully expanded alternatives and similar. You'll probably find the comments on how SDFs and Combine composites are expanded interesting. GroupByKeys and Flattens are pretty straight foward so far and do the appropriate in memory work. That would change depending on how data services evolve (eg. Stop being strictly in memory). But as it ism this is an OK abstraction to start. The code living in an internal package gives us the freedom to evolve the abstractions we're using as needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
