kennknowles commented on PR #29924: URL: https://github.com/apache/beam/pull/29924#issuecomment-1942968028
> What did core-construction-java originally do? I see it moved to "org.apache.beam.sdk.util.construction", Is it a utility? Thinking about this module or "org.apache.beam.sdk.construction" looks better. As a similar module "org.apache.beam.sdk.expansion" just has its top level namespace after sdk. @Abacn the `util` folder in the SDK is "not for users" and has its javadoc hidden. It is kind of a hacky way but it has a long history. The history of core construction is a few different things: 1. `runners/core-java` was created for execution helpers for runners, so they could all build their workers. 2. later we created transform overrides that are for runners, but only _before_ job submission and I think that is when `runners/core-construction-java` was created. Specifically, things like `DataflowRunner` were allowed to depend on core-construction-java but they were not allowed to depend on core-java. Only the worker is allowed to depend on core-java. 3. The really big reason was to keep proto and gRPC dependencies out of the core SDK. We wanted the core SDK to be "pure" for each language, and portability would be an optional plugin/add-on. Now we have a totally different philosophy where portability comes first and SDKs come second, and everything depends on proto so much there is no point. I realize now that actually I could have just added `PTransform.toProto` to the Java SDK without doing this merge 😅 and just deleted the translation from core-construction-java. But to summarize my intention with all these changes, it is to make it much much easier to add portability stuff to every transform in the Java SDK, but just implementing `PTransform.getURN()` and `PTransform.toProto()` basically. Get rid of all the `@AutoService` stuff and all the interfaces involved there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
