chamikaramj commented on code in PR #36515:
URL: https://github.com/apache/beam/pull/36515#discussion_r2430480569
##########
sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceOptions.java:
##########
@@ -65,6 +65,13 @@ public interface ExpansionServiceOptions extends
PipelineOptions {
void setUseAltsServer(boolean useAltsServer);
+ @Description(
Review Comment:
Persistent. This will allow service to use the default expansion even for
managed which will allow better container merging when expanding multiple
managed transforms.
##########
sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/TransformProvider.java:
##########
@@ -127,7 +134,30 @@ default String getTransformUniqueID(RunnerApi.FunctionSpec
spec) {
ExternalTransforms.SchemaTransformPayload payload;
try {
payload =
ExternalTransforms.SchemaTransformPayload.parseFrom(spec.getPayload());
- return payload.getIdentifier();
+ if
(PTransformTranslation.MANAGED_TRANSFORM_URN.equals(payload.getIdentifier())) {
+ try {
+ // ManagedSchemaTransform includes a schema field
transform_identifier that includes the
+ // underlying schema
+ // transform ID so we special case that here.
+ Row configRow =
+
RowCoder.of(SchemaTranslation.schemaFromProto(payload.getConfigurationSchema()))
+ .decode(new
ByteArrayInputStream(payload.getConfigurationRow().toByteArray()));
Review Comment:
All fields used here are required (should be specified all cases) I believe.
Do you see specific calls that we should guard against ? I explicitly fail if
the "transform_identifier" field is not found which also has to be present
based on the schema (so we guard against changes to that schema).
https://github.com/apache/beam/blob/7a9a4e6afa52c2093fa9ce253090703e592af2c0/sdks/java/managed/src/main/java/org/apache/beam/sdk/managed/ManagedSchemaTransformProvider.java#L90
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
