[
https://issues.apache.org/jira/browse/BEAM-12464?focusedWorklogId=637105&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-637105
]
ASF GitHub Bot logged work on BEAM-12464:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Aug/21 20:23
Start Date: 11/Aug/21 20:23
Worklog Time Spent: 10m
Work Description: ibzib commented on a change in pull request #14974:
URL: https://github.com/apache/beam/pull/14974#discussion_r687139294
##########
File path:
sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoSchemaTranslator.java
##########
@@ -172,17 +173,24 @@ static Schema getSchema(Descriptors.Descriptor
descriptor) {
enumIds.putIfAbsent(fieldDescriptor.getName(),
fieldDescriptor.getNumber()) == null);
}
FieldType oneOfType = FieldType.logicalType(OneOfType.create(subFields,
enumIds));
- fields.add(Field.of(oneofDescriptor.getName(), oneOfType));
+ oneOfFieldLocationMap.put(
+ oneofDescriptor.getFields().get(0).getNumber(),
+ Field.of(oneofDescriptor.getName(), oneOfType));
}
for (Descriptors.FieldDescriptor fieldDescriptor : descriptor.getFields())
{
- if (!oneOfFields.contains(fieldDescriptor.getNumber())) {
+ int fieldDescriptorNumber = fieldDescriptor.getNumber();
+ if (!oneOfFields.contains(fieldDescriptorNumber)) {
// Store proto field number in metadata.
FieldType fieldType = beamFieldTypeFromProtoField(fieldDescriptor);
fields.add(
- withFieldNumber(
- Field.of(fieldDescriptor.getName(), fieldType),
fieldDescriptor.getNumber())
+ withFieldNumber(Field.of(fieldDescriptor.getName(), fieldType),
fieldDescriptorNumber)
.withOptions(getFieldOptions(fieldDescriptor)));
+ } else if (oneOfFieldLocationMap.containsKey(fieldDescriptorNumber)) {
+ Field oneOfField = oneOfFieldLocationMap.get(fieldDescriptorNumber);
+ if (oneOfField != null) {
Review comment:
oneOfField should never be null. Is the null checker not able to tell?
##########
File path:
sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoSchemaTranslator.java
##########
@@ -157,6 +157,7 @@ static Schema getSchema(Class<? extends Message> clazz) {
static Schema getSchema(Descriptors.Descriptor descriptor) {
Set<Integer> oneOfFields = Sets.newHashSet();
+ Map<Integer, Field> oneOfFieldLocationMap = Maps.newHashMap();
Review comment:
Nit: don't include the name of the type in the variable name.
```suggestion
Map<Integer, Field> oneOfFieldLocation = Maps.newHashMap();
```
##########
File path:
sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoSchemaTranslator.java
##########
@@ -157,6 +157,7 @@ static Schema getSchema(Class<? extends Message> clazz) {
static Schema getSchema(Descriptors.Descriptor descriptor) {
Set<Integer> oneOfFields = Sets.newHashSet();
Review comment:
Since the variable names oneOfFields and oneOfFieldLocationMap are very
similar, we should disambiguate them somehow. Maybe emphasize that this one
refers to the components of a oneof field, not the oneof field itself.
We should also add comments explaining the meaning of these variables.
```suggestion
Set<Integer> oneOfComponentFields = Sets.newHashSet();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 637105)
Time Spent: 2h 10m (was: 2h)
> Change ProtoSchemaTranslator beam schema creation to match the order for
> protobufs containing Oneof fields
> ----------------------------------------------------------------------------------------------------------
>
> Key: BEAM-12464
> URL: https://issues.apache.org/jira/browse/BEAM-12464
> Project: Beam
> Issue Type: Improvement
> Components: extensions-java-protobuf
> Reporter: Reuben van Ammers
> Priority: P2
> Labels: stale-P2
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently, when ProtoSchemaTranslator creates the beam schema from a protobuf
> definition it always puts the Oneofs at the start of the beam schema due to
> Oneofs being created from the code first. This means that the order of the
> fields doesn't match the order of the protobuf defintion. As the schema
> generation is used when converting from beam rows to protobufs, it
> additionally means that it is impossible to convert from a beam row where the
> oneof fields are not the first fields in the beamrow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
