damondouglas commented on code in PR #25927:
URL: https://github.com/apache/beam/pull/25927#discussion_r1146688175
##########
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java:
##########
@@ -38,6 +39,28 @@ public class SchemaAwareJavaBeans {
private static final DefaultSchemaProvider DEFAULT_SCHEMA_PROVIDER = new
DefaultSchemaProvider();
+ /** Convenience method for {@link AllPrimitiveDataTypes} instantiation. */
Review Comment:
```
/** Convenience method for {@link AvroPrimitiveDataTypes} instantiation. */
```
##########
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java:
##########
@@ -270,6 +400,55 @@ public static SerializableFunction<Row,
DoublyNestedDataTypes> doublyNestedDataT
return
DEFAULT_SCHEMA_PROVIDER.fromRowFunction(DOUBLY_NESTED_DATA_TYPES_TYPE_DESCRIPTOR);
}
+ /**
+ * Contains all primitive Java types supported by Avro. The purpose of this
class is to test
+ * schema-aware PTransforms with flat {@link Schema} {@link Row}s.
+ */
+ @DefaultSchema(AutoValueSchema.class)
+ @AutoValue
+ public abstract static class AvroPrimitiveDataTypes implements Serializable {
Review Comment:
I'll let you decide based on my comment and I'll LGTM either way.
What I like about AvroPrimitiveDataTypes is that it makes it easy to write
tests against field types we know would be compatible with Avro encodings.
What I struggle with is:
1. The name implies a coupling of data encoding though I get it
2. Writing tests against AllPrimitiveDataTypes that fail is valuable
information on how we expose FieldType
3. Dealing with annoying FieldTypes that aren't compatible with a Schema
related use case is valuable information to figure out how a real user of Beam
would have to deal with this when they don't have the ability to change the
original class but do have the ability to change the Schema using, for example,
https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/schemas/transforms/DropFields.html.
I dealt with this problem when writing tests for CsvIO and it was valuable
discovery. Writing tests around this problem of incompatible fields provided
for me a usability test not just does this code work test.
Another alternative would be just to get rid of the Byte field in
AllPrimitiveDataTypes but that would require you to refactor the tests that
break as a result.
This comment also applies to the other additions to SchemaAwareJavaBeans and
I didn't comment on those individually as it would have been redundant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
