ahmedabu98 commented on code in PR #25927:
URL: https://github.com/apache/beam/pull/25927#discussion_r1149210074
##########
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java:
##########
@@ -270,6 +400,55 @@ public static SerializableFunction<Row,
DoublyNestedDataTypes> doublyNestedDataT
return
DEFAULT_SCHEMA_PROVIDER.fromRowFunction(DOUBLY_NESTED_DATA_TYPES_TYPE_DESCRIPTOR);
}
+ /**
+ * Contains all primitive Java types supported by Avro. The purpose of this
class is to test
+ * schema-aware PTransforms with flat {@link Schema} {@link Row}s.
+ */
+ @DefaultSchema(AutoValueSchema.class)
+ @AutoValue
+ public abstract static class AvroPrimitiveDataTypes implements Serializable {
Review Comment:
Tried this and some formats
[failed](https://ci-beam.apache.org/job/beam_PreCommit_Java_File-schema-transform_IO_Direct_Commit/1/testReport/org.apache.beam.sdk.io.fileschematransform/JsonFileWriteSchemaTransformFormatProviderTest/doublyNestedDataTypesRepeat/);
looks like XML, JSON, CSV don't support bytes (as in a sequence of bytes), but
XML and JSON do support a single byte type. Opposite to the case of Avro and
Parquet, which support a byte sequence but not a single byte.
Thinking of proceeding by taking not including any form of bytes in
AllPrimitiveDataTypes since there is no common overlap. Instead, we can have
two new classes, ByteType and ByteSequenceType that will test for byte, byte[].
I can create new tests for these in both file read and write transforms. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
