[
https://issues.apache.org/jira/browse/BEAM-4454?focusedWorklogId=171386&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-171386
]
ASF GitHub Bot logged work on BEAM-4454:
----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Dec/18 16:55
Start Date: 02/Dec/18 16:55
Worklog Time Spent: 10m
Work Description: kanterov commented on a change in pull request #7181:
[BEAM-4454] Add more AVRO utilities to convert between Beam and Avro.
URL: https://github.com/apache/beam/pull/7181#discussion_r238102608
##########
File path:
sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/AvroUtilsTest.java
##########
@@ -49,15 +59,14 @@
public void supportsAnyAvroSchema(
@From(RecordSchemaGenerator.class) org.apache.avro.Schema avroSchema) {
// not everything is possible to translate
- assumeThat(avroSchema,
not(containsField(AvroUtilsTest::hasArrayOrMapOfNullable)));
assumeThat(avroSchema, not(containsField(AvroUtilsTest::hasNonNullUnion)));
- Schema schema = AvroUtils.toSchema(avroSchema);
+ Schema schema = AvroUtils.toBeamSchema(avroSchema);
Iterable iterable = new RandomData(avroSchema, 10);
List<GenericRecord> records = Lists.newArrayList((Iterable<GenericRecord>)
iterable);
for (GenericRecord record : records) {
- AvroUtils.toRowStrict(record, schema);
+ AvroUtils.toBeamRowStrict(record, schema);
Review comment:
With the conversion from `Row` to `GenericRecord`, now it's possible to add
an assertion:
```java
// roundtrip for enums returns strings because Beam doesn't have enum
type
assumeThat(avroSchema, not(containsField(x -> x.getType() ==
Type.ENUM)));
// roundtrip for fixed returns bytes because Beam doesn't have FIXED type
assumeThat(avroSchema, not(containsField(x -> x.getType() ==
Type.BYTES)));
...
for (GenericRecord record : records) {
GenericRecord out =
AvroUtils.toGenericRecord(AvroUtils.toBeamRowStrict(record,
schema), avroSchema);
assertEquals(record, out);
}
```
As well as the opposite property:
```
forall x: toBeamRowStrict(toGenericRecord(x)) === x
```
For the second property there is generator in `QuickCheckGenerators`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 171386)
Time Spent: 50m (was: 40m)
> Provide automatic schema registration for AVROs
> -----------------------------------------------
>
> Key: BEAM-4454
> URL: https://issues.apache.org/jira/browse/BEAM-4454
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Need to make sure this is a compatible change
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
