[
https://issues.apache.org/jira/browse/BEAM-9041?focusedWorklogId=367699&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367699
]
ASF GitHub Bot logged work on BEAM-9041:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Jan/20 19:39
Start Date: 07/Jan/20 19:39
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #10492:
[BEAM-9041, BEAM-9042] SchemaCoder equals should not rely on from/toRowFunction
equality
URL: https://github.com/apache/beam/pull/10492#discussion_r363916612
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/AvroUtils.java
##########
@@ -431,7 +465,47 @@ public static GenericRecord toGenericRecord(
*/
public static SerializableFunction<Row, GenericRecord>
getRowToGenericRecordFunction(
@Nullable org.apache.avro.Schema avroSchema) {
- return g -> toGenericRecord(g, avroSchema);
+ return new RowToGenericRecordFn(avroSchema);
+ }
+
+ private static class RowToGenericRecordFn implements
SerializableFunction<Row, GenericRecord> {
+ private transient org.apache.avro.Schema avroSchema;
+
+ RowToGenericRecordFn(@Nullable org.apache.avro.Schema avroSchema) {
+ this.avroSchema = avroSchema;
+ }
+
+ @Override
+ public GenericRecord apply(Row input) {
+ return toGenericRecord(input, avroSchema);
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (this == other) {
+ return true;
+ }
+ if (other == null || getClass() != other.getClass()) {
+ return false;
+ }
+ RowToGenericRecordFn that = (RowToGenericRecordFn) other;
+ return avroSchema.equals(that.avroSchema);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(avroSchema);
+ }
+
+ private void writeObject(ObjectOutputStream out) throws IOException {
+ final String avroSchemaAsString = (avroSchema == null) ? null :
avroSchema.toString();
+ out.writeObject(avroSchemaAsString);
+ }
+
+ private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
+ final String avroSchemaAsString = (String) in.readObject();
+ avroSchema = new
org.apache.avro.Schema.Parser().parse(avroSchemaAsString);
Review comment:
I'm not 100% sure but it looks like the failures are occurring when
avroSchema is null. Either way I think you need to check if
`avroSchemaAsString` is null here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 367699)
Time Spent: 2h (was: 1h 50m)
> SchemaCoder equals should not rely on from/toRowFunction equality
> -----------------------------------------------------------------
>
> Key: BEAM-9041
> URL: https://issues.apache.org/jira/browse/BEAM-9041
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: Minor
> Fix For: 2.18.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> SchemaCoder equals implementation relies on SerializableFunction equals
> method, this is error-prone because users rarely implement the equals method
> for a SerializableFunction. One alternative would be to rely on bytes
> equality for this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
