Certain recursive schemas can prevent Schema.RecordSchema.hashCode() and
.equals() from returning
-------------------------------------------------------------------------------------------------
Key: AVRO-685
URL: https://issues.apache.org/jira/browse/AVRO-685
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.4.1
Environment: all
Reporter: Richard Ahrens
Priority: Critical
I am creating a protocol in memory by building up Schema objects, then writing
the avpr file to disk and running SpecificCompiler against it to generate Java
sources. My protocol file causes SpecificCompiler to hang. Running in the
debugger, I can see a long stack trace emanating from
SpecificCompiler.enqueue() (see debugger stack trace at end of this text).
What appears to be happening is that Schema.RecordSchema.hashCode() is removing
itself from the SEEN_HASHCODE map prematurely; schemas with circular references
in multiple fields are added and removed from SEEN_HASHCODE causing the code to
bounce around between fields without ever unwinding to the root object.
I'm unable to include attachments to this Jira issue, but here's a patch that
fixes the problem. If this patch is accepted, I'd like to request an
incremental release as this is a showstopper for us.
Index: src/java/org/apache/avro/Schema.java
===================================================================
--- src/java/org/apache/avro/Schema.java (revision 1028064)
+++ src/java/org/apache/avro/Schema.java (working copy)
@@ -587,21 +587,27 @@
Set seen = SEEN_EQUALS.get();
SeenPair here = new SeenPair(this, o);
if (seen.contains(here)) return true; // prevent stack overflow
+ boolean first = seen.isEmpty();
try {
seen.add(here);
return fields.equals(((RecordSchema)o).fields);
} finally {
- seen.remove(here);
+ if(first) {
+ seen.clear();
+ }
}
}
public int hashCode() {
Map seen = SEEN_HASHCODE.get();
if (seen.containsKey(this)) return 0; // prevent stack overflow
+ boolean first = seen.isEmpty();
try {
seen.put(this, this);
return super.hashCode() + fields.hashCode();
} finally {
- seen.remove(this);
+ if(first) {
+ seen.clear();
+ }
}
}
void toJson(Names names, JsonGenerator gen) throws IOException {
I can also provide a sample avpr file to reproduce the issue-- please contact
me directly.
Debugger stack trace referenced above:
org.apache.avro.specific.SpecificCompiler at localhost:3273
Thread [main] (Suspended)
System.identityHashCode(Object) line: not available [native
method]
IdentityHashMap<K,V>.hash(Object, int) line: 284
IdentityHashMap<K,V>.put(K, V) line: 412
Schema$RecordSchema.hashCode() line: 601
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$ArraySchema.hashCode() line: 703
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$UnionSchema.hashCode() line: 781
Schema$ArraySchema.hashCode() line: 703
Schema$Field.hashCode() line: 421
Schema$LockableArrayList<E>(AbstractList<E>).hashCode() line:
527
Schema$RecordSchema.hashCode() line: 602
HashMap<K,V>.getEntry(Object) line: 344
HashMap<K,V>.containsKey(Object) line: 335
HashSet<E>.contains(Object) line: 184
SpecificCompiler.enqueue(Schema) line: 134
SpecificCompiler.<init>(Protocol) line: 70
SpecificCompiler.compileProtocol(File, File) line: 114
SpecificCompiler.main(String[]) line: 399
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
