Hi Lewis, After fixing GORA-188 [0] managing union filelds in a record when comparing, test passes. Now I think my first guess was wrong. The problem was on "content". Now "content" is optional (nullable), and was not being compared right at PersistentBase#getFieldHashCode().
Before unions, map serialization and deserialization was resulting in different elements order, but that should not be a problem since conceptually maps don't have order and Map.hashCode() seems to return the same value as expected. Anyway, if you wish to get the same order after deserializing, we should check serialization and deserialization. Maybe the problem is in the serialization. Anyway, must no rely on order in maps. I will add the fix to GORA-174 and fix another bug before applying the patch to /branches/GORA_174. Will do it today :) Regards, Alfonso Nishikawa [0] https://issues.apache.org/jira/browse/GORA-188 2013/3/10 Lewis John Mcgibbney <[email protected]>: > Hi, > > I have a question relating to this test which fails after applying GORA-174, > 1. Within IOUtils#deserialize(Configuration conf, DataInput in, T obj , > Class<T> objClass) [0] we use Hadoop's > serializationFactory.getDeserializer(objClass) [1] to pick the correct > serializer for the persistent class. > 2. In this particular instance, as expected, we are returned Gora's > PersistentDezerializer[2]. > 3. In PersistentDeserializer#deserialize(PersistentBase persistent) we rely > upon Gora's PersistentDatumReader[3] to read(reuseObjects ? persistent : > null, decoder) > 4. PersistentDatumReader leverages Avro's SpecificDatumReader[4] to read, > which in turn is inherited from GenericDatumReader[5]. > > The question I have... Are we guaranteed that data structure is maintained > when using GenericDatumReader for retrieving data? > The problem in TestPersistentSerialization is that we get the 'same' data > but it is not structured identically before and after serialization... this > is a problem for us. To what extent I do not know, but it is a problem > none-the-less. > Can anyone confirm > > 1. Am I barking up the wrong tree, is the problem related to something else? > 2. Should this be asked on user@avro > > [0] > http://gora.apache.org/docs/current/apidocs-0.2.1/index.html?org/apache/gora/util/IOUtils.html > [1] > http://hadoop.apache.org/docs/r1.0.4/api/index.html?org/apache/hadoop/io/serializer/SerializationFactory.html > [2] > http://gora.apache.org/docs/current/apidocs-0.2.1/index.html?org/apache/gora/mapreduce/PersistentDeserializer.html > [3] > http://gora.apache.org/docs/current/apidocs-0.2.1/index.html?org/apache/gora/avro/PersistentDatumReader.html > [4] > http://avro.apache.org/docs/1.3.3/api/java/index.html?org/apache/avro/specific/SpecificDatumReader.html > [5] > http://avro.apache.org/docs/1.3.3/api/java/index.html?org/apache/avro/generic/GenericDatumReader.html > > -- > *Lewis* -- "Drinking bloody marys all night will make you feel like a corpse in the morning."
