[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158095#comment-14158095 ] Ben Roling commented on HIVE-5865: -- Hey [~brocknoland] - the HCatLoader tests failed again but as I stated previously, those tests are failing without any of the changes from this JIRA. Is there anything more you want me to do on this? Are those test failures something someone else is already looking at? I would have to assume they are being seen in other builds as well. AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, HIVE-5865.2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156692#comment-14156692 ] Ben Roling commented on HIVE-5865: -- Yeah, I'll take a look at the test failures and also see if I can create a new test specific to the issue. It was lazy of me not to have done that from the start -- I just didn't take the time to familiarize myself with the full build and testing setup of the project to do it as this is my first contribution to Hive. AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Roling updated HIVE-5865: - Attachment: HIVE-5865-v2.patch Uploading a new patch with a unit test that fails without the fix for this issue. AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156883#comment-14156883 ] Ben Roling commented on HIVE-5865: -- I don't know what exactly is the cause of the TestHCatalogLoader failures but I don't think it is related to my changes. That test seems to fail even with my changes backed out. AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Roling updated HIVE-5865: - Status: Patch Available (was: Open) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.12.0, 0.11.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Roling updated HIVE-5865: - Status: Open (was: Patch Available) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.12.0, 0.11.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157207#comment-14157207 ] Ben Roling commented on HIVE-5865: -- Thanks [~brocknoland]! AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, HIVE-5865.2.patch, HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Roling updated HIVE-5865: - Attachment: HIVE-5865.patch Attaching patch with fix AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ben Roling Attachments: HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Roling updated HIVE-5865: - Status: Patch Available (was: Open) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.13.0, 0.12.0, 0.11.0 Reporter: Ben Roling Attachments: HIVE-5865.patch AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
Ben Roling created HIVE-5865: Summary: AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' Key: HIVE-5865 URL: https://issues.apache.org/jira/browse/HIVE-5865 Project: Hive Issue Type: Bug Affects Versions: 0.12.0, 0.11.0 Reporter: Ben Roling AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys will always be 'org.apache.avro.util.Utf8'. If the reader schema defines avro.java.string=String, this assumption does not hold, resulting in a ClassCastException. I think a simple fix would be to define 'mapDatum' with type MapCharSequence,Object instead of MapUtf8,Object. Assuming the key has the more general type of 'CharSequence' avoids the need to make an assumption of either String or Utf8. I discovered the issue when using Hive 0.11.0. Looking at the tags it is also there is in 0.12.0 and trunk: https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 The reason I saw this issue was because I pointed my Hive table to a schema file I populated based on pulling the schema from the SCHEMA$ attribute of an Avro generated Java class and I used stringType=String in the configuration of the avro-maven-plugin when generating my Java classes. If I alter the schema my Hive table points to such that it doesn't have the avro.java.string attribute on my map type objects then queries work fine but if I leave those in there I get the ClassCastException anytime I try to query the table. -- This message was sent by Atlassian JIRA (v6.1#6144)
