[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162690#comment-14162690 ] Brock Noland commented on HIVE-5865: I think a release note is appropriate... Added! > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling >Assignee: Ben Roling > Fix For: 0.15.0 > > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161323#comment-14161323 ] Lefty Leverenz commented on HIVE-5865: -- Doc query: Should this be documented in the Avro SerDe wikidoc for release 0.15.0? * [Avro SerDe | https://cwiki.apache.org/confluence/display/Hive/AvroSerDe] Gentle nudge: How about a release note? > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling >Assignee: Ben Roling > Fix For: 0.15.0 > > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158173#comment-14158173 ] Brock Noland commented on HIVE-5865: Those tests are failing due to: https://issues.apache.org/jira/browse/HIVE-8271?focusedCommentId=14157581&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14157581 I will commit this today. Thanks!! > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158095#comment-14158095 ] Ben Roling commented on HIVE-5865: -- Hey [~brocknoland] - the HCatLoader tests failed again but as I stated previously, those tests are failing without any of the changes from this JIRA. Is there anything more you want me to do on this? Are those test failures something someone else is already looking at? I would have to assume they are being seen in other builds as well. > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[
https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157885#comment-14157885
]
Hive QA commented on HIVE-5865:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12672645/HIVE-5865.2.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 6542 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testGetInputBytes[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadPartitionedBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadComplex[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadPrimitiveTypes[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsNoDataInDataNoSpec[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testEmptyStore[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testNoAlias[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testPartitionPublish[5]
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1099/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1099/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1099/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12672645
> AvroDeserializer incorrectly assumes keys to Maps will always be of type
> 'org.apache.avro.util.Utf8'
>
>
> Key: HIVE-5865
> URL: https://issues.apache.org/jira/browse/HIVE-5865
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
>Reporter: Ben Roling
> Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch,
> HIVE-5865.2.patch, HIVE-5865.patch
>
>
> AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys
> will always be 'org.apache.avro.util.Utf8'. If the reader schema defines
> "avro.java.string"="String", this assumption does not hold, resulting in a
> ClassCastException.
> I think a simple fix would be to define 'mapDatum' with type
> Map instead of Map. Assuming the key has
> the more general type of 'CharSequence' avoids the need to make an assumption
> of either String or Utf8.
> I discovered the issue when using Hive 0.11.0. Looking at the tags it is
> also there is in 0.12.0 and trunk:
> https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313
> The reason I saw this issue was because I pointed my Hive table to a schema
> file I populated based on pulling the schema from the SCHEMA$ attribute of an
> Avro generated Java class and I used stringType=String in the configuration
> of the avro-maven-plugin when generating my Java classes.
> If I alter the schema my Hive table points to such that it doesn't have the
> "avro.java.string" attribute on my "map" type objects then queries work fine
> but if I leave those in there I get the ClassCastException anytime I try to
> query the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157207#comment-14157207 ] Ben Roling commented on HIVE-5865: -- Thanks [~brocknoland]! > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157198#comment-14157198 ] Brock Noland commented on HIVE-5865: +1 pending tests > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865-v2.patch, HIVE-5865.2.patch, > HIVE-5865.2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156883#comment-14156883 ] Ben Roling commented on HIVE-5865: -- I don't know what exactly is the cause of the TestHCatalogLoader failures but I don't think it is related to my changes. That test seems to fail even with my changes backed out. > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865-v2.patch, HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156692#comment-14156692 ] Ben Roling commented on HIVE-5865: -- Yeah, I'll take a look at the test failures and also see if I can create a new test specific to the issue. It was lazy of me not to have done that from the start -- I just didn't take the time to familiarize myself with the full build and testing setup of the project to do it as this is my first contribution to Hive. > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[
https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156243#comment-14156243
]
Hive QA commented on HIVE-5865:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12672440/HIVE-5865.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 6503 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.pig.TestHCatLoader.testColumnarStorePushdown[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testConvertBooleanToInt[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testGetInputBytes[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadPartitionedBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadBasic[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadComplex[5]
org.apache.hive.hcatalog.pig.TestHCatLoader.testSchemaLoadPrimitiveTypes[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsNoDataInDataNoSpec[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testEmptyStore[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testNoAlias[5]
org.apache.hive.hcatalog.pig.TestHCatStorer.testPartitionPublish[5]
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1083/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1083/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1083/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12672440
> AvroDeserializer incorrectly assumes keys to Maps will always be of type
> 'org.apache.avro.util.Utf8'
>
>
> Key: HIVE-5865
> URL: https://issues.apache.org/jira/browse/HIVE-5865
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.11.0, 0.12.0, 0.13.0
>Reporter: Ben Roling
> Attachments: HIVE-5865.patch
>
>
> AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys
> will always be 'org.apache.avro.util.Utf8'. If the reader schema defines
> "avro.java.string"="String", this assumption does not hold, resulting in a
> ClassCastException.
> I think a simple fix would be to define 'mapDatum' with type
> Map instead of Map. Assuming the key has
> the more general type of 'CharSequence' avoids the need to make an assumption
> of either String or Utf8.
> I discovered the issue when using Hive 0.11.0. Looking at the tags it is
> also there is in 0.12.0 and trunk:
> https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313
> The reason I saw this issue was because I pointed my Hive table to a schema
> file I populated based on pulling the schema from the SCHEMA$ attribute of an
> Avro generated Java class and I used stringType=String in the configuration
> of the avro-maven-plugin when generating my Java classes.
> If I alter the schema my Hive table points to such that it doesn't have the
> "avro.java.string" attribute on my "map" type objects then queries work fine
> but if I leave those in there I get the ClassCastException anytime I try to
> query the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-5865) AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
[ https://issues.apache.org/jira/browse/HIVE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155729#comment-14155729 ] Sean Busbey commented on HIVE-5865: --- Can you add a test case for a table backed by a schema that requires this patch to avoid failing? > AvroDeserializer incorrectly assumes keys to Maps will always be of type > 'org.apache.avro.util.Utf8' > > > Key: HIVE-5865 > URL: https://issues.apache.org/jira/browse/HIVE-5865 > Project: Hive > Issue Type: Bug >Affects Versions: 0.11.0, 0.12.0, 0.13.0 >Reporter: Ben Roling > Attachments: HIVE-5865.patch > > > AvroDeserializer. deserializeMap() incorrectly assumes the type of they keys > will always be 'org.apache.avro.util.Utf8'. If the reader schema defines > "avro.java.string"="String", this assumption does not hold, resulting in a > ClassCastException. > I think a simple fix would be to define 'mapDatum' with type > Map instead of Map. Assuming the key has > the more general type of 'CharSequence' avoids the need to make an assumption > of either String or Utf8. > I discovered the issue when using Hive 0.11.0. Looking at the tags it is > also there is in 0.12.0 and trunk: > https://github.com/apache/hive/blob/99f5bfcdf64330d062a30c0c9d83be1fbee54c34/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L313 > The reason I saw this issue was because I pointed my Hive table to a schema > file I populated based on pulling the schema from the SCHEMA$ attribute of an > Avro generated Java class and I used stringType=String in the configuration > of the avro-maven-plugin when generating my Java classes. > If I alter the schema my Hive table points to such that it doesn't have the > "avro.java.string" attribute on my "map" type objects then queries work fine > but if I leave those in there I get the ClassCastException anytime I try to > query the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
