[
https://issues.apache.org/jira/browse/HCATALOG-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422804#comment-13422804
]
Travis Crawford commented on HCATALOG-364:
------------------------------------------
Now that pig 0.10. is released I think requiring it is fair game. Its both
reasonable and easy too :)
In that mail I noted how you can just remove the parts that make boolean fields
throw exceptions and queries work. I've been running pig 0.11.0 (not 10,
actually 11) queries with that stuff hacked into a local branch a bunch today
and it just seems to work correctly.
At runtime, I "register /usr/lib/hive/lib/*" and the antlr stuff does not seem
to cause issues. It may when running tests however if we simply put both
versions on the classpath. Maybe try bumping our pig dependency to 10 and see
if the tests pass?
> HCatalog should handle boolean fields
> -------------------------------------
>
> Key: HCATALOG-364
> URL: https://issues.apache.org/jira/browse/HCATALOG-364
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
>
> Currently HCatalog does not support boolean fields, explicitly looking for
> them and throwing an exception:
> {code}
> Caused by: org.apache.pig.PigException: ERROR 1115: Incompatible type found
> in hcat table schema: boolean
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:403)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:408)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatTableSchemaFollowsPigRules(PigHCatUtil.java:422)
> at org.apache.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:158)
> {code}
> This is a big problem as many of our schemas use boolean.
> I propose we take the same approach as elephant-bird's
> [ThriftToPig.java|https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/util/ThriftToPig.java#L106]
> and translate into an Integer in the pig layer. This approach allows
> processing frameworks with boolean support to take advantage of that feature,
> and frameworks without boolean support could use the translated value
> (versions of pig people are likely to have deployed; newer pig versions with
> boolean support could use the type directly). We've been using this technique
> for some time and in practice it works well.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
