[
https://issues.apache.org/jira/browse/HCATALOG-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249447#comment-13249447
]
Travis Crawford commented on HCATALOG-364:
------------------------------------------
I just checked PIG-1429 and boolean support was added in 0.10.0.
HCatalog currently depends on pig 0.8.0 - supporting booleans would require
bumping the dependency to an unreleased pig version. I know the release branch
is starting soon (has started already?) but it will take some time for adoption.
Given that no currently released version of pig supports booleans, I'm in favor
of the translation approach. When pig 0.10.0 is released, a change could be
made to check for boolean support and use it if present.
Thoughts?
> HCatalog should handle boolean fields
> -------------------------------------
>
> Key: HCATALOG-364
> URL: https://issues.apache.org/jira/browse/HCATALOG-364
> Project: HCatalog
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
>
> Currently HCatalog does not support boolean fields, explicitly looking for
> them and throwing an exception:
> {code}
> Caused by: org.apache.pig.PigException: ERROR 1115: Incompatible type found
> in hcat table schema: boolean
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:403)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHcatFieldFollowsPigRules(PigHCatUtil.java:408)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatSchemaFollowsPigRules(PigHCatUtil.java:391)
> at
> org.apache.hcatalog.pig.PigHCatUtil.validateHCatTableSchemaFollowsPigRules(PigHCatUtil.java:422)
> at org.apache.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:158)
> {code}
> This is a big problem as many of our schemas use boolean.
> I propose we take the same approach as elephant-bird's
> [ThriftToPig.java|https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/pig/util/ThriftToPig.java#L106]
> and translate into an Integer in the pig layer. This approach allows
> processing frameworks with boolean support to take advantage of that feature,
> and frameworks without boolean support could use the translated value
> (versions of pig people are likely to have deployed; newer pig versions with
> boolean support could use the type directly). We've been using this technique
> for some time and in practice it works well.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
