[
https://issues.apache.org/jira/browse/HCATALOG-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229682#comment-13229682
]
Sushanth Sowmyan commented on HCATALOG-308:
-------------------------------------------
As an update, this jira was first of all intended to move the
configureInputJobProperties call from the Partition level to the table level,
since we were passing a TableDesc to it, and the idea was to eventually
introduce a new configureInputPartitionJobProperties at a partition level.
Much of this is due to StorageHandler being a Table-level concept in hive, and
us attempting to use that to replace StorageDrivers, which were a partition
level concept in HCat. Hive does not use a storage handler for simpler IF/OF
based partitions, and thus allows partitions to use different file formats.
With our change, however, we tried to unify code under FosterStorageHandler,
and have it do any and all initialization. This works, as long as the table's
IF is the same through it.
So, now, given that we have to step back on this, this is missing the time
window to release with 0.4, which means we should attempt to solve this in a
more complete manner, starting with changing hive's StorageHandlers to
understand partitions and then migrating HCat calls to use appropriately.
Other fixes that this mammoth patch also tried to address were:
a) Consistency between input and output - will be addressed with the SH
partition revamp.
b) Change of method signatures to (TableDesc) rather than
(TableDesc,Map<String,String>) to increase readability and simplicity - will
likely be abandoned because we will have backward compatibility headaches
later, we'll stick to the older signature
c) Replacement of Properties with Map<String,String> in InputJobInfo and
OutputJobInfo - will have to be abandoned for backward compatibility reasons.
At some point, we might decide to introduce something easier to use than
InputJobInfo/OutputJobInfo (maybe even with a builder interface!) to make it
easier/simpler to interface with the user, and deprecate the current one, but
that's at least a few revisions away.
d) Removal of mapred IF/OF from HCat - can easily be done, and should. -
Created https://issues.apache.org/jira/browse/HCATALOG-319 for it.
> TableDesc and jobProperties related changes to configureInputJobProperties
> and configureOutputJobProperties
> -----------------------------------------------------------------------------------------------------------
>
> Key: HCATALOG-308
> URL: https://issues.apache.org/jira/browse/HCATALOG-308
> Project: HCatalog
> Issue Type: Sub-task
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.4
>
> Attachments: HCATALOG-308.2.patch, HCATALOG-308.4.patch,
> HCATALOG-308.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
