[
https://issues.apache.org/jira/browse/HCATALOG-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398613#comment-13398613
]
Travis Crawford commented on HCATALOG-434:
------------------------------------------
Started working on this and its actually pretty complex. It might make sense to
reorganize things in a couple separate changes to limit risk
I ran this to move the pig classes into their own subproject. This would get
published as {{org.apache.hcatalog-hcatalog-pig-adapter}}. At first I just
called it {{pig-adapter}} but that means {{pig-adapter-0.5.0-dev.jar}} does not
say hcatalog anywhere which I think is confusing.
The general approach is adding {{build-common-new.xml}} that contains macros &
template tasks. Then inside the subproject {{build.xml}} we basically set
properties/attributes that are used by the template tasks. I created a "new"
common xml file so the existing build is not affected. We could move stuff like
the oozie package to this format, and convert the storage handlers. We could
move the "core" stuff to a subproject too, so the top-level build mainly has
packaging tasks.
I'm probably missing something, so posting this patch to get some feedback on
the general approach.
{code}
mkdir -p hcatalog-pig-adapter/src/main/java/org/apache/hcatalog
svn add hcatalog-pig-adapter
svn mv src/java/org/apache/hcatalog/pig
hcatalog-pig-adapter/src/main/java/org/apache/hcatalog
mkdir -p hcatalog-pig-adapter/src/test/java/org/apache/hcatalog
svn add hcatalog-pig-adapter/src/test
svn mv src/test/org/apache/hcatalog/pig/
hcatalog-pig-adapter/src/test/java/org/apache/hcatalog
# this test uses pig classes
svn mv src/test/org/apache/hcatalog/mapreduce/TestHCatHiveCompatibility.java \
src/test/org/apache/hcatalog/mapreduce/TestHCatHiveCompatibility.java.broken
{code}
> Package HCatalog pig support as a separate jar
> ----------------------------------------------
>
> Key: HCATALOG-434
> URL: https://issues.apache.org/jira/browse/HCATALOG-434
> Project: HCatalog
> Issue Type: Task
> Reporter: Travis Crawford
>
> HCatalog's pig support should be moved into a separate jar.
> DETAILS:
> One of HCatalog's major goals is making the HiveMetaStore accessible to other
> processing frameworks (pig for example). If we bundle everything inside a
> single hcatalog.jar it will have a large number of processing framework
> dependencies.
> For example, let's say HCatalog has the core classes, and adaptors for pig &
> scalding all bundled inside a single hcatalog.jar. The maven dependencies
> will include both pig & scalding. Projects integrating with hcat either end
> up with way more dependencies than necessary, or they need to pick & choose
> which dependencies to include in their project.
> Packaging pig support as a separate jar allows users to get the pig support
> without dependencies from other processing frameworks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
