[
https://issues.apache.org/jira/browse/BIGTOP-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060705#comment-14060705
]
Sean Mackrory commented on BIGTOP-1376:
---------------------------------------
While I like the idea of eliminating the amount of duplication that happens in
Oozie's build, I would have some concerns about this (I've thought about this
problem a lot myself, and have not found a solution I felt entirely comfortable
pursuing). Just a few thoughts to keep in mind:
* I believe mistakes in maintaining the sharelib are by far the number 1 cause
of user problems with Oozie. We need to be sure that any change we make is
going to be an improvement in how users can work with the sharelib, and not a
regression - not even a small one.
* Currently oozie-sharelib.tar.gz contains all the libraries that Oozie was
built against. You would be switching to maintaining the sharelib based on
which libraries are installed on a particular node in your cluster, which might
not be everything want to use with Oozie, and *might* be different versions.
Personally, I like the idea of doing away with the tarball and the duplication,
but I still think any change in the status quo here needs to be decided upon
very carefully. Would you mind if I emailed the oozie-dev list and asked for
their comments on this issue to make sure we have lots of well-informed
feedback?
{quote}I think this proposal aligned with what we have done in Hue
packaging{quote}
Which isn't necessarily a good thing - I've had a few people complain to me
about how it's done in Bigtop :) Modularity does seem like a good thing, and in
the case of Bigtop specifically it's a bit of a necessity for Hue because Hue
includes apps that that Bigtop stack alone does not yet support, but modularity
also brings complexity which can cause (and has caused) users grief. So I'm not
-1'ing, I'm just suggesting you proceed cautiously.
> Split Oozie package into several small packages
> -----------------------------------------------
>
> Key: BIGTOP-1376
> URL: https://issues.apache.org/jira/browse/BIGTOP-1376
> Project: Bigtop
> Issue Type: Improvement
> Reporter: Guodong Wang
>
> Oozie can manage hive/pig/sqoop actions, while user do not need to enable
> them all. It is better to provide a way for user to enable them.
> My proposal is like what bigtop has done in Hue packaging, split the
> functions as separate plugin package.
> # provide packages: oozie-hive, oozie-sqoop, oozie-pig. Each of them depends
> on hive, sqoop and pig respectively.
> # In each package, use the symlinks to manage the dependency jars. For
> example, if oozie-hive is installed, some symlinks are created under
> /usr/lib/oozie/share/lib/hive/.
> # After install oozie-hive/oozie-sqoop/oozie-pig packages, users need to do
> sharelib upgrade with oozie-setup script.
> We can have the following benifits with this proposal
> # No dependency jars are in Oozie packages. All dependency jars are managed
> by symlinks.
> # Recently, BIGTOP-1349 was found in our cluster. If using this proposal,
> this issue can be solved. Because oozie-hive depends on hive. If user enable
> oozie-hive plugin, hive jars are installed on Oozie node.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
