Tuning the POM only mitigates the problem. The problem of one HDFS jar is that you can't rule out all unnecessary dependency. For example, NamenodeWebHdfsMethods depends on jersey-server and servlet. The Apache Falcon project has clients for HDFS, Hive, Pig, Oozie, thus it pulls in the dependency. It also pulls in Tomcat as well.
Here is the bad part: the dependency from the client and Tomcat results in three different versions of servlet jars in the classpath. They have the same package name but they obviously have different behavior. We have to be extremely careful on the orders of the jars in the classpath to make things work. One might be able to hand craft a hdfs-client POM to do the trick, but it requires a lot of tuning and testing, and sometimes it might not be feasible due to the dependency between the classes in the jar. Pulling out the client to another module seems much cleaner and easier to maintain. ~Haohui On Thu, Apr 3, 2014 at 5:06 AM, Steve Loughran <ste...@hortonworks.com>wrote: > to follow up with an example, > > JIRA on updating dependencies and tuning the POMs > https://issues.apache.org/jira/browse/HADOOP-9991 > > > here's a JIRA on dropping ZK from the hadoop-client POM > > https://issues.apache.org/jira/browse/HADOOP-9905 > > And there's an mr-client POM where we've been slowly cutting down on what > it pulls in > https://issues.apache.org/jira/browse/MAPREDUCE-5624 > > This shows that > 1. we can given maven/ivy projects what they need -and no more- through > POM-only projects. > 2. its an ongoing project to keep those dependencies cut down. > 3. there's always the risk that you drop too much and some project > discovers that while their code builds, it doesn't run any more. > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
