I filed HDFS-6200 to demonstrate the feasibility of the approach. ~Haohui
On Fri, Apr 4, 2014 at 11:46 AM, Haohui Mai <h...@hortonworks.com> wrote: > I agree with Nicholas, Steve and Alejandro that it might require some > nontrivial to achieve the goal. Here is my high-level plan: > > 1. Create a new hdfs-client package, and gradually move classes from hdfs > to hdfs-client. Fortunately IDEs like Eclipse and IntelliJ can do most of > the heavy-liftings. > 2. The hdfs package depends on the hdfs-client, because the webhdfs server > uses a DFSClient internally. > 3. Downstream projects can continue to depend on hdfs in the meantime, but > they can start using the new hdfs-client projects once we make enough > progress. > > That way the work can be done incrementally. Tackling the dependency of > hadoop-common requires more work thus I plan to take care of it after we > have a separate hdfs-client. Does it sound a reasonable plan? > > ~Haohui > > > On Thu, Apr 3, 2014 at 10:16 AM, Alejandro Abdelnur <t...@cloudera.com>wrote: > >> Haouhi's suggestion of a hdfs-client JAR with client dependencies only, >> would be IMO the 'correct' way of doing things, we should have a >> hdfs-server and hdfs-client JARs. >> >> Doing this is practice is not trivial as classes are not properly >> segregated. So, Steven's suggestion of an hdfs-client seems the best bet >> short term. >> >> thx >> >> >> On Thu, Apr 3, 2014 at 5:06 AM, Steve Loughran <ste...@hortonworks.com >> >wrote: >> >> > to follow up with an example, >> > >> > JIRA on updating dependencies and tuning the POMs >> > https://issues.apache.org/jira/browse/HADOOP-9991 >> > >> > >> > here's a JIRA on dropping ZK from the hadoop-client POM >> > >> > https://issues.apache.org/jira/browse/HADOOP-9905 >> > >> > And there's an mr-client POM where we've been slowly cutting down on >> what >> > it pulls in >> > https://issues.apache.org/jira/browse/MAPREDUCE-5624 >> > >> > This shows that >> > 1. we can given maven/ivy projects what they need -and no more- through >> > POM-only projects. >> > 2. its an ongoing project to keep those dependencies cut down. >> > 3. there's always the risk that you drop too much and some project >> > discovers that while their code builds, it doesn't run any more. >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or >> entity to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the >> reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> >> >> >> -- >> Alejandro >> > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
