Re: hadoop JARs not in lib/ directory of layout

2011-08-10 Thread Alejandro Abdelnur
Eric, I've just copied and pasted the example you written in your previous answer and added 'lib/' to it. My concern with the current approach is that complicates the packaging logic requiring special handling of artifacts (JARs) and it complicates the products scripts. Thanks. Alejandro On W

Re: hadoop JARs not in lib/ directory of layout

2011-08-10 Thread Eric Yang
On Aug 10, 2011, at 11:10 AM, Alejandro Abdelnur wrote: > Eric, > > I'd argue that including the JAR as you suggest will most likely break > because of required dependencies of the Hadoop JAR that may not be part of > HBase (ie the jackson JARs). > > But if you want to still do that you can alwa

Re: hadoop JARs not in lib/ directory of layout

2011-08-10 Thread Alejandro Abdelnur
Eric, I'd argue that including the JAR as you suggest will most likely break because of required dependencies of the Hadoop JAR that may not be part of HBase (ie the jackson JARs). But if you want to still do that you can always include the jar from the lib directory, for example: $HBASE_PREFIX/

Re: hadoop JARs not in lib/ directory of layout

2011-08-05 Thread Arindam Khaled
Please unsubscribe me. Thanks. Kind regards, Arindam Khaled On Aug 4, 2011, at 4:23 PM, Alejandro Abdelnur wrote: Allen, I agree 100% with you, there will be setups that will break. I was just pointing out that it may be a bit less that expected because of the scenarios I described.

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Eric Yang
It is easier to write shell script to import jar files by directory instead of explicitly reference to a few jars with specific versions. The common use case is: HBase needs to use hadoop jar files, but HBase depends on more recent version of log4j. The construction of the class path would be:

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Alejandro Abdelnur
[moving to core-dev@, general@ BCCed] Eric, Even if the JAR is in lib/ you could import/use that JAR only. How would you use Hadoop JARs without its dependencies? Many things will break unless you add the dependency JARs. Granted, there are JARs that are used by Hadoop server side only (JT/NN/T

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Alejandro Abdelnur
Allen, I agree 100% with you, there will be setups that will break. I was just pointing out that it may be a bit less that expected because of the scenarios I described. But yes, you are right, something/somewhere will break. And being more clear than before, yes, it MUST GO in the release notes

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Allen Wittenauer
On Aug 4, 2011, at 1:59 PM, Alejandro Abdelnur wrote: > Pig, Hive bundle Hadoop JARs with distributions, so no issue there. Re-read what I said: >> I suspect lots of pig, hive, and hbase installations will also break. It still remains a potential issue for those of us who

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Alejandro Abdelnur
Allen, Pig, Hive bundle Hadoop JARs with distributions, so no issue there. And for those who use Hadoop JARs from HADOOP_HOME, they normally add all dirs under ${HADOOP_HOME}/lib And finally, with the new layout of Hadoop trunk, things will be broken anyhow for them, so work will have to be done

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Allen Wittenauer
On Aug 4, 2011, at 1:06 PM, Alejandro Abdelnur wrote: > [moving to core-dev@] > > A big release note is doable. > > Still, people normally use 'hadoop' script when submitting jobs and 'hadoop' > would take care of having the JAR in the classpath. What other things would > break? > Eve

Re: hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Alejandro Abdelnur
[moving to core-dev@] A big release note is doable. Still, people normally use 'hadoop' script when submitting jobs and 'hadoop' would take care of having the JAR in the classpath. What other things would break? I'll start working on a patch then. Thxs. Alejandro On Thu, Aug 4, 2011 at 12:52

hadoop JARs not in lib/ directory of layout

2011-08-04 Thread Alejandro Abdelnur
[Using the core-dev@ alias now] -- Forwarded message -- From: Alejandro Abdelnur Date: Thu, Aug 4, 2011 at 11:03 AM Subject: hadoop JARs not in lib/ directory of layout To: gene...@hadoop.apache.org What is the rationale for having the hadoop JARs outside of the lib/ directory