OK. On Tue, Feb 19, 2013 at 12:53 PM, kaveh minooie <[email protected]> wrote:
> Thanks Lewis > > it turned out that slf4j 1.6.1 is being pulled by both hbase 0.90.4 > and zookeeper 3.4.5: > > Required by > Organisation Name Revision In Configurations Asked > Revision > org.apache.nutch nutch working@localhost default, test, > master 1.4.3 > org.apache.hbase hbase 0.90.4 compile, runtime 1.6.1 > org.apache.zookeeper zookeeper 3.4.5 default, compile, runtime, > master 1.6.1 > Dependencies > Module Revision Status Resolver Default Licenses > Size > log4j by log4j 1.2.15 release maven2 false The Apache Software > License, Version 2.0 0 kB > log4j by log4j 1.2.16 release maven2 false The Apache Software > License, Version 2.0 470 kB > slf4j-api by org.slf4j 1.6.1 release maven2 false 25 kB > > > I think I am better off upgrading the slf4j across my hadoop cluster :) > > > thanks, > > > On 02/18/2013 10:26 PM, Lewis John Mcgibbney wrote: > >> Hi, >> So by the sounds of it the slf4j is being pulled transitively and you need >> to determine from where. >> You can use the ant report task which makes ivy generate a nice dependency >> report for you. You can then see which direct dependency includes slf4j. I >> know for a fact that all gora-core dependency will pull it. >> >> Once you've highlighted where it is coming from you can make the relevant >> exclusion ok. >> >> On Saturday, February 16, 2013, kaveh minooie <[email protected]> wrote: >> >>> So when you say "prune the dependencies", I am not sure what you are >>> >> talking about cause what I could think of is not working. let me explain >> the situation again. nutch 2.x ivy file ( ivy/ivy.xml ) has this in it: >> >>> >>> <dependencies> >>> <dependency org="org.elasticsearch" name="elasticsearch" >>> rev="0.19.4" >>> conf="*->default,sources"/> >>> <dependency org="org.apache.solr" name="solr-solrj" rev="3.4.0" >>> conf="*->default" /> >>> <dependency org="org.slf4j" name="slf4j-log4j12" rev="1.6.1" >>> conf="*->master" /> >>> >>> hadoop 1.1.1 ships with slf4j 1.4.3. these 2 are not compatible. now, i >>> >> rather not mess with my hadoop cluster so I tried to downgrade slf4j in >> nutch. I changed the above lines to : >> >>> >>> <dependencies> >>> <!-- >>> <dependency org="org.elasticsearch" name="elasticsearch" >>> rev="0.19.4" >>> conf="*->default,sources"/> >>> --> >>> <dependency org="org.apache.zookeeper" name="zookeeper" rev="3.4.5" >>> conf="*->default"/> >>> <dependency org="org.apache.solr" name="solr-solrj" rev="3.6.2" >>> conf="*->default" /> >>> <dependency org="org.slf4j" name="slf4j-log4j12" rev="1.4.3" >>> >>> as you can see I am upgrading the solr and zookeeper and removing the >>> >> elasticsearch, and all of these changes work fine since I can see the >> appropriate files in the build/lib directory after ant is done. but it >> doesn't work for slf4j, and the files copied to build/lib ( and >> subsequently in my job file ) are : >> >>> kaveh@d1r2n2:/source/nutch/**nutch$ ll build/lib/slf* >>> -rw-r--r-- 1 kaveh kaveh 25496 Jul 5 2010 build/lib/slf4j-api-1.6.1.jar >>> -rw-r--r-- 1 kaveh kaveh 9753 Jul 5 2010 >>> >> build/lib/slf4j-log4j12-1.6.1.**jar >> >>> >>> since i need the job file i can't just manually change the files in >>> >> build/lib, won't do me any good. now I don't know ant very well, and that >> is mostly why I am asking this from you guys. I have to say that I also >> changed the same thing in pom.xml as well: >> >>> >>> <dependency> >>> <groupId>org.slf4j</groupId> >>> <artifactId>slf4j-log4j12</**artifactId> >>> <version>1.4.3</version> >>> <optional>true</optional> >>> </dependency> >>> >>> but I still end up with the 1.6.1 version. I don't know how exactly ant >>> >> and ivy and pom work together, so I am asking if there is any other config >> file that I am missing, or why while it is working fine for solr and >> zookeeper it is not affecting the slf4j? >> >>> >>> thanks, >>> >>> >>> On 02/16/2013 09:42 AM, Lewis John Mcgibbney wrote: >>> >>>> >>>> A solution would be to manually prune the dependencies which are fetched >>>> via Ivy. If old slf4j dependencies are fetched for Hadoop via Ivy then >>>> maybe we need to make the exclusions explicit within ivy.xml. if you are >>>> able , then please provide a patch which fixes this if it is really a >>>> problem. >>>> It is important to note that pom.xml will most likely be outdated. You >>>> should build nutch with ant + ivy for the time being as this is stable. >>>> Thank you >>>> Lewis >>>> >>>> On Saturday, February 16, 2013, kaveh minooie <[email protected]> wrote: >>>> >>>>> >>>>> unfortunately your links have been removed from the email that i got so >>>>> >>>> i >> >>> >>>> am not sure what [0] and [1] are, but this is what i am using : >>>> >>>>> >>>>> kaveh@d1r2n2:/source/nutch/**nutch.git$ git remote -v >>>>> origin >>>>> https://github.com/apache/**nutch.git<https://github.com/apache/nutch.git>(fetch) >>>>> origin >>>>> https://github.com/apache/**nutch.git<https://github.com/apache/nutch.git>(push) >>>>> kaveh@d1r2n2:/source/nutch/**nutch.git$ git branch -v >>>>> * 2.x f02dcf6 NUTCH-XX remove unused db.max.inlinks from >>>>> >>>> >>>> nutch-default.xml >>>> >>>>> >>>>> trunk a7a1b41 NUTCH-1521 CrawlDbFilter pass null url to >>>>> >>>> urlNormalizers >> >>> kaveh@d1r2n2:/2locos/source/**nutch/nutch.git$ >>>>> >>>>> >>>>> i am using branch 2.x >>>>> >>>>> On 02/15/2013 06:02 PM, Lewis John Mcgibbney wrote: >>>>> >>>>>> >>>>>> Hi Kaveh, >>>>>> >>>>>> Two seconds please. First lets set some thing straight. >>>>>> Nutch trunk is from here [0] >>>>>> Nutch 2,x is from here [1] >>>>>> Which one do you use? >>>>>> >>>>>> On Fri, Feb 15, 2013 at 4:53 PM, kaveh minooie <[email protected]> >>>>>> >>>>> wrote: >> >>> >>>>>> but here is my problem. I tried to build the nutch using ver 1.4.3 of >>>>>>> >>>>>> >>>> the >>>> >>>>> >>>>>>> slf4j. i changed the version in both ivy.xml and pom.xml and cleaned >>>>>>> >>>>>> my >> >>> >>>> ivy >>>> >>>>> >>>>>>> cache but ant still fetches the version 1.6.1 when it builds the >>>>>>> >>>>>> >>>> project. >>>> >>>>> >>>>>>> what am I missing? >>>>>>> >>>>>>> >>>>>>> We can progress with the problem once we know what's actually going >>>>>> on. >>>>>> Thanks >>>>>> Lewis >>>>>> >>>>>> >>>>> >>> >>> >> > -- > Kaveh Minooie > > www.plutoz.com > -- *Lewis*
