To add to chaitanya's suggestions earlier - you can set the version number in libraries.properties / ivy.xml to be something else (for the patched version you are working on, say 0.22.1 or 0.23.0 ) and publish using the 'ant mvn-install' task.
Then in mapreduce's ivy.xml - just change the libraries.properties to the new version that you had put and build ,to include your new version. Let us know how it goes ( no need to go , --offline etc. ). On Thu, Mar 18, 2010 at 7:54 PM, Rodrigo Schmidt <[email protected]>wrote: > > Ant clean is not erasing ~/.ivy2/cache, so I have to explicitly remove the > files if I want to replace the jar for a newly compiled one. This doesn't > sound right, though. > > Cheers, > Rodrigo > > On 3/18/10 12:54 AM, "chaitanya krishna" <[email protected]> > wrote: > > > Hi Rodrigo, > > > > AFAIK, I dont find any other way of cross-project debugging as of now. > > Also, -Doffline=true is not working (a bug). Once this works, I think we > can > > work around by putting hadoop-core.jar in > > mapred-trunk/build/ivy/lib/Hadoop/common (not sure though). > > > > Also, I think doing an ant clean before running ant test > > -Dresolvers=internal might be the actual solution instead of deleting > > hadoop-core.jar in ~/.ivy2/cache, because the cached jar might be the one > in > > build/ivy/lib/Hadoop/common. > > > > Regards, > > Chaitanya. > > > > On Thu, Mar 18, 2010 at 1:01 PM, Rodrigo Schmidt <[email protected] > >wrote: > > > >> > >> Thanks Chaitanya! > >> > >> Didn't work at first. I tried many times in different ways, connected to > >> the > >> Internet and not. > >> > >> When I was about to give up I realized ivy might be caching some stuff, > >> which turned out to be true. I had to delete the hadoop-core.jar file > >> inside > >> ~/ivy2/cache for it to work. > >> > >> This whole ivy2 and maven dependencies look too complicated for offline > use > >> and cross-project debugging. Isn't there an easier way to do things? > >> > >> Cheers, > >> Rodrigo > >> > >> > >> On 3/17/10 11:27 PM, "chaitanya krishna" <[email protected]> > >> wrote: > >> > >>> In case you want to use hadoop-core.jar containing your modifications > in > >>> common, i think the following works: > >>> > >>> 1. ant mvn-install in common project. This will create hadoop-core.jar > >> and > >>> installs it in the local m2 repository. > >>> 2. ant test -Dresolvers=internal in mapreduce project. This will make > >> sure > >>> that the hadoop-core.jar is taken from local m2 repository. > >>> > >>> Hope this helps. > >>> > >>> -Chaitanya. > >>> > >>> On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt < > [email protected] > >>> wrote: > >>> > >>>> > >>>> I tried that but my logs still didn't show up. To test if it really > >> worked, > >>>> I disabled my Internet connection and tried it, which failed. Did I > miss > >>>> anything? > >>>> > >>>> > >>>> Output below: > >>>> > >>>> > >>>> $ant -Doffline=yes test > >>>> Buildfile: build.xml > >>>> > >>>> ivy-download: > >>>> > >>>> ivy-init-dirs: > >>>> > >>>> ivy-probe-antlib: > >>>> > >>>> ivy-init-antlib: > >>>> > >>>> ivy-init: > >>>> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 :: > >>>> http://ant.apache.org/ivy/ :: > >>>> :: loading settings :: file = > >>>> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml > >>>> > >>>> ivy-resolve-common: > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] You probably access the destination server through a > proxy > >>>> server that is not well configured. > >>>> [ivy:resolve] > >>>> [ivy:resolve] :: problems summary :: > >>>> [ivy:resolve] :::: WARNINGS > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> > >> > hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac > >>>> > >> > he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0 > >>>> -SNAPSHOT/maven-metadata.xml> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom< > >> https://re > >>>> > >> > pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo > >>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar< > >> https://re > >>>> > >> > pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo > >>>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS > >>>> HOT/hadoop-core-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/org/apache/ > >>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS > >>>> HOT/hadoop-core-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/org/apache/ > >>>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] module not found: > >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT > >>>> [ivy:resolve] ==== apache-snapshot: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom< > >> https://reposi > >>>> > >> > tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co > >>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar< > >> https://reposi > >>>> > >> > tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co > >>>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] ==== maven2: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/ > >>>> hadoop-core-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/org/apache/hado > >>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/ > >>>> hadoop-core-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/org/apache/hado > >>>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml< > >> https://repository > >>>> . > >> > apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t > >>>> est/0.22.0-SNAPSHOT/maven-metadata.xml> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> > >>>> > >> > hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom< > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha > >>>> > >> > doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> > >>>> > >> > hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar< > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha > >>>> > >> > doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0- > >>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/o > >>>> > >> > rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0- > >>>> SNAPSHOT.pom> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0- > >>>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/o > >>>> > >> > rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0- > >>>> SNAPSHOT.jar> > >>>> [ivy:resolve] module not found: > >>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT > >>>> [ivy:resolve] ==== apache-snapshot: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> > >> > op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http > >>>> s:// > >> > repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop > >>>> > /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> > org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> > >> > op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http > >>>> s:// > >> > repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop > >>>> > /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] ==== maven2: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP > >>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/org/a > >>>> > >> > pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP > >>>> SHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> > org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP > >>>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/org/a > >>>> > >> > pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP > >>>> SHOT.jar> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> > >> > hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac > >>>> > >> > he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0 > >>>> -SNAPSHOT/maven-metadata.xml> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom< > >> https://re > >>>> > >> > pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo > >>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] Host repository.apache.org not found. > >>>> url= > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/ > >>>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar< > >> https://re > >>>> > >> > pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo > >>>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS > >>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/org/apache/ > >>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] Host repo1.maven.org not found. > >>>> url= > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS > >>>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/org/apache/ > >>>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] module not found: > >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT > >>>> [ivy:resolve] ==== apache-snapshot: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom< > >> https://reposi > >>>> > >> > tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd > >>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar< > >> https://reposi > >>>> > >> > tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd > >>>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] ==== maven2: tried > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/ > >>>> hadoop-hdfs-0.22.0-SNAPSHOT.pom< > >> http://repo1.maven.org/maven2/org/apache/hado > >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom> > >>>> [ivy:resolve] -- artifact > >>>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar: > >>>> [ivy:resolve] > >>>> > >>>> > >> > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/ > >>>> hadoop-hdfs-0.22.0-SNAPSHOT.jar< > >> http://repo1.maven.org/maven2/org/apache/hado > >>>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar> > >>>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > >>>> [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: > >>>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > >>>> [ivy:resolve] :: > org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT: > >> not > >>>> found > >>>> [ivy:resolve] :: > >>>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT: > >>>> not found > >>>> [ivy:resolve] :: > org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: > >> not > >>>> found > >>>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > >>>> [ivy:resolve] > >>>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > >>>> > >>>> > >>>> > >>>> On 3/17/10 11:14 PM, "Amar Kamat" <[email protected]> wrote: > >>>> > >>>>> Use -Doffline=yes switch while running the tests. > >>>>> Amar > >>>>> > >>>>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <[email protected]> wrote: > >>>>> > >>>>> > >>>>> > >>>>> How do I disable the ivy download to run unit tests? I need to > replace > >>>>> hadoop-core.jar for one I created with some extra logging to debug a > >> unit > >>>> test > >>>>> for mapreduce. > >>>>> > >>>>> Right now I'm just erasing some lines from build.xml, but I'm quite > >> sure > >>>> there > >>>>> is an easier way to do that. > >>>>> > >>>>> Thanks, > >>>>> Rodrigo > >>>>> > >>>> > >>>> > >> > >> > >
