Re: Loading mechnism of plugin classes and singleton objects

Enzo Michelangeli Fri, 08 Jun 2007 03:54:37 -0700

----- Original Message -----From: "Doğacan Güney" <[EMAIL PROTECTED]>

Sent: Friday, June 08, 2007 3:49 PM

[...]

Any idea?


This will certainly help a lot. If it is not too much trouble, can you
add debug outputs for hashCodes of conf objects (both for the one in
the cache and for the parameter, because it seems Configuration object
is created more than once so their hashCode may be different, which in
turn causes the change in CACHE's hashCode(*)) and a stack trace?
A stack trace of depth 2-3 will probably suffice, I am just wondering
what is calling PluginRepository.get(conf).


OK, I changed my debug code as follows:

 public static synchronized PluginRepository get(Configuration conf) {
   PluginRepository result = CACHE.get(conf);
       /* --- start debug code */
       String tr = "";
       StackTraceElement[] tes = Thread.currentThread().getStackTrace();
       for(int j=2; j<tes.length; j++)
           tr = tr+"\n    "+tes[j].toString();
       LOG.info("In thread "+Thread.currentThread()+
                " a static method of the class "+
                (new CurrentClassGetter()).getCurrentClass()+
                " called CACHE.get("+conf+
                "), where CACHE is "+CACHE+
                " and CACHE.hashCode() = "+CACHE.hashCode()+
                " - got result = "+result+
                " conf.hashCode() was: "+conf.hashCode()+
                " hashCode was: "+conf.hashCode()+
                " Stack Trace:"+tr);
       /* end debug code --- */
   if (result == null) {
     result = new PluginRepository(conf);
     CACHE.put(conf, result);
   }
   return result;
 }

 /* --- start debug code */
 public static class CurrentClassGetter extends SecurityManager {
   public String getCurrentClass() {
     Class cl = super.getClassContext()[1];
     return cl.toString() + "@" + cl.hashCode();
   }
 }
 /* end debug code --- */

(With full stack trace: bytes are cheap ;-) )

I did not bother to print the hashCode of the keys in CACHE because it's
become evident why CACHE.get(conf) returns null: the hashCode of conf
changes! This is strange, because, as you can see below, the strings that

make keys and values of conf appears unchanged. Perhaps we should overridethe equals() method in org.apache.hadoop.conf.Configuration (invoked byCACHE.get(), according to the specs of the java.util.Map interface), so thatthe hashCode()s of the keys get ignored, and conf1.equals(conf2) return trueif and only if:


1. conf1.size() == conf2.size(),

2. for each key k1 of conf1 there is a key k2 in conf2 such as:
 2.1 k1.equals(k2)
 2.2 conf1.get(k1).equals(conf2.get(k2))

Anyway, I'm attaching the log below.

Thanks for the detailed analysis!


Glad to be of help!

Enzo

2007-06-08 17:24:39,211 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:39,231 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:39,802 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:39,802 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
27058272 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:43,618 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 23315571 - got result = null conf.hashCode() was: 7461949
Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:46)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:43,848 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:43,858 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:43,918 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:43,918 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
7461949 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:44,299 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 54703198 - got result = null conf.hashCode() was:
19647819 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:46)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:44,499 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:44,509 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.urlfilter.geoip.GeoIpURLFilter.setConf(GeoIpURLFilter.java:252)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:44,599 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:24:44,599 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result =
[EMAIL PROTECTED] conf.hashCode() was:
19647819 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   
org.apache.nutch.plugin.PluginDescriptor.collectLibs(PluginDescriptor.java:309)
   
org.apache.nutch.plugin.PluginDescriptor.getDependencyLibs(PluginDescriptor.java:298)
   
org.apache.nutch.plugin.PluginDescriptor.getClassLoader(PluginDescriptor.java:277)
   org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:155)
   org.apache.nutch.net.URLFilters.<init>(URLFilters.java:54)
   org.apache.nutch.crawl.CrawlDbFilter.configure(CrawlDbFilter.java:66)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)

2007-06-08 17:25:22,694 INFO  plugin.PluginRepository - In thread
Thread[Thread-0,5,main] a static method of the class class
[EMAIL PROTECTED] called
CACHE.get(Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal: hadoop-site.xml), where CACHE is {Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED],
Configuration: defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop-ADMIN/mapred/local/localRunner/job_rafehc.xml ,
mapred-default.xmlfinal:
[EMAIL PROTECTED] and
CACHE.hashCode() = 64604955 - got result = null conf.hashCode() was:
27692793 Stack Trace:
   org.apache.nutch.plugin.PluginRepository.get(PluginRepository.java:99)
   org.apache.nutch.scoring.ScoringFilters.<init>(ScoringFilters.java:59)
   org.apache.nutch.crawl.CrawlDbReducer.configure(CrawlDbReducer.java:46)
   org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
   org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82)
   org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:217)
   org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:155)

Re: Loading mechnism of plugin classes and singleton objects

Reply via email to