FYI: Re: Adding new plugin and classloading issues
One useful piece of info about using mail.jar as a Nutch plugin: If mail.jar is installed as a plugin, then the META-INF/mailcap file may not be found from mail.jar. javax.activation.MailcapCommandMap searches mailcap from a number of places, but it will call SecuritySupport.getContextClassLoader() to load mailcap from 'resources'. This returns Thread.currentThread().getContextClassLoader() to get the classloader, which is not the PluginClassLoader, so does not find it. Antony Antony Bowesman wrote: Argh, sorry for the noise! After lots of digging, I found that my plugin classes were also in my app jar file, so it was loaded by the parent class loader, not the plugin class loader. Regards Antony
Re: Adding new plugin and classloading issues
Argh, sorry for the noise! After lots of digging, I found that my plugin classes were also in my app jar file, so it was loaded by the parent class loader, not the plugin class loader. Regards Antony
Re: Adding new plugin and classloading issues
D:\Toss-1.3.3x\plugins>ls -l parse-mime total 2120 -rwxrwxrwx 1 adb None 217843 Jan 23 15:42 bcmail-jdk16-141.jar -rwxrwxrwx 1 adb None 1680122 Jan 23 15:42 bcprov-jdk16-141.jar -rwxrwxrwx 1 adb None 371264 May 28 2008 mail.jar -rwxrwxrwx 1 adb None5648 Jan 23 15:44 parse-mime.jar -rwxrwxrwx 1 adb None1057 Jan 23 18:30 plugin.xml Rgds Antony Doğacan Güney wrote: Can you check build/plugins/ to see if mail.jar is there? dogacan-mbp:~/Documents/kod/nutch-svn$ ls build/plugins/parse-pdf/ FontBox-0.1.0-dev.jar parse-pdf.jar PDFBox-0.7.3.jarplugin.xml It should be there, based on the files you sent, but maybe there is a bug there in nutch we are missing. On Sun, Jan 25, 2009 at 11:47 PM, Antony Bowesman wrote: Otis Gospodnetic wrote: Step one is to identify the exact jar where this class lives. Are you sure it's in mail.jar? Maybe it's in activate.jar? $ jar tvf mail.jar | grep ParseException 385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class It's javamail 1.4.1. If I put mail.jar into my application's lib directory, where all jars are put on the classpath it finds the class. However, then it fails on another class from the BouncyCastle jars, also part of the plugin, so it's some Nutch issue. In the debugger, it shows the plugin descriptor's classloader having the 4 libraries in the loader.ucp.path ArrayList. ExtensionClass.getExtensionInstance() does the following Class extensionClazz = loader.loadClass(getClazz()); // lazy loading of Plugin in case there is no instance of the plugin // already. this.pluginRepository.getPluginInstance(getDescriptor()); Object object = extensionClazz.newInstance(); and it is the call to newInstance() which gives the CNF exception. The final entry in the stack is Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the arg is javax.mail.internet.ParseException. I've not managed to figure out what part of the object creation the Nutch classload takes part in, but it seems it doesn't have a lot to do with the classloading for the plugins. Antony Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Antony Bowesman To: nutch-user@lucene.apache.org Sent: Friday, January 23, 2009 2:49:48 AM Subject: Adding new plugin and classloading issues I'm using the 0.9 plugin framework and am trying to add a new plugin and am having classloading issues. I want to use Javamail's mail.jar and BouncyCastle jars. My plugin implemnentation catches javax.mail.internet.ParseException and when it tries to create the new instance of my Parser, it gives java.lang.NoClassDefFoundError: javax/mail/internet/ParseException at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389) at java.lang.Class.getConstructor0(Class.java:2699) at java.lang.Class.newInstance0(Class.java:326) at java.lang.Class.newInstance(Class.java:308) at org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160) at org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130) My plugin.xml has I have tried it with the export for all libraries, also by having a lib-javamail plugin, which does an export for mail.jar and with my plugin having the but it just doesn't work. It seems identical to the standard PDF parsing or the POI based parsers, so I'm at a loss... It's a stand alone app, no Tomcat or anything else. Any ideas? Antony
Re: Adding new plugin and classloading issues
Can you check build/plugins/ to see if mail.jar is there? dogacan-mbp:~/Documents/kod/nutch-svn$ ls build/plugins/parse-pdf/ FontBox-0.1.0-dev.jar parse-pdf.jar PDFBox-0.7.3.jarplugin.xml It should be there, based on the files you sent, but maybe there is a bug there in nutch we are missing. On Sun, Jan 25, 2009 at 11:47 PM, Antony Bowesman wrote: > Otis Gospodnetic wrote: >> >> Step one is to identify the exact jar where this class lives. Are you >> sure it's in mail.jar? Maybe it's in activate.jar? > > $ jar tvf mail.jar | grep ParseException > 385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class > > It's javamail 1.4.1. If I put mail.jar into my application's lib directory, > where all jars are put on the classpath it finds the class. However, then > it fails on another class from the BouncyCastle jars, also part of the > plugin, so it's some Nutch issue. > > In the debugger, it shows the plugin descriptor's classloader having the 4 > libraries in the loader.ucp.path ArrayList. > > ExtensionClass.getExtensionInstance() does the following > >Class extensionClazz = loader.loadClass(getClazz()); >// lazy loading of Plugin in case there is no instance of the plugin >// already. >this.pluginRepository.getPluginInstance(getDescriptor()); >Object object = extensionClazz.newInstance(); > > and it is the call to newInstance() which gives the CNF exception. The > final entry in the stack is > > Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the > arg is javax.mail.internet.ParseException. > > I've not managed to figure out what part of the object creation the Nutch > classload takes part in, but it seems it doesn't have a lot to do with the > classloading for the plugins. > > Antony > > >> >> Otis >> -- >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >> >> >> >> - Original Message >>> >>> From: Antony Bowesman >>> To: nutch-user@lucene.apache.org >>> Sent: Friday, January 23, 2009 2:49:48 AM >>> Subject: Adding new plugin and classloading issues >>> >>> I'm using the 0.9 plugin framework and am trying to add a new plugin and >>> am having classloading issues. >>> >>> I want to use Javamail's mail.jar and BouncyCastle jars. My plugin >>> implemnentation catches javax.mail.internet.ParseException and when it tries >>> to create the new instance of my Parser, it gives >>> >>> java.lang.NoClassDefFoundError: javax/mail/internet/ParseException >>>at java.lang.Class.getDeclaredConstructors0(Native Method) >>>at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389) >>>at java.lang.Class.getConstructor0(Class.java:2699) >>>at java.lang.Class.newInstance0(Class.java:326) >>>at java.lang.Class.newInstance(Class.java:308) >>>at >>> org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160) >>>at >>> org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130) >>> >>> My plugin.xml has >>> >>> >>> I have tried it with the export for all libraries, also by having a >>> lib-javamail plugin, which does an export for mail.jar and with my plugin >>> having the >>> >>> >>> but it just doesn't work. It seems identical to the standard PDF parsing >>> or the POI based parsers, so I'm at a loss... >>> >>> It's a stand alone app, no Tomcat or anything else. >>> >>> Any ideas? >>> Antony >> >> > > -- Doğacan Güney
Re: Adding new plugin and classloading issues
Otis Gospodnetic wrote: Step one is to identify the exact jar where this class lives. Are you sure it's in mail.jar? Maybe it's in activate.jar? $ jar tvf mail.jar | grep ParseException 385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class It's javamail 1.4.1. If I put mail.jar into my application's lib directory, where all jars are put on the classpath it finds the class. However, then it fails on another class from the BouncyCastle jars, also part of the plugin, so it's some Nutch issue. In the debugger, it shows the plugin descriptor's classloader having the 4 libraries in the loader.ucp.path ArrayList. ExtensionClass.getExtensionInstance() does the following Class extensionClazz = loader.loadClass(getClazz()); // lazy loading of Plugin in case there is no instance of the plugin // already. this.pluginRepository.getPluginInstance(getDescriptor()); Object object = extensionClazz.newInstance(); and it is the call to newInstance() which gives the CNF exception. The final entry in the stack is Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the arg is javax.mail.internet.ParseException. I've not managed to figure out what part of the object creation the Nutch classload takes part in, but it seems it doesn't have a lot to do with the classloading for the plugins. Antony Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Antony Bowesman To: nutch-user@lucene.apache.org Sent: Friday, January 23, 2009 2:49:48 AM Subject: Adding new plugin and classloading issues I'm using the 0.9 plugin framework and am trying to add a new plugin and am having classloading issues. I want to use Javamail's mail.jar and BouncyCastle jars. My plugin implemnentation catches javax.mail.internet.ParseException and when it tries to create the new instance of my Parser, it gives java.lang.NoClassDefFoundError: javax/mail/internet/ParseException at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389) at java.lang.Class.getConstructor0(Class.java:2699) at java.lang.Class.newInstance0(Class.java:326) at java.lang.Class.newInstance(Class.java:308) at org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160) at org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130) My plugin.xml has I have tried it with the export for all libraries, also by having a lib-javamail plugin, which does an export for mail.jar and with my plugin having the but it just doesn't work. It seems identical to the standard PDF parsing or the POI based parsers, so I'm at a loss... It's a stand alone app, no Tomcat or anything else. Any ideas? Antony
Re: Adding new plugin and classloading issues
Step one is to identify the exact jar where this class lives. Are you sure it's in mail.jar? Maybe it's in activate.jar? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Antony Bowesman > To: nutch-user@lucene.apache.org > Sent: Friday, January 23, 2009 2:49:48 AM > Subject: Adding new plugin and classloading issues > > I'm using the 0.9 plugin framework and am trying to add a new plugin and am > having classloading issues. > > I want to use Javamail's mail.jar and BouncyCastle jars. My plugin > implemnentation catches javax.mail.internet.ParseException and when it tries > to > create the new instance of my Parser, it gives > > java.lang.NoClassDefFoundError: javax/mail/internet/ParseException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389) > at java.lang.Class.getConstructor0(Class.java:2699) > at java.lang.Class.newInstance0(Class.java:326) > at java.lang.Class.newInstance(Class.java:308) > at > org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160) > at > org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130) > > My plugin.xml has > > > > > > > > > > > I have tried it with the export for all libraries, also by having a > lib-javamail > plugin, which does an export for mail.jar and with my plugin having the > > > > > > but it just doesn't work. It seems identical to the standard PDF parsing or > the > POI based parsers, so I'm at a loss... > > It's a stand alone app, no Tomcat or anything else. > > Any ideas? > Antony
