Solr 4.3 fails to load MySQL driver
Hi, in my attempt to migrate for m 3.6.x to 4.3.0 I stumbled upon an issue loading the MySQL driver from the [instance]/lib dir: Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.dataimport.DataImportHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:266) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:448) ... 18 more To narrow it down, I use the plain example configuration with the following changes: - Add a dataimport requestHandler to example/conf/solrconfig.xml (copied from a working solr 3.6.x) - Created example/conf/data-config.xml with dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver ... and SQL statement (both copied from a working solr 3.6.x) - placed the current driver mysql-connector-java-5.1.25-bin.jar in example/lib As to my knowledge the lib dir is included automatically to the path. To make sure I tried to: - add lib dir=./lib / to explicit to solrconf.xml - add absolute path to solrarconf.xml - changed solr.xml to use solr persistent=true sharedLib=lib All to no avail. System Info: - OpenJDK Runtime Environmentm 1.7.0_19 - Solr 4.3.0 - mysql-connector-java-5.1.25-bin.jar The same configuration run fine with a solr 3.6.x on the very same machine. Any help is appreciated! Cheers Chris -- Christian Köhler
Re: Solr 4.3 fails to load MySQL driver
Check the Solr log on startup - it will explicitly state which lib directories/files will be used. Make sure they agree with where the DIH jars reside. Keep in mind that the directory structure of Solr changed - use the lib from 4.3 solrconfig. Try to use DIH in the standard Solr 4.3 example first. Then mimic that in your customization. -- Jack Krupansky -Original Message- From: Christian Köhler Sent: Thursday, May 23, 2013 8:25 AM To: solr-user@lucene.apache.org Subject: Solr 4.3 fails to load MySQL driver Hi, in my attempt to migrate for m 3.6.x to 4.3.0 I stumbled upon an issue loading the MySQL driver from the [instance]/lib dir: Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.dataimport.DataImportHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:266) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:448) ... 18 more To narrow it down, I use the plain example configuration with the following changes: - Add a dataimport requestHandler to example/conf/solrconfig.xml (copied from a working solr 3.6.x) - Created example/conf/data-config.xml with dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver ... and SQL statement (both copied from a working solr 3.6.x) - placed the current driver mysql-connector-java-5.1.25-bin.jar in example/lib As to my knowledge the lib dir is included automatically to the path. To make sure I tried to: - add lib dir=./lib / to explicit to solrconf.xml - add absolute path to solrarconf.xml - changed solr.xml to use solr persistent=true sharedLib=lib All to no avail. System Info: - OpenJDK Runtime Environmentm 1.7.0_19 - Solr 4.3.0 - mysql-connector-java-5.1.25-bin.jar The same configuration run fine with a solr 3.6.x on the very same machine. Any help is appreciated! Cheers Chris -- Christian Köhler
Re: Solr 4.3 fails to load MySQL driver
On 5/23/2013 6:25 AM, Christian Köhler wrote:
in my attempt to migrate for m 3.6.x to 4.3.0 I stumbled upon an issue
loading the MySQL driver from the [instance]/lib dir:
Caused by: java.lang.ClassNotFoundException:
org.apache.solr.handler.dataimport.DataImportHandler
The best thing to do is take the lib directives out of solrconfig.xml
and put your extra jars in ${solr.solr.home}/lib, where solr.solr.home
is the directory where solr.xml lives. NB: There might be two solr.xml
files in your setup, but if there are, one of them will tell your
servlet container how to start solr, the correct file tells solr about
cores.
Normally, you can set up another global lib directory, absolute or
relative to solr.solr.home, with the sharedLib attribute in solr.xml,
but that doesn't work in 4.3.0 - only ${solr.solr.home}/lib works in
that specific version. Here's the bug report:
https://issues.apache.org/jira/browse/SOLR-4791
I discovered another glitch last night in the 4.4 development version
and filed a bug report, but I've been informed that I've been doing it
wrong for the last couple of years:
https://issues.apache.org/jira/browse/SOLR-4852
Thanks,
Shawn
Re: Solr 4.3 fails to load MySQL driver
Hi, thanx for pointing this out to me. 1152 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrConfig – Adding specified lib dirs to ClassLoader org.apache.solr.core.SolrResourceLoader – Adding 'file:/home/christian/zfmk/solr/solr-4.3.0/example/lib/mysql-connector-java-5.1.25-bin.jar' to classloader The mysql-connector-java DOES get loaded, but is not available to org.apache.solr.core.SolrResourceLoader.findClass Has something changed for the syntax creating a dataimport handler? solrconfig.xml: --- requestHandler name=/dataimport class=org.apache.solr.handler.dataimport.DataImportHandler lst name=defaults str name=configdata-config.xml/str /lst /requestHandler data-config.xml: dataConfig dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost/koehler_zfmk user=my_user password=secret/ document name=content entity name=rawidentificationid query=SELECT * FROM foobar; /entity /document /dataConfig I use this configuration successfully with 3.6 Regards Chris Am 23.05.2013 14:39, schrieb Jack Krupansky: Check the Solr log on startup - it will explicitly state which lib directories/files will be used. Make sure they agree with where the DIH jars reside. Keep in mind that the directory structure of Solr changed - use the lib from 4.3 solrconfig. Try to use DIH in the standard Solr 4.3 example first. Then mimic that in your customization. -- Jack Krupansky -Original Message- From: Christian Köhler Sent: Thursday, May 23, 2013 8:25 AM To: solr-user@lucene.apache.org Subject: Solr 4.3 fails to load MySQL driver Hi, in my attempt to migrate for m 3.6.x to 4.3.0 I stumbled upon an issue loading the MySQL driver from the [instance]/lib dir: Caused by: java.lang.ClassNotFoundException: org.apache.solr.handler.dataimport.DataImportHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:266) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:448) ... 18 more To narrow it down, I use the plain example configuration with the following changes: - Add a dataimport requestHandler to example/conf/solrconfig.xml (copied from a working solr 3.6.x) - Created example/conf/data-config.xml with dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver ... and SQL statement (both copied from a working solr 3.6.x) - placed the current driver mysql-connector-java-5.1.25-bin.jar in example/lib As to my knowledge the lib dir is included automatically to the path. To make sure I tried to: - add lib dir=./lib / to explicit to solrconf.xml - add absolute path to solrarconf.xml - changed solr.xml to use solr persistent=true sharedLib=lib All to no avail. System Info: - OpenJDK Runtime Environmentm 1.7.0_19 - Solr 4.3.0 - mysql-connector-java-5.1.25-bin.jar The same configuration run fine with a solr 3.6.x on the very same machine. Any help is appreciated! Cheers Chris -- Christian Köhler ganzgraph gmbh Bornheimer Straße 37 53111 Bonn koeh...@ganzgraph.de http://www.ganzgraph.de/ Tel.: +49-(0)228-227 99 400 Fax : +49-(0)228-227 99 409 Geschäftsführer: Christian Köhler, Thorsten Orth Unternehmenssitz: Bonn Handelsregister-Nummer: HRB 19066 beim Amtsgericht: Bonn UstId-Nr: DE 280482111
Re: Solr 4.3 fails to load MySQL driver
: in my attempt to migrate for m 3.6.x to 4.3.0 I stumbled upon an issue loading : the MySQL driver from the [instance]/lib dir: : : Caused by: java.lang.ClassNotFoundException: : org.apache.solr.handler.dataimport.DataImportHandler one of us is mistaken by what that error means. you say it means that the MySQL driver isn't being loaded, but nothing in your mail suggests to me that there is a problem loading hte MySql driver. what i see is that Solr can't seem to load the DIH class, suggesting that the dataimporthandler jar is not getting loaded. There may or nay not also be a problem loading the MySQL driver, but nothing is even going to attempt to do so unless Solr can successfully construct an instance of the DataImportHandler. So unless there are more details to your error that start mentioning the MySql classes, i would check your lib settings for loading the DIH jars and make sure those are right. -Hoss
Re: Solr 4.3 fails to load MySQL driver
Hi
one of us is mistaken by what that error means. you say it means that
the MySQL driver isn't being loaded, but nothing in your mail suggests
to me that there is a problem loading hte MySql driver. what i see is
that Solr can't seem to load the DIH class, suggesting that the
dataimporthandler jar is not getting loaded.
I corrected myself in my last mail: the MySQL driver IS loaded (thanx
for pointing out to me where to look).
There may or nay not also be a problem loading the MySQL driver, but
I only SUSPECT of the MySQL driver being the culprit for the
dataimporthandler jar is not getting loaded. Not sure!
MySql classes, i would check your lib settings for loading the DIH
jars
I am not using DIH. IMHO its just the plain example code in
solr-4.3.0/example/solr/collection1/ that is being called.
I include the full trace back to clarify my problem (hopefully)
Cheers
Chris
/home/solr-4.3.0/example# java -jar start.jar
0[main] INFO org.eclipse.jetty.server.Server – jetty-8.1.8.v20121106
19 [main] INFO org.eclipse.jetty.deploy.providers.ScanningAppProvider
– Deployment monitor /home/solr/solr-4.3.0/example/contexts at interval 0
24 [main] INFO org.eclipse.jetty.deploy.DeploymentManager –
Deployable added:
/home/solr/solr-4.3.0/example/contexts/solr-jetty-context.xml
653 [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor
– NO JSP Support for /solr, did not find
org.apache.jasper.servlet.JspServlet
Null identity service, trying login service: null
Finding identity service: null
674 [main] INFO org.eclipse.jetty.server.handler.ContextHandler –
started
o.e.j.w.WebAppContext{/solr,file:/home/solr/solr-4.3.0/example/solr-webapp/webapp/},/home/solr/solr-4.3.0/example/webapps/solr.war
674 [main] INFO org.eclipse.jetty.server.handler.ContextHandler –
started
o.e.j.w.WebAppContext{/solr,file:/home/solr/solr-4.3.0/example/solr-webapp/webapp/},/home/solr/solr-4.3.0/example/webapps/solr.war
688 [main] INFO org.apache.solr.servlet.SolrDispatchFilter –
SolrDispatchFilter.init()
703 [main] INFO org.apache.solr.core.SolrResourceLoader – JNDI not
configured for solr (NoInitialContextEx)
704 [main] INFO org.apache.solr.core.SolrResourceLoader – solr home
defaulted to 'solr/' (could not find system property or JNDI)
713 [main] INFO org.apache.solr.core.CoreContainer – looking for solr
config file: /home/solr/solr-4.3.0/example/solr/solr.xml
715 [main] INFO org.apache.solr.core.CoreContainer – New
CoreContainer 1857140958
716 [main] INFO org.apache.solr.core.CoreContainer – Loading
CoreContainer using Solr Home: 'solr/'
716 [main] INFO org.apache.solr.core.SolrResourceLoader – new
SolrResourceLoader for directory: 'solr/'
962 [main] INFO org.apache.solr.core.CoreContainer – loading shared
library: /home/solr/solr-4.3.0/example/solr/lib
962 [main] ERROR org.apache.solr.core.SolrResourceLoader – Can't find
(or read) file to add to classloader: solr/lib
971 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
socketTimeout to: 0
973 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
urlScheme to: http://
973 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
connTimeout to: 0
974 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
maxConnectionsPerHost to: 20
974 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
corePoolSize to: 0
974 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
maximumPoolSize to: 2147483647
974 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
maxThreadIdleTime to: 5
974 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
sizeOfQueue to: -1
975 [main] INFO
org.apache.solr.handler.component.HttpShardHandlerFactory – Setting
fairnessPolicy to: false
980 [main] INFO org.apache.solr.client.solrj.impl.HttpClientUtil –
Creating new http client,
config:maxConnectionsPerHost=20maxConnections=1socketTimeout=0connTimeout=0retry=false
1073 [main] INFO org.apache.solr.core.CoreContainer – Registering Log
Listener
1087 [coreLoadExecutor-3-thread-1] INFO
org.apache.solr.core.CoreContainer – Creating SolrCore 'collection1'
using instanceDir: solr/collection1
1088 [coreLoadExecutor-3-thread-1] INFO
org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for
directory: 'solr/collection1/'
1143 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrConfig
– Adding specified lib dirs to ClassLoader
1144 [coreLoadExecutor-3-thread-1] INFO
org.apache.solr.core.SolrResourceLoader – Adding
'file:/home/solr/solr-4.3.0/example/lib/jetty-util-8.1.8.v20121106.jar'
to classloader
1144 [coreLoadExecutor-3-thread-1] INFO
org.apache.solr.core.SolrResourceLoader – Adding
'file:/home/solr/solr-4.3.0/example/lib/servlet-api-3.0.jar' to
Re: Solr 4.3 fails to load MySQL driver
: I only SUSPECT of the MySQL driver being the culprit for the dataimporthandler : jar is not getting loaded. Not sure! the dataimporthandler *class* is not getting loaded the dataimporthandler *jar* is not getting loaded. : MySql classes, i would check your lib settings for loading the DIH : jars : : I am not using DIH. IMHO its just the plain example code in : solr-4.3.0/example/solr/collection1/ that is being called. i'm totally confused ... DIH == DataImportHandler ... it's just an acronym, you say you aren't using DIH, but you are having a problem loading DIH, so DIH is used in your configs. : I include the full trace back to clarify my problem (hopefully) ... : org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for : directory: 'solr/collection1/' : 1143 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrConfig – : Adding specified lib dirs to ClassLoader : 1144 [coreLoadExecutor-3-thread-1] INFO : org.apache.solr.core.SolrResourceLoader – Adding : 'file:/home/solr/solr-4.3.0/example/lib/jetty-util-8.1.8.v20121106.jar' to : classloader ...ok, for starters this makes no sense, and may be the cause of some problems. you aparently have your collection1 configs setup to load all of the classes from the /home/solr/solr-4.3.0/example/example/lib directory as part of the collection1 classloader. you really don't want to do that. It will most likeley cause you all sorts of problems, even if it's unrelated to the current problem. Second, note in particular all of the lines that look like that line above -- specifically lines that say org.apache.solr.core.SolrResourceLoader - Addming to classloader. besides the ones refering to /home/solr/solr-4.3.0/example/lib/ (which is almost certainly not what you want) you then have a bunch refering to contrib/extraction and contrib/langid, and contrib/velocity -- all of which is great, those plugins and their dependencies are now available to use. but no where does it ever say anything about adding contrib/dataimporthandler jars to the classloader. which means your config isn't setup to load any of hte dataimporthandler jars as plugins which means when it's done loading plugins, and it starts to initialize things like RequestHandlers, and it finds a refrence to the DataImportHandler, it doesn't know what that means... : Caused by: java.lang.ClassNotFoundException: : org.apache.solr.handler.dataimport.DataImportHandler if you look at the 4.3 DIH examples, you'll note that the only solrconfig.xml files that mention DataImportHandler also include lib directives like the following in order to load dataimporthandler as a plugin... lib dir=../../../../dist/ regex=solr-dataimporthandler-.*\.jar / ... requestHandler name=/dataimport class=org.apache.solr.handler.dataimport.DataImportHandler -Hoss
Re: Solr 4.3 fails to load MySQL driver
Hi, i'm totally confused ... DIH == DataImportHandler ... it's just an acronym, you say you aren't using DIH, but you are having a problem loading DIH, so DIH is used in your configs. sorry for the confusion. I was just trying to say: I use the example code from solr-4.3.0/example/solr and not from solr-4.3.0/example/example-DIH ...ok, for starters this makes no sense, and may be the cause of some problems. you aparently have your collection1 configs setup to load all of the classes from the /home/solr/solr-4.3.0/example/example/lib directory as part of the collection1 classloader. you really don't want to do that. It will most likeley cause you all sorts of problems, even if it's unrelated to the current problem. For solr is was recomended to place the MySQL driver in solr_3.6.2/example/lib/ This dir is load by default in 3.6 (as I did not add any additional lib dirs). Thats why I did this in 4.3 as well. What's the best practice to place third party libs? I added example/lib/ to collection1/conf/solrconfig.xml as lib dir Without this, the MySQL driver is not loaded according to the org.apache.solr.core.SolrResourceLoader – Adding xxx messages but no where does it ever say anything about adding contrib/dataimporthandler jars to the classloader. collection1/conf/solrconfig.xml has the following lib dirs by default: lib dir=../../../contrib/extraction/lib regex=.*\.jar / lib dir=../../../dist/ regex=solr-cell-\d.*\.jar / lib dir=../../../contrib/clustering/lib/ regex=.*\.jar / lib dir=../../../dist/ regex=solr-clustering-\d.*\.jar / lib dir=../../../contrib/langid/lib/ regex=.*\.jar / lib dir=../../../dist/ regex=solr-langid-\d.*\.jar / lib dir=../../../contrib/velocity/lib regex=.*\.jar / lib dir=../../../dist/ regex=solr-velocity-\d.*\.jar / Looks the same to me as in 3.6. which means your config isn't setup to load any of hte dataimporthandler jars as plugins That means I have to configure the dataimporthandler manually in 4.3? If yes, this is the root of all problems ... which means when it's done loading plugins, and it starts to initialize things like RequestHandlers, and it finds a refrence to the DataImportHandler, it doesn't know what that means... : Caused by: java.lang.ClassNotFoundException: : org.apache.solr.handler.dataimport.DataImportHandler if you look at the 4.3 DIH examples, you'll note that the only solrconfig.xml files that mention DataImportHandler also include lib directives like the following in order to load dataimporthandler as a plugin... lib dir=../../../../dist/ regex=solr-dataimporthandler-.*\.jar / included this ... to no avail. requestHandler name=/dataimport class=org.apache.solr.handler.dataimport.DataImportHandler requestHandler name=/dataimport class=org.apache.solr.handler.dataimport.DataImportHandler still does not load. Regards Chris
