Jacques Le Roux created OFBIZ-11156:
---------------------------------------
Summary: Issue loading solr component (JNDI timeout)
Key: OFBIZ-11156
URL: https://issues.apache.org/jira/browse/OFBIZ-11156
Project: OFBiz
Issue Type: Bug
Components: solr
Affects Versions: Trunk
Reporter: Jacques Le Roux
This follows the discussion in dev ML at [https://s.apache.org/358zz]
Trunk HEAD, R17 and R18 are affected. R16 has a much lesser issue that can be
neglected.
{noformat}
2019-08-16 09:14:14,453 |main |ConfigXMLReader
|I| controller loaded: 0.003s, 207 requests, 81 views in
file:/C:/projectsASF/release18.12/plugins/scrum/webapp/scrum/WEB-INF/controller.xml
2019-08-16 09:17:16,061 |main |ServiceDispatcher
|I| Registering dispatcher: solr
2019-08-16 09:24:10,541 |main |ConfigXMLReader
|I| controller loaded: 0.004s, 207 requests, 81 views in
file:/C:/projectsASF/release17.12/plugins/scrum/webapp/scrum/WEB-INF/controller.xml
2019-08-16 09:27:12,125 |main |ServiceDispatcher
|I| Registering dispatcher: solr
{noformat}
2 minutes to register solr on my machine!
The much lesser issue on my machine and demo with R16:
{noformat}
2019-08-16 09:30:54,464 |0.0.0.0-startStop-1 |ConfigXMLReader
|I| controller loaded: 0.004s, 207 requests, 81 views in
file:/C:/projectsASF/release16.11/specialpurpose/scrum/webapp/scrum/WEB-INF/controller.xml
2019-08-16 09:31:55,069 |0.0.0.0-startStop-1 |SolrResourceLoader
|W| Can't find (or read) directory to add to classloader: lib (resolved as:
C:\projectsASF\release16.11\specialpurpose\solr\home\lib).
2019-08-16 09:31:55,239 |0.0.0.0-startStop-1 |ServiceDispatcher
|I| Registering dispatcher: solr
2019-08-16 03:12:33,016 |0.0.0.0-startStop-1 |ConfigXMLReader
|I| controller loaded: 0.005s, 207 requests, 81 views in
file:/home/ofbizDemo/branch16.11/specialpurpose/scrum/webapp/scrum/WEB-INF/controller.xml
2019-08-16 03:12:33,655 |0.0.0.0-startStop-1 |SolrResourceLoader
|W| Can't find (or read) directory to add to classloader: lib (resolved as:
/home/ofbizDemo/branch16.11/specialpurpose/solr/home/lib).
2019-08-16 03:12:33,879 |0.0.0.0-startStop-1 |ServiceDispatcher
|I| Registering dispatcher: solr
{noformat}
600ms is barely an issue. So I think we can agree it's OK there, anyway nobody
never complained.
In trunk, R17 and R18, it's due to SolrResourceLoader.locateSolrHome() trying
to locate Solr home dir by 1st doing a JNDI lookup, then a system property and
eventually harcode it to "solr/"
[JNDI lookups can be slow|https://www.beyondjava.net/slow-jndi-lookups]. But
here we speak about minutes because in Solr there are several calls w/o caching.
I thought that by avoiding the JNDI lookup done in
{{OFBizSolrContextFilter::init}} (through {{super.init(config);}} I'd fix the
issue. So I wrote this
{code:java}
Index: config/solrconfig.properties
===================================================================
--- config/solrconfig.properties (revision 1864966)
+++ config/solrconfig.properties (working copy)
@@ -50,3 +50,7 @@
solr.log.dir=runtime/logs/solr
# Defines Solr log level
solr.log.level=INFO
+
+# Defines Solr home directory
+solr.solr.home=plugins/solr/home
+
Index: src/main/java/org/apache/ofbiz/solr/webapp/OFBizSolrContextFilter.java
===================================================================
--- src/main/java/org/apache/ofbiz/solr/webapp/OFBizSolrContextFilter.java
(revision 1864966)
+++ src/main/java/org/apache/ofbiz/solr/webapp/OFBizSolrContextFilter.java
(working copy)
@@ -66,6 +66,7 @@
Properties props = System.getProperties();
props.setProperty("solr.log.dir",
UtilProperties.getPropertyValue("solrconfig", "solr.log.dir",
"runtime/logs/solr"));
props.setProperty("solr.log.level",
UtilProperties.getPropertyValue("solrconfig", "solr.log.level", "INFO"));
+ config.getServletContext().setAttribute("solr.solr.home",
UtilProperties.getPropertyValue("solrconfig", "solr.solr.home",
"plugins/solr"));
super.init(config);
}
{code}
to prevent the {{SolrResourceLoader.locateSolrHome()}} calls in
{{SolrDispatchFilter::init}}
It worked but it's obviously not enough. It's then 1 vs 3 minutes before, JNDI
timeout I guess. Here it is seen with {{<logger name="org.apache.solr"
level="all"/>}} in log4j2.xml)
{noformat}
2019-08-16 12:03:39,402 |main |SolrDispatchFilter
|I| ___ _ Welcome to Apache Solr? version 8.2.0
2019-08-16 12:03:39,402 |main |SolrDispatchFilter
|I| / __| ___| |_ _ Starting in standalone mode on port null
2019-08-16 12:03:39,402 |main |SolrDispatchFilter
|I| \__ \/ _ \ | '_| Install dir: null
2019-08-16 12:03:39,403 |main |SolrDispatchFilter
|I| |___/\___/_|_| Start time: 2019-08-16T10:03:39.402Z
2019-08-16 12:03:39,403 |main |SolrDispatchFilter
|I| Log level override, property solr.log.level=INFO
2019-08-16 12:03:39,412 |main |SolrResourceLoader
|D| new SolrResourceLoader for directory:
'C:\projectsASF\ofbiz\plugins\solr\home'
2019-08-16 12:04:39,421 |main |SolrResourceLoader
|D| No /solr/home in JNDI
2019-08-16 12:04:39,421 |main |SolrResourceLoader
|I| solr home defaulted to 'solr/' (could not find system property or JNDI)
2019-08-16 12:04:39,428 |main |SolrXmlConfig
|I| Loading container configuration from
C:\projectsASF\ofbiz\plugins\solr\home\solr.xml
{noformat}
There are 3 JNDI lookups in Solr (at least with our use of Solr) that locate
the Solr home dir, hence the 2 minutes before my change. The JNDI context nor
result are cached. BTW I believe Solr could do better by using
[https://docs.oracle.com/javase/8/docs/api/javax/naming/InitialContext.html#doLookup-java.lang.String-]
in SolrResourceLoader.locateSolrHome() and caching but that's out of subject.
I then tried to find a way to [Configure Solr Home with
JNDI|https://cwiki.apache.org/confluence/display/solr/SolrTomcat#SolrTomcat-ConfiguringSolrHomewithJNDI].
I tried many ways but none worked so far. Among them:
# As a try: add an
[<env-entry>|https://www.oracle.com/webfolder/technetwork/tutorials/obe/java/env_entry/env_entry.html#t3]
in solr component web.xml file
{code:xml}
IIndex: web.xml
===================================================================
--- web.xml (revision 1864966)
+++ web.xml (working copy)
@@ -20,6 +20,12 @@
<display-name>Apache OFBiz - Solr Component</display-name>
<description>Solr Component of the Apache OFBiz Project</description>
+ <env-entry>
+ <env-entry-name>solr/home</env-entry-name>
+
<env-entry-value>C:\projectsASF\ofbiz\plugins\solr\home</env-entry-value>
+ <env-entry-type>java.lang.String</env-entry-type>
+ </env-entry>
+
<context-param>
<param-name>entityDelegatorName</param-name>
<param-value>default</param-value>
{code}
Not sure why it did not work
# I tried to set in jndi.properties (though I believe it's not for JNDI env
but ressource), like
{noformat}
solr/home= C:\projectsASF\ofbiz\plugins\solr\home
{noformat}
# Used OFBIZ-9484 to grab the content of a context.xml file:
{code:java}
@@ -514,6 +517,23 @@
context.setDisplayName(appInfo.name);
context.setPath(getWebappMountPoint(appInfo));
context.addLifecycleListener(new ContextConfig());
+
+ // adding webapp's META-INF/context.xml begin
+ String contextXmlFilePath = new
StringBuilder().append("file:///").append(location).append("/").append(Constants.ApplicationContextXml).toString();
+ URL contextXmlUrl = null;
+ try {
+ contextXmlUrl =
FlexibleLocation.resolveLocation(contextXmlFilePath);
+ contextXmlFilePath = new
StringBuilder().append(location).append("/").append(Constants.ApplicationContextXml).toString();
+ File contextXmlFile = FileUtil.getFile(contextXmlFilePath);
+ if(contextXmlFile.exists() && contextXmlFile.isFile()) {
+ Debug.logInfo(contextXmlFilePath + " found and will be
loaded.", module);
+ context.setConfigFile(contextXmlFilePath);
+ }
+ } catch (MalformedURLException e) {
+ Debug.logInfo(contextXmlFilePath+ " not found.", module);
+ }
+ // add webapp's META-INF/context.xml end
+
context.setJ2EEApplication("OFBiz");
context.setJ2EEServer("OFBiz Container");
context.setLoader(new
WebappLoader(Thread.currentThread().getContextClassLoader()));
{code}
And following
[https://tomcat.apache.org/tomcat-9.0-doc/config/context.html#Environment_Entries],
added
{code:xml}
<Context>
<Environment name="solr/home"
value="C:\projectsASF\ofbiz\plugins\solr\home" type="java.lang.String"
description="Solr home for JNDI"/>
</Context>
{code}
But reading
[https://docs.oracle.com/javase/jndi/tutorial/beyond/env/update.html] I'm
unsure {{context.setConfigFile(contextXmlFilePath);}} does the work of loading
this Environment
I don't see a way to set a short timeout on the JNDI lookup. A [solution like
that|https://stackoverflow.com/questions/45837807/how-to-set-jndi-lookup-timeout-on-glassfish]
could be used. That would need to be in Solr...
Getting back to R16 for lucene and solr component could maybe be a way, but
it's really bad then. Another option is to not use Solr if you don't need it ;)
Just before sending this comment I looked into our archive and BAM! Just 6
years ago: point 2 of my comment: [https://s.apache.org/358zz]
If I miss something it's very well hidden, all ideas are welcome.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
