[
https://issues.apache.org/jira/browse/SOLR-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658215#action_12658215
]
Kay Kay commented on SOLR-922:
------------------------------
The value add of Solr by providing a single point of ExecutorService is to
essentially have a single place of ExecutorPolicy / performance tuning for
thread dump / out of resource issue.
* SolrCore: I am a bit reluctant on this one especially with the design of
multi SolrCore because that would mean managing the ExecutorService member when
it is shut down (close ). This becomes all the more pronounced when SolrCores
are re-registered for the same name. I am bit reluctant to go via this path.
* ServletFilter / Outside the web-app : This would be the ideal place for this
- but ideally if it were a non-Singleton pattern - it would be great. This
needs more investigation though.
* CoreContainer : Given that CoreContainer is more or less a singleton
instance (except when used by non-HTTP filter - when another instance seems to
be created ) - this can serve the purpose for most of the cases. (at least 4 of
the 6 mentioned to begin with ).
This becomes all the more pronounced especially on DataImporter.java , when a
new thread is started for a delta-import / full-import commands. For one of our
cases - we send frequent delta imports and creating thread objects and
launching them seems to be taking a big hit.
Attached here is a profiling program - that shows a ~80% improvement in the
launch of concurrent units , using ExecutorService ( FixedThreadPool
implementation ) as compared to the raw Thread(). start() .
> Solr WebApp wide Executor for better efficient management of threads ,
> separating the logic in the thread from the launch of the same.
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-922
> URL: https://issues.apache.org/jira/browse/SOLR-922
> Project: Solr
> Issue Type: Improvement
> Components: clients - java
> Environment: Tomcat 6, JRE 6
> Reporter: Kay Kay
> Priority: Minor
> Attachments: SOLR-922.patch
>
>
> For a different jira - when we were discussing bringing in parallelism
> through threads and using Executors - encountered a case of using a webapp
> wide Executor for reusing thread pools for better use of thread resources ,
> instead of thread.start() .
> pros: Custom Request Handlers and other plugins to the Solr App server can
> use this Executor API to retrieve the executor and just submit the Runnable /
> Callable impls to get the job done while getting the benefits of a thread
> pool . This might be necessary as we continue to write plugins to the core
> architecture and centralizing the threadpools might make it easy to control /
> prevent global Executor objects across the codebase / recreating them locally
> ( as they might be expensive ).
> $ find . -name *.java | xargs grep -nr 'start()' | grep "}"
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/XPathEntityProcessor.java:377:
> }.start();
> ./contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/DataImporter.java:368:
> }.start();
> ./src/java/org/apache/solr/handler/SnapPuller.java:382: }.start();
> ./src/java/org/apache/solr/handler/SnapShooter.java:52: }.start();
> ./src/java/org/apache/solr/handler/ReplicationHandler.java:134:
> }.start();
> ./src/common/org/apache/solr/common/util/ConcurrentLRUCache.java:112:
> }.start();
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
