[
https://issues.apache.org/jira/browse/SOLR-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028498#comment-14028498
]
Mikhail Khludnev commented on SOLR-3585:
----------------------------------------
[~dsmiley]
bq. I think it's crazy that to load Solr faster you have to make it a
client-side concern to know how many threads
I saw quite often that a producer is a slowpoke (single thread SELECT), and
fast consumer doesn't help at all. That's why I prefer to burden client for
threads. Look at SolrCloud if client doesn't care about performance, it can
send update to any node, but if needs performance, it have to concern about
cluster topology, and send to nodes concurrently;
bq. Unless you are using ConcurrentUpdateSolrServer then
in this case I'd rather say, this guy doesn't bother about performance, or
neither cure help here.
bq. Should it know how many threads are on the box and how concurrent its IO
hardware is?
ok. here are two separate concerns: throttling and performance. the former one
is addressed by j2ee container thread pool so far. However, I can agree that it
might make sense to introduce separate thread pool for handling updates, but
only for limiting number of threads to avoid unnecessary index segmentation due
to concurrent flush. Also, don't you think that this should be handled even
deeper, somewhere around Lucene IndexWriter? Coming back to performance, I'm
indented to blame client-side as slowpoke.
bq. threads might be changed dynamically.
again. throttling interleaves with j2ee container functionality
Ok Guys, if somebody need it, let it be. One note, now the code here does the
same as ConcurrentUpdateSolrServer, it's not good at all. I'd prefer to extract
core thread code from CUSS, and reuse it in this update processor with
in-process sink for sure.
> processing updates in multiple threads
> --------------------------------------
>
> Key: SOLR-3585
> URL: https://issues.apache.org/jira/browse/SOLR-3585
> Project: Solr
> Issue Type: Improvement
> Components: update
> Affects Versions: 4.0-ALPHA, 5.0
> Reporter: Mikhail Khludnev
> Attachments: SOLR-3585.patch, SOLR-3585.patch, multithreadupd.patch,
> report.tar.gz
>
>
> Hello,
> I'd like to contribute update processor which forks many threads which
> concurrently process the stream of commands. It may be beneficial for users
> who streams many docs through single request.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
