[
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-4816:
---------------------------------
Description:
This issue adds the following enhancements to CloudSolrServer's update logic:
1) Document routing: Updates are routed directly to the correct shard leader
eliminating document routing at the server.
2) Parallel update execution: Updates for each shard executed in a separate
thread so parallel indexing can occur on each shard.
3) Javabin transport: The requests are sent via javabin transport.
These enhancements should allow for near linear scalability on indexing
throughput.
Usage:
CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
SolrInputDocument doc1 = new SolrInputDocument();
doc1.addField(id, "0");
doc1.addField("a_t", "hello1");
SolrInputDocument doc2 = new SolrInputDocument();
doc2.addField(id, "2");
doc2.addField("a_t", "hello2");
UpdateRequest request = new UpdateRequest();
request.add(doc1);
request.add(doc2);
request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
NamedList response = cloudClient.request(request); // Returns a backwards
compatible condensed response.
//To get more detailed response down cast to RouteResponse:
CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;
NamedList responses = rr.getRouteResponse();
was:
This issue adds a new Solr Cloud client called the
ConcurrentUpdateCloudSolrServer. This Solr Cloud client implements document
routing in the client so that document routing overhead is eliminated on the
Solr servers. Documents are batched up for each shard and then each batch is
sent in it's own thread.
With this client, Solr Cloud indexing throughput should scale linearly with
cluster size.
This client also has robust failover built-in because the actual requests are
made using the LBHttpSolrServer. The list of urls used for the request to each
shard begins with the leader and is followed by that shard's replicas. So the
leader will be tried first and if it fails it will try the replicas.
Sample usage:
ConcurrentUpdateCloudServer client = new
ConcurrentUpdateCloudSolrServer(zkHostAddress);
UpdateRequest request = new UpdateRequest();
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", 2);
doc.addField("manu","BMW");
request.add(doc);
NamedList response = client.request(request);
NamedList exceptions = response.get("exceptions"); // contains any exceptions
from the shards
NamedList responses = response.get("responses"); // contains the responses from
shards without exception.
> Add document routing to CloudSolrServer
> ---------------------------------------
>
> Key: SOLR-4816
> URL: https://issues.apache.org/jira/browse/SOLR-4816
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Affects Versions: 4.3
> Reporter: Joel Bernstein
> Priority: Minor
> Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
> SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
> SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
> SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
> SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
> SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch
>
>
> This issue adds the following enhancements to CloudSolrServer's update logic:
> 1) Document routing: Updates are routed directly to the correct shard leader
> eliminating document routing at the server.
> 2) Parallel update execution: Updates for each shard executed in a separate
> thread so parallel indexing can occur on each shard.
> 3) Javabin transport: The requests are sent via javabin transport.
> These enhancements should allow for near linear scalability on indexing
> throughput.
> Usage:
> CloudSolrServer cloudClient = new CloudSolrServer(zkAddress);
> SolrInputDocument doc1 = new SolrInputDocument();
> doc1.addField(id, "0");
> doc1.addField("a_t", "hello1");
> SolrInputDocument doc2 = new SolrInputDocument();
> doc2.addField(id, "2");
> doc2.addField("a_t", "hello2");
> UpdateRequest request = new UpdateRequest();
> request.add(doc1);
> request.add(doc2);
> request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false);
> NamedList response = cloudClient.request(request); // Returns a backwards
> compatible condensed response.
> //To get more detailed response down cast to RouteResponse:
> CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response;
> NamedList responses = rr.getRouteResponse();
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
