[
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anshum Gupta updated SOLR-5477:
-------------------------------
Attachment: SOLR-5477.patch
Still a lot of stuff to be added/fixed but here's another patch so that if
someone is interested, he/she could get some idea on the direction in which I'm
moving.
Here's what's been fixed/changed in this patch:
* Standardized the request parameters for CoreAdmin and Collection level
request status. Parameters now are : action=REQUESTSTATUS&requestid=XX
* Added async option for all other calls (not just splitshard and
createcollection).
* Fixed the encoding related stuff and the potential NPEs where ever I could
spot those.
* Removed unused code and cleaned up some debugging stuff.
* Used a ThreadPool in case of coreadmin in a rather raw manner i.e. static
(and without the mode check etc). Have added todo's to add those checks and to
also call shutdown for the threadpool.
* Changed names of the queues to the ones suggested by Shalin.
* Items are no longer being put in the completed queue in Overseer.
* Ran ant check-forbidden-apis to fix reported issues.
Here's what still lies with me incomplete/not-working:
* Make shard requests async (pass the param to coreadmin requests and poll for
completion/failure) from the OverseerCollectionProcessor.
* Have a threadpool for OverseerCollectionProcessor so that the long running
tasks don't block everything else.
* Improve current tests and add more tests.
> Async execution of OverseerCollectionProcessor tasks
> ----------------------------------------------------
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Noble Paul
> Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch,
> SOLR-5477.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to
> have the requests get timed out. It is more of a problem if the cluster is
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id.
> as separate collection admin command will be added to poll the status of the
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are
> completed the queue entry is removed. OverSeerColectionProcessor will perform
> these tasks in multiple threads
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
