[
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873554#comment-13873554
]
Shalin Shekhar Mangar commented on SOLR-5477:
---------------------------------------------
{quote}
Questions:
This move would mean having a lot more clients talk to and write to zk. Does
this approach make sense as far as the intended direction of SolrCloud is
concerned?
Any suggestions/concerns about scalability of zk as far as having multiple
updates coming into zk is concerned.
{quote}
# I don't think we need to use a ZooKeeper queue for communication. The only
reason where that is required is when we need fail over. That is desired for
the overseer actions but not required for core admin actions. For example, I
don't think you should be able to submit a core admin action when the target
node is not up.
# This approach only works with a cloud aware client such as SolrJ i.e. you
can't submit an async core admin action with HTTP. This in itself is a strong
reason to warrant a -1 from me.
# This a very intrusive change compared to other approaches. I think the
benefits you have outlined can be achieved in simpler ways (see my comments on
approach #2 below)
{quote}
Approach #2:
Continue accepting the request like right now, but just :
Get the call to return immediately
Use zk to only track/store the status (persistence). The request status calls
still comes to the core and the status is fetched from zk by the core instead
of the client being intelligent and talking directly to zk.
{quote}
This is much more acceptable to me. Clients should not have to worry about ZK
to submit a request.
{quote}
This approach is certainly less intrusive but then also doesn't come with the
benefit of having the client just watch over a particular zk node for task
state change etc.
{quote}
In my mind, the proposal to have the client watch nodes for task state change
is orthogonal to the way the task is invoked. There is no reason why the
request can be made via HTTP _and_ the response from core container can't
contain the request id (is this the complete zookeeper node name?) which can be
used by a client to setup a watch directly if it so desires. In any case a HTTP
based status/polling API must exist.
> Async execution of OverseerCollectionProcessor tasks
> ----------------------------------------------------
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Noble Paul
> Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to
> have the requests get timed out. It is more of a problem if the cluster is
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id.
> as separate collection admin command will be added to poll the status of the
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are
> completed the queue entry is removed. OverSeerColectionProcessor will perform
> these tasks in multiple threads
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
