[
https://issues.apache.org/jira/browse/SOLR-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15589143#comment-15589143
]
Joel Bernstein commented on SOLR-9417:
--------------------------------------
Manual testing is looking good. The following expression worked well and the
daemons terminated when they finished processing the topic. So this the basic
template of a batch job, that kicks off then terminates when it's finished:
{code}
parallel(emails1,
workers=2,
sort="daemonOp asc",
daemon(id="daemon1",
terminate="true",
update(emails,
batchSize=250,
topic(checkpoints,
emails1,
q="*:*",
id="topic2",
initialCheckpoint=0,
partitionKeys="id",
fl="id, from, to, body"
)
)
)
)
{code}
The code above uses parallel workers to iterate over a topic and index the
content to another SolrCloud collection.
> Allow daemons to terminate when they finish iterating a topic
> -------------------------------------------------------------
>
> Key: SOLR-9417
> URL: https://issues.apache.org/jira/browse/SOLR-9417
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Joel Bernstein
> Assignee: Joel Bernstein
> Fix For: 6.3
>
> Attachments: SOLR-9417.patch, SOLR-9417.patch, SOLR-9417.patch
>
>
> The daemon expression currently runs until it's killed. This ticket will add
> a new *terminate* parameter to the daemon expression that will allow the
> daemon to shut itself down when it's finished processing a topic.
> There are a couple of small changes that need to be made to allow the daemon
> to terminate on it's own:
> 1) The daemon will need to be passed the Map of all daemons in the /stream
> handler. This will allow the DaemonStream to remove itself from the Map when
> it terminates.
> 2) Logic needs to be added for the daemon to exit it's run loop if the topic
> signals it had a zero Tuple run. The *sleepMillis* value in the EOF Tuple can
> be used for this purpose. If sleepMillis is greater then 0 then this signals
> a zero Tuple run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
