[
https://issues.apache.org/jira/browse/CONNECTORS-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102008#comment-13102008
]
Karl Wright commented on CONNECTORS-202:
----------------------------------------
I looked briefly at how you'd want to do this. The current way output
connectors are designed requires either the following:
- The commit-within parameter is part of configuration information, in which
case it is per-connection, not per job. But in this case a change to the
commit-within info will not cause any documents to be reindexed.
- The commit-within parameter is part of output specification information, in
which case it is per-job. However, any changes to the parameter will cause all
documents associated with that job to be reindexed the next time the job is run.
It is also the case that the Solr output connector already has a configuration
tab where a commit-within parameter would logically fit, but if output
specification were used, a new tab would probably need to be introduced.
While it is possible to change the output connector API so that specification
information is available directly at the time the request to add to the index
is made, all this together argues that maybe we should consider the parameter
to be configuration not specification information. It is, after all, "how"
information and not "what". If a user needs both "urgent" and "lazy" commits,
they can readily do this by creating two Solr connections. Doesn't seem like
there would be too much of a downside to this approach. What do you think?
> SOLR connector suport for commitWithin
> --------------------------------------
>
> Key: CONNECTORS-202
> URL: https://issues.apache.org/jira/browse/CONNECTORS-202
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Lucene/SOLR connector
> Affects Versions: ManifoldCF 0.2, ManifoldCF 0.3
> Reporter: Jan Høydahl
> Assignee: Karl Wright
> Labels: commit
> Fix For: ManifoldCF 0.4
>
>
> The output connection must support commitWithin
> (http://wiki.apache.org/solr/UpdateXmlMessages#Optional_attributes_for_.22add.22)
> in addition to sending a commit() at the end of a job.
> This allows for efficient handling of commits on the Solr side.
> The parameter should ideally be configurable per job. In that way you could
> say that for "Important job" commitWithin=10s while for "Big crawl job",
> commitWithin=600s.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
