[
https://issues.apache.org/jira/browse/CONNECTORS-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771130#comment-13771130
]
Karl Wright commented on CONNECTORS-13:
---------------------------------------
Hi Jan,
Thanks for the update.
ManifoldCF's synchronization is currently separate from its configuration, so
the first step would be to fold all that under a single interface that could be
implemented with zookeeper. But obviously *some* configuration external to
zookeeper still must exist, in order to configure zookeeper itself. Do you
know where Solr puts that?
My immediate goal would be to replace most of the the properties.xml, and maybe
the logging.ini file too, with zookeeper configuration. But I'm still fuzzy as
to how that would work. It would be great to talk it through a bit more to
settle the design.
> We should move to eliminate process synchronization via shared file system,
> and use a process/service instead
> -------------------------------------------------------------------------------------------------------------
>
> Key: CONNECTORS-13
> URL: https://issues.apache.org/jira/browse/CONNECTORS-13
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Framework core
> Affects Versions: ManifoldCF 0.1, ManifoldCF 0.2
> Reporter: Karl Wright
> Fix For: ManifoldCF next
>
>
> The current implementation relies on the file system to synchronize activity
> between various LCF processes. This has several downsides: first, it is
> possible to get the file system into a state that is corrupted (by killing
> processes); second, this limits the future ability to spread crawler workload
> over multiple machines.
> It should be reasonably straightforward, and probably more resilient, to
> introduce a "synchronization process", which all other LCF processes talk to
> in order to manage locks, shared data, and other synchronization activities.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
