[
https://issues.apache.org/jira/browse/CONNECTORS-727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13692356#comment-13692356
]
Karl Wright edited comment on CONNECTORS-727 at 6/24/13 9:00 PM:
-----------------------------------------------------------------
I had a look at this again.
New comments:
(1) Have a look at the class org.apache.manifoldcf.core.common.DateParser.java.
It may be helpful to you for handling ISO8601 dates. DateAdapter could well
make use of it.
(2) InterruptedException is likely to be thrown whenever the agents process is
shutting down. It is critical to throw the equivalent of that exception
upstream so that the worker thread shuts down properly. To do that, you should
"throw new
ManifoldCFException(e.getMessage,e,ManifoldCFException.INTERRUPTED)", not just
the generic ManifoldCFException. For example:
{code}
} catch (InterruptedException ex) {
throw new ManifoldCFException("addSeedDocuments error: " +
ex.getMessage(), ex);
}
{code}
... should really be:
{code}
} catch (InterruptedException ex) {
throw new ManifoldCFException(ex.getMessage(), ex,
ManifoldCFException.INTERRUPTED);
}
{code}
(3) You can't call methods from any IActivities object inside your background
threads, because that will grab and free locks - exactly the kind of behavior
which you want the background thread to be free of. For example:
{code}
ExecuteSeedingThread seedingThread = new ExecuteSeedingThread(client,
activities, url.toString());
{code}
There are a whole range of objects in org.apache.manifoldcf.core.common which
are designed to help you pass data back from a background thread to a worker
thread for this very purpose. In the seeding case you will want
XThreadStringBuffer. In a streaming case you use XThreadInputStream. See how
it is used in the GoogleDrive connector or DropBox connector.
The rest looks pretty good as far as I can tell at the moment...
was (Author: [email protected]):
I had a look at this again.
New comments:
(1) Have a look at the class org.apache.manifoldcf.core.common.DateParser.java.
It may be helpful to you for handling ISO8601 dates. DateAdapter could well
make use of it.
(2) InterruptedException is likely to be thrown whenever the agents process is
shutting down. It is critical to throw the equivalent of that exception
upstream so that the worker thread shuts down properly. To do that, you should
"throw new
ManifoldCFException(e.getMessage,e,ManifoldCFException.INTERRUPTED)", not just
the generic ManifoldCFException. For example:
{code}
} catch (InterruptedException ex) {
throw new ManifoldCFException("addSeedDocuments error: " +
ex.getMessage(), ex);
}
{code}
... should really be:
{code}
} catch (InterruptedException ex) {
throw new ManifoldCFException(ex.getMessage(), ex,
ManifoldCFException.INTERRUPTED);
}
{code}
(3) You can't call methods from any IActivities object inside your background
threads, because that will throw and free locks - exactly the kind of behavior
which you want the background thread to be free of. For example:
{code}
ExecuteSeedingThread seedingThread = new ExecuteSeedingThread(client,
activities, url.toString());
{code}
There are a whole range of objects in org.apache.manifoldcf.core.common which
are designed to help you pass data back from a background thread to a worker
thread for this very purpose. In the seeding case you will want
XThreadStringBuffer. In a streaming case you use XThreadInputStream. See how
it is used in the GoogleDrive connector or DropBox connector.
The rest looks pretty good as far as I can tell at the moment...
> generic connector
> -----------------
>
> Key: CONNECTORS-727
> URL: https://issues.apache.org/jira/browse/CONNECTORS-727
> Project: ManifoldCF
> Issue Type: Improvement
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
>
> OK, this is tricky, but really nice idea. I was thinking about indexing some
> sources which do not have API, or API does not provide information needed by
> Manifold, or there is dedicated system and IT team that can easily add some
> API.
> Now you have to write dedicated connector and probably some API extension,
> plugin, etc that would talk with each other to provide seeds, versions and
> documents. Which requires knowledge on how to write Manifold connectors AND
> knowledge about system - there is no so many programmers that know both
> systems :)
> So lets make things easier - provide "generic" Manifold connector that works
> with "generic" API (ie XML over HTTP which is *really* easy to implement in
> any language). This API and protocol are strictly defined and specified. Then
> to integrate with the custom document repository one has to only implement
> API entry point which follow those specifications.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
