btw, I've just raised and fixed SM-776 [1], [2]
This should ensure that the file is processed only once if you use
a persistent locking mechanism -- which is not implemented yet :(
If the file is not processed correctly, the lock is removed so that it
should be re-processed later. If the file is processed but not deleted
(because it has been configured or because the deletion failed), the
lock is not removed, so that it should not be re-processed.
This should ensure the closest match to once-and-only-once delivery,
even across clusters.
The only problem is to implement the distributed locking, while taking
into account the fact that a node holding a lock may crash (the lock
should be freed in such a case).
[1] http://issues.apache.org/activemq/browse/SM-776
[2]
http://svn.apache.org/viewvc/incubator/servicemix/trunk/deployables/bindingcomponents/servicemix-ftp/src/main/java/org/apache/servicemix/ftp/FtpPollingEndpoint.java?revision=486860&view=markup
On 12/13/06, Guillaume Nodet <[EMAIL PROTECTED]> wrote:
On 12/13/06, RNobrega <[EMAIL PROTECTED]> wrote:
>
>
> gnodet wrote:
> >
> >
> > What's you exact use case for both ? It would help finding a solution.
> >
> >
>
> Use case #1: In broad terms: we're trying to deploy something today,
> and perhaps "slightly" change it tomorrow without having to
> re-deploy. Slight changes could be, for example, overriding properties
> of beans created by spring from the xbean.xml files within the service
> assemblies
>
> For this, we have a configuration endpoint that hooks up to the
> container using the propertyListener, and when it notices that the
> container changed to STARTED, loads a bunch of properties from a set
> of tables and (re-)configures other endpoints (is this an abuse of
> spring and/or servicemix? is there a better way to achieve the same?)
How do you "reconfigure" endpoints ?
Could you do that individually for each endpoint using the EndpointListener ?
You can intercept all endpoints activation / deactivation.
>
> Use case #2: we're fetching files using ftp. Out FTPGet endpoint
> accepts a simple xml message with the url to be downloaded. The files
> are on external sites not under our control; we can't, for example,
> remove them, or rename them; they're quite large (100Mb) and our
> servicemix is clustered. We're trying to avoid downloading the same
> file over and over again; we're also not interested in downloading the
> same file from two nodes at the same time.
>
> We're trying to solve this by persisting all urls successfully
> downloaded to a db table; a url is added to the table as soon as the
> download starts, but is removed in case of failure (could be several
> hours later). I thought about having a PersistentXPathPredicate handle
> the database access part, and putting an eip filter in front of the
> FTPGet endpoint. The tricky bit is the "removed in case of failure"
> one:
>
> public boolean matches(MessageExchange exchange) {
>
> step 1) use an (injected) xpath expression to find out the string to
> be persisted. In this example, the string is the url to be
> downloaded, but it could be anything else
>
> step 2) use an (injected) datasource and check whether this string
> is on the table. if it is, return false
>
> step 3) add the string to the table, and add a listener (this) to
> the exchange; if the exchange fails, remove the string from
> the table
> }
>
> If I understand your suggestion correctly, PersistentXPathPredicate
> should implement ExchangeListener, be added to the container, and do
> something when the exchange fails, correct?
>
This is all about using a clustered servicemix with an ftp binding, right ?
The latest ftp component (not the one included in servicemix-components),
but the jbi compliant one has a starting point to implement this.
An endpoint can be given a o.a.s.locks.LockManager. If the implementation
of this LockManager is clustered (using a database for example), two endpoints
won't be able to process the same file at the same time.
Unfortunately this does
not solve your entire problem as there are only two states, and there is no way
to determine if a given file has already been processed. I guess it could be
enhanced to use 3 states: free / processing / processed, where a processing
failure would put back the lock in the free state. Or we could just
imagine that
if the processing has been successful, no unlock happen (which should be easier
to implement).
Anyway, i think your solution could work. The EIP filter has been written for
things like that, but you may need a more dedicated duplicates remover
which would be able to change the state depending on the outcome of the
exchange processing.
I don't think using an ExchangeListener would be the best option, as you
just need to write a modified EIP filter for that.
> --
> View this message in context:
http://www.nabble.com/listeners-in-container-and-MessageExchange-tf2815007s12049.html#a7859785
> Sent from the ServiceMix - User mailing list archive at Nabble.com.
>
>
--
Cheers,
Guillaume Nodet
--
Cheers,
Guillaume Nodet
