Responding to Claus' request for feedback on how to store EIP stats and expose it in several ways...
With the current architecture where stats are kept in memory there are a few downsides, mainly: (1) not being able to store rich data, only counters due to memory usage considerations and (2) stats are lost between restarts. A legitimate requirement is to be able to browse the list of exchange IDs marked as duplicates by the idempotent consumer, but storing those in memory would cause an immediate memory leak due to an ever-growing list. Good thing is that at least we can set skipDuplicate to false and react to the DUPLICATE_MESSAGE storing the information somewhere, but it would be easier and less error prone if there was a centralised place to resort to to obtain better richer visibility and governance, without the user having to implement hooks here and there to react when things happen inside Camel (tracer, interceptor, etc.). In my mind, the ideal solution is to create a generic "management engine" to which components, EIPs, services, etc. publish their stats. This represents an important overhaul, but in my opinion it is imperative for Camel to advance to the next generation ;) Features of my proposed "management engine": * pluggable persistence repositories, to choose what back-end your stats will be persisted to (JDBC, MongoDB, Redis, In-memory, etc.) * flexible external access via pluggable management front-ends (JMX, REST, raw API), so that any type of client can access the info from the outside * persistence flushing policy, so that the user can control how often the management engine will persist in-memory data to the back-end * purge policy, so that old stats are flushed out on a schedule, i.e. it's self-cleaning * centralised filtering of management events, so you can turn off stats for specific EIPs, Components, etc. from one point only. E.g. (*, !idempotentConsumer, !camel-jpa) would enable all stats except idempotent consumer and the camel-jpa component. * query capabilities based on the Simple language * async/sync persistence behaviour selection ... based on how important your stats are to you * etc. Note that achieving blazing-fast persistence nowadays is a piece of cake with NoSQL, Big Data and Data Grids. I realise the extent of the impact on the codebase to implement this feature, but seeing that Camel 3.0 is coming up in conversations, I wanted to drop my 2 cents ;) Regards, Raúl. On 5 January 2012 16:14, Babak Vahdat <babak.vah...@swissonline.ch> wrote: > As Claus has already suggested [1] I would like to share the question with > you for a better possible third approach! > > For all the details regarding the proposed improvement please take a look > at > the comments and also the 2 attached patches on it and let us know of any > possible third approach you could think of. Thanks! > > [1] https://issues.apache.org/jira/browse/CAMEL-4782 > > Babak > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/DISCUSS-How-to-better-enlist-the-duplicate-message-count-by-the-IdempotentConsumer-on-JMX-tp5123118p5123118.html > Sent from the Camel Development mailing list archive at Nabble.com. >
