Hi Dmitriy Thank you, for your reply. I didn't realize you had called into our office, but then I have been on-site for a long time. I initiated the conversation with Jim Leonard some time ago and would have liked to have met.
I have looked through all the examples and am struggling to understand the appropriate Ignite technology to use. I am already using the DataStreamer to load files into a cache. I need to check that a bundle of different files have been cached within a time limit. I have a listener being triggered on put. So when an xml file arrives I need to check the cache for the other files in the bundle so they can be packaged. I propose the listener puts the xml file key into a queue and have many distributed processes "taking" from the queue to perform the bundling if all the files have arrived in the cache. This would become a general purpose pattern for a machine learning workflow I think this should be a simple task for Ignite. Bets wishes, Andy Andrew Rendle Big Data Partnership Mob: +44 (0)7973 878 454 Tel: +44 (0)20 7 205 2550 [email protected] twitter.com/bigdataexperts www.bigdatapartnership.com On 13 October 2015 at 00:14, dsetrakyan [via Apache Ignite Users] < [email protected]> wrote: > Hi, > > Please properly subscribe to the user list, so everyone in the community > could get the notifications. All you need to do is sent an empty email to " > [email protected]" and follow the instructions in the > reply. > > andy@bdp wrote > I need to create an architecture for Ignite that allows: > > Data to be streamed into a cache. > > You will likely need to use IgniteDataStreamer API for this. > > andy@bdp wrote > A cache listener to put selected entries in a queue. One or more > distributed processes to wait for queue entries, do some work and put the > results back in a cache. > > The streamer comes with StreamReceiver which allows you to process the > data before it gets put in a cache. I am not sure what you mean by > processing "waiting" for queue entries. Would it be sufficient to have this > processes implement their logic in the StreamReceiver? > > andy@bdp wrote > Also, point me in the direction of an example(s) of a listener putting > something on a queue and 1 or more processes consuming the queue entries. > > Ignite comes with many examples. I think in your case you should take a > look at the streamer examples here: > > https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/streaming > > https://github.com/apache/ignite/tree/master/examples/src/main/java8/org/apache/ignite/examples/java8/streaming > > andy@bdp wrote > Does anyone have a sample architecture diagram that Ignite conforms to? > > There are many architectural diagrams in the documentation. If you are > asking how the keys are distributed, perhaps you will find this > documentation useful: > https://apacheignite.readme.io/docs/cache-modes > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-ignite-users.70518.x6.nabble.com/Ignite-architecture-tp1588p1595.html > To unsubscribe from Ignite architecture, click here > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1588&code=YW5kcmV3LnJlbmRsZUBiaWdkYXRhcGFydG5lcnNoaXAuY29tfDE1ODh8MTYzODgzNzc0Ng==> > . > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- *NOTICE AND DISCLAIMER* This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. Business Address: Eagle House, 163 City Road, London, EC1V 1NR. United Kingdom Registered Office: Finsgate, 5-7 Cranwood Street, London, EC1V 9EE. United Kingdom Big Data Partnership Limited is a company registered in England & Wales with Company No 7904824 -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-architecture-tp1588p1600.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
