Nitin, If I am not mistaken rivers in ES will be deprecated. So in any case river is not a good way to go for design of future application. We are using rivers quite a lot in our project and we will be moving the functionality to middleware layer.
Regards, Lukáš Dne 31.8.2014 3:58 "Nitin Maheshwari" <[email protected]> napsal(a): > Thanks for the reference, its helpful. > > > On Thu, Aug 28, 2014 at 12:33 PM, [email protected] < > [email protected]> wrote: > >> I work on such a component which can handle jobs and delegate them to >> other nodes but no working implementation available yet. >> >> For a service component that is able to maintain state in the cluster >> state, see RiverState class in JDBC plugin. >> >> Jörg >> >> >> On Thu, Aug 28, 2014 at 8:08 AM, Nitin Maheshwari <[email protected]> >> wrote: >> >>> Thanks Jörg for your quick and timely response. >>> >>> I am new to ES, can you point me to any reference implementation for the >>> ES service component? >>> >>> Once again thanks for the help. >>> >>> Nitin >>> >>> >>> On Tuesday, 26 August 2014 15:07:05 UTC+5:30, Jörg Prante wrote: >>> >>>> For multi tenant, the river concept is awkward. River is a singleton >>>> and is bound to single user execution, and you are right, creating river >>>> instances per DB and per index does not scale. >>>> >>>> There are several options: >>>> >>>> - write a more sophisticated plugin which acts as a service and not as >>>> a singleton. The ES service component, which would maintain state in the >>>> cluster state, could accept job requests where each job request is >>>> equivalent to a JDBC pull. The job requests are delegated to a node which >>>> is not very busy with jobs (load balancing). The code of the JDBC river can >>>> be reused for that. >>>> >>>> - write a separate middleware for your tenants where they can have >>>> separate access to the DB and prepare ES JSON bulk files from (maybe be by >>>> REST API calls similar in style to ES). This would be a domain specific >>>> solution but offers most flexibility to the tenants, they are free to >>>> decide how and when to create and index the data from DB. >>>> >>>> Jörg >>>> >>>> >>>> >>>> >>>> On Tue, Aug 26, 2014 at 11:21 AM, Nitin Maheshwari <[email protected]> >>>> wrote: >>>> >>>>> Hi Jörg, >>>>> >>>>> I am working on a multi tenant application where each tenant has its >>>>> own database. I am planning to use ES for indexing the data, and JDBC >>>>> river >>>>> for doing periodic bulk indexing. I do not want to create one river per DB >>>>> per object type. This will lead to too many rivers. >>>>> >>>>> I wanted to modify the JDBC river so that I can give parent DB >>>>> location, where all tenant db connection information is available. And >>>>> then >>>>> inside the river, modify it such that a feader thread is created for each >>>>> river. >>>>> >>>>> Do you see any issue with this or do you have any other recommendation? >>>>> >>>>> Thanks, >>>>> Nitin >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473% >>>>> 40googlegroups.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/76780b26-5dfa-4de4-9d14-5e1a9cacb2d3%40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/76780b26-5dfa-4de4-9d14-5e1a9cacb2d3%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/w88yWcbx4lM/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGTXjgHFeV9ip4n0vO_TJNthcvD1RK-pHWruwL0gx_Rew%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGTXjgHFeV9ip4n0vO_TJNthcvD1RK-pHWruwL0gx_Rew%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Nitin (Nits) > http://nitinmaheshwari.in > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAHDjFTEe5cxJMX6v%3DS1HzGATmnyA7jvHa-EKEQhE%3Dv5E8Nuvjg%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAHDjFTEe5cxJMX6v%3DS1HzGATmnyA7jvHa-EKEQhE%3Dv5E8Nuvjg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUZzsJpeZ5ZDKsAWp8Ko6hpOPjRh4nowTh9nbjrPUn9tfw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
