I work on such a component which can handle jobs and delegate them to other nodes but no working implementation available yet.
For a service component that is able to maintain state in the cluster state, see RiverState class in JDBC plugin. Jörg On Thu, Aug 28, 2014 at 8:08 AM, Nitin Maheshwari <[email protected]> wrote: > Thanks Jörg for your quick and timely response. > > I am new to ES, can you point me to any reference implementation for the > ES service component? > > Once again thanks for the help. > > Nitin > > > On Tuesday, 26 August 2014 15:07:05 UTC+5:30, Jörg Prante wrote: > >> For multi tenant, the river concept is awkward. River is a singleton and >> is bound to single user execution, and you are right, creating river >> instances per DB and per index does not scale. >> >> There are several options: >> >> - write a more sophisticated plugin which acts as a service and not as a >> singleton. The ES service component, which would maintain state in the >> cluster state, could accept job requests where each job request is >> equivalent to a JDBC pull. The job requests are delegated to a node which >> is not very busy with jobs (load balancing). The code of the JDBC river can >> be reused for that. >> >> - write a separate middleware for your tenants where they can have >> separate access to the DB and prepare ES JSON bulk files from (maybe be by >> REST API calls similar in style to ES). This would be a domain specific >> solution but offers most flexibility to the tenants, they are free to >> decide how and when to create and index the data from DB. >> >> Jörg >> >> >> >> >> On Tue, Aug 26, 2014 at 11:21 AM, Nitin Maheshwari <[email protected]> >> wrote: >> >>> Hi Jörg, >>> >>> I am working on a multi tenant application where each tenant has its own >>> database. I am planning to use ES for indexing the data, and JDBC river for >>> doing periodic bulk indexing. I do not want to create one river per DB per >>> object type. This will lead to too many rivers. >>> >>> I wanted to modify the JDBC river so that I can give parent DB location, >>> where all tenant db connection information is available. And then inside >>> the river, modify it such that a feader thread is created for each river. >>> >>> Do you see any issue with this or do you have any other recommendation? >>> >>> Thanks, >>> Nitin >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/771fc3a8-2203-4db8-a07b-067430e7a473%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/76780b26-5dfa-4de4-9d14-5e1a9cacb2d3%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/76780b26-5dfa-4de4-9d14-5e1a9cacb2d3%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGTXjgHFeV9ip4n0vO_TJNthcvD1RK-pHWruwL0gx_Rew%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
