I could switch away from plugin mode, but that would mean people must adapt and sell another JDBC bridge component to their boss and worse, that will keep the SPOF weakness of rivers but with more burden of responsibility on myself. Not in my sparetime.
Maybe a compromise will work better: external service, but also attached to the ES nodes as a "lightweight" restartable plugin. It looks attractive to me to move over to Groovy and run SQL / JDBC from there. The picture is an enhanced deploy plugin and a general harvester job distribution module with UI, where JDBC is just one option. With the deploy plugin it is easy to move JDBC driver jars and other dependencies around in the cluster, for stop/start, keep state etc. Jörg On Fri, Jan 9, 2015 at 9:22 AM, Radu Gheorghe <[email protected]> wrote: > Yeah, my general approach so far was to stay away from plugins as much as > possible and use external services instead. Something that your JDBC river > can do as well, as far as I understand. > > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > On Fri, Jan 9, 2015 at 10:20 AM, [email protected] < > [email protected]> wrote: > >> No I didn't :( >> >> I still struggle with stateful plugins - there is no good solution I know >> of and that is the reason of trouble. >> >> Jörg >> >> On Fri, Jan 9, 2015 at 9:01 AM, Radu Gheorghe <[email protected] >> > wrote: >> >>> Thanks Ben and Joerg. Too bad there was no good solution this particular >>> upgrade, at least we all learned something (well, aside from Joerg who knew >>> it all from the start :D). >>> >>> Best regards, >>> Radu >>> >>> -- >>> Performance Monitoring * Log Analytics * Search Analytics >>> Solr & Elasticsearch Support * http://sematext.com/ >>> >>> On Fri, Jan 9, 2015 at 12:28 AM, [email protected] < >>> [email protected]> wrote: >>> >>>> Hi Radu, >>>> >>>> the deploy plugin is somewhat limited to "restartable" plugins as it >>>> can only restart services of a plugin, i.e. lifecycle components with >>>> carefully implemented doStart()/doStop() methods. >>>> >>>> Most plugins come with modules that add REST endpoints, actions, >>>> parsers, functions etc. as modules by the onModule...() mechanism, so they >>>> get "baked" into the ES node as immutable objects before the node is going >>>> up. This means, the deploy plugin can not change or undo this behavior. >>>> (Maybe some evil hackery will do to a certain degree) >>>> >>>> For my use case, I embed a Ratpack server and start my own HTTP web >>>> server that does not use baked-in ES modules, just a restartable service. >>>> It is really delightful to just use a single curl PUT command for rapid >>>> prototyping a Ratpack web app embedded in ES without node restart. >>>> >>>> A restartable plugin is: >>>> >>>> https://github.com/jprante/elasticsearch-plugin-ratpack >>>> >>>> Jörg >>>> >>>> >>>> >>>> On Thu, Jan 8, 2015 at 5:24 PM, Radu Gheorghe < >>>> [email protected]> wrote: >>>> >>>>> Hello Ben, >>>>> >>>>> Maybe it works if you uninstall the plugin from one node at a time and >>>>> do a rolling restart (sticking to 1.3.2), then do the upgrade with another >>>>> rolling restart, then install the plugin back again with yet another >>>>> rolling restart? >>>>> >>>>> I would understand if you said "no way I do 3 restarts!" :) But maybe >>>>> this will help in future: >>>>> https://github.com/jprante/elasticsearch-plugin-deploy >>>>> >>>>> Best regards, >>>>> Radu >>>>> -- >>>>> Performance Monitoring * Log Analytics * Search Analytics >>>>> Solr & Elasticsearch Support * http://sematext.com/ >>>>> >>>>> On Wed, Jan 7, 2015 at 5:15 AM, Ben Berg <[email protected]> wrote: >>>>> >>>>>> Hello, >>>>>> I am attempting to upgrade a 10 node cluster from 1.3.2 to 1.4.2 - I >>>>>> upgraded the first node,removed and reinstalled latest versions of >>>>>> plugins >>>>>> - two non-site plugins are river-twitter (version 2.4.1) and jdbc >>>>>> (version >>>>>> 1.4.0.8 and also tried 1.4.0.7) - and when starting the node I see the >>>>>> errors below in logs on upgraded node and node does not join cluster. If >>>>>> i >>>>>> downgrade to 1.3.2, uninstall plugins and reinstall jdbc river plugin >>>>>> version 1.3.0.4 the node properly joins the cluster. >>>>>> Errors from logs: >>>>>> [2015-01-06 21:29:39,970][INFO ][node ] [bi-es1] >>>>>> version[1.4.2], pid[5910], build[927caff/2014-12-16T14:11:12Z] >>>>>> [2015-01-06 21:29:39,971][INFO ][node ] [bi-es1] >>>>>> initializing ... >>>>>> [2015-01-06 21:29:40,009][INFO ][plugins ] [bi-es1] >>>>>> loaded [river-twitter, jdbc-1.4.0.7-a875ced], sites [head, kopf, bigdesk, >>>>>> paramedic, HQ, whatson] >>>>>> [2015-01-06 21:29:45,587][INFO ][node ] [bi-es1] >>>>>> initialized >>>>>> [2015-01-06 21:29:45,588][INFO ][node ] [bi-es1] >>>>>> starting ... >>>>>> [2015-01-06 21:29:45,960][INFO ][transport ] [bi-es1] >>>>>> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/ >>>>>> 192.168.83.231:9300]} >>>>>> [2015-01-06 21:29:45,982][INFO ][discovery ] [bi-es1] >>>>>> bi-cluster1/VyTzSKWnQAS9ZCk971fJEg >>>>>> [2015-01-06 21:29:51,906][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28805963] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:29:51,915][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:29:57,028][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28807127] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:29:57,036][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:30:02,245][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28808254] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:30:02,252][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:30:07,576][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28809377] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:30:07,583][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:30:12,689][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28810050] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:30:12,696][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:30:15,989][WARN ][discovery ] [bi-es1] >>>>>> waited for 30s and no initial state was set by the discovery >>>>>> [2015-01-06 21:30:16,002][INFO ][http ] [bi-es1] >>>>>> bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/ >>>>>> 192.168.83.231:9200]} >>>>>> [2015-01-06 21:30:16,003][INFO ][node ] [bi-es1] >>>>>> started >>>>>> [2015-01-06 21:30:18,277][WARN ][transport.netty ] [bi-es1] >>>>>> Message not fully read (request) for [28811164] and action >>>>>> [discovery/zen/join/validate], resetting >>>>>> [2015-01-06 21:30:18,292][INFO ][discovery.zen ] [bi-es1] >>>>>> failed to send join request to master >>>>>> [[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}], >>>>>> reason >>>>>> [RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]]; >>>>>> nested: >>>>>> RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]]; >>>>>> nested: ElasticsearchIllegalArgumentException[No custom index metadata >>>>>> factory registered for type [rivers]]; ] >>>>>> [2015-01-06 21:30:19,406][DEBUG][action.admin.cluster.state] [bi-es1] >>>>>> no known master node, scheduling a retry >>>>>> >>>>>> I have tried removing all plugins from the upgraded node and that >>>>>> results in same errors. I have tried closing the _river index and >>>>>> removing >>>>>> all defined rivers (wasn't hopeful that would work but gave it a shot). >>>>>> >>>>>> I am not sure what else to try - thoughts? ideas? >>>>>> >>>>>> I am safely downgraded back to 1.3.2 and node rejoined and things >>>>>> seem good except still on 1.3.2. Would prefer not to do a full cluster >>>>>> shutdown to upgrade all nodes and jdbc plugin but that seems maybe it >>>>>> would >>>>>> solve this? >>>>>> >>>>>> I have had issues in past with upgrades/jdbc river plugin (really >>>>>> love the plugin though) so my brain is focusing on that and might not be >>>>>> the issue? I used same process on a test cluster that I just realized did >>>>>> not have the jdbc plugin installed on it and that went smooth. >>>>>> >>>>>> Any help would be greatly appreciated! >>>>>> Thanks >>>>>> Ben >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elasticsearch" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elasticsearch/ef4334fc-b6c9-4802-ab8a-934989901efd%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/elasticsearch/ef4334fc-b6c9-4802-ab8a-934989901efd%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elasticsearch/CAHXA0_16LKxArbm7K3xLv0ewbnaOAnZZLQx5ZZqLezssfebkfA%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/CAHXA0_16LKxArbm7K3xLv0ewbnaOAnZZLQx5ZZqLezssfebkfA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elasticsearch" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGsq6W1rkfjaofGFYY%2Bp8eOJBZ3hKr%2BZrxUtV9jP0R%3DHg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGsq6W1rkfjaofGFYY%2Bp8eOJBZ3hKr%2BZrxUtV9jP0R%3DHg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3xVB-ZKRm2SC4Dwdv0WxNAERCskCVicX_7DAqGROuo0A%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3xVB-ZKRm2SC4Dwdv0WxNAERCskCVicX_7DAqGROuo0A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEm8bNRTsJ%3DQEJi5a_otLToz4Fbhf%2BFoU-FnUJnF9gmGg%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEm8bNRTsJ%3DQEJi5a_otLToz4Fbhf%2BFoU-FnUJnF9gmGg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3PfMbU6oEtOyoT3zgiG5gbVt973jOf6B6q8ReJ8C0yXg%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3PfMbU6oEtOyoT3zgiG5gbVt973jOf6B6q8ReJ8C0yXg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEs%3D8BFHS59eXES0gSwY2Z6M1K9Ctz3J7Z1ZZ8CJk1K7g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
