Can you try to remove activemq from karaf and host it externally? Just to see if it really is the reason.
Christian 2018-03-11 5:28 GMT+01:00 Matthew Zipay <[email protected]>: > Yes, the Karaf instances are on different machines, and I have removed > Cellar entirely from the system. > > If I simply start up both instances, then everything *except* the data > sources comes up fine on both instances. Because of this, I am temporarily > running this system as active/failover (karaf.lock=true) until I can get to > the bottom of this data source issue. > > As far as what I see in the logs... on the second node (the one where the > data sources do not initialize), I do see that the cfg's are found, and > there are logging entries stating that they will be initialized, but then > nothing happens. No errors, nothing. > > Then if I then stop node 1, I almost immediately see the data sources > initialize on node 2. (But, of course, if I then restart node 1, same issue > - the data sources do not initialize on *that* node). > > Throughout all of the variations I have tested, there has been one > constant: - ActiveMQ. It is configured using JDBC master/slave. It doesn't > seem like this should cause Pax JDBC data sources to not initialize > (ActiveMQ is not sharing the data source - entirely separate > configuration), but that's the only thing I can think of: when ActiveMQ > master releases the lock and allows the slave to become the new master, > that's when I see Pax JDBC initialize the data sources. Thoughts? > > > On Fri, Mar 9, 2018 at 2:38 AM, Christian Schneider < > [email protected]> wrote: > >> Are the karaf instances on different machines? >> If they are on different machines and you do not use cellar or specially >> configured locking then both should come up completely independent from >> each other. >> >> Do you see anything in the log of the second instance when you stop the >> first one? I wonder what could trigger the data source come up in this >> moment. >> >> Christian >> >> 2018-03-08 17:30 GMT+01:00 Matthew Zipay <[email protected]>: >> >>> Thanks for the quick replies. With respect to Cellar, I should clarify >>> that the Pax JDBC issue persists whether or not Cellar is involved (Cellar >>> was a recent addition to the system, specifically to see if its explicit >>> support for active/active would resolve the Pax JDBC issue - it did not). >>> >>> I have tried two variations: >>> (1) Without Cellar, bring up both Karaf nodes without Karaf locking >>> (karaf.lock=false). (i.e. "unsupported" active/active) >>> (2) With Cellar, bring up both Karaf nodes without Karaf locking >>> (karaf.lock=false). (i.e. "supported" active/active) >>> >>> In either scenario, the data sources only initialize on the *first* node >>> that comes up. >>> >>> In either scenario, I can confirm that Pax JDBC bundles are started on >>> both nodes: >>> >>> Node 1: >>> 228 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Generic Driver Extender >>> 229 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Config >>> 230 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Pooling Support Base >>> 270 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC MSSQL Driver Adapter >>> 271 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Oracle Driver Adapter >>> 272 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Pooling DBCP2 >>> >>> Node 2: >>> 228 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Generic Driver Extender >>> 229 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Config >>> 230 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Pooling Support Base >>> 270 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC MSSQL Driver Adapter >>> 271 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Oracle Driver Adapter >>> 272 | Active | 80 | 1.0.1 | OPS4J Pax >>> JDBC Pooling DBCP2 >>> >>> However, jdbc:ds-list only shows the initialized data sources on node 1; >>> on node 2 *no* data sources are initialized, and I have multiple bundles in >>> GracePeriod waiting for data sources that never show up. For example, on >>> node 2: >>> >>> admin@root>diag 331 >>> SKU Resolution (331) >>> ----------------------------------------------- >>> Status: GracePeriod >>> Blueprint >>> 3/8/18 11:10 AM >>> Missing dependencies: >>> (&(dataSourceName=PRODUCTDS)(objectClass=javax.sql.DataSource)) >>> >>> >>> The "org.ops4j.datasource-PRODUCT.cfg" config file is present and >>> identical on both nodes. I see the data source initialization occur in node >>> 1's log, but nothing in node 2's log. As soon as I bring down node 1, then >>> I see the data sources on node 2 initialize. >>> >>> Could this have anything to do with ActiveMQ master/slave? That's the >>> only constant here. It seems unlikely, but I'm out of ideas. >>> >>> >>> >>> On Thursday, March 8, 2018 at 2:32:39 AM UTC-5, Jean-Baptiste Onofré >>> wrote: >>>> >>>> Hi, >>>> >>>> Cellar syncs the bundles and config on the cluster. >>>> >>>> By default, it doesn't sync "local" config (basically the etc/*.cfg >>>> files). To do so, you have to enable the local listener in etc/ >>>> org.apache.karaf.cellar.groups.cfg. >>>> >>>> Else, you have to use cluster:config-property-set to create the config >>>> on the cluster. >>>> >>>> Regards >>>> JB >>>> >>>> On 03/08/2018 08:22 AM, Christian Schneider wrote: >>>> >>>> I am not experienced with cellar but generally I would expect that in >>>> an active/active setup both machines start the same services. >>>> >>>> So first thing to check is if the pax-jdbc features and bundles are >>>> active on the second node. If they are active you can check the log to see >>>> if pax-jdbc reports that something is missing. >>>> >>>> When you describe that the datasources on the second machine come up >>>> when the first machine goes down it sounds like you have setup something >>>> like a master slave setup in cellar. >>>> >>>> Christian >>>> >>>> 2018-03-07 22:55 GMT+01:00 Matthew Zipay <[email protected]>: >>>> >>>>> I really like the approach that Pax JDBC introduced for managing data >>>>> sources, but I am running into an issue that may require me to abandon it >>>>> if I can't get it resolved. >>>>> >>>>> My setup is as follows: >>>>> ServiceMix 7.0.1 (Karaf 4.0.9) running on two nodes, clustered with >>>>> Cellar (active/active). ActiveMQ is JDBC master/slave. Using Pax JDBC >>>>> 1.0.1 >>>>> (config, pool, and oracle and mssql adapters). >>>>> >>>>> I have five (5) data sources configured for the various databases in >>>>> use by this system. What I see is that the data sources are only available >>>>> on the *first* node in the cluster that comes up. When the second node >>>>> comes up, even though it also has the data source cfg's, the data sources >>>>> never get initialized, and all of my bundles that use the data sources are >>>>> stuck perpetually in GracePeriod waiting on the data sources (confirmed >>>>> with bundle:diag). >>>>> >>>>> If I bring down the first node, *then* the data sources on the second >>>>> node suddenly spring to life and all's well. But this is not the behavior >>>>> I would desire or expect, and it may be showstopper for me w/r/t Pax JDBC. >>>>> I need those data sources available on both nodes. >>>>> >>>>> Is this expected? If not, any ideas how I can work around it? >>>>> -- >>>>> -- >>>>> ------------------ >>>>> OPS4J - http://www.ops4j.org - [email protected] >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OPS4J" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Christian Schneider >>>> http://www.liquid-reality.de >>>> >>>> Computer Scientist >>>> http://www.adobe.com >>>> >>>> -- >>>> -- >>>> ------------------ >>>> OPS4J - http://www.ops4j.org - [email protected] >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OPS4J" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>> -- >>> ------------------ >>> OPS4J - http://www.ops4j.org - [email protected] >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OPS4J" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> -- >> Christian Schneider >> http://www.liquid-reality.de >> >> Computer Scientist >> http://www.adobe.com >> >> -- >> -- >> ------------------ >> >> OPS4J - http://www.ops4j.org - [email protected] >> >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "OPS4J" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/to >> pic/ops4j/LHv7sqWkZTg/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > -- > ------------------ > OPS4J - http://www.ops4j.org - [email protected] > > --- > You received this message because you are subscribed to the Google Groups > "OPS4J" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- -- Christian Schneider http://www.liquid-reality.de Computer Scientist http://www.adobe.com -- -- ------------------ OPS4J - http://www.ops4j.org - [email protected] --- You received this message because you are subscribed to the Google Groups "OPS4J" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
