Yes, the Karaf instances are on different machines, and I have removed Cellar entirely from the system.
If I simply start up both instances, then everything *except* the data sources comes up fine on both instances. Because of this, I am temporarily running this system as active/failover (karaf.lock=true) until I can get to the bottom of this data source issue. As far as what I see in the logs... on the second node (the one where the data sources do not initialize), I do see that the cfg's are found, and there are logging entries stating that they will be initialized, but then nothing happens. No errors, nothing. Then if I then stop node 1, I almost immediately see the data sources initialize on node 2. (But, of course, if I then restart node 1, same issue - the data sources do not initialize on *that* node). Throughout all of the variations I have tested, there has been one constant: - ActiveMQ. It is configured using JDBC master/slave. It doesn't seem like this should cause Pax JDBC data sources to not initialize (ActiveMQ is not sharing the data source - entirely separate configuration), but that's the only thing I can think of: when ActiveMQ master releases the lock and allows the slave to become the new master, that's when I see Pax JDBC initialize the data sources. Thoughts? On Fri, Mar 9, 2018 at 2:38 AM, Christian Schneider <ch...@die-schneider.net > wrote: > Are the karaf instances on different machines? > If they are on different machines and you do not use cellar or specially > configured locking then both should come up completely independent from > each other. > > Do you see anything in the log of the second instance when you stop the > first one? I wonder what could trigger the data source come up in this > moment. > > Christian > > 2018-03-08 17:30 GMT+01:00 Matthew Zipay <mzi...@gmail.com>: > >> Thanks for the quick replies. With respect to Cellar, I should clarify >> that the Pax JDBC issue persists whether or not Cellar is involved (Cellar >> was a recent addition to the system, specifically to see if its explicit >> support for active/active would resolve the Pax JDBC issue - it did not). >> >> I have tried two variations: >> (1) Without Cellar, bring up both Karaf nodes without Karaf locking >> (karaf.lock=false). (i.e. "unsupported" active/active) >> (2) With Cellar, bring up both Karaf nodes without Karaf locking >> (karaf.lock=false). (i.e. "supported" active/active) >> >> In either scenario, the data sources only initialize on the *first* node >> that comes up. >> >> In either scenario, I can confirm that Pax JDBC bundles are started on >> both nodes: >> >> Node 1: >> 228 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Generic Driver Extender >> 229 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Config >> 230 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Pooling Support Base >> 270 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC MSSQL Driver Adapter >> 271 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Oracle Driver Adapter >> 272 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Pooling DBCP2 >> >> Node 2: >> 228 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Generic Driver Extender >> 229 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Config >> 230 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Pooling Support Base >> 270 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC MSSQL Driver Adapter >> 271 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Oracle Driver Adapter >> 272 | Active | 80 | 1.0.1 | OPS4J Pax >> JDBC Pooling DBCP2 >> >> However, jdbc:ds-list only shows the initialized data sources on node 1; >> on node 2 *no* data sources are initialized, and I have multiple bundles in >> GracePeriod waiting for data sources that never show up. For example, on >> node 2: >> >> admin@root>diag 331 >> SKU Resolution (331) >> ----------------------------------------------- >> Status: GracePeriod >> Blueprint >> 3/8/18 11:10 AM >> Missing dependencies: >> (&(dataSourceName=PRODUCTDS)(objectClass=javax.sql.DataSource)) >> >> >> The "org.ops4j.datasource-PRODUCT.cfg" config file is present and >> identical on both nodes. I see the data source initialization occur in node >> 1's log, but nothing in node 2's log. As soon as I bring down node 1, then >> I see the data sources on node 2 initialize. >> >> Could this have anything to do with ActiveMQ master/slave? That's the >> only constant here. It seems unlikely, but I'm out of ideas. >> >> >> >> On Thursday, March 8, 2018 at 2:32:39 AM UTC-5, Jean-Baptiste Onofré >> wrote: >>> >>> Hi, >>> >>> Cellar syncs the bundles and config on the cluster. >>> >>> By default, it doesn't sync "local" config (basically the etc/*.cfg >>> files). To do so, you have to enable the local listener in etc/ >>> org.apache.karaf.cellar.groups.cfg. >>> >>> Else, you have to use cluster:config-property-set to create the config >>> on the cluster. >>> >>> Regards >>> JB >>> >>> On 03/08/2018 08:22 AM, Christian Schneider wrote: >>> >>> I am not experienced with cellar but generally I would expect that in an >>> active/active setup both machines start the same services. >>> >>> So first thing to check is if the pax-jdbc features and bundles are >>> active on the second node. If they are active you can check the log to see >>> if pax-jdbc reports that something is missing. >>> >>> When you describe that the datasources on the second machine come up >>> when the first machine goes down it sounds like you have setup something >>> like a master slave setup in cellar. >>> >>> Christian >>> >>> 2018-03-07 22:55 GMT+01:00 Matthew Zipay <mzi...@gmail.com>: >>> >>>> I really like the approach that Pax JDBC introduced for managing data >>>> sources, but I am running into an issue that may require me to abandon it >>>> if I can't get it resolved. >>>> >>>> My setup is as follows: >>>> ServiceMix 7.0.1 (Karaf 4.0.9) running on two nodes, clustered with >>>> Cellar (active/active). ActiveMQ is JDBC master/slave. Using Pax JDBC 1.0.1 >>>> (config, pool, and oracle and mssql adapters). >>>> >>>> I have five (5) data sources configured for the various databases in >>>> use by this system. What I see is that the data sources are only available >>>> on the *first* node in the cluster that comes up. When the second node >>>> comes up, even though it also has the data source cfg's, the data sources >>>> never get initialized, and all of my bundles that use the data sources are >>>> stuck perpetually in GracePeriod waiting on the data sources (confirmed >>>> with bundle:diag). >>>> >>>> If I bring down the first node, *then* the data sources on the second >>>> node suddenly spring to life and all's well. But this is not the behavior >>>> I would desire or expect, and it may be showstopper for me w/r/t Pax JDBC. >>>> I need those data sources available on both nodes. >>>> >>>> Is this expected? If not, any ideas how I can work around it? >>>> -- >>>> -- >>>> ------------------ >>>> OPS4J - http://www.ops4j.org - op...@googlegroups.com >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OPS4J" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to ops4j+un...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> -- >>> Christian Schneider >>> http://www.liquid-reality.de >>> >>> Computer Scientist >>> http://www.adobe.com >>> >>> -- >>> -- >>> ------------------ >>> OPS4J - http://www.ops4j.org - op...@googlegroups.com >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OPS4J" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to ops4j+un...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >> -- >> ------------------ >> OPS4J - http://www.ops4j.org - ops4j@googlegroups.com >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OPS4J" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to ops4j+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > -- > Christian Schneider > http://www.liquid-reality.de > > Computer Scientist > http://www.adobe.com > > -- > -- > ------------------ > > OPS4J - http://www.ops4j.org - ops4j@googlegroups.com > > --- > You received this message because you are subscribed to a topic in the > Google Groups "OPS4J" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/ops4j/LHv7sqWkZTg/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > ops4j+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- ------------------ OPS4J - http://www.ops4j.org - ops4j@googlegroups.com --- You received this message because you are subscribed to the Google Groups "OPS4J" group. To unsubscribe from this group and stop receiving emails from it, send an email to ops4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
