Hi, thanks for the your detailed update on the scheduler.
I did the change in Karaf 4.2.x to be able to support Quartz scheduler in cluster mode (especially when used with Cellar). I worked fine when I tested, using dynamic import, so I'm a bit surprised it doesn't work out of the box for you. Let me take a look on your patch file to identify the missing parts. In the mean time, do you mind to prepare a PR ? Thanks ! Regards JB On 07/09/2018 14:22, Miroslav Beranič wrote: > Hi guys, > > I am working and talking about Apache Karaf version 4.2.2-SNAPSHOT ( > current master on Github ). > > I was rally happy to see Scheduler service with Quartz as default > implementation. Without much thinking, I've hooked up Quartz with JDBC ( > I would really like Karaf Scheduler ( Quartz ) to use existing > Datasource and JPA but no luck ). > > When I moved forward, I did not care about JPA that much any more, as > even JDBC was not working. To begin, this is what I've changed in my > Karaf configuration: > > file: etc/org.apache.karaf.scheduler.quartz.cfg > org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX > org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate > org.quartz.jobStore.useProperties=false > org.quartz.jobStore.dataSource=MyQuartzDS > org.quartz.jobStore.isClustered=true > org.quartz.jobStore.tablePrefix=q_ # Oracle has 30 char limit on > identifier, I started to cry, when I saw error message in SQL terminal. > I use Oracle 12c1 > org.quartz.jobStore.clusterCheckinInterval=30000 > > org.quartz.dataSource.MyQuartzDS.driver=oracle.jdbc.driver.OracleDriver > org.quartz.dataSource.MyQuartzDS.URL=jdbc:oracle:thin:@//oracle.local:1521/test > org.quartz.dataSource.MyQuartzDS.user=APP > org.quartz.dataSource.MyQuartzDS.password=app > org.quartz.dataSource.MyQuartzDS.maxConnections=5 > org.quartz.dataSource.MyQuartzDS.validationQuery=select * from dual; > > org.quartz.scheduler.jmx.export=true > org.quartz.scheduler.rmi.export=false > org.quartz.scheduler.rmi.proxy=false > > This seems to be legit. As I saw in > org.apache.karaf.scheduler.core.Activator#doStart method, all / only > properties with key prefix "org.quartz" are forwarded to Quartz. Looks > OK, but I guess "org.quartz." ( with a dot at the end ) would be more > correct. But, minor issue. > > Lets start it up. Well, no - it will not. To not write all here, all > error messages, all missing error messages, but strange behavior, let me > go to, what I think, is a solution. > > So steps to make your Karaf Scheduler ( 4.2.x ) work with Quartz ( > 2.2.x ) , I did this: > > 1.) Karaf 4.2.2 is using Quartz 2.2.1, that is OK, it could be upgraded > to 2.3.0, but I do not need it ( as I am porting application from Quartz > 1.8.5, this is way of my list of worries ) > 2.) scheduler/pom.xml has package imports defined as: > <Import-Package> > !com.mchange.*, > !oracle.*, > !org.quartz.*, > !weblogic.*, > !javax.transaction, > javax.servlet*;resolution:=optional, > org.jboss.*;resolution:=optional, > * > </Import-Package> > > ( I really am not OSGi expert, and all, but this looked strange to me ), > turned out be "true" ( I guess, as I am open to option, I am wrong, but > end of the day - this works for me, so for me -t his is correct ) > > I changed this to: > <Import-Package> > com.mchange.*;resolution:=optional, > oracle.*;resolution:=optional, > org.quartz.*, > !weblogic.*, > javax.transaction, > javax.servlet*;resolution:=optional, > org.jboss.*;resolution:=optional, > * > </Import-Package> > > ( removed all the ! in front of the packages and added > resolution:=optional, as not all users want persistent Quartz jobs , I > guess ) > > This is not all, now, when I deploy to Karaf, all sort of issue is > reported. Turned out I need to install > org.apache.servicemix.bundles.quartz bundle, from this url: > > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > > ( I did not install it from the URL, Karaf shell is doing this ... ) > > but look, here is > <!-- > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > --> > <dependency> > <groupId>org.apache.servicemix.bundles</groupId> > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > <version>2.2.1_1</version> > </dependency> > > Nope, it will not work. As, same problem with Apache Karaf Scheduler's > POM file. Wrong imports. I guess someone found out this is wrong, and > created this version ( not direct next version, but few versions later on ): > > <!-- > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > --> > <dependency> > <groupId>org.apache.servicemix.bundles</groupId> > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > <version>2.3.0_2</version> > </dependency> > > But wait ... sure, this has different Quartz version. How nice , why not > ... If we can upgrade, we should do it. But at this point, I was really > crying ... for muliple reasons, ... I was to much focused into my path > and versions I've decided to clone repository and change Quartz version > and name it 2.2.5. > So: > <dependency> > <groupId>org.apache.servicemix.bundles</groupId> > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > <version>2.3.0_2</version> > </dependency> > with changed Quartz version to 2.2.1 is "my" > <dependency> > <groupId>org.apache.servicemix.bundles</groupId> > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > <version>2.2.5</version> > </dependency> > > ( was this good or not, again: it works for me now, and I ran out of > hair to pull them out ... so, keep calm and move forward ) > > But this is not end of problems. I forgot what exactly was wrong at this > point. > > I've installed feature: > > scr transaction jndi jpa hibernate pac-jdbc-oracle ( my own oracle jdbc > bundle ) aries-blueprint scheduler karaf-scheduler-example > > and bundles: > bundle:install > mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.c3p0/0.9.5.2_3-SNAPSHOT > > Looook, I am starting to see something. Yeey baby: > > 2018-09-07T09:43:58,678 | INFO | activator-1-thread-1 | > JobStoreTX | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Using db table-based data access locking > (synchronization). > 2018-09-07T09:43:58,680 | INFO | activator-1-thread-1 | > JobStoreTX | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | JobStoreTX initialized. > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > QuartzScheduler | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Scheduler meta-data: Quartz Scheduler (v2.2.1) > 'MyScheduler' with instanceId 'my' > Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. > NOT STARTED. > Currently in standby mode. > Number of jobs executed: 0 > Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 20 threads. > Using job-store 'org.quartz.impl.jdbcjobstore.JobStoreTX' - which > supports persistence. and is clustered. > > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > StdSchedulerFactory | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Quartz scheduler 'MyScheduler' initialized from an > externally provided properties instance. > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > StdSchedulerFactory | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Quartz scheduler version: 2.2.1 > 2018-09-07T09:43:58,713 | INFO | activator-1-thread-1 | > AbstractPoolBackedDataSource | 107 - > org.apache.servicemix.bundles.c3p0 - 0.9.5.2_3-SNAPSHOT | Initializing > c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ > acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> > 1000, autoCommitOnClose -> false, automaticTestTable -> null, > breakAfterAcquireFailure -> false, checkoutTimeout -> 0, > connectionCustomizerClassName -> null, connectionTesterClassName -> > com.mchange.v2.c3p0.impl.DefaultConnectionTester, > contextClassLoaderSource -> caller, dataSourceName -> > z8kfsx9x1dnli4w1yck8tp|3c42672a, debugUnreturnedConnectionStackTraces -> > false, description -> null, driverClass -> > oracle.jdbc.driver.OracleDriver, extensions -> {}, factoryClassLocation > -> null, forceIgnoreUnresolvedTransactions -> false, > forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, > identityToken -> z8kfsx9x1dnli4w1yck8tp|3c42672a, > idleConnectionTestPeriod -> 50, initialPoolSize -> 3, jdbcUrl -> > jdbc:oracle:thin:@//oracle.local:1521/test, maxAdministrativeTaskTime -> > 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections > -> 0, maxPoolSize -> 5, maxStatements -> 0, maxStatementsPerConnection > -> 120, minPoolSize -> 1, numHelperThreads -> 3, preferredTestQuery -> > select * from dual;, privilegeSpawnedThreads -> false, properties -> > {user=******, password=******}, propertyCycle -> 0, > statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> > true, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> > 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ] > 2018-09-07T09:43:58,820 | INFO | activator-1-thread-1 | > JobStoreTX | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | ClusterManager: detected 1 failed or restarted instances. > 2018-09-07T09:43:58,821 | INFO | activator-1-thread-1 | > JobStoreTX | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | ClusterManager: Scanning for instance "my"'s failed > in-progress jobs. > 2018-09-07T09:43:58,826 | INFO | activator-1-thread-1 | > QuartzScheduler | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Scheduler MyScheduler$_my started. > 2018-09-07T09:43:58,837 | INFO | activator-1-thread-1 | > CommandExtension | 35 - org.apache.karaf.shell.core - > 4.2.2.SNAPSHOT | Registering commands for bundle > org.apache.karaf.scheduler.core/4.2.2.SNAPSHOT > > This is it! I did it. Time for big coffee. And than I came back at the > terminal and saw this: > > 2018-09-07T09:43:59,021 | WARN | activator-1-thread-1 | > WhiteboardHandler | 105 - org.apache.karaf.scheduler.core > - 4.2.2.SNAPSHOT | Error scheduling job > org.apache.karaf.scheduler.SchedulerError: > org.quartz.JobPersistenceException: Couldn't store job: Unable to > serialize JobDataMap for insertion into database because the value of > property 'QuartzJobScheduler.Logger' is not serializable: > org.ops4j.pax.logging.slf4j.Slf4jLogger [See nested exception: > java.io.NotSerializableException: Unable to serialize JobDataMap for > insertion into database because the value of property > 'QuartzJobScheduler.Logger' is not serializable: > org.ops4j.pax.logging.slf4j.Slf4jLogger] > at > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:239) > ~[105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.apache.karaf.scheduler.core.WhiteboardHandler.register(WhiteboardHandler.java:140) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.apache.karaf.scheduler.core.WhiteboardHandler.access$100(WhiteboardHandler.java:37) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.apache.karaf.scheduler.core.WhiteboardHandler$1.addingService(WhiteboardHandler.java:66) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:941) > [?:?] > at > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:870) > [?:?] > at > org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256) > [?:?] > at > org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183) > [?:?] > at > org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:318) [?:?] > at > org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:261) [?:?] > at > org.apache.karaf.scheduler.core.WhiteboardHandler.<init>(WhiteboardHandler.java:71) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.apache.karaf.scheduler.core.Activator.doStart(Activator.java:60) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:275) > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:?] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:?] > at java.lang.Thread.run(Thread.java:748) [?:?] > Caused by: org.quartz.JobPersistenceException: Couldn't store job: > Unable to serialize JobDataMap for insertion into database because the > value of property 'QuartzJobScheduler.Logger' is not serializable: > org.ops4j.pax.logging.slf4j.Slf4jLogger > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1115) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058) > ~[?:?] > at > org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886) ~[?:?] > at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249) > ~[?:?] > at > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:237) > ~[?:?] > ... 17 more > Caused by: java.io.NotSerializableException: Unable to serialize > JobDataMap for insertion into database because the value of property > 'QuartzJobScheduler.Logger' is not serializable: > org.ops4j.pax.logging.slf4j.Slf4jLogger > at > org.quartz.impl.jdbcjobstore.StdJDBCDelegate.serializeJobData(StdJDBCDelegate.java:3083) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.insertJobDetail(OracleDelegate.java:154) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1112) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93) > ~[?:?] > at > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058) > ~[?:?] > at > org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886) ~[?:?] > at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249) > ~[?:?] > at > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:237) > ~[?:?] > ... 17 more > > > WAAAT!! No no no no. Well, this was just the beginning and I did not > know this at that time. So I started to look for > 'QuartzJobScheduler.Logger' in Karaf soruce code. Waaat, who thought it > is smart to store non-serializable data in Job data, whaaat. > At that point I started to realize, why it was so much pain to get > persistence working ... ( I guess ) no one tried or even wanted to make > Quartz persistent in Karaf - since this code is in from 2014. > > Scheduling is like one of the five most important features that I need > from the platform. I started to cry again. I did not test features, > before I pushed for the migration. There is no way back, so what can be > done now here. > > Let's see. > > I basically "re-wrote" Scheduler service in Karaf. What I did: > > 1.) Make sure Job data has only Serializable data - > - ScheduleOptions was made Serializable, with addition to move Trigger > creation from InternalScheduleOptions's constructor to method compile(). > - getJobs() method from Scheduler interface returns String names and not > Objects > - Updated QuartzJobExecutor to remove non-serializable items from > JobDataMap storage > > 2.) > - Added custom StdScheduler and StdSchedulerFactory classes, to > implemented changes > - Added SchedulerStorage as support for "backward compatibility" and > "short lived" jobs. This is not good place to use it. > > 3.) > - Updated Scheduler Shell commands to use changed Scheduler API ( job > names are as keys ) > > Patch file is in this email attachements, I've also pushed changes to my > Github repository, with only this changes: > > https://github.com/mibesis/apache-karaf/tree/karaf-4.2.2-quartz-jdbc > > > Anyone that is willing to use it -- please do, I would only ask Karf > core development team to "make sure" Quartz Job data is always fill with > Serializable data, as this makes all the difference and there is no need > for method getJobs() from Sheculer to return > Map<Object, ScheduleOptions> as it can not make data Serializable -- to > return Map<Object, ScheduleOptions> is OK, as all API calls are made > over keys, names, etc. > > I need few days, to come over this and coffee cups, it is Friday now, so > I guess I will grab some cold drink ... it was big learing experiance > for me, to make this work. > > So this is my first patch file, if there is something else needed or I > can support in someway, let me know. ( I know SchedulerStorage is not > perfect solution, but again: it works for me ; but I agree, this can be > made better ) > > > I know I do not say it much ( ever ) -- Apache Karaf is really great > platform, it is just that I send email, when I have problems :D ... > > Kind Regards, > Miroslav > > > -- > Miroslav Beranič > MIBESIS > +386(0)40/814-843 > [email protected] <mailto:[email protected]> > http://www.mibesis.si -- Jean-Baptiste Onofré [email protected] http://blog.nanthrax.net Talend - http://www.talend.com
