Hi JB, Scheduler with use on Cellar is my semi-next task, scheduled for beginning of October.
PR? Sorry, I am not into this, so PR stands for? Kind Regards, Miroslav V V pet., 7. sep. 2018 ob 14:48 je oseba Jean-Baptiste Onofré < [email protected]> napisala: > Hi, > > thanks for the your detailed update on the scheduler. > > I did the change in Karaf 4.2.x to be able to support Quartz scheduler > in cluster mode (especially when used with Cellar). > > I worked fine when I tested, using dynamic import, so I'm a bit > surprised it doesn't work out of the box for you. > > Let me take a look on your patch file to identify the missing parts. In > the mean time, do you mind to prepare a PR ? > > Thanks ! > Regards > JB > > On 07/09/2018 14:22, Miroslav Beranič wrote: > > Hi guys, > > > > I am working and talking about Apache Karaf version 4.2.2-SNAPSHOT ( > > current master on Github ). > > > > I was rally happy to see Scheduler service with Quartz as default > > implementation. Without much thinking, I've hooked up Quartz with JDBC ( > > I would really like Karaf Scheduler ( Quartz ) to use existing > > Datasource and JPA but no luck ). > > > > When I moved forward, I did not care about JPA that much any more, as > > even JDBC was not working. To begin, this is what I've changed in my > > Karaf configuration: > > > > file: etc/org.apache.karaf.scheduler.quartz.cfg > > org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX > > > org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.oracle.OracleDelegate > > org.quartz.jobStore.useProperties=false > > org.quartz.jobStore.dataSource=MyQuartzDS > > org.quartz.jobStore.isClustered=true > > org.quartz.jobStore.tablePrefix=q_ # Oracle has 30 char limit on > > identifier, I started to cry, when I saw error message in SQL terminal. > > I use Oracle 12c1 > > org.quartz.jobStore.clusterCheckinInterval=30000 > > > > org.quartz.dataSource.MyQuartzDS.driver=oracle.jdbc.driver.OracleDriver > > org.quartz.dataSource.MyQuartzDS.URL=jdbc:oracle:thin:@ > //oracle.local:1521/test > > org.quartz.dataSource.MyQuartzDS.user=APP > > org.quartz.dataSource.MyQuartzDS.password=app > > org.quartz.dataSource.MyQuartzDS.maxConnections=5 > > org.quartz.dataSource.MyQuartzDS.validationQuery=select * from dual; > > > > org.quartz.scheduler.jmx.export=true > > org.quartz.scheduler.rmi.export=false > > org.quartz.scheduler.rmi.proxy=false > > > > This seems to be legit. As I saw in > > org.apache.karaf.scheduler.core.Activator#doStart method, all / only > > properties with key prefix "org.quartz" are forwarded to Quartz. Looks > > OK, but I guess "org.quartz." ( with a dot at the end ) would be more > > correct. But, minor issue. > > > > Lets start it up. Well, no - it will not. To not write all here, all > > error messages, all missing error messages, but strange behavior, let me > > go to, what I think, is a solution. > > > > So steps to make your Karaf Scheduler ( 4.2.x ) work with Quartz ( > > 2.2.x ) , I did this: > > > > 1.) Karaf 4.2.2 is using Quartz 2.2.1, that is OK, it could be upgraded > > to 2.3.0, but I do not need it ( as I am porting application from Quartz > > 1.8.5, this is way of my list of worries ) > > 2.) scheduler/pom.xml has package imports defined as: > > <Import-Package> > > !com.mchange.*, > > !oracle.*, > > !org.quartz.*, > > !weblogic.*, > > !javax.transaction, > > javax.servlet*;resolution:=optional, > > org.jboss.*;resolution:=optional, > > * > > </Import-Package> > > > > ( I really am not OSGi expert, and all, but this looked strange to me ), > > turned out be "true" ( I guess, as I am open to option, I am wrong, but > > end of the day - this works for me, so for me -t his is correct ) > > > > I changed this to: > > <Import-Package> > > com.mchange.*;resolution:=optional, > > oracle.*;resolution:=optional, > > org.quartz.*, > > !weblogic.*, > > javax.transaction, > > javax.servlet*;resolution:=optional, > > org.jboss.*;resolution:=optional, > > * > > </Import-Package> > > > > ( removed all the ! in front of the packages and added > > resolution:=optional, as not all users want persistent Quartz jobs , I > > guess ) > > > > This is not all, now, when I deploy to Karaf, all sort of issue is > > reported. Turned out I need to install > > org.apache.servicemix.bundles.quartz bundle, from this url: > > > > > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > > > > ( I did not install it from the URL, Karaf shell is doing this ... ) > > > > but look, here is > > <!-- > > > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > > --> > > <dependency> > > <groupId>org.apache.servicemix.bundles</groupId> > > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > > <version>2.2.1_1</version> > > </dependency> > > > > Nope, it will not work. As, same problem with Apache Karaf Scheduler's > > POM file. Wrong imports. I guess someone found out this is wrong, and > > created this version ( not direct next version, but few versions later > on ): > > > > <!-- > > > https://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.quartz > > --> > > <dependency> > > <groupId>org.apache.servicemix.bundles</groupId> > > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > > <version>2.3.0_2</version> > > </dependency> > > > > But wait ... sure, this has different Quartz version. How nice , why not > > ... If we can upgrade, we should do it. But at this point, I was really > > crying ... for muliple reasons, ... I was to much focused into my path > > and versions I've decided to clone repository and change Quartz version > > and name it 2.2.5. > > So: > > <dependency> > > <groupId>org.apache.servicemix.bundles</groupId> > > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > > <version>2.3.0_2</version> > > </dependency> > > with changed Quartz version to 2.2.1 is "my" > > <dependency> > > <groupId>org.apache.servicemix.bundles</groupId> > > <artifactId>org.apache.servicemix.bundles.quartz</artifactId> > > <version>2.2.5</version> > > </dependency> > > > > ( was this good or not, again: it works for me now, and I ran out of > > hair to pull them out ... so, keep calm and move forward ) > > > > But this is not end of problems. I forgot what exactly was wrong at this > > point. > > > > I've installed feature: > > > > scr transaction jndi jpa hibernate pac-jdbc-oracle ( my own oracle jdbc > > bundle ) aries-blueprint scheduler karaf-scheduler-example > > > > and bundles: > > bundle:install > > > mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.c3p0/0.9.5.2_3-SNAPSHOT > > > > Looook, I am starting to see something. Yeey baby: > > > > 2018-09-07T09:43:58,678 | INFO | activator-1-thread-1 | > > JobStoreTX | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Using db table-based data access locking > > (synchronization). > > 2018-09-07T09:43:58,680 | INFO | activator-1-thread-1 | > > JobStoreTX | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | JobStoreTX initialized. > > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > > QuartzScheduler | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Scheduler meta-data: Quartz Scheduler (v2.2.1) > > 'MyScheduler' with instanceId 'my' > > Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. > > NOT STARTED. > > Currently in standby mode. > > Number of jobs executed: 0 > > Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 20 > threads. > > Using job-store 'org.quartz.impl.jdbcjobstore.JobStoreTX' - which > > supports persistence. and is clustered. > > > > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > > StdSchedulerFactory | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Quartz scheduler 'MyScheduler' initialized from an > > externally provided properties instance. > > 2018-09-07T09:43:58,689 | INFO | activator-1-thread-1 | > > StdSchedulerFactory | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Quartz scheduler version: 2.2.1 > > 2018-09-07T09:43:58,713 | INFO | activator-1-thread-1 | > > AbstractPoolBackedDataSource | 107 - > > org.apache.servicemix.bundles.c3p0 - 0.9.5.2_3-SNAPSHOT | Initializing > > c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ > > acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> > > 1000, autoCommitOnClose -> false, automaticTestTable -> null, > > breakAfterAcquireFailure -> false, checkoutTimeout -> 0, > > connectionCustomizerClassName -> null, connectionTesterClassName -> > > com.mchange.v2.c3p0.impl.DefaultConnectionTester, > > contextClassLoaderSource -> caller, dataSourceName -> > > z8kfsx9x1dnli4w1yck8tp|3c42672a, debugUnreturnedConnectionStackTraces -> > > false, description -> null, driverClass -> > > oracle.jdbc.driver.OracleDriver, extensions -> {}, factoryClassLocation > > -> null, forceIgnoreUnresolvedTransactions -> false, > > forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, > > identityToken -> z8kfsx9x1dnli4w1yck8tp|3c42672a, > > idleConnectionTestPeriod -> 50, initialPoolSize -> 3, jdbcUrl -> > > jdbc:oracle:thin:@//oracle.local:1521/test, maxAdministrativeTaskTime -> > > 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections > > -> 0, maxPoolSize -> 5, maxStatements -> 0, maxStatementsPerConnection > > -> 120, minPoolSize -> 1, numHelperThreads -> 3, preferredTestQuery -> > > select * from dual;, privilegeSpawnedThreads -> false, properties -> > > {user=******, password=******}, propertyCycle -> 0, > > statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> > > true, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> > > 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ] > > 2018-09-07T09:43:58,820 | INFO | activator-1-thread-1 | > > JobStoreTX | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | ClusterManager: detected 1 failed or restarted > instances. > > 2018-09-07T09:43:58,821 | INFO | activator-1-thread-1 | > > JobStoreTX | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | ClusterManager: Scanning for instance "my"'s failed > > in-progress jobs. > > 2018-09-07T09:43:58,826 | INFO | activator-1-thread-1 | > > QuartzScheduler | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Scheduler MyScheduler$_my started. > > 2018-09-07T09:43:58,837 | INFO | activator-1-thread-1 | > > CommandExtension | 35 - org.apache.karaf.shell.core - > > 4.2.2.SNAPSHOT | Registering commands for bundle > > org.apache.karaf.scheduler.core/4.2.2.SNAPSHOT > > > > This is it! I did it. Time for big coffee. And than I came back at the > > terminal and saw this: > > > > 2018-09-07T09:43:59,021 | WARN | activator-1-thread-1 | > > WhiteboardHandler | 105 - org.apache.karaf.scheduler.core > > - 4.2.2.SNAPSHOT | Error scheduling job > > org.apache.karaf.scheduler.SchedulerError: > > org.quartz.JobPersistenceException: Couldn't store job: Unable to > > serialize JobDataMap for insertion into database because the value of > > property 'QuartzJobScheduler.Logger' is not serializable: > > org.ops4j.pax.logging.slf4j.Slf4jLogger [See nested exception: > > java.io.NotSerializableException: Unable to serialize JobDataMap for > > insertion into database because the value of property > > 'QuartzJobScheduler.Logger' is not serializable: > > org.ops4j.pax.logging.slf4j.Slf4jLogger] > > at > > > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:239) > > ~[105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > > org.apache.karaf.scheduler.core.WhiteboardHandler.register(WhiteboardHandler.java:140) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > > org.apache.karaf.scheduler.core.WhiteboardHandler.access$100(WhiteboardHandler.java:37) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > > org.apache.karaf.scheduler.core.WhiteboardHandler$1.addingService(WhiteboardHandler.java:66) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:941) > > [?:?] > > at > > > org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:870) > > [?:?] > > at > > > org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256) > > [?:?] > > at > > > org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183) > > [?:?] > > at > > org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:318) [?:?] > > at > > org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:261) [?:?] > > at > > > org.apache.karaf.scheduler.core.WhiteboardHandler.<init>(WhiteboardHandler.java:71) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > org.apache.karaf.scheduler.core.Activator.doStart(Activator.java:60) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:275) > > [105:org.apache.karaf.scheduler.core:4.2.2.SNAPSHOT] > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > [?:?] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?] > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > [?:?] > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > [?:?] > > at java.lang.Thread.run(Thread.java:748) [?:?] > > Caused by: org.quartz.JobPersistenceException: Couldn't store job: > > Unable to serialize JobDataMap for insertion into database because the > > value of property 'QuartzJobScheduler.Logger' is not serializable: > > org.ops4j.pax.logging.slf4j.Slf4jLogger > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1115) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93) > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058) > > ~[?:?] > > at > > org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886) > ~[?:?] > > at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249) > > ~[?:?] > > at > > > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:237) > > ~[?:?] > > ... 17 more > > Caused by: java.io.NotSerializableException: Unable to serialize > > JobDataMap for insertion into database because the value of property > > 'QuartzJobScheduler.Logger' is not serializable: > > org.ops4j.pax.logging.slf4j.Slf4jLogger > > at > > > org.quartz.impl.jdbcjobstore.StdJDBCDelegate.serializeJobData(StdJDBCDelegate.java:3083) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.insertJobDetail(OracleDelegate.java:154) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1112) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787) > > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93) > ~[?:?] > > at > > > org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058) > > ~[?:?] > > at > > org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886) > ~[?:?] > > at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249) > > ~[?:?] > > at > > > org.apache.karaf.scheduler.core.QuartzScheduler.schedule(QuartzScheduler.java:237) > > ~[?:?] > > ... 17 more > > > > > > WAAAT!! No no no no. Well, this was just the beginning and I did not > > know this at that time. So I started to look for > > 'QuartzJobScheduler.Logger' in Karaf soruce code. Waaat, who thought it > > is smart to store non-serializable data in Job data, whaaat. > > At that point I started to realize, why it was so much pain to get > > persistence working ... ( I guess ) no one tried or even wanted to make > > Quartz persistent in Karaf - since this code is in from 2014. > > > > Scheduling is like one of the five most important features that I need > > from the platform. I started to cry again. I did not test features, > > before I pushed for the migration. There is no way back, so what can be > > done now here. > > > > Let's see. > > > > I basically "re-wrote" Scheduler service in Karaf. What I did: > > > > 1.) Make sure Job data has only Serializable data - > > - ScheduleOptions was made Serializable, with addition to move Trigger > > creation from InternalScheduleOptions's constructor to method compile(). > > - getJobs() method from Scheduler interface returns String names and not > > Objects > > - Updated QuartzJobExecutor to remove non-serializable items from > > JobDataMap storage > > > > 2.) > > - Added custom StdScheduler and StdSchedulerFactory classes, to > > implemented changes > > - Added SchedulerStorage as support for "backward compatibility" and > > "short lived" jobs. This is not good place to use it. > > > > 3.) > > - Updated Scheduler Shell commands to use changed Scheduler API ( job > > names are as keys ) > > > > Patch file is in this email attachements, I've also pushed changes to my > > Github repository, with only this changes: > > > > https://github.com/mibesis/apache-karaf/tree/karaf-4.2.2-quartz-jdbc > > > > > > Anyone that is willing to use it -- please do, I would only ask Karf > > core development team to "make sure" Quartz Job data is always fill with > > Serializable data, as this makes all the difference and there is no need > > for method getJobs() from Sheculer to return > > Map<Object, ScheduleOptions> as it can not make data Serializable -- to > > return Map<Object, ScheduleOptions> is OK, as all API calls are made > > over keys, names, etc. > > > > I need few days, to come over this and coffee cups, it is Friday now, so > > I guess I will grab some cold drink ... it was big learing experiance > > for me, to make this work. > > > > So this is my first patch file, if there is something else needed or I > > can support in someway, let me know. ( I know SchedulerStorage is not > > perfect solution, but again: it works for me ; but I agree, this can be > > made better ) > > > > > > I know I do not say it much ( ever ) -- Apache Karaf is really great > > platform, it is just that I send email, when I have problems :D ... > > > > Kind Regards, > > Miroslav > > > > > > -- > > Miroslav Beranič > > MIBESIS > > +386(0)40/814-843 > > [email protected] <mailto:[email protected]> > > http://www.mibesis.si > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com > -- Miroslav Beranič MIBESIS +386(0)40/814-843 [email protected] http://www.mibesis.si
