Hi Romain!
Sorry that it takes that long.
But now I got the tread dump – see attachment jstackTomEE.txt
Best regards,
Thomas
From: Romain Manni-Bucau [via OpenEJB]
[mailto:[email protected]]
Sent: Montag, 12. November 2012 15:35
To: Thomas Schuler
Subject: Re: Use external database for persistent timer service
Hi,
if it hangs can you please take a thread dump at this moment (if you have a
jdk type in a console "jps" -> find Boostrap id then "jstack <id>") please?
*Romain Manni-Bucau*
*Twitter: @rmannibucau <https://twitter.com/rmannibucau>*
*Blog: **http://rmannibucau.wordpress.com/*<http://rmannibucau.wordpress.com/>
*LinkedIn: **http://fr.linkedin.com/in/rmannibucau*
*Github: https://github.com/rmannibucau*
2012/11/12 tschuler <[hidden
email]</user/SendEmail.jtp?type=node&node=4658531&i=0>>
> Hi Romain!
>
> Taking the snapshot from today (Monday 12) looks better but still not good
> enough I think.
>
> The following situation: An automticTimer is running and a programmatic
> timer is added (but yet not fired).
> End TomEE. Wait till trigger time of programmatic trigger passed. Start
> TomEE.
>
> è Startup sequence seems to hang, no access to SchedulerBeanPersisted.
>
> 12.11.2012 13:24:27 org.quartz.core.QuartzScheduler initialize
> INFO: Scheduler meta-data: Quartz Scheduler (v2.1.6) 'MyScheduler' with
> instanceId 'DE064071352723067406'
> Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
> NOT STARTED.
> Currently in standby mode.
> Number of jobs executed: 0
> Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 4 threads.
> Using job-store 'org.quartz.impl.jdbcjobstore.JobStoreCMT' - which
> supports persistence. and is clustered.
> 12.11.2012 13:24:27 org.quartz.impl.StdSchedulerFactory instantiate
> INFO: Quartz scheduler 'MyScheduler' initialized from an externally
> provided properties instance.
> 12.11.2012 13:24:27 org.quartz.impl.StdSchedulerFactory instantiate
> INFO: Quartz scheduler version: 2.1.6
> 12.11.2012 13:24:27 org.quartz.impl.jdbcjobstore.JobStoreSupport
> logWarnIfNonZero
> INFO: ClusterManager: detected 1 failed or restarted instances.
> 12.11.2012 13:24:27 org.quartz.impl.jdbcjobstore.JobStoreSupport
> clusterRecover
> INFO: ClusterManager: Scanning for instance "DE064071352722853574"'s
> failed in-progress jobs.
> 12.11.2012 13:24:27 org.quartz.impl.jdbcjobstore.JobStoreSupport
> logWarnIfNonZero
> INFO: ClusterManager: ......Freed 1 acquired trigger(s).
> 12.11.2012 13:24:27 org.quartz.core.QuartzScheduler start
> INFO: Scheduler MyScheduler_$_DE064071352723067406 started.
> 12.11.2012 13:24:27 org.quartz.impl.jdbcjobstore.JobStoreSupport
> recoverMisfiredJobs
> INFO: Handling 1 trigger(s) that missed their scheduled fire-time.
>
> All following startups of TomEE show the same behavior, but slightly
> different log messages at the end:
> […]
> 12.11.2012 13:45:03 org.quartz.impl.jdbcjobstore.JobStoreSupport
> logWarnIfNonZero
> INFO: ClusterManager: detected 1 failed or restarted instances.
> 12.11.2012 13:45:03 org.quartz.impl.jdbcjobstore.JobStoreSupport
> clusterRecover
> INFO: ClusterManager: Scanning for instance "DE064071352724225831"'s
> failed in-progress jobs.
> 12.11.2012 13:45:03 org.quartz.core.QuartzScheduler start
> INFO: Scheduler MyScheduler_$_DE064071352724303131 started.
> 12.11.2012 13:45:03 org.quartz.impl.jdbcjobstore.JobStoreSupport
> recoverMisfiredJobs
> INFO: Handling 2 trigger(s) that missed their scheduled fire-time.
>
> A cleanup of the quartz tables is needed to get the application running
> again.
>
> Best regards,
> Thomas
>
> From: Romain Manni-Bucau [via OpenEJB] [mailto:
> [hidden email]</user/SendEmail.jtp?type=node&node=4658531&i=1>]
> Sent: Freitag, 9. November 2012 15:30
> To: Thomas Schuler
> Subject: RE: Use external database for persistent timer service
>
> I pushed some stuff yesterday about 2 (well can be linked even if not
> direct) so it should be in the snapshot
> Le 9 nov. 2012 13:16, "tschuler" <[hidden
> email]</user/SendEmail.jtp?type=node&node=4658497&i=0>> a écrit :
>
> > Hi Romain!
> >
> > About 1: It may be a config issue, but I don't think so - doesn't matter
> > right now.
> > About 2: If you have a solution for it, please let me know. Would be
> > included in a future snapshot, I think?
> > About 3: We need to be sure that timer issues using e.g. the actual TomEE
> > version can be executed using the next TomEE version.
> > As there are serialized OpenEJB classes in the quartz
> > database, they may be incompatible with the class definition included in
> a
> > future TomEE version.
> >
> > Best regards,
> > Thomas
> >
> >
> > From: Romain Manni-Bucau [via OpenEJB] [mailto:
> > [hidden email]</user/SendEmail.jtp?type=node&node=4658497&i=1>]
> > Sent: Donnerstag, 8. November 2012 22:38
> > To: Thomas Schuler
> > Subject: Re: Use external database for persistent timer service
> >
> > about 1: maybe just a quartz config
> > about 2: looking into it, think i got why but thinking to the best
> > solution*
> > about 3: we can store it as we want. That said not sure it'd be used, if
> > you change your tomee instance you often reset some datas (and i think
> > these timers can be resetted in such a case, no?)
> >
> > *Romain Manni-Bucau*
> > *Twitter: @rmannibucau <https://twitter.com/rmannibucau>*
> > *Blog: **http://rmannibucau.wordpress.com/*<
> > http://rmannibucau.wordpress.com/>
> > *LinkedIn: **http://fr.linkedin.com/in/rmannibucau*
> > *Github: https://github.com/rmannibucau*
> >
> >
> >
> >
> > 2012/11/8 tschuler <[hidden
> > email]</user/SendEmail.jtp?type=node&node=4658490&i=0>>
> >
> > > Hi!
> > >
> > > I tried todays snapshot and it worked better if TomEE is running all
> the
> > > time.
> > > But in case of TomEE restart a few errors happen for persisted timers.
> > >
> > > 1) automatic timeout:
> > > It takes two minutes until the timer fires again
> > >
> > > 08.11.2012 14:28:46 org.apache.catalina.startup.Catalina start
> > > INFO: Server startup in 2357 ms
> > > 08.11.2012 14:29:45
> org.quartz.impl.jdbcjobstore.JobStoreSupport
> > > logWarnIfNonZero
> > > INFO: ClusterManager: detected 1 failed or restarted instances.
> > > 08.11.2012 14:29:45
> org.quartz.impl.jdbcjobstore.JobStoreSupport
> > > clusterRecover
> > > INFO: ClusterManager: Scanning for instance
> > > "DE064071352380642408"'s
> > > failed in-progress jobs.
> > > 08.11.2012 14:29:45
> org.quartz.impl.jdbcjobstore.JobStoreSupport
> > > logWarnIfNonZero
> > > INFO: ClusterManager: ......Freed 1 acquired trigger(s).
> > > 08.11.2012 14:30:45
> org.quartz.impl.jdbcjobstore.JobStoreSupport
> > > recoverMisfiredJobs
> > > INFO: Handling 1 trigger(s) that missed their scheduled
> > fire-time.
> > > 08.11.2012 14:30:45
> timerPersisted.server.SchedulerBeanAutomated
> > > automaticTimeout
> > > INFO: Automatic timeout occured
> > >
> > > 2) automatic and programmatic timeouts:
> > > TomEE kill prevented the execution of five programmatic timeouts,
> > while
> > > server restart the first programmatic timeout causes an error and is
> > never
> > > executed
> > >
> > > INFO: OpenWebBeans Container has started, it took 10 ms.
> > > 08.11.2012 16:59:02
> > > org.apache.openejb.core.timer.EjbTimerServiceImpl ejbTimeout
> > > WARNUNG: Exception from ejbTimeout on SchedulerBeanPersisted
> > > org.apache.openejb.OpenEJBException: Deployment does not exist
> in
> > > this container. Deployment(id='SchedulerBeanPersisted'),
> Container(id='My
> > > Singleton Container')
> > > at
> > >
> > >
> >
> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:180)
> > > at
> > >
> > >
> >
> org.apache.openejb.core.timer.EjbTimerServiceImpl.ejbTimeout(EjbTimerServiceImpl.java:552)
> > > at
> > >
> >
> org.apache.openejb.core.timer.EjbTimeoutJob.execute(EjbTimeoutJob.java:39)
> > > at
> org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> > > at
> > >
> > >
> >
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> > > 08.11.2012 16:59:02
> > org.apache.openejb.assembler.classic.Assembler
> > > createApplication
> > > INFO: Created Ejb(deployment-id=SchedulerBeanAutomated,
> > > ejb-name=SchedulerBeanAutomated, container=My Singleton Container)
> > > 08.11.2012 16:59:02
> > > org.apache.openejb.core.timer.EjbTimerServiceImpl ejbTimeout
> > > WARNUNG: Exception from ejbTimeout on SchedulerBeanPersisted
> > > org.apache.openejb.OpenEJBException: Deployment does not exist
> in
> > > this container. Deployment(id='SchedulerBeanPersisted'),
> Container(id='My
> > > Singleton Container')
> > > at
> > >
> > >
> >
> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:180)
> > > at
> > >
> > >
> >
> org.apache.openejb.core.timer.EjbTimerServiceImpl.ejbTimeout(EjbTimerServiceImpl.java:552)
> > > at
> > >
> >
> org.apache.openejb.core.timer.EjbTimeoutJob.execute(EjbTimeoutJob.java:39)
> > > at
> org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> > > at
> > >
> > >
> >
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> > > 08.11.2012 16:59:02
> > > org.apache.openejb.core.timer.EjbTimerServiceImpl ejbTimeout
> > > WARNUNG: Exception from ejbTimeout on SchedulerBeanPersisted
> > > org.apache.openejb.OpenEJBException: Deployment does not exist
> in
> > > this container. Deployment(id='SchedulerBeanPersisted'),
> Container(id='My
> > > Singleton Container')
> > > at
> > >
> > >
> >
> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:180)
> > > at
> > >
> > >
> >
> org.apache.openejb.core.timer.EjbTimerServiceImpl.ejbTimeout(EjbTimerServiceImpl.java:552)
> > > at
> > >
> >
> org.apache.openejb.core.timer.EjbTimeoutJob.execute(EjbTimeoutJob.java:39)
> > > at
> org.quartz.core.JobRunShell.run(JobRunShell.java:213)
> > > at
> > >
> > >
> >
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> > > 08.11.2012 16:59:02
> > > org.apache.openejb.core.timer.EjbTimerServiceImpl ejbTimeout
> > > WARNUNG: Failed to execute ejbTimeout on SchedulerBeanPersisted
> > > successfully within 1 attempts
> > > 08.11.2012 16:59:02
> > org.apache.openejb.assembler.classic.Assembler
> > > createApplication
> > > INFO: Created Ejb(deployment-id=SchedulerBeanPersisted,
> > > ejb-name=SchedulerBeanPersisted, container=My Singleton Container)
> > > 08.11.2012 16:59:02
> > org.apache.openejb.assembler.classic.Assembler
> > > createApplication
> > > INFO: Started Ejb(deployment-id=SchedulerBeanAutomated,
> > > ejb-name=SchedulerBeanAutomated, container=My Singleton Container)
> > > 08.11.2012 16:59:02
> > org.apache.openejb.assembler.classic.Assembler
> > > createApplication
> > > INFO: Started Ejb(deployment-id=SchedulerBeanPersisted,
> > > ejb-name=SchedulerBeanPersisted, container=My Singleton Container)
> > > 08.11.2012 16:59:02
> > org.apache.openejb.assembler.classic.Assembler
> > > createApplication
> > > INFO: Deployed
> > >
> > >
> >
> Application(path=C:\Java\tomee-examples\tomee-timerPersisted\webapps\timerPersisted)
> > > 08.11.2012 16:59:02
> timerPersisted.server.SchedulerBeanPersisted
> > > programmaticTimeout
> > > INFO: Programmatic timeout occurred.
> > >
> > > 3) Having a look into the quartz database tables, I can see that
> JOB_DATA
> > > column of table QRTZ_TRIGGERS includes the, let me call it timer
> context.
> > > This timer context includes serialized objects.
> > > What I am wondering about is how you can garantee that the timer
> > events
> > > can be executed with an updated TomEE version that may include an
> update
> > of
> > > the serialzed objects?
> > >
> > > Best regards,
> > > Thomas
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658484.html
> > > Sent from the OpenEJB User mailing list archive at Nabble.com.
> > >
> >
> > ________________________________
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658490.html
> > To unsubscribe from Use external database for persistent timer service,
> > click here<
> > >.
> > NAML<
> >
> http://openejb.979440.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> > >
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658495.html
> > Sent from the OpenEJB User mailing list archive at Nabble.com.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658497.html
> To unsubscribe from Use external database for persistent timer service,
> click here<
> >.
> NAML<
> http://openejb.979440.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
>
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658529.html
> Sent from the OpenEJB User mailing list archive at Nabble.com.
>
________________________________
If you reply to this email, your message will be added to the discussion below:
http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658531.html
To unsubscribe from Use external database for persistent timer service, click
here<http://openejb.979440.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4658321&code=dGhvbWFzLnNjaHVsZXJAb3BlbnRleHQuY29tfDQ2NTgzMjF8LTE4NTIyNTQ0OTI=>.
NAML<http://openejb.979440.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
jstackTomEE.txt (31K)
<http://openejb.979440.n4.nabble.com/attachment/4658605/0/jstackTomEE.txt>
--
View this message in context:
http://openejb.979440.n4.nabble.com/Use-external-database-for-persistent-timer-service-tp4658321p4658605.html
Sent from the OpenEJB User mailing list archive at Nabble.com.
