If it runs again then look at the logic where it's gotta decide whether this is the initial call or if the job is already running and this is just a status check. Is it looking at the request to figure that out? If so, and the expected info isn't found, walk the data path to figure out where it is getting lost (sounds like the session filter might be a good place to start). Is it looking in a database/shared memory/file system? Perhaps these are not in sync? Perhaps the long running job is doing everything in a single transaction, not committed until the very end, and the "I'm running!" flag set within the transaction is not visible unless the query is READ_UNCOMMITTED?
This run/not decision is critical, so it should be determined atomically. Perhaps one transaction for the decision (which either concludes "report status" or ensures its record is visible to any other query) and others for each separable unit of work within the long running job? Oh, yeah, you said Redis. I've somehow managed to avoid that, so I don't know what persistent data check/update can be done atomically, but that's where you're hanging your hat. Maybe the response to your "is anyone else already running this job?" query is being cached, and so the first "no" response is being returned every time? So many places this decision could go wrong in a distributed platform. Every possible race condition will eventually happen. Can you confidently say that two concurrent requests on distinct hosts will always result in a single "winner"? (Then later on there's a "liveness" detector you'll need to figure out in case the winner fails. Perhaps it should periodically update that record to show when it was last known to be running? (How far into the job it is right then would also be great to snapshot.) if that time stamp is older than X, is it safe to assume it failed and to elect another leader to take its place? Make sure time zone differences on different hosts can't impact the timeout checks. Is it safe, even in the face of a network disconnection, so eventually the evicted leader can reappear? Would one of them detect the issue and kill itself?) -Dale > On Sep 23, 2023, at 4:23 PM, Burton Rhodes <burtonrho...@gmail.com> wrote: > > I am attempting to implement a centralized session store to an existing > application using Spring Session (Redis) and Struts. I have everything > working on a basic level, but I am running into an issue when a Struts Action > uses ExecuteAndWait. The refresh attempts don't seem to pick up the running > action using the provided Struts Token (at least I think this is the case). > So on each refresh, the action seems to run again as if it was the first time > duplicating database inserts, etc. It becomes an infinite process, and > Struts never returns the final "SUCCESS" result JSP page to the browser. > It's worth noting that if I disable the Spring Session filter, everything > works fine. Anyone have a clue as to what might be going on or how I might > begin to troubleshoot? > > I've included a thread dump so you can see the filters involved. The > "SessionRepositoryFilter" is where the HttpSession is wrapped by Spring > (Redis) and is before any Struts filters. I've also included a basic action > definition that is causing the issue (although this is happening on all > actions that use the ExecuteAndWait logic). Any help is appreciated. > > ** Example Action Definition ** > > <action name="ContactBatchCategories_update" > class="com.afs.web.struts.action.contact.ContactBatchCategoriesAction" > method="update"> > <interceptor-ref name="myDefaultStack"/> > <interceptor-ref name="openSessionExecuteAndWaitInterceptor" > > <param name="delay">0</param> > </interceptor-ref> > <result > name="input">/struts/search/actions/contactBatchCategories_modal.jsp</result> > <result name="wait">/struts/common/progressMonitorWait_modal.jsp</result> > <result>/struts/common/progressMonitorSuccess_modal.jsp</result> > </action> > > > ** Thread Dump ** > > java.lang.Thread.State: RUNNABLE > at > com.afs.web.struts.action.contact.ContactBatchCategoriesAction.prepare(ContactBatchCategoriesAction.java:62) > at > com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:171) > at > com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99) > at > com.opensymphony.xwork2.DefaultActionInvocation.executeConditional(DefaultActionInvocation.java:299) > at > com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:253) > at > org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:154) > at > com.opensymphony.xwork2.DefaultActionInvocation.executeConditional(DefaultActionInvocation.java:299) > at > com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:253) > at > com.afs.web.config.struts.StrutsAccountInterceptor.intercept(StrutsAccountInterceptor.java:67) > at > com.opensymphony.xwork2.DefaultActionInvocation.executeConditional(DefaultActionInvocation.java:299) > at > com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:253) > at > com.afs.web.config.struts.StrutsSessionInterceptor.intercept(StrutsSessionInterceptor.java:44) > at > com.opensymphony.xwork2.DefaultActionInvocation.executeConditional(DefaultActionInvocation.java:299) > at > com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:253) > at > com.afs.web.common.struts.interceptor.ExceptionInterceptor.intercept(ExceptionInterceptor.java:33) > at > com.opensymphony.xwork2.DefaultActionInvocation.executeConditional(DefaultActionInvocation.java:299) > at > com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:253) > at > org.apache.struts2.factory.StrutsActionProxy.execute(StrutsActionProxy.java:48) > at > org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:651) > at > org.apache.struts2.dispatcher.ExecuteOperations.executeAction(ExecuteOperations.java:79) > at > org.apache.struts2.dispatcher.filter.StrutsPrepareAndExecuteFilter.handleRequest(StrutsPrepareAndExecuteFilter.java:157) > at > org.apache.struts2.dispatcher.filter.StrutsPrepareAndExecuteFilter.tryHandleRequest(StrutsPrepareAndExecuteFilter.java:140) > at > org.apache.struts2.dispatcher.filter.StrutsPrepareAndExecuteFilter.doFilter(StrutsPrepareAndExecuteFilter.java:128) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > com.afs.web.config.filter.MyAppSessionFilter.doFilter(MyAppSessionFilter.java:85) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:352) > at > org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:131) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:85) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:110) > at > org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:164) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:151) > at > org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:129) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) > at > org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:117) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:102) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:225) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:190) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:186) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > org.eclipse.jetty.websocket.servlet.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:170) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:142) > at > org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) > at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149) > at > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:51) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) > at org.eclipse.jetty.server.Server.handle(Server.java:563) > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) > at > org.eclipse.jetty.server.HttpChannel$$Lambda$1353.1731450897.dispatch(Unknown > Source:-1) > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) > at > org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) > at > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:416) > at > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:385) > at > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:272) > at > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:140) > at > org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy$$Lambda$1338.510268842.run(Unknown > Source:-1) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) > at java.lang.Thread.run(Thread.java:834) --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
