To anyone who runs into this in the future: After long, painful debugging sessions the problem was found: Creating/modifying the session in JASPIC login code. We moved all the code needing to init session vars to a javax.servlet.http.HttpSessionListener which resolved this issue.
On Wed, Jan 11, 2017 at 1:41 PM, Romain Manni-Bucau <[email protected]> wrote: > 2017-01-11 19:21 GMT+01:00 Adam Cornett <[email protected]>: > > > I don't have firm evidence that my application is trying to make > concurrent > > requests, although I can run the server with a debugger attached and try > to > > catch it. There are not many changes from owb 1.6.3 to 1.7.0, and this > > does fit the bill in terms of the difficulty to reproduce and the > symptoms, > > but it is an unproven theory. > > > > If you look at the commit though there is a change in behavior if two > > different threads using the same session both call initSessionContext "at > > the same time". I assume this happens and is expected based on the > > existence of the synchronized block and double check. > > > > > Yes but part of the code also assume it doesn't happen. Not sure it is an > issue or this synchronized was actually around something else. > > > > Should this discussion be moved to the OWB dev list? > > > > > Not sure, until it gets proven it is maybe not that. It is quite hard ot > find issues from code review, in particular in that area. > > If you can reproduce it in an acceptable time (even if it needs 2 h) but in > a reliable way you can patch this part to ensure it is the cause maybe? > > Can you try something like that before we move over OWB - or we can move > but we would still need to be certain of the cause? > > > > On Wed, Jan 11, 2017 at 1:01 PM, Romain Manni-Bucau < > [email protected] > > > > > wrote: > > > > > we can lock "more" but this shouldn't happen too in a normal app since > > > session is not thread safe, do you know why you are in this case? > > > > > > > > > Romain Manni-Bucau > > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > <https://blog-rmannibucau.rhcloud.com> | Old Blog > > > <http://rmannibucau.wordpress.com> | Github <https://github.com/ > > > rmannibucau> | > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory > > > <https://javaeefactory-rmannibucau.rhcloud.com> > > > > > > 2017-01-11 18:50 GMT+01:00 Adam Cornett <[email protected]>: > > > > > > > That is what I suspect is happening, yes. > > > > > > > > On Wed, Jan 11, 2017 at 12:45 PM, Romain Manni-Bucau < > > > > [email protected]> > > > > wrote: > > > > > > > > > Do you mean you have 2 threads accessing the session at the same > > time? > > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > > > <https://blog-rmannibucau.rhcloud.com> | Old Blog > > > > > <http://rmannibucau.wordpress.com> | Github <https://github.com/ > > > > > rmannibucau> | > > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE > Factory > > > > > <https://javaeefactory-rmannibucau.rhcloud.com> > > > > > > > > > > 2017-01-11 18:42 GMT+01:00 Adam Cornett <[email protected]>: > > > > > > > > > > > I have not had much luck creating a reproducible test case, so in > > > > > > desperation I began reading the full diff of OWB 1.6.3 -> 1.7.0 > > > looking > > > > > for > > > > > > any change that might have introduced this issue. > > > > > > > > > > > > I suspect that the changes made for OWB-1136 create a scenario > > where > > > > the > > > > > > session context is not setup correctly. Specifically the change > to > > > > > > the org.apache.webbeans.web.context.WebContextsService. > > > > > > initSessionContext(Object) > > > > > > method in commit 1753758. > > > > > > > > > > > > In the new version of the code, if a there is contention and a > > thread > > > > is > > > > > > waiting on the OWB_SESSION_CONTEXT_ATTRIBUTE_NAME lock and while > > > > waiting > > > > > > another thread creates the session inside the synchronized block > > then > > > > > this > > > > > > new code will perform no further actions in this method. > > > > > > In the previous version, this method would always > > > > > > call sessionContexts.set(currentSessionContext) (even if there > was > > > no > > > > > > active session!) > > > > > > > > > > > > I think the solution would be to add an 'else' block to the end > of > > > the > > > > if > > > > > > statement on line 478 that correctly calls the same logic in the > > else > > > > > block > > > > > > which starts on line 493 since it should have the same behavior > in > > > the > > > > > case > > > > > > of the session created while waiting as it would if it came in > and > > > > found > > > > > > the session context already existed. > > > > > > > > > > > > The locking/ordering required to trigger this bug makes it very > > hard > > > to > > > > > > test for and reproduce, but I think a careful study of this > method > > > > shows > > > > > > that the change does introduce a situation where the session > > context > > > is > > > > > not > > > > > > put into the sessionContexts ThreadLocal, thus causing the error > > I've > > > > > > reported above. > > > > > > > > > > > > > > > > > > On Sat, Nov 19, 2016 at 3:46 PM, Adam Cornett < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > I will see if I can get a reproduceable test project, might be > a > > > few > > > > > > days. > > > > > > > > > > > > > > On Nov 19, 2016 2:32 PM, "Romain Manni-Bucau" < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Ok, so next step is to try to reproduce it without jsf i > guess. > > > Any > > > > > > >> hint/help? > > > > > > >> > > > > > > >> Le 19 nov. 2016 19:54, "Adam Cornett" <[email protected] > > > > a > > > > > écrit > > > > > > : > > > > > > >> > > > > > > >> I have removed the custom error page and the issue will occur > > > > randomly > > > > > > on > > > > > > >> any page using a ViewScoped bean. > > > > > > >> > > > > > > >> On Nov 19, 2016 11:53 AM, "Romain Manni-Bucau" < > > > > [email protected] > > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > >> > @Adam: can you confirm it is only for error pages (or very > > > linked > > > > to > > > > > > >> it)? > > > > > > >> > > > > > > > >> > > > > > > > >> > Romain Manni-Bucau > > > > > > >> > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > > > > >> > <https://blog-rmannibucau.rhcloud.com> | Old Blog > > > > > > >> > <http://rmannibucau.wordpress.com> | Github < > > > https://github.com/ > > > > > > >> > rmannibucau> | > > > > > > >> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE > > > > Factory > > > > > > >> > <https://javaeefactory-rmannibucau.rhcloud.com> > > > > > > >> > > > > > > > >> > 2016-11-18 22:31 GMT+01:00 Adam Cornett < > > [email protected] > > > >: > > > > > > >> > > > > > > > >> > > To cut down on the number of places to look I rolled back > to > > > > TomEE > > > > > > >> 7.0.1 > > > > > > >> > > and pulled forward specific libraries: > > > > > > >> > > TomEE 7.0.1 + MyFaces 2.2.11 = no @ViewScoped bug > > > > > > >> > > TomEE 7.0.1 + MyFaces 2.2.11 + OWB 1.7.0 = @ViewScoped bug > > > shows > > > > > up. > > > > > > >> > > > > > > > > >> > > This seems to indicate that some change in OWB between > 1.6.3 > > > and > > > > > > 1.7.0 > > > > > > >> > may > > > > > > >> > > be the culprit. > > > > > > >> > > > > > > > > >> > > On Wed, Nov 16, 2016 at 4:04 PM, Romain Manni-Bucau < > > > > > > >> > [email protected] > > > > > > >> > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > >> > > > 2016-11-16 21:29 GMT+01:00 Adam Cornett < > > > > [email protected] > > > > > >: > > > > > > >> > > > > > > > > > >> > > > > Just coming up with ideas here: what if the listeners > > > being > > > > > > >> iterated > > > > > > >> > > > > through in org.apache.catalina.core.StandardContext. > > > > > > >> > > > > fireRequestDestroyEvent > > > > > > >> > > > > are the issue? If the facesContext/request/session is > > > torn > > > > > down > > > > > > >> > before > > > > > > >> > > > the > > > > > > >> > > > > owb listener OWB gets a chance to clean up the beans > it > > > > could > > > > > be > > > > > > >> > > causing > > > > > > >> > > > a > > > > > > >> > > > > problem? If the order of the iteration is not stable > > that > > > > > could > > > > > > >> also > > > > > > >> > > > > explain why the issue is intermittent. > > > > > > >> > > > > > > > > > > >> > > > > There is actually a note about this in the MyFaces > > source > > > > > > >> > > > > (org.apache.myfaces.cdi.view.ViewScopeBeanHolder. > > > > > > >> > > > destroyBeansOnPreDestroy) > > > > > > >> > > > > where they are dealing with differences between CDI > > > > > > >> implementations > > > > > > >> > and > > > > > > >> > > > > mention Tomcat + OWB. > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > OWB listener insertion is normally forced (it is a tomee > > > > custom > > > > > > one > > > > > > >> > BTW) > > > > > > >> > > so > > > > > > >> > > > this is stable. > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > >> > > > > On Wed, Nov 16, 2016 at 12:09 PM, Adam Cornett < > > > > > > >> > [email protected] > > > > > > >> > > > > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > > > > >> > > > > > Wouldn't that be easy :) > > > > > > >> > > > > > > > > > > > >> > > > > > I have not been able to reproduce this in a simple > > test > > > > > case. > > > > > > >> After > > > > > > >> > > > some > > > > > > >> > > > > > more testing on our app I have found that the first > > > stack > > > > > > trace > > > > > > >> I > > > > > > >> > > > posted > > > > > > >> > > > > is > > > > > > >> > > > > > actually generated when trying to render our custom > > > error > > > > > > page. > > > > > > >> > > When I > > > > > > >> > > > > > take the custom error page out the second error in > OP > > > > > (posted > > > > > > >> again > > > > > > >> > > > > below) > > > > > > >> > > > > > is the exception causing the problem. > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > org.apache.webbeans.component.ManagedBean - > Exception > > > > > thrown > > > > > > >> while > > > > > > >> > > > > > destroying bean instance : [ViewScopeBeanHolder, > > > > > > >> > > WebBeansType:MANAGED, > > > > > > >> > > > > > Name:null, API Types:[java.io.Serializable, > > > > > > >> > > > java.lang.Object,org.apache. > > > > > > >> > > > > > myfaces.cdi.view.ViewScopeBeanHolder], > > > > > > >> > Qualifiers:[javax.enterprise. > > > > > > >> > > > > > inject.Default,javax.enterprise.inject.Any]] > > > > > > >> > > > > > java.lang.NullPointerException: null > > > > > > >> > > > > > at org.apache.myfaces.cdi.view. > ViewScopeContextImpl. > > > > > > >> > > destroyAllActive( > > > > > > >> > > > > > ViewScopeContextImpl.java:229) > > > > > > >> > > > > > at org.apache.myfaces.cdi.view. > ViewScopeContextImpl. > > > > > > >> > > destroyAllActive( > > > > > > >> > > > > > ViewScopeContextImpl.java:223) > > > > > > >> > > > > > at org.apache.myfaces.cdi.view.ViewScopeBeanHolder. > > > > > > >> > > > > > destroyBeansOnPreDestroy( > > ViewScopeBeanHolder.java:221) > > > > > > >> > > > > > at sun.reflect.NativeMethodAccessorImpl. > > invoke0(Native > > > > > > Method) > > > > > > >> > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke( > > > > > > >> > > > > > NativeMethodAccessorImpl.java:62) > > > > > > >> > > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > > > > > >> > > > > > DelegatingMethodAccessorImpl.java:43) > > > > > > >> > > > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > > > > >> > > > > > at org.apache.webbeans.intercept. > > > > > > LifecycleInterceptorInvocation > > > > > > >> > > > > > Context.proceed(LifecycleInterceptorInvocation > > > > > > Context.java:103) > > > > > > >> > > > > > at org.apache.webbeans.portable. > InjectionTargetImpl. > > > > > > preDestroy( > > > > > > >> > > > > > InjectionTargetImpl.java:352) > > > > > > >> > > > > > at org.apache.webbeans.component. > > > AbstractOwbBean.destroy( > > > > > > >> > > > > > AbstractOwbBean.java:179) > > > > > > >> > > > > > at org.apache.webbeans.context.AbstractContext. > > > > > > destroyInstance( > > > > > > >> > > > > > AbstractContext.java:206) > > > > > > >> > > > > > at org.apache.webbeans.context.AbstractContext. > > > > > > destroyInstance( > > > > > > >> > > > > > AbstractContext.java:192) > > > > > > >> > > > > > at org.apache.webbeans.context. > > AbstractContext.destroy( > > > > > > >> > > > > > AbstractContext.java:218) > > > > > > >> > > > > > at org.apache.webbeans.web. > > context.WebContextsService. > > > > > > >> > > > > > destroyRequestContext(WebContextsService.java:408) > > > > > > >> > > > > > at org.apache.openejb.cdi.CdiAppContextsService. > > > > > > >> > > destroyRequestContext( > > > > > > >> > > > > > CdiAppContextsService.java:113) > > > > > > >> > > > > > at org.apache.webbeans.web. > > context.WebContextsService. > > > > > > >> > > > > > endContext(WebContextsService.java:223) > > > > > > >> > > > > > at org.apache.openejb.server. > > > httpd.BeginWebBeansListener. > > > > > > >> > > > > requestDestroyed( > > > > > > >> > > > > > BeginWebBeansListener.java:99) > > > > > > >> > > > > > at org.apache.catalina.core.StandardContext. > > > > > > >> > fireRequestDestroyEvent( > > > > > > >> > > > > > StandardContext.java:5965) > > > > > > >> > > > > > at org.apache.catalina.core. > StandardHostValve.invoke( > > > > > > >> > > > > > StandardHostValve.java:182) > > > > > > >> > > > > > at org.apache.catalina.valves. > > ErrorReportValve.invoke( > > > > > > >> > > > > > ErrorReportValve.java:79) > > > > > > >> > > > > > at org.apache.tomee.catalina. > OpenEJBSecurityListener$ > > > > > > >> > > > > > RequestCapturer.invoke( > OpenEJBSecurityListener.java: > > 97) > > > > > > >> > > > > > at org.apache.catalina.core. > > StandardEngineValve.invoke( > > > > > > >> > > > > > StandardEngineValve.java:87) > > > > > > >> > > > > > at org.apache.catalina.connector. > > CoyoteAdapter.service( > > > > > > >> > > > > > CoyoteAdapter.java:349) > > > > > > >> > > > > > at org.apache.coyote.http11. > Http11Processor.service( > > > > > > >> > > > > > Http11Processor.java:784) > > > > > > >> > > > > > at org.apache.coyote.AbstractProcessorLight. > process( > > > > > > >> > > > > > AbstractProcessorLight.java:66) > > > > > > >> > > > > > at org.apache.coyote.AbstractProtocol$ > > > ConnectionHandler. > > > > > > >> process( > > > > > > >> > > > > > AbstractProtocol.java:802) > > > > > > >> > > > > > at org.apache.tomcat.util.net. > > > > NioEndpoint$SocketProcessor. > > > > > > >> > > > > > doRun(NioEndpoint.java:1410) > > > > > > >> > > > > > at org.apache.tomcat.util.net. > > SocketProcessorBase.run( > > > > > > >> > > > > > SocketProcessorBase.java:49) > > > > > > >> > > > > > at java.util.concurrent. > ThreadPoolExecutor.runWorker( > > > > > > >> > > > > > ThreadPoolExecutor.java:1142) > > > > > > >> > > > > > at java.util.concurrent. > > ThreadPoolExecutor$Worker.run( > > > > > > >> > > > > > ThreadPoolExecutor.java:617) > > > > > > >> > > > > > at org.apache.tomcat.util.threads.TaskThread$ > > > > > > WrappingRunnable. > > > > > > >> run( > > > > > > >> > > > > > TaskThread.java:61) > > > > > > >> > > > > > at java.lang.Thread.run(Thread.java:745) > > > > > > >> > > > > > > > > > > > >> > > > > > Looking an the MyFaces code it looks like the there > is > > > no > > > > > > active > > > > > > >> > > faces > > > > > > >> > > > > > instance (FacesContext.getCurrentInstance()) when > it > > > > tries > > > > > to > > > > > > >> > > destroy > > > > > > >> > > > > the > > > > > > >> > > > > > beans. > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > On Wed, Nov 16, 2016 at 10:51 AM, Romain > Manni-Bucau < > > > > > > >> > > > > > [email protected]> wrote: > > > > > > >> > > > > > > > > > > > >> > > > > >> 2016-11-16 16:48 GMT+01:00 Adam Cornett < > > > > > > >> [email protected]>: > > > > > > >> > > > > >> > > > > > > >> > > > > >> > I don't think so. This error will happen for > even > > > the > > > > > > >> simplest > > > > > > >> > > > bean: > > > > > > >> > > > > we > > > > > > >> > > > > >> > have one 40 line class that just loads a list of > > JPA > > > > > > objects > > > > > > >> > from > > > > > > >> > > > the > > > > > > >> > > > > db > > > > > > >> > > > > >> > and sorts it for presentation and it triggers the > > > issue > > > > > > too. > > > > > > >> So > > > > > > >> > > > far I > > > > > > >> > > > > >> have > > > > > > >> > > > > >> > seen this happen on at least 8 different view > > scoped > > > > > beans > > > > > > >> > across > > > > > > >> > > > > >> different > > > > > > >> > > > > >> > wars in our product. > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> Does that mean it is easy to reproduce on a github > > > > project? > > > > > > >> 0:-) > > > > > > >> > > > > >> > > > > > > >> > > > > >> > > > > > > >> > > > > >> > I've been trying to sort this out for two days > now > > > and > > > > > > can't > > > > > > >> > > figure > > > > > > >> > > > > out > > > > > > >> > > > > >> if > > > > > > >> > > > > >> > it is caused by a change in our app or if there > is > > > > > > something > > > > > > >> > going > > > > > > >> > > > on > > > > > > >> > > > > in > > > > > > >> > > > > >> > OWB/MyFaces/TomEE. I've disabled tomcat session > > > > > > replication > > > > > > >> and > > > > > > >> > > the > > > > > > >> > > > > >> issue > > > > > > >> > > > > >> > still persists (although sometimes it takes many > > > tries > > > > to > > > > > > get > > > > > > >> > the > > > > > > >> > > > > issue > > > > > > >> > > > > >> to > > > > > > >> > > > > >> > manifest). > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > We have session beans all over our application > and > > > > those > > > > > > work > > > > > > >> > > fine, > > > > > > >> > > > it > > > > > > >> > > > > >> is > > > > > > >> > > > > >> > just he viewscoped ones that can trigger the > issue. > > > > The > > > > > > >> > > > intermittent > > > > > > >> > > > > >> > nature of the problem is also very confusing. > What > > > > sort > > > > > of > > > > > > >> > things > > > > > > >> > > > > >> could I > > > > > > >> > > > > >> > do in our application code that would cause the > > > > container > > > > > > to > > > > > > >> > loose > > > > > > >> > > > the > > > > > > >> > > > > >> > request? > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > On Wed, Nov 16, 2016 at 10:35 AM, Romain > > Manni-Bucau > > > < > > > > > > >> > > > > >> > [email protected]> > > > > > > >> > > > > >> > wrote: > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > Normally the session is captured from the > > request. > > > Is > > > > > it > > > > > > >> > > possible > > > > > > >> > > > > you > > > > > > >> > > > > >> > loose > > > > > > >> > > > > >> > > the request when touching this bean somehow? > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > Romain Manni-Bucau > > > > > > >> > > > > >> > > @rmannibucau <https://twitter.com/rmannibucau> > | > > > > Blog > > > > > > >> > > > > >> > > <https://blog-rmannibucau.rhcloud.com> | Old > > Blog > > > > > > >> > > > > >> > > <http://rmannibucau.wordpress.com> | Github < > > > > > > >> > > https://github.com/ > > > > > > >> > > > > >> > > rmannibucau> | > > > > > > >> > > > > >> > > LinkedIn <https://www.linkedin.com/in/ > > rmannibucau> > > > | > > > > > > >> JavaEE > > > > > > >> > > > Factory > > > > > > >> > > > > >> > > <https://javaeefactory-rmannibucau.rhcloud.com > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > 2016-11-16 15:51 GMT+01:00 Adam Cornett < > > > > > > >> > [email protected] > > > > > > >> > > >: > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > Using TomEE 7.0.2 we are seeing intermittent > > > > problems > > > > > > >> with > > > > > > >> > > beans > > > > > > >> > > > > >> > > > using javax.faces.view.ViewScoped. > > > > > > >> > > > > >> > > > We have many pages that use @ViewScoped > beans, > > > but > > > > > the > > > > > > >> > > behavior > > > > > > >> > > > of > > > > > > >> > > > > >> this > > > > > > >> > > > > >> > > > issue is not predictable. Sometimes we can > > login > > > > and > > > > > > not > > > > > > >> > have > > > > > > >> > > > any > > > > > > >> > > > > >> > issue > > > > > > >> > > > > >> > > at > > > > > > >> > > > > >> > > > all, sometimes it will only impact a single > > bean > > > > and > > > > > > all > > > > > > >> > other > > > > > > >> > > > > >> > > @ViewScoped > > > > > > >> > > > > >> > > > beans will work fine, but when a bean starts > > > > throwing > > > > > > an > > > > > > >> > > error, > > > > > > >> > > > it > > > > > > >> > > > > >> will > > > > > > >> > > > > >> > > > keep throwing the error until the user logs > out > > > > (ends > > > > > > the > > > > > > >> > > > servlet > > > > > > >> > > > > >> > > session) > > > > > > >> > > > > >> > > > and then when logging back it in may or may > not > > > > > happen > > > > > > >> > again. > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > The primary error is below: > > > > > > >> > > > > >> > > > javax.enterprise.context. > > > > ContextNotActiveException: > > > > > > >> > WebBeans > > > > > > >> > > > > >> context > > > > > > >> > > > > >> > > with > > > > > > >> > > > > >> > > > scope type annotation @SessionScoped does not > > > exist > > > > > > >> within > > > > > > >> > > > current > > > > > > >> > > > > >> > thread > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.container. > > > > > > >> BeanManagerImpl.getContext( > > > > > > >> > > > > >> > > > BeanManagerImpl.java:331) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.intercept. > > > > > > >> > NormalScopedBeanInterceptorHan > > > > > > >> > > > > >> > > > dler.getContextualInstance(Nor > > > > > > >> malScopedBeanInterceptorHan > > > > > > >> > dle > > > > > > >> > > > > >> r.java:89) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.intercept. > > > > > > >> > SessionScopedBeanInterceptorHa > > > > > > >> > > > > >> > > > ndler.getContextualInstance(Se > > > > > > >> ssionScopedBeanInterceptorHa > > > > > > >> > > > > >> > ndler.java:76) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.intercept. > > > > > > >> > NormalScopedBeanInterceptorHan > > > > > > >> > > > > >> dler.get( > > > > > > >> > > > > >> > > > NormalScopedBeanInterceptorHandler.java:71) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.cdi.view. > > > ViewScopeBeanHolder$$ > > > > > > >> > OwbNormalSc > > > > > > >> > > > > >> opeProxy0. > > > > > > >> > > > > >> > > > generateUniqueViewScopeId(org/ > > > > > apache/myfaces/cdi/view/ > > > > > > >> > > > > >> > > > ViewScopeBeanHolder.java) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.cdi.impl. > > > > > CDIManagedBeanHandlerImpl. > > > > > > >> > > > > >> > > generateViewScopeId( > > > > > > >> > > > > >> > > > CDIManagedBeanHandlerImpl.java:92) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.view. > > > > > ViewScopeProxyMap.getWrapped( > > > > > > >> > > > > >> > > > ViewScopeProxyMap.java:79) > > > > > > >> > > > > >> > > > at org.apache.myfaces.view. > > > ViewScopeProxyMap.size( > > > > > > >> > > > > >> > > > ViewScopeProxyMap.java:99) > > > > > > >> > > > > >> > > > at javax.faces.component. > UIViewRoot.saveState( > > > > > > >> > > > > UIViewRoot.java:1502) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.renderkit. > ErrorPageWriter._ > > > > > > >> > writeComponent( > > > > > > >> > > > > >> > > > ErrorPageWriter.java:851) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.renderkit. > > > > > > ErrorPageWriter.debugHtml( > > > > > > >> > > > > >> > > > ErrorPageWriter.java:352) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.renderkit. > > > > ErrorPageWriter.handle( > > > > > > >> > > > > >> > > > ErrorPageWriter.java:471) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.context.MyF > > > > > > >> acesExceptionHandlerWrapper > > > > > > >> > Imp > > > > > > >> > > > > >> l.handle( > > > > > > >> > > > > >> > > > MyFacesExceptionHandlerWrapperImpl.java:301) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > javax.faces.context.ExceptionHandlerWrapper. > > > > handle( > > > > > > >> > > > > >> > > > ExceptionHandlerWrapper.java:61) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.jsf. > > > > OwbExceptionHandler.handle( > > > > > > >> > > > > >> > > > OwbExceptionHandler.java:61) > > > > > > >> > > > > >> > > > at org.apache.myfaces.lifecycle. > > > > > LifecycleImpl.render( > > > > > > >> > > > > >> > > > LifecycleImpl.java:287) > > > > > > >> > > > > >> > > > at javax.faces.webapp.FacesServlet.service( > > > > > > >> > > > FacesServlet.java:200) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:230) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at org.apache.tomcat.websocket. > > > > > > server.WsFilter.doFilter( > > > > > > >> > > > > >> > > WsFilter.java:52) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > > ApplicationDispatcher.invoke( > > > > > > >> > > > > >> > > > ApplicationDispatcher.java:726) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationDispatcher. > > > > > > >> > > processRequest( > > > > > > >> > > > > >> > > > ApplicationDispatcher.java:469) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationDispatcher.doForward( > > > > > > >> > > > > >> > > > ApplicationDispatcher.java:394) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > > > ApplicationDispatcher.forward( > > > > > > >> > > > > >> > > > ApplicationDispatcher.java:311) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > com.achalert.alert.security.se > > > > > > >> rvlets.LoginHandlerServlet. > > > > > > >> > > > > >> > processRequest( > > > > > > >> > > > > >> > > > LoginHandlerServlet.java:186) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > com.achalert.alert.security.se > > > > > > >> rvlets.LoginHandlerServlet. > > > > > > >> > > > > >> > > > doPost(LoginHandlerServlet.java:371) > > > > > > >> > > > > >> > > > at javax.servlet.http.HttpServlet.service( > > > > > > >> > > HttpServlet.java:648) > > > > > > >> > > > > >> > > > at javax.servlet.http.HttpServlet.service( > > > > > > >> > > HttpServlet.java:729) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:230) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at org.apache.tomcat.websocket. > > > > > > server.WsFilter.doFilter( > > > > > > >> > > > > >> > > WsFilter.java:52) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at org.apache.openejb.server. > > > > > httpd.EEFilter.doFilter( > > > > > > >> > > EEFilter. > > > > > > >> > > > > >> java:65) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > com.achalert.alert.security.filters. > > > > > > >> > SessionValidationFilter. > > > > > > >> > > > > >> doFilter( > > > > > > >> > > > > >> > > > SessionValidationFilter.java:134) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > com.achalert.alert.http. > OmnibusAchAlertFilter. > > > > > > doFilter( > > > > > > >> > > > > >> > > > OmnibusAchAlertFilter.java:161) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > ApplicationFilterChain. > > > > > > >> > > > internalDoFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.Appli > > > > > > >> cationFilterChain.doFilter( > > > > > > >> > > > > >> > > > ApplicationFilterChain.java:165) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > > StandardWrapperValve.invoke( > > > > > > >> > > > > >> > > > StandardWrapperValve.java:198) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > > StandardContextValve.invoke( > > > > > > >> > > > > >> > > > StandardContextValve.java:108) > > > > > > >> > > > > >> > > > at org.apache.tomee.catalina. > > > OpenEJBValve.invoke( > > > > > > >> > > OpenEJBValve. > > > > > > >> > > > > >> java:44) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.authentica > > > > > > >> tor.AuthenticatorBase.invoke( > > > > > > >> > > > > >> > > > AuthenticatorBase.java:472) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > StandardHostValve.invoke( > > > > > > >> > > > > >> > > > StandardHostValve.java:140) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.valves. > > > > ErrorReportValve.invoke( > > > > > > >> > > > > >> > > > ErrorReportValve.java:79) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomee.catalina. > > > OpenEJBSecurityListener$ > > > > > > >> > > > > >> > > RequestCapturer.invoke( > > > > > > >> > > > > >> > > > OpenEJBSecurityListener.java:97) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > StandardEngineValve.invoke( > > > > > > >> > > > > >> > > > StandardEngineValve.java:87) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.connector. > > > > CoyoteAdapter.service( > > > > > > >> > > > > >> > > > CoyoteAdapter.java:349) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.http11. > > > Http11Processor.service( > > > > > > >> > > > > >> > > Http11Processor.java:784) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.AbstractProcessorLight. > > > process( > > > > > > >> > > > > >> > > > AbstractProcessorLight.java:66) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.AbstractProtocol$ > > > > > > >> > ConnectionHandler.process( > > > > > > >> > > > > >> > > > AbstractProtocol.java:802) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.net. > > > > > > NioEndpoint$SocketProcessor. > > > > > > >> > > > > >> > > > doRun(NioEndpoint.java:1410) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.net. > > > > SocketProcessorBase.run( > > > > > > >> > > > > >> > > > SocketProcessorBase.java:49) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > java.util.concurrent. > > > ThreadPoolExecutor.runWorker( > > > > > > >> > > > > >> > > > ThreadPoolExecutor.java:1142) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > java.util.concurrent. > > > > ThreadPoolExecutor$Worker.run( > > > > > > >> > > > > >> > > > ThreadPoolExecutor.java:617) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.threads.TaskThread$ > > > > > > >> > > WrappingRunnable.run( > > > > > > >> > > > > >> > > > TaskThread.java:61) > > > > > > >> > > > > >> > > > at java.lang.Thread.run(Thread.java:745) > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > I've also noticed that there is another error > > > which > > > > > may > > > > > > >> be > > > > > > >> > > > > related. > > > > > > >> > > > > >> > This > > > > > > >> > > > > >> > > > one is usually somewhere in the logs before > the > > > > above > > > > > > >> error > > > > > > >> > > > > happens: > > > > > > >> > > > > >> > > > org.apache.webbeans.component.ManagedBean - > > > > > Exception > > > > > > >> > thrown > > > > > > >> > > > > while > > > > > > >> > > > > >> > > > destroying bean instance : > > [ViewScopeBeanHolder, > > > > > > >> > > > > >> WebBeansType:MANAGED, > > > > > > >> > > > > >> > > > Name:null, API > > > > > > >> > > > > >> > > > Types:[org.apache.myfaces.cdi. > > > > > > >> > view.ViewScopeBeanHolder,java. > > > > > > >> > > > > >> > > > io.Serializable,java.lang.Object], > > > > > > >> > > > > >> > > > Qualifiers:[javax.enterprise. > > > inject.Default,javax. > > > > > > >> > > > > >> > > enterprise.inject.Any]] > > > > > > >> > > > > >> > > > java.lang.NullPointerException: null > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.cdi.view. > > > ViewScopeContextImpl. > > > > > > >> > > > > destroyAllActive( > > > > > > >> > > > > >> > > > ViewScopeContextImpl.java:229) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.cdi.view. > > > ViewScopeContextImpl. > > > > > > >> > > > > destroyAllActive( > > > > > > >> > > > > >> > > > ViewScopeContextImpl.java:223) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.myfaces.cdi.view. > > ViewScopeBeanHolder. > > > > > > >> > > > > >> > > destroyBeansOnPreDestroy( > > > > > > >> > > > > >> > > > ViewScopeBeanHolder.java:221) > > > > > > >> > > > > >> > > > at sun.reflect.NativeMethodAccessorImpl. > > > > > invoke0(Native > > > > > > >> > > Method) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke( > > > > > > >> > > > > >> > > NativeMethodAccessorImpl.java: > > > > > > >> > > > > >> > > > 62) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > sun.reflect.DelegatingMethodAccessorImpl. > > invoke( > > > > > > >> > > > > >> > > > DelegatingMethodAccessorImpl.java:43) > > > > > > >> > > > > >> > > > at java.lang.reflect.Method. > > > > invoke(Method.java:498) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.intercept. > > > > > > >> > LifecycleInterceptorInvocation > > > > > > >> > > > > >> > > > Context.proceed( > LifecycleInterceptorInvocation > > > > > > >> > > Context.java:103) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.portable. > > > InjectionTargetImpl. > > > > > > >> > preDestroy( > > > > > > >> > > > > >> > > > InjectionTargetImpl.java:352) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.component. > > > > > AbstractOwbBean.destroy( > > > > > > >> > > > > >> > > > AbstractOwbBean.java:179) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.context.AbstractContext. > > > > > > >> > destroyInstance( > > > > > > >> > > > > >> > > > AbstractContext.java:206) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.context.AbstractContext. > > > > > > >> > destroyInstance( > > > > > > >> > > > > >> > > > AbstractContext.java:192) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.context. > > > > AbstractContext.destroy( > > > > > > >> > > > > >> > > > AbstractContext.java:218) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.web. > > > > context.WebContextsService. > > > > > > >> > > > > >> > > destroyRequestContext( > > > > > > >> > > > > >> > > > WebContextsService.java:408) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.openejb.cdi. > CdiAppContextsService. > > > > > > >> > > > > destroyRequestContext( > > > > > > >> > > > > >> > > > CdiAppContextsService.java:113) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.webbeans.web. > > > > context.WebContextsService. > > > > > > >> > > > > >> > > > endContext(WebContextsService.java:223) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.openejb.server. > > > > > httpd.BeginWebBeansListener. > > > > > > >> > > > > >> > requestDestroyed( > > > > > > >> > > > > >> > > > BeginWebBeansListener.java:99) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core.StandardContext. > > > > > > >> > > > fireRequestDestroyEvent( > > > > > > >> > > > > >> > > > StandardContext.java:5965) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > StandardHostValve.invoke( > > > > > > >> > > > > >> > > > StandardHostValve.java:182) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.valves. > > > > ErrorReportValve.invoke( > > > > > > >> > > > > >> > > > ErrorReportValve.java:79) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomee.catalina. > > > OpenEJBSecurityListener$ > > > > > > >> > > > > >> > > RequestCapturer.invoke( > > > > > > >> > > > > >> > > > OpenEJBSecurityListener.java:97) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.core. > > > > StandardEngineValve.invoke( > > > > > > >> > > > > >> > > > StandardEngineValve.java:87) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.catalina.connector. > > > > CoyoteAdapter.service( > > > > > > >> > > > > >> > > > CoyoteAdapter.java:349) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.http11. > > > Http11Processor.service( > > > > > > >> > > > > >> > > Http11Processor.java:784) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.AbstractProcessorLight. > > > process( > > > > > > >> > > > > >> > > > AbstractProcessorLight.java:66) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.coyote.AbstractProtocol$ > > > > > > >> > ConnectionHandler.process( > > > > > > >> > > > > >> > > > AbstractProtocol.java:802) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.net. > > > > > > NioEndpoint$SocketProcessor. > > > > > > >> > > > > >> > > > doRun(NioEndpoint.java:1410) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.net. > > > > SocketProcessorBase.run( > > > > > > >> > > > > >> > > > SocketProcessorBase.java:49) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > java.util.concurrent. > > > ThreadPoolExecutor.runWorker( > > > > > > >> > > > > >> > > > ThreadPoolExecutor.java:1142) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > java.util.concurrent. > > > > ThreadPoolExecutor$Worker.run( > > > > > > >> > > > > >> > > > ThreadPoolExecutor.java:617) > > > > > > >> > > > > >> > > > at > > > > > > >> > > > > >> > > > org.apache.tomcat.util.threads.TaskThread$ > > > > > > >> > > WrappingRunnable.run( > > > > > > >> > > > > >> > > > TaskThread.java:61) > > > > > > >> > > > > >> > > > at java.lang.Thread.run(Thread.java:745) > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > Adam Cornett > > > > > > >> > > > > >> > > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > -- > > > > > > >> > > > > >> > Adam Cornett > > > > > > >> > > > > >> > [email protected] > > > > > > >> > > > > >> > (678) 296-1150 > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > -- > > > > > > >> > > > > > Adam Cornett > > > > > > >> > > > > > [email protected] > > > > > > >> > > > > > (678) 296-1150 > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > -- > > > > > > >> > > > > Adam Cornett > > > > > > >> > > > > [email protected] > > > > > > >> > > > > (678) 296-1150 > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > -- > > > > > > >> > > Adam Cornett > > > > > > >> > > [email protected] > > > > > > >> > > (678) 296-1150 > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Adam Cornett > > > > > > [email protected] > > > > > > (678) 296-1150 > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Adam Cornett > > > > [email protected] > > > > (678) 296-1150 > > > > > > > > > > > > > > > -- > > Adam Cornett > > [email protected] > > (678) 296-1150 > > > -- Adam Cornett [email protected] (678) 296-1150
