Doug, I think the solution to synchronize all the HTTP requests on a session makes sense. In most environments where I have worked, I found that once the user is "tied" to one server, he/she typically stays with the same server unless something drastic happens. Since not all the processing goes through a servlet, I think a Filter might be a better mechanism to do such synchronization.
Thanks for your detailed reply. -AP_ http://www.alexparansky.com Java/J2EE Architect/Consultant http://www.myprofiles.com/member/profile/apara_personal -----Original Message----- From: Doug Bateman [mailto:[EMAIL PROTECTED]] Sent: Monday, May 06, 2002 5:06 PM To: [EMAIL PROTECTED]; Alex Paransky Cc: Doug Bateman Subject: Re: The problem with deadlocking transactions... I'll address the database deadlock issue in a separate follow up up e-mail. However, the specific scenario where are user clicks on a single link rapidly in succession will generally cause negative effects in most Servlet applications, not just ones using EJB. I'll present a short description to a solution to this problem at the end of this e-mail. What is happening is that multiple threads all belonging to the same HTTP Session are attempting to access the same shared stateful resources at nearly the exact same time. Any one of 5 very negative scenarios can result: (a) Deadlocks. Two threads each require locks that the other thread has, and wait indefinitely (or until timeout) for the other thread to release the resource. Deadlocks of this sort can either occur in the database or in the application server. (b) Optimistic Locking Rollbacks. Here, the database doesn't prevent concurrent access to the data, but detects conditions where two transactions have overlapping edits to a record, and forces the 2nd transaction to rollback in order to protect the integrity of the database. (This can most often occur when a SELECT rather than a SELECT FOR UPDATE is used to read the field.) If the entire transaction occurs within a single SQL request, the database is able to hide the rollback and automatically retry. But in cases where the transaction spans multiple database requests, the database lacks the necessary understanding of the relationship between the requests, and is unable to do the retry. In this case, a java.sql.SQLException is thrown. The client code responsible for starting the transaction has to catch the exception, inspect the SQL ERROR CODE FIELD of the exception to identify the exception was caused by a concurrency violation, and MANUALLY retry the transaction. While it has been suggested that JDBC provided a subclass of java.sql.SQLException for exceptions where a retry is warranted, the spec designers of JDBC rejected this suggestion, feeling a big CASE statement to determine the nature of the SQL Exception to be better design. Also note that EJB provides no facilities to automate this kind of transaction retry, although it could very easily be done by the container. (c) Thread Pool Exhaustion. In this scenario, the first thread to request a resource beings using it while all the other threads from the same thread pool are stuck waiting in line. What this means is that not only is performance poor for the CURRENT USER who clicked on the link multiple times, queueing up the requests blocking on that user's resource, but performance ALL USERS is negatively impacted because NO USER is able to have their request serviced until a thread is finally made available again in the thread pool. In effect, ALL USERS are stuck waiting on that ONE GREEDY USER. (d) Rather than queue up requests for the shared resource, the 2nd and subsequent concurrent requests are summarily rejected, and exceptions are thrown. This is often the case when STATEFUL SESSION BEANS are used, as the EJB SPEC states: "Clients are not allowed to make concurrent calls to a stateful session object. If a client-invoked business method is in progress on an instance when another client-invoked call, from the same or different client, arrives at the same instance of a stateful session bean class, the container may throw the java.rmi.RemoteException to the second client [4] , if the client is a remote client, or the javax.ejb.EJBException, if the client is a local client." Some application servers allow an option where concurrent requests to STATEFUL SESSION BEANS to be queued, rather than be rejected. However, now we're back to the problem where a single user can tie up all the threads in the thread pool, causing the one greedy user to degrade the performance of all users. (e) Data corruption & loss. This last scenario occurs when concurrent requests to state are neither queued nor rejected/rolled back. For example, two threads may attempt to at the same time read and write to the contents a java.util.Hashmap or a java.util.LinkedList stored on the HTTP Session, which can result in corruption or loss of the underlying data structure or data. In another example, two threads may concurrently be reading from and writing to the same database record, without the protect of an overall transaction, which breaks isolation and can cause corrupt or incorrect data to be stored in the database. So what do we do? a) Allow concurrent manipulation of the data, corrupting it? b) Reject concurrent requests with rollbacks or RemoteExceptions, causing the end user to see a broken app? c) Queue requests, eating up the thread pool, degrading performance of all users, not just the one greedy user? Most of the time requests will be quick to process, and thread pool starvation from queuing won't be a big deal. In this case, consider synchronizing all HTTP service requests on the HTTP Session. For example: public class MyServlet extends javax.http.HttpServlet { public void service(...) { HttpSession session = request.getSession(); synchronized(session) { //service the request here. } } } (Note: For distributed web applications where the same session may be accessed on multiple machines, it's a bit more complex than this.) And of course, in the case of READ-ONLY operations, corrupting data is a non-issue, and we need not be concerned with concurrent access. So if your web request is of this type, consider using an isModified flag to avoid the ejbStore() operation in your entity beans if nothing has changed. (Most CMP engines these days can do this for you.) But if you really are concerned about queuing, what you may really want is a policy that: a) Properly protects our data against corruption b) Restricts the number of queued requests associated with the same user (ie the same HTTP Session). and c) Allows the MOST RECENT request to proceed (as this is the one the end user will see) while rejecting the EARLIER requests still sitting in the queue. The way to do this is with a thread apartment in the servlet for each HTTP Session. When a new request arrives, the thread apartment for that HTTP Session is obtained. If it's the only current request for that apartment, the request is allowed to proceed. If it is the 2nd request and there is already a request in processing, the first request can't be easily aborted and must be allowed to complete, so the 2nd request waits for the first to complete. If a 3rd request arrives while the 2nd request is still waiting, the 2nd request is removed from the wait queue and aborted (as it hasn't begun it's processing yet), and the 3rd request waits on the 1st requests completion instead. So here the policy is that a maximum of 1 request can be waiting for a particular user at a time. In applications with frames or multiple windows, you've got to allow queue sizes to be a little bit larger. Coding a thread apartment with a policy like this is rather tricky, and involves a lot of the subtleties of thread-safe programming. So unless requests take a long time to process and thread pool starvation is a real issue, I recommend you go with a simpler approach. However, on previous projects, I have experienced situations where thread pool exhaustion was occurring as a result of this form of queuing. If there's interest, I can post the apartment pattern on Theserverside.com. Doug The Middleware Company On Sun, 5 May 2002 11:12:32 -0700, Alex Paransky <[EMAIL PROTECTED]> wrote: >I understand what you are saying in the first possible solution. However, >in this case the user is clicking on the same link. There are multiple >requests which are being fired off at the same time. The access is the >same, since it's the same link that is being pressed. > >The second solution would work at the cost of having to maintain duplicate >code. The navigation relationships and security management are quite >difficult, thus I would have to duplicate a lot of complicated code. > >I am wondering if this is a bug on the application server. > >-AP_ > > >-----Original Message----- >From: A mailing list for Enterprise JavaBeans development >[mailto:[EMAIL PROTECTED]]On Behalf Of Mike Bresnahan >Sent: Sunday, May 05, 2002 10:52 AM >To: [EMAIL PROTECTED] >Subject: Re: The problem with deadlocking transactions... > > >> When we wrap a transaction around our read operations the >> container does not >> have to ejbLoad/ejbStore for every call into the entity bean, however, we >> noticed that users clicking on the same link in the browser in rapid >> succession create multiple requests which all promptly deadlock on each >> other. > >First possible solution: Analyze your data access paths and see if you can >ensure that users always access data in the same order. For example, given >entities X and Y, make sure that users always lock X and then Y. If you can >do this, you can eliminate the possibility of a deadlock. > >Second possible solution: Use a lightweight mechanism for read-only data. >I.e. don't use EJBs for data that you are only reading, use a Data Access >Object instead. > >Mike Bresnahan > > ========================================================================== >To unsubscribe, send email to [EMAIL PROTECTED] and include in the body >of the message "signoff EJB-INTEREST". For general help, send email to >[EMAIL PROTECTED] and include in the body of the message "help". > > ========================================================================== >To unsubscribe, send email to [EMAIL PROTECTED] and include in the body >of the message "signoff EJB-INTEREST". For general help, send email to >[EMAIL PROTECTED] and include in the body of the message "help". > =========================================================================== To unsubscribe, send email to [EMAIL PROTECTED] and include in the body of the message "signoff EJB-INTEREST". For general help, send email to [EMAIL PROTECTED] and include in the body of the message "help".
