Re: Why "/ 1000 * 1000
I'm assuming ifModifiedSince is already a multiple of 1000 (these are times in msec). In that case, the following comparison would do the same thing: if (ifModifiedSince < (lastModified - 999)) or if (ifModifiedSince <= (lastModified - 1000)) But I suppose that /1000*1000 might be considered more "intuitive". Okay, I admit that's not exactly intuitive either. But they're both better than the suggestion in bug 33933 (no offense intended). http://issues.apache.org/bugzilla/show_bug.cgi?id=33933 Maybe a comment in the code would have prevented this whole thread altogether. ;-) ~Tom On Sep 14, 2005, at 10:41 PM, Bill Barker wrote: HTTP headers only send times to the second, so yes, to drop off some precision. - Original Message - From: "Yaakov Chaikin" <[EMAIL PROTECTED]> To: Sent: Wednesday, September 14, 2005 8:37 PM Subject: Why "/ 1000 * 1000 Hi, While reading the Tomcat's source code, I noticed the following line in the service method of the HttpServlet class: if (ifModifiedSince < (lastModified / 1000 * 1000)) What's the point of "/ 1000 * 1000"? To drop off some precision? Thanks, Yaakov. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This message is intended only for the use of the person(s) listed above as the intended recipient(s), and may contain information that is PRIVILEGED and CONFIDENTIAL. If you are not an intended recipient, you may not read, copy, or distribute this message or any attachment. If you received this communication in error, please notify us immediately by e-mail and then delete all copies of this message and any attachments. In addition you should be aware that ordinary (unencrypted) e-mail sent through the Internet is not secure. Do not send confidential or sensitive information, such as social security numbers, account numbers, personal identification numbers and passwords, to us via ordinary (unencrypted) e-mail. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native/common jk_lb_worker.c jk_shm.h jk_status.c
Mladen,
I appreciate you addressing this issue. But isn't this fix just
delaying the problem? An IEEE double has 52 bits of mantissa versus
32 bits for size_t. So the problem will now occur at 4 PB
(petabytes). Sure, that's a lot of bytes and my webservers would
never see this kind of load. But there might be some sites serving
lots of large files that could have a problem with this ... maybe ...
sometime.
So, perhaps it's something that can be overlooked for now. But
maybe you should consider doing it like the Request method. In that
method it adds the lbfactor from each worker's value and subtracts
the total from the worker that is used. For the Traffic method I
imaging adding and adding/subtracting the lbfactor*bytes_read (or
lbfactor*bytes_transferred) might accomplish the same thing.
~Tom
On Jun 13, 2005, at 1:55 AM, [EMAIL PROTECTED] wrote:
mturk 2005/06/13 00:55:51
Modified:jk/native/common jk_lb_worker.c jk_shm.h jk_status.c
Log:
Use double instead size_t for trensferred/read, so that lb doesn't
break on trnasferred mode when 2G of data has been send/read.
Revision ChangesPath
1.90 +5 -5 jakarta-tomcat-connectors/jk/native/common/
jk_lb_worker.c
Index: jk_lb_worker.c
===
RCS file: /home/cvs/jakarta-tomcat-connectors/jk/native/common/
jk_lb_worker.c,v
retrieving revision 1.89
retrieving revision 1.90
diff -u -r1.89 -r1.90
--- jk_lb_worker.c15 May 2005 16:33:47 -1.89
+++ jk_lb_worker.c13 Jun 2005 07:55:51 -1.90
@@ -225,8 +225,8 @@
{
unsigned int i;
int total_factor = 0;
-size_t mytraffic = 0;
-size_t curmin = 0;
+double mytraffic = 0;
+double curmin = 0;
worker_record_t *candidate = NULL;
if (p->lblock == JK_LB_LOCK_PESSIMISTIC)
@@ -313,8 +313,8 @@
jk_logger_t *l)
{
unsigned int i;
-size_t mytraffic = 0;
-size_t curmin = 0;
+double mytraffic = 0;
+double curmin = 0;
worker_record_t *candidate = NULL;
if (p->lblock == JK_LB_LOCK_PESSIMISTIC)
1.21 +3 -3 jakarta-tomcat-connectors/jk/native/common/
jk_shm.h
Index: jk_shm.h
===
RCS file: /home/cvs/jakarta-tomcat-connectors/jk/native/common/
jk_shm.h,v
retrieving revision 1.20
retrieving revision 1.21
diff -u -r1.20 -r1.21
--- jk_shm.h15 May 2005 11:23:50 -1.20
+++ jk_shm.h13 Jun 2005 07:55:51 -1.21
@@ -81,9 +81,9 @@
/* Statistical data */
volatile time_t error_time;
/* Number of bytes read from remote */
-volatile size_t readed;
+volatile double readed;
/* Number of bytes transferred to remote */
-volatile size_t transferred;
+volatile double transferred;
/* Number of times the worker was elected */
volatile size_t elected;
/* Number of non 200 responses */
1.42 +4 -15 jakarta-tomcat-connectors/jk/native/common/
jk_status.c
Index: jk_status.c
===
RCS file: /home/cvs/jakarta-tomcat-connectors/jk/native/common/
jk_status.c,v
retrieving revision 1.41
retrieving revision 1.42
diff -u -r1.41 -r1.42
--- jk_status.c15 May 2005 15:22:05 -1.41
+++ jk_status.c13 Jun 2005 07:55:51 -1.42
@@ -166,11 +166,10 @@
}
/* Actually APR's apr_strfsize */
-static char *status_strfsize(size_t size, char *buf)
+static char *status_strfsize(double size, char *buf)
{
const char ord[] = "KMGTPE";
const char *o = ord;
-int remain;
if (size < 0) {
return strcpy(buf, " - ");
@@ -181,22 +180,12 @@
return buf;
}
do {
-remain = (int)(size & 1023);
-size >>= 10;
+size /= 1024;
if (size >= 973) {
++o;
continue;
}
-if (size < 9 || (size == 9 && remain < 973)) {
-if ((remain = ((remain * 5) + 256) / 512) >= 10)
-++size, remain = 0;
-if (sprintf(buf, "%d.%d%c", (int) size, remain, *o)
< 0)
-return strcpy(buf, "");
-return buf;
-}
-if (remain >= 512)
-++size;
-if (sprintf(buf, "%3d%c", (int) size, *o) < 0)
+if (sprintf(buf, "%.2f%c", size, *o) < 0)
return strcpy(buf, "");
return buf;
} while (1);
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For add
Re: mod_jk glitches
I should have trusted my instincts and not my math. A size_t (32 bits on most machines) rolls over at 4 GB, not 4 MB... d'oh! So this falls apart under a decent load, after a day or two in my case. I guess I'll be going back to the "request" method. For me, that should last about 1000 days before rolling over. I suggest that maybe doubles would be better for the read/write bytes. Although I still prefer a model that doesn't break at rollover (reset all counters or moving averages for example). On Jun 9, 2005, at 8:13 PM, Tom Anderson wrote: At first I thought maybe it was because transferred, readed (sic) and mytraffic are size_t and maybe one of them rolled over. But that would rollover at 4MB right?
Re: mod_jk glitches
It didn't paste as well as I'd hoped... the relevant information, formatted in text is: NameFAccWrRdBusyMax webl7100495464 239M 2.4G 1 41 webl4130648407 312M 3.1G 1 50 webl6169 1056555 507M 1.1G 7 56 < now getting ALL requests webl8220 41631672.0G 3.8G 28 109 webl5 25 12457160M 606M 1 19 Where the headings come from the status tool: NameWorker route name FLoad Balancer Factor AccNumber of requests Wr Number of bytes transferred Rd Number of bytes read Busy Current number of busy connections Max Maximum number of busy connections Hoping this time it's readable... On Jun 9, 2005, at 8:13 PM, Tom Anderson wrote: I hope that the below "snapshot" of my jkstatus shows up okay. This is from my current setup using mod_jk 1.2.13 and using the method=Traffic setting. What's not obvious from this static snapshot is that the middle webserver (webl6) is currently getting all requests. This is in spite of the fact that the 4th server shows 28 busy connections... don't believe it, it's not getting any connections. The only webserver getting requests of the 5 is the middle one. I don't know how that happened but it appears that the Rd (bytes read) got reset so in order to "balance" things out it is now sending everything to the one server. At first I thought maybe it was because transferred, readed (sic) and mytraffic are size_t and maybe one of them rolled over. But that would rollover at 4MB right? Since I'm not fluent in this code, maybe someone who is could comment. From my cursory look, it appears there might be a couple of issues here: 1. Relying on total bytes (requests) can lead to situations where all requests go to a single worker if one of counters gets messed up. Perhaps it would be more reliable to keep a moving average instead which might only temporarily disrupt normal operations. 2. Based on the Busy counts being incorrect, there doesn't appear to be any semaphore locking of the shared memory. Could that be why the Rd value got reset? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
mod_jk glitches
I hope that the below "snapshot" of my jkstatus shows up okay. This is from my current setup using mod_jk 1.2.13 and using the method=Traffic setting. What's not obvious from this static snapshot is that the middle webserver (webl6) is currently getting all requests. This is in spite of the fact that the 4th server shows 28 busy connections... don't believe it, it's not getting any connections. The only webserver getting requests of the 5 is the middle one. I don't know how that happened but it appears that the Rd (bytes read) got reset so in order to "balance" things out it is now sending everything to the one server. At first I thought maybe it was because transferred, readed (sic) and mytraffic are size_t and maybe one of them rolled over. But that would rollover at 4MB right? Since I'm not fluent in this code, maybe someone who is could comment. From my cursory look, it appears there might be a couple of issues here: 1. Relying on total bytes (requests) can lead to situations where all requests go to a single worker if one of counters gets messed up. Perhaps it would be more reliable to keep a moving average instead which might only temporarily disrupt normal operations. 2. Based on the Busy counts being incorrect, there doesn't appear to be any semaphore locking of the shared memory. Could that be why the Rd value got reset? ~Tom Worker Status for loadbalance Type Sticky session Force Sticky session Retries Method Lock lb True False 3 Traffic Optimistic Name Type Host Addr Stat F V Acc Err Wr Rd Busy Max RR Cd webl7 ajp13 webl7:8009 172.18.7.100:8009 OK 100 100 495464 242 239M 2.4G 1 41 webl4 ajp13 webl4:8009 172.18.4.100:8009 OK 130 130 648407 368 312M 3.1G 1 50 webl6 ajp13 webl6:8009 172.18.6.100:8009 OK 169 169 1056555 305 507M 1.1G 7 56 webl8 ajp13 webl8:8009 172.18.8.100:8009 OK 220 220 4163167 1662 2.0G 3.8G 28 109 webl5 ajp13 webl5:8009 172.18.5.100:8009 OK 25 25 124571 53 60M 606M 1 19
Re: Sudden Shutdown - EXCEPTION_ACCESS_VIOLATION
I've been getting this behavior on exactly one of our several web servers. I haven't narrowed it down but suspect a possible hardware issue (RAM?). It always occurs within the jvm (jvm.dll in your case) so it could be a bug in the JVM too I suppose. We're going to try to move our disks to another box and see if the problem persists or not. Sorry I don't have any solutions but I thought I'd let you know that you're not alone. ~Tom On Mar 11, 2005, at 4:31 AM, Irineu Avanço Jr. wrote: Hello people... I was doing a load testing with my Web application running under Tomcat, and suddenly Tomcat has stopped. So, I went to the sdtout.log file and got the fatal error message listed below. Then I started the server again, restarted the load testing and got the same problem a couple of times. Has anybody ever faced a problem like this before? Does anybody know how to handle that? Softwares installed in the server. S.O.: Windows XP 2002. Tomcat 5.0.27 J2SDK 1.4.2_07 Error I've been taken from the stdout.log file Unexpected Signal : EXCEPTION_ACCESS_VIOLATION (0xc005) occurred at PC=0x8036580 Function=[Unknown.] Library=C:\j2sdk1.4.2_07\jre\bin\client\jvm.dll NOTE: We are unable to locate the function name symbol for the error just occurred. Please refer to release documentation for possible reason and solutions. Current Java thread: at java.util.Hashtable.get(Hashtable.java:332) - locked <0x107c9bc8> (a java.util.Hashtable) at org.apache.turbine.services.BaseServiceBroker.getServiceInstance(BaseSe rvice Broker.java:347) at org.apache.turbine.services.BaseServiceBroker.getService(BaseServiceBro ker.j ava:296) at org.apache.turbine.services.resources.TurbineResources.getService(Turbi neRes ources.java:109) at org.apache.turbine.services.resources.TurbineResourceService.interpolat e(Tur bineResourceService.java:249) at org.apache.turbine.services.resources.TurbineResourceService.getVector( Turbi neResourceService.java:527) at org.apache.turbine.services.resources.TurbineResources.getVector(Turbin eReso urces.java:326) at org.apache.turbine.modules.ScreenLoader.getInstance(ScreenLoader.java: 192) at org.apache.turbine.services.template.TurbineTemplateService.getParsedMo duleN ame(TurbineTemplateService.java:655) at org.apache.turbine.services.template.TurbineTemplateService.getCachedNa me(Tu rbineTemplateService.java:581) at org.apache.turbine.services.template.TurbineTemplateService.getScreenNa me(Tu rbineTemplateService.java:371) at org.apache.turbine.services.template.TurbineTemplate.getScreenName(Turb ineTe mplate.java:131) at org.apache.jetspeed.modules.pages.JetspeedTemplatePage.doBuildAfterActi on(Je tspeedTemplatePage.java:207) at org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java: 155) at org.apache.turbine.modules.Page.build(Page.java:90) at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:123) at org.apache.turbine.Turbine.doGet(Turbine.java:563) at org.apache.turbine.Turbine.doPost(Turbine.java:658) at javax.servlet.http.HttpServlet.service(HttpServlet.java:709) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic ation FilterChain.java:237) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil terCh ain.java:157) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal ve.ja va:214) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveC ontex t.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java: 520) at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardCo ntext Valve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal ve.ja va:152) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveC ontex t.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java: 520) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav a:137 ) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveC ontex t.java:104) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav a:117 ) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveC ontex t.java:102) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java: 520) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve .java :109) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveC ontex t.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java: 520) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java: 799) at org.apache.coyote.http1
Re: JDBCStore Downgrade (Tomcat4 vs. Tomcat 5)
It looks like the list stripped the image attachments. Let me know if you're interested and I'll forward you the original email with the pictures. ~Tom On Jan 5, 2005, at 4:59 PM, Tom Anderson wrote: FYI, We applied the 2 patches described in this thread with dramatic effects on our network bandwidth. Our JDBCStore talks to a database on another machine and this graph shows how the bandwidth usage dropped dramatically after these patches went up last night at 17:00 (most recent time is on the LEFT): I beleive that green represents bytes/sec FROM the database machine and the blue represents bytes/sec TO the database. The graph over the last week shows the same thing: I hope this picture really illustrates the problem and demonstrates the worthiness of the patch. Again, I volunteer to make the patch to whatever branches it's needed on. ~Tom p.s. I hope sending a picture to the list isn't taboo... if so, sorry. On Dec 30, 2004, at 8:53 PM, Tom Anderson wrote: And to fix the loading of too many sessions while processing expires, I suggest the attached patch to only load in sessions that you KNOW need to be expired (I believe it's better to err in loading too few sessions during store expiration since they will get removed eventually anyways). Here's that patch: FYI, I am testing both patches in my development environment and they seem to work well and achieve what I need to go forward with Tomcat 5. Hopefully you will accept one or both to make Tomcat more robust! Let me know if you want me to apply either or both patches to the HEAD (or whatever). And I would also be willing to do doc patches as needed. Thanks, ~Tom On Dec 29, 2004, at 7:16 PM, Tom Anderson wrote: I would like to share my experiences in upgrading from Tomcat 4 to Tomcat 5 and offer some suggestions for improvement (to Tomcat 5) based on them. This message got a little long but I hope somebody in the know will find the time to read it and help me figure out how to rectify my problems so I can upgrade to Tomcat 5 (again). BACKGROUND We have been using Tomcat 4 successfully for a couple of years (or more?). On some machines we have quite a few contexts running (approx. 75) so making sure they all work well together is important.We have been using the "experimental" JDBCStore mechanism with very good results and I have also contributed some patches to achieve this success. One problem we discovered early on with Tomcat 4 was the JDBCStore.processExpires() being called on all of our many contexts at roughly the same time. The effect was that the database would slow and/or crash. Fortunately, we were able to work around this problem using the checkInterval settings in JDBCStore. By staggering these checkIntervals to different values we could ensure that different contexts would hit the database at various times and spread the load out. Also, in our installation, the JDBCStore checkInterval has been set about 15 times higher than the PersistentManager checkInterval because it is slower and simply doesn't need to be run as often (clearing out the store isn't as important or fast as clearing out memory). TOMCAT 5 Recently we tried to upgrade to Tomcat 5 and discovered that our database was periodically choking and causing massive problems with Tomcat's ability to process requests. So we have had to rollback to Tomcat 4. I have spent the last several days going through the Tomcat 5 code to try to figure out what has been happening. I have discovered that the checkInterval values are no longer used (despite what the documentation says) and are instead replaced by the backgroundProcessorDelay value in the Container class. This means that I now only have one value to set for the equivalent of the 2 checkInterval values we had in Tomcat 4. I suspect that this was done to decrease the number of threads necessary which seems like a noble goal. But the downside is that I now am forced to run my JDBCStore.processExpires() much more often than I'd like (15 times more often in my case) which is a hardship on the database. Furthermore, in Tomcat 4 JDBCStore overrode StoreBase.processExpires() to only select older sessions from the database something like this: SELECT id FROM tomcat_sessions WHERE app = '/Standalone/localhost/acme' AND 1104288671296 > (lastaccess + maxinactive*1000))) This was a good thing because each session is loaded into memory before deciding whether or not to expire it, so you don't want to load every Stored session if possible. In my case we often have tens of thousands of sessions in Store so loading all those older sessions every 60 seconds is quite a hardship, especially when combined with the fact that it is running 15 times as often! To summarize the problems: 1. no control of the StoreBase.processExpires() interval means that processing ex
Re: JDBCStore Downgrade (Tomcat4 vs. Tomcat 5)
FYI, We applied the 2 patches described in this thread with dramatic effects on our network bandwidth. Our JDBCStore talks to a database on another machine and this graph shows how the bandwidth usage dropped dramatically after these patches went up last night at 17:00 (most recent time is on the LEFT): I beleive that green represents bytes/sec FROM the database machine and the blue represents bytes/sec TO the database. The graph over the last week shows the same thing: I hope this picture really illustrates the problem and demonstrates the worthiness of the patch. Again, I volunteer to make the patch to whatever branches it's needed on. ~Tom p.s. I hope sending a picture to the list isn't taboo... if so, sorry. On Dec 30, 2004, at 8:53 PM, Tom Anderson wrote: And to fix the loading of too many sessions while processing expires, I suggest the attached patch to only load in sessions that you KNOW need to be expired (I believe it's better to err in loading too few sessions during store expiration since they will get removed eventually anyways). Here's that patch: FYI, I am testing both patches in my development environment and they seem to work well and achieve what I need to go forward with Tomcat 5. Hopefully you will accept one or both to make Tomcat more robust! Let me know if you want me to apply either or both patches to the HEAD (or whatever). And I would also be willing to do doc patches as needed. Thanks, ~Tom On Dec 29, 2004, at 7:16 PM, Tom Anderson wrote: I would like to share my experiences in upgrading from Tomcat 4 to Tomcat 5 and offer some suggestions for improvement (to Tomcat 5) based on them. This message got a little long but I hope somebody in the know will find the time to read it and help me figure out how to rectify my problems so I can upgrade to Tomcat 5 (again). BACKGROUND We have been using Tomcat 4 successfully for a couple of years (or more?). On some machines we have quite a few contexts running (approx. 75) so making sure they all work well together is important. We have been using the "experimental" JDBCStore mechanism with very good results and I have also contributed some patches to achieve this success. One problem we discovered early on with Tomcat 4 was the JDBCStore.processExpires() being called on all of our many contexts at roughly the same time. The effect was that the database would slow and/or crash. Fortunately, we were able to work around this problem using the checkInterval settings in JDBCStore. By staggering these checkIntervals to different values we could ensure that different contexts would hit the database at various times and spread the load out. Also, in our installation, the JDBCStore checkInterval has been set about 15 times higher than the PersistentManager checkInterval because it is slower and simply doesn't need to be run as often (clearing out the store isn't as important or fast as clearing out memory). TOMCAT 5 Recently we tried to upgrade to Tomcat 5 and discovered that our database was periodically choking and causing massive problems with Tomcat's ability to process requests. So we have had to rollback to Tomcat 4. I have spent the last several days going through the Tomcat 5 code to try to figure out what has been happening. I have discovered that the checkInterval values are no longer used (despite what the documentation says) and are instead replaced by the backgroundProcessorDelay value in the Container class. This means that I now only have one value to set for the equivalent of the 2 checkInterval values we had in Tomcat 4. I suspect that this was done to decrease the number of threads necessary which seems like a noble goal. But the downside is that I now am forced to run my JDBCStore.processExpires() much more often than I'd like (15 times more often in my case) which is a hardship on the database. Furthermore, in Tomcat 4 JDBCStore overrode StoreBase.processExpires() to only select older sessions from the database something like this: SELECT id FROM tomcat_sessions WHERE app = '/Standalone/localhost/acme' AND 1104288671296 > (lastaccess + maxinactive*1000))) This was a good thing because each session is loaded into memory before deciding whether or not to expire it, so you don't want to load every Stored session if possible. In my case we often have tens of thousands of sessions in Store so loading all those older sessions every 60 seconds is quite a hardship, especially when combined with the fact that it is running 15 times as often! To summarize the problems: 1. no control of the StoreBase.processExpires() interval means that processing expired persisted sessions is done as frequently as sessions in memory which is too often in a production environment 2. the processing of Stored expires is loading all old sessions into memory before decidin
Re: JDBCStore Downgrade (Tomcat4 vs. Tomcat 5)
And to fix the loading of too many sessions while processing expires, I
suggest the attached patch to only load in sessions that you KNOW need
to be expired (I believe it's better to err in loading too few sessions
during store expiration since they will get removed eventually
anyways). Here's that patch:
Index: StoreBase.java
===
RCS file:
/home/cvspublic/jakarta-tomcat-catalina/catalina/src/share/org/apache/catalina/session/StoreBase.java,v
retrieving revision 1.8
diff -r1.8 StoreBase.java
189a190,198
> /**
> * By default oldKeys() returns all keys but may be overridden,
> * for efficiency, to provide a list of keys that are ready to be
> * expired.
> */
> public String[] oldKeys() throws IOException {
> return keys();
> }
>
207c216,217
< keys = keys();
---
> // only expire old sessions from the store
> keys = oldKeys();
Index: JDBCStore.java
===
RCS file:
/home/cvspublic/jakarta-tomcat-catalina/catalina/src/share/org/apache/catalina/session/JDBCStore.java,v
retrieving revision 1.9
diff -r1.9 JDBCStore.java
47c47
< * @version $Revision: 1.9 $, $Date: 2004/03/20 11:11:07 $
---
> * @version $Revision: 1.1 $, $Date: 2004/12/21 23:45:59 $
442c442,464
< public String[] keys() throws IOException {
---
> public String[] keys() throws IOException
> {
> return keys(false);
> }
>
> /**
> * Return an array containing the session identifiers of all Sessions
> * currently saved in this Store that are older than dirt.
> * If there are no such Sessions, a zero-length array is returned.
> *
> * @exception IOException if an input/output error occurred
> */
> public String[] oldKeys() throws IOException {
> return keys(true);
> }
>
>
> /**
> * Return all keys or just old keys depending on the 'old' argument.
> *
> * @exception IOException if an input/output error occurred
> */
> protected String[] keys(boolean old) throws IOException {
445a468,475
> if (old) {
> // old sessions are those where the current time is
> // greater than the last access time plus max inactive time
> keysSql += " AND ? > (" +
> sessionLastAccessedCol + " + (" + sessionMaxInactiveCol +
> "*1000))";
> }
>
461a492,494
> if (old) {
> preparedKeysSql.setLong(2,
> System.currentTimeMillis());
> }
FYI, I am testing both patches in my development environment and they
seem to work well and achieve what I need to go forward with Tomcat 5.
Hopefully you will accept one or both to make Tomcat more robust!
Let me know if you want me to apply either or both patches to the HEAD
(or whatever). And I would also be willing to do doc patches as
needed.
Thanks,
~Tom
On Dec 29, 2004, at 7:16 PM, Tom Anderson wrote:
I would like to share my experiences in upgrading from Tomcat 4 to
Tomcat 5 and offer some suggestions for improvement (to Tomcat 5)
based on them. This message got a little long but I hope somebody in
the know will find the time to read it and help me figure out how to
rectify my problems so I can upgrade to Tomcat 5 (again).
BACKGROUND
We have been using Tomcat 4 successfully for a couple of years (or
more?). On some machines we have quite a few contexts running
(approx. 75) so making sure they all work well together is important.
We have been using the "experimental" JDBCStore mechanism with very
good results and I have also contributed some patches to achieve this
success.
One problem we discovered early on with Tomcat 4 was the
JDBCStore.processExpires() being called on all of our many contexts at
roughly the same time. The effect was that the database would slow
and/or crash. Fortunately, we were able to work around this problem
using the checkInterval settings in JDBCStore. By staggering these
checkIntervals to different values we could ensure that different
contexts would hit the database at various times and spread the load
out. Also, in our installation, the JDBCStore checkInterval has been
set about 15 times higher than the PersistentManager checkInterval
because it is slower and simply doesn't need to be run as often
(clearing out the store isn't as important or fast as clearing out
memory).
TOMCAT 5
Recently we tried to upgrade to Tomcat 5 and discovered that our
database was periodically choking and causing massive problems with
Tomcat's ability to process requests. So we have had to rollback to
Tom
Re: JDBCStore Downgrade (Tomcat4 vs. Tomcat 5)
I thought it might be easiest to show my suggested solutions via
patches. These are against 5.0.28 but if the idea is accepted I could
make the patch against any version. I would also be happy to patch
the docs if this is deemed a good thing.
So, my solution to the Store interval problem (#1 in my original email)
is to emulate the managerChecksFrequency setting in the
StandardContext. In other words, I suggest a storeChecksFrequency
setting for the PersistenManager.The patch to accomplish this is
straightforward:
Index: PersistentManagerBase.java
===
RCS file:
/home/cvspublic/jakarta-tomcat-catalina/catalina/src/share/org/apache/catalina/session/PersistentManagerBase.java,v
retrieving revision 1.19
diff -r1.19 PersistentManagerBase.java
212a213,227
> /**
> * Frequency of the session cleanup, and related store operations.
> * Store operations will be done once for the specified amount of
> * backgroundProcess calls (ie, the lower the amount, the most often the
> * checks will occur).
> */
> private int storeChecksFrequency = 10;
>
>
> /**
> * Iteration count for store background processing.
> */
> private int storeCount = 0;
>
>
227c242,248
< ((StoreBase) this.getStore()).processExpires();
---
>
> // only process store expires every Nth time
> storeCount = (storeCount + 1) % storeChecksFrequency;
>
> if (storeCount == 0) {
> ((StoreBase) this.getStore()).processExpires();
> }
545a567,595
>
> /**
> * Return the frequency of store checks.
> */
> public int getStoreChecksFrequency() {
>
> return (this.storeChecksFrequency);
>
> }
>
>
> /**
> * Set the store checks frequency.
> *
> * @param storeChecksFrequency the new store checks frequency
> */
> public void setStoreChecksFrequency(int storeChecksFrequency) {
>
> if (storeChecksFrequency <= 0) {
> return;
> }
>
> int oldStoreChecksFrequency = this.storeChecksFrequency;
> this.storeChecksFrequency = storeChecksFrequency;
> support.firePropertyChange("storeChecksFrequency",
>new Integer(oldStoreChecksFrequency),
>new Integer(this.storeChecksFrequency));
>
> }
~Tom
On Dec 29, 2004, at 7:16 PM, Tom Anderson wrote:
I would like to share my experiences in upgrading from Tomcat 4 to
Tomcat 5 and offer some suggestions for improvement (to Tomcat 5)
based on them. This message got a little long but I hope somebody in
the know will find the time to read it and help me figure out how to
rectify my problems so I can upgrade to Tomcat 5 (again).
BACKGROUND
We have been using Tomcat 4 successfully for a couple of years (or
more?). On some machines we have quite a few contexts running
(approx. 75) so making sure they all work well together is important.
We have been using the "experimental" JDBCStore mechanism with very
good results and I have also contributed some patches to achieve this
success.
One problem we discovered early on with Tomcat 4 was the
JDBCStore.processExpires() being called on all of our many contexts at
roughly the same time. The effect was that the database would slow
and/or crash. Fortunately, we were able to work around this problem
using the checkInterval settings in JDBCStore. By staggering these
checkIntervals to different values we could ensure that different
contexts would hit the database at various times and spread the load
out. Also, in our installation, the JDBCStore checkInterval has been
set about 15 times higher than the PersistentManager checkInterval
because it is slower and simply doesn't need to be run as often
(clearing out the store isn't as important or fast as clearing out
memory).
TOMCAT 5
Recently we tried to upgrade to Tomcat 5 and discovered that our
database was periodically choking and causing massive problems with
Tomcat's ability to process requests. So we have had to rollback to
Tomcat 4. I have spent the last several days going through the
Tomcat 5 code to try to figure out what has been happening.
I have discovered that the checkInterval values are no longer used
(despite what the documentation says) and are instead replaced by the
backgroundProcessorDelay value in the Container class. This means
that I now only have one value to set for the equivalent of the 2
checkInterval values we had in Tomcat 4. I suspect that this was
done to decrease the number of threads necessary which seems like a
noble goal. But the downside is that I now am forced to run my
JDBCStore.proc
JDBCStore Downgrade (Tomcat4 vs. Tomcat 5)
I would like to share my experiences in upgrading from Tomcat 4 to Tomcat 5 and offer some suggestions for improvement (to Tomcat 5) based on them. This message got a little long but I hope somebody in the know will find the time to read it and help me figure out how to rectify my problems so I can upgrade to Tomcat 5 (again). BACKGROUND We have been using Tomcat 4 successfully for a couple of years (or more?). On some machines we have quite a few contexts running (approx. 75) so making sure they all work well together is important. We have been using the "experimental" JDBCStore mechanism with very good results and I have also contributed some patches to achieve this success. One problem we discovered early on with Tomcat 4 was the JDBCStore.processExpires() being called on all of our many contexts at roughly the same time. The effect was that the database would slow and/or crash. Fortunately, we were able to work around this problem using the checkInterval settings in JDBCStore. By staggering these checkIntervals to different values we could ensure that different contexts would hit the database at various times and spread the load out. Also, in our installation, the JDBCStore checkInterval has been set about 15 times higher than the PersistentManager checkInterval because it is slower and simply doesn't need to be run as often (clearing out the store isn't as important or fast as clearing out memory). TOMCAT 5 Recently we tried to upgrade to Tomcat 5 and discovered that our database was periodically choking and causing massive problems with Tomcat's ability to process requests. So we have had to rollback to Tomcat 4. I have spent the last several days going through the Tomcat 5 code to try to figure out what has been happening. I have discovered that the checkInterval values are no longer used (despite what the documentation says) and are instead replaced by the backgroundProcessorDelay value in the Container class. This means that I now only have one value to set for the equivalent of the 2 checkInterval values we had in Tomcat 4. I suspect that this was done to decrease the number of threads necessary which seems like a noble goal. But the downside is that I now am forced to run my JDBCStore.processExpires() much more often than I'd like (15 times more often in my case) which is a hardship on the database. Furthermore, in Tomcat 4 JDBCStore overrode StoreBase.processExpires() to only select older sessions from the database something like this: SELECT id FROM tomcat_sessions WHERE app = '/Standalone/localhost/acme' AND 1104288671296 > (lastaccess + maxinactive*1000))) This was a good thing because each session is loaded into memory before deciding whether or not to expire it, so you don't want to load every Stored session if possible. In my case we often have tens of thousands of sessions in Store so loading all those older sessions every 60 seconds is quite a hardship, especially when combined with the fact that it is running 15 times as often! To summarize the problems: 1. no control of the StoreBase.processExpires() interval means that processing expired persisted sessions is done as frequently as sessions in memory which is too often in a production environment 2. the processing of Stored expires is loading all old sessions into memory before deciding which ones need to be removed from Store instead of loading older sessions 3. the documentation still suggests that the checkInterval values are used in Tomcat 5 SOLUTIONS: There are many ways to solve problem #1. Here are my thoughts: 1a. We could reinstate the checkInterval for the Store. But I can appreciate the desire to keep the number of threads down so maybe this isn't the best idea. 1b. Invent a new parameter in StoreBase, something like checkRatio (TBR), that indicates how often to processExpires in relation to the PersistentManager. For example, if I set checkRatio="15" that means only run StoreBase.processExpires on every 15th call to PersistentManager.backgroundProcess(). 1c. Have ManagerBase and StoreBase both extend ContainerBase so that we can set different backgroundProcessorDelay values for them. This one makes me a bit nervous because I'm not familiar with the architectural issues it might raise but it seems like it could work. As for #2: 2a. Override processExpires() in JDBCStore so that it only loads sessions that have a good chance of needing expiration. 2b. Define a new method in Store called something like oldKeys() that returns keys that are past expiration and call that method in StoreBase.processExpires() And the solution to #3 is obvious... remove "checkInterval" from the documentation that appears here: http://jakarta.apache.org/tomcat/tomcat-5.0-doc/config/manager.html If you read this far, thank you very much. I look forward to hearing back from somebody who can help. And I would be happy to make patch(es) f
Re: [VOTE] 4.1.31 maintenance release
The 4.1.31 maintenance release should happen: [ X ] Yes [ ] No I would benefit directly from many of the fixes. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Where's 4.1.31?
Patching Tomcat 4.1.30 is pretty much what I have done. I spent a lot of effort getting our installation working in a stable way. And a lot of that effort was in applying patches similar to the ones that are in the baseline but have never been released. As far as moving to Tomcat 5.x, I have a ton of applications running on 4.1.x and moving them forward is no small task for me. In fact, the effort of stabilizing Tomcat 4.1.x has me gun shy. I will probably wait a year or so more until I know things are REALLY good and stable first. Eventually I will have to bite the bullet and do so but I thought it might be nice to get a 4.1.31 release (which adds a ton of stability fixes) in the interim so I could remove my custom patches. Charlie Cox hit it on the head with his response: Because a 4.1.x upgrade is not an api change. There is much more testing involved in upgrading to a new major version than a point release. The problem is finding the time to review the (possible)effects of 5.x on your installation and all your applications when you could roll out a point release with much less effort. I appreciate everyone's feedback and understand why you don't want to release a new 4.1.x. I don't necessarily agree it's a good thing since there are a lot of installations of 4.1 out there that would benefit, but I can live with it. Maybe I'll be back next year with questions about 5.1. ;-) Sorry for rocking the boat and thanks, ~Tom On Aug 20, 2004, at 10:35 AM, Jess Holle wrote: I've said too much on this already, but if you really need 4.1.x and bug fixes thereto, then why not take Tomcat 4.1.30 and patch it as necessary to address bugs of sufficient concern and deliver that to your customer/user base? That seems like a better solution for all than a 4.1.31 release. -- Jess Holle Shapira, Yoav wrote: Hi, I agree with Jess, this is the wrong direction in principle. We're encouraging users to stick with 4.1.x if we do this release. Normally we have just an informal "if everyone is OK with this, I'd like to push out release X on this day" and then a vote on labeling it as stable. The latter being the only official vote. But in this case, I'd want to have a more general vote of should we have 4.1.x maintenance releases, given the reasons stated earlier in this thread. If we have such a vote, and if it passes, and if you decide to go ahead with this release, then you will probably assume responsibility for bugs filed against 4.1.x. This is of course unofficial, but nonetheless this type of arrangement exists with Tomcat 3.3.x. None of us care much for the 4.1.x issues now, except that Mark moved the relevant Connector-related issues from 4.1.x to 5.0.x so that they don't get dropped. This type of move, which I didn't like originally, should definitely be stopped if 4.1.x is still in active development as indicated by regular releases. Man this is a bummer going into the weekend ;) Yoav Shapira Millennium Research Informatics - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Where's 4.1.31?
There have been some important bug fixes in the baseline since April that haven't made it to a release version yet. Are there plans to release another version of Tomcat 4.1? The changes I am interested in some fixes that were done by Glenn and Markt but some thought they might be too risky. But I applaud those changes since they could allow me to discard my own internal patches that I have had to make to address these same problems.Some of the problems/fixes I'm talking about are: 1. Inefficient database queries for persistent sessions 2. Session expiration problems caused by differing interpretations of the getLastAccessedTime() method 3. NPEs in StoreBase 4. Remove non-serializable attributes from sessions 5. Changed classes throw InvalidClassExceptions on de-serialization These seem like pretty significant issues to me and I doubt I'm the only one to have encountered them. Can I expect a new release any time soon? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-4.0/catalina/src/share/org/apache/catalina/session JDBCStore.java PersistentManagerBase.java StandardManager.java StandardSession.java StoreBase.java
I'm just a list lurker (so my opinion doesn't "count") but I'm really happy to see these fixes. I have had to override JDBCStore and PersistentManagerBase at my site to make similar fixes so that things work acceptably (yes, I submitted patches but they weren't accepted). Having these fixes in would allow me to upgrade to 4.1.x and get rid of my old overridden classes. Thanks Glenn! ~Tom On Apr 30, 2004, at 1:15 PM, [EMAIL PROTECTED] wrote: glenn 2004/04/30 12:15:26 Modified:catalina/src/share/org/apache/catalina/session JDBCStore.java PersistentManagerBase.java StandardManager.java StandardSession.java StoreBase.java Log: The JDBCStore required a great deal of unnecessary db queries to manage the persisted data. This could severly impact its ability to scale to large numbers of sessions. 1. When a JSESSIONID cookie was submitted with a request where the Session no longer exists multiple queries of the db occurred to try and load a persisted Session from the Store. I was seeing four attempts to load from the persistence store each request when a Session did not exist for a JSESSIONID. PersistentManagerBase swapIn() and swapOut() were patched to maintain a Hashtable of JSESSIONID's which do not exist in the Store so that they don't have to be checked multiple times. Each checkInterval the Hashtable is cleared to prevent it from consuming too much memory. 2. The StoreBase.processExpires() method triggers a load of each Session persisted to the db each checkInterval to perform its test to determine if the Session has expired. This incurred alot of overhead on the db, especially if there was a large amount of session data. The number of queries performed each checkInterval is 1 + number of sessions persisted to the db + number of expired sessions removed. The StoreBase.processExpires() method was overridden in JDBCStore. The method in JDBCStore performs a query of the db to find only those Sessions which should be expired. The number of queries performed here is 1 + 2 * the number of expired sessions (load then remove of expired session). 3. JDBCStore.remove() is being called sometimes with a null sessionid String causing an unnecessary synchronization and db query. Added a check for a null sessionid String at top of method. Problems with expiring sessions have been reported numerous times. The basic problem is as follows, starting at time 0 min and with a max inactive interval of 30 minutes: 00 min: First request, new session created, LastAccessedTime 0 02 min: Second request, reuse session, LastAccessedTime 0 31 min: Third request, reuse session, LastAccessedTime now 2 33 min: Background manager thread expires session even though it has only been two minutes since the remote clients last request. The argument for not changing how this works is based on how the Servlet Spec defines Session.getLastAccessedTime(). But I agree with all those who have complained about this behaviour that Tomcat session timeouts are buggy. So I came up with a compromise that still allows the HttpSession.getLastAccessedTime() to return the time of the previous request for those who are Servlet Spec purists. But internally sessions are expired when current time > last request + max inactive interval. When we do a major revision we should consider adding the StandardSession.getLastUsedTime() method to the org.apache.catalina.Session interface. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [PATCH] JDBCStore-howto.html Updated
The JDBCStore-howto.html document is in catalina/docs.But I see that manager.xml is NOT up-to-date (does not document the "app" column in JDBCStore). Also, many of the attributes that are listed as required actually have default values. And, shouldn't the example of the created table contain the default column names? Here is a patch to manager.xml with my corrections. Take it for what it's worth. ;-) ~Tom On Monday, August 25, 2003, at 05:13 PM, Glenn Nielsen wrote: Thanks for the patch Tom. I don't see where we include this document in our release docs anymore. The JDBCStore is documented in /webapps/tomcat-docs/config/manager.xml . And those docs are up to date. Perhaps this file should be removed from CVS. Regards, Glenn Tom Anderson wrote: I noticed that the JDBCStore-howto.html no longer reflects reality so I updated it. Here's my patch. ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[PATCH] JDBCStore-howto.html Updated
I noticed that the JDBCStore-howto.html no longer reflects reality so I updated it. Here's my patch. ~Tom Index: JDBCStore-howto.html === RCS file: /home/cvspublic/jakarta-tomcat-4.0/catalina/docs/JDBCStore-howto.html,v retrieving revision 1.1 diff -u -r1.1 JDBCStore-howto.html --- JDBCStore-howto.html27 Apr 2001 22:11:03 - 1.1 +++ JDBCStore-howto.html25 Aug 2003 21:46:33 - @@ -44,6 +44,7 @@ create table tomcat$sessions ( id varchar(100) not null primary key, +app varchar(100) not null, valid char(1) not null, maxinactive int not null, lastaccess bigint, @@ -75,6 +76,7 @@ mysql> create table tomcat$sessions -> ( -> id varchar(100) not null primary key, +-> app varchar(100) not null, -> valid char(1) not null, -> maxinactive int not null, -> lastaccess bigint, @@ -109,6 +111,7 @@ connectionURL="jdbc:mysql://localhost/tomcat?user=test&password=test" sessionTable="tomcat$sessions" sessionIdCol="id" +sessionAppCol="app" sessionDataCol="data" sessionValidCol="valid" sessionMaxInactiveCol="maxinactive" @@ -155,6 +158,11 @@ sessionIdCol The column in the session table that contains the session ID + + + +sessionAppCol + The column in the session table that identifies the webapp (built from Engine, Host and Context). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: StandardSession class question
I agree. It also messes with session persistence.
On Wednesday, March 5, 2003, at 06:21 PM, Michael Tildahl wrote:
It looks like the StandardSession class, in the 4.x line, uses two
variables
thisAccessedTime and lastAccessedTime to keep track of the sessions
last
accessed time. The method access(), which is called in the
StandardHostValue
class every time a user makes a request, contains the following code:
public void access() {
this.isNew = false;
this.lastAccessedTime = this.thisAccessedTime;
this.thisAccessedTime = System.currentTimeMillis();
}
The problem is that lastAccessedTime is used to determine if the
session has
expired. So it takes two "clicks" by a user to keep their session
active.
The variable thisAccessedTime is private and not really used except in
this
method. My question is why is this done? Is there some reason why the
thisAccessedTime variable is needed at all? It seems like setting
lastAccessedTime equal the System.currentTimeMillis() would work as
expected
and keep the session alive with only one "click". I've made this
change and
it looks to work.
Thanks
- Michael Tildahl
Aplia Inc.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC on Suse 8.1, urgent, please
Have you read this document? http://jakarta.apache.org/tomcat/tomcat-4.1-doc/class-loader-howto.html On Thursday, March 6, 2003, at 09:58 AM, Uros Kotnik wrote: Hi I know this is for tomcat-user but maybe I can get faster answer here. I'm using TC 4.1.18 I developed app in W2K environment, but have problems to configure TC on Suse 8.1 Linux. I have my base classes with my realm implementation and other stuff in framework.jar, framework.jar is in TChome/server/lib On windows everything is working OK but when I try to start TC on Linux I first get : - java.lang.ClassNotFoundException: org.asterius.servlet.InitListener InitListener is my class in framework.jar OK, then I put framework.jar in classpath Afther that I get this : - java.lang.NoClassDefFoundError: org/apache/catalina/realm/RealmBase because I'm using RealmBase in framework.jar to make my own realm Than, when I put RealmBase in classpath I get java.lang.NoClassDefFoundError for other classes and so on... It seems that I have to put all TC classes(jar) in classpath ? Any solution to that ? Thanx - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: UTF-8 characters
It would be nice if Tomcat detected the encoding. I posted a similar
question with no response.
Here's how I think you should handle this.
// call before getting anything from the reqest
request.setCharacterEncoding("UTF-8");
Then all strings will be pulled as UTF-8 which is equivalent to the
work-around below.
FYI, from the Servlet 2.3 specification:
SRV.4.9 Request data encoding
Currently, many browsers do not send a char encoding qualifier with the
Content-
Type header, leaving open the determination of the character encoding
for reading
HTTP requests. The default encoding of a request the container uses to
create the
request reader and parse POST data must be ТISO-8859-1У, if none has
been
specified by the client request. However, in order to indicate to the
developer in this
case the failure of the client to send a character encoding, the
container returns null
from the getCharacterEncoding method.
If the client hasnХt set character encoding and the request data is
encoded with
a different encoding than the default as described above, breakage can
occur. To
remedy this situation, a new method setCharacterEncoding(String enc) has
been added to the ServletRequest interface. Developers can override the
character encoding supplied by the container by calling this method. It
must be
called prior to parsing any post data or reading any input from the
request. Calling
this method once data has been read will not affect the encoding.
What I don't understand is why Tomcat isn't detecting and using the
Content-Type header when it IS specified. I have done tests to show
that Content-Type is being set but I still am forced to do the
request.setCharacterEncoding() to get things out correctly.
~Tom
On Tuesday, March 4, 2003, at 03:03 AM, Oxley, David wrote:
Cheers,
Worked a treat. Should this code be done automatically by Tomcat?
Dave.
-Original Message-
From: Kent Degrano [mailto:[EMAIL PROTECTED]
Sent: 04 March 2003 09:59
To: Tomcat Developers List
Subject: RE: UTF-8 characters
try this code, Dave.
String str = new
String(request.getParameter("key").getBytes("ISO-8859-
1"), "UTF-8");
-Original Message-
From: Oxley, David [mailto:[EMAIL PROTECTED]
Sent: Tuesday, March 04, 2003 5:30 PM
To: '[EMAIL PROTECTED]'
Subject: UTF-8 characters
When a response to the browser is in UTF-8 encoding form responses
from
said
page are giving UTF-8 characters when req.getParameter is called. i.e.
instead of getting é, I get the UTF-8 encoding é returned to our
servlet.
Surely the getParameter method should return é as it does with other
encodings. Is this a bug?
Dave.
__
__
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
__
__
---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.456 / Virus Database: 256 - Release Date: 2/18/2003
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.456 / Virus Database: 256 - Release Date: 2/18/2003
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
__
__
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
__
__
___
_
This e-mail has been scanned for all viruses by Star Internet. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
___
_
Character encoding
I have a question about the Tomcat implementation of the Java Servlet Spec (2.3) with regards to request character encoding. Section 4.9 of the spec reads as follows: SRV.4.9 Request data encoding Currently, many browsers do not send a char encoding qualifier with the Content- Type header, leaving open the determination of the character encoding for reading HTTP requests. The default encoding of a request the container uses to create the request reader and parse POST data must be "ISO-8859-1", if none has been specified by the client request. However, in order to indicate to the developer in this case the failure of the client to send a character encoding, the container returns null from the getCharacterEncoding method. My interpretation of this is that, if "charset" in the Content-Type header is set to something, then javax.servlet.ServletRequest.getCharacterEncoding() should return that encoding. And if that header is not set, then it would return null and use "ISO-8859-1". However, my tests of Tomcat 4.1.18 show that, for example, I can set charset to "Big5" (and I have verified that the Content-Type header is "text/html;charset=Big5" but a call to getCharacterEncoding() returns null. Am I misinterpreting the spec or is this a bug in Tomcat's implementation? ~Tom
[PATCH] Added "app" column to tomcat$sessions
I wrote a patch to JDBCStore.java that adds a new column to the
tomcat$sessions table to hold information about the webapp that is
trying to save the session.
The need for this comes when you run multiple webapps which each may
have classes in session that are unique to the webapp. As written,
JDBCStore tries to pull all webapps from the database regardless of who
generated them. Exceptions ensue.
With this patch, a webapp will only read and write sessions that belong
to it. I much prefer this to creating multiple tomcat$session tables
because of the testing and maintenance aspects. Hopefully somebody
will agree enough to consider applying this patch.
Cheers,
~Tom
p.s. With all the fixes I've submitted, my version of Tomcat has a much
more stable implementation of JDBCStore than the controlled version ...
good for me but bad for everyone else ... what's the trick to getting
the patches accepted?
Index: JDBCStore.java
===
RCS file:
/home/cvspublic/jakarta-tomcat-4.0/catalina/src/share/org/apache/catalina/session/JDBCStore.java,v
retrieving revision 1.9
diff -r1.9 JDBCStore.java
11c11
< * Copyright (c) 1999 The Apache Software Foundation. All rights
---
> * Copyright (c) 1999, 2003 The Apache Software Foundation. All rights
82a83
> import java.util.ArrayList;
125a127,131
>
> /**
> * name associated with this app
> */
> private String myName = null;
144a151,155
> * App column to use.
> */
> protected String sessionAppCol = "app";
>
> /**
219a231,245
> * Return the name for this instance (built from container name)
> */
> public String getName() {
> if (myName == null) {
> Container container = manager.getContainer();
> if (container.getParent() == null) {
> myName = container.getName();
> } else {
> myName = container.getParent().getName() + container.getName();
> }
> }
> return myName;
> }
>
> /**
300a327,346
> * Set the App column for the table.
> *
> * @param sessionAppCol the column name
> */
> public void setSessionAppCol(String sessionAppCol) {
> String oldSessionAppCol = this.sessionAppCol;
> this.sessionAppCol = sessionAppCol;
> support.firePropertyChange("sessionAppCol",
>oldSessionAppCol,
>this.sessionAppCol);
> }
>
> /**
> * Return the Id column for the table.
> */
> public String getSessionAppCol() {
> return(this.sessionAppCol);
> }
>
> /**
391,393c437,438
< "SELECT COUNT(s."+sessionIdCol+"), c."+sessionIdCol+
< " FROM "+sessionTable+" s, "+sessionTable+" c"+
< " GROUP BY c."+sessionIdCol;
---
> "SELECT "+sessionIdCol+" FROM "+sessionTable +
> " WHERE " + sessionAppCol + " = ?";
406a452
> preparedKeysSql.setString(1, getName());
408,414c454,460
< if (rst != null && rst.next()) {
< keys = new String[rst.getInt(1)];
< keys[0] = rst.getString(2);
< i=1;
<
< while(rst.next())
< keys[i++] = rst.getString(2);
---
> if (rst != null) {
> ArrayList ids = new ArrayList();
> while(rst.next()) {
> String id = rst.getString(1);
> ids.add(id);
> }
> keys = (String[])ids.toArray(new String[ids.size()]);
445c491,492
< ") FROM ".concat(sessionTable);
---
> ") FROM ".concat(sessionTable) +
> " WHERE " + sessionAppCol + " = ?";
455a503
> preparedSizeSql.setString(1, getName());
497c545
< " WHERE "+sessionIdCol+" = ?";
---
> " WHERE "+sessionIdCol+" = ? AND " + sessionAppCol + " = ?";
506a555
> preparedLoadSql.setString(2, getName());
630a680
> sessionAppCol+", "+
634c684
< sessionLastAccessedCol+") VALUES (?, ?, ?, ?, ?)";
---
> sessionLastAccessedCol+") VALUES (?, ?, ?, ?, ?, ?)";
665,668c715,719
< preparedSaveSql.setBinaryStream(2, in, size);
< preparedSaveSql.setString(3, session.isValid()?"1":"0");
< preparedSaveSql.setInt(4, session.getMaxInactiveInterval());
< preparedSaveSql.setLong(5, session.getLastAccessedTime());
---
> preparedSaveSql.setString(2, getName());
> preparedSaveSql.setBinaryStream(3, in, size);
> preparedSaveSql.setString(4, session.isValid()?"1":"0");
> preparedSaveSql.setInt(5, session.getMaxInactiveInterval());
> preparedSaveSql.setLong(6, session.getLastAccessedTime());
-
To unsubscribe, e-mail: [EMAIL P
session persistence with multiple webapps
I just discovered another feature of the the JDBCStore. If your webapps save their own classes in the Session (as webapps are wont to do), then when JDBCStore runs its checks to clean the database, it will try to pull in ALL sessions including those from other webapps. Becasue the classes from other webapps are not accessible from each webapp, this causes ClassNotFoundExceptions when it tries to load the Sessions (before it deletes them). Now, I realize that I could set up a different table for each webapp to avoid this problem. But this is a bit unwieldy in my environment because we try to put as many webapps on each box as possible with lots of redundant boxes. So maintenance of all these tables is an issue. I would like to propose modifying the JDBCStore to include to a column in the database for the webapp so that you could set up as many webapps you want. Thoughts? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: immediate Session backup
On Saturday, February 8, 2003, at 10:59 PM, Tom Anderson wrote: I would like the PersistentManagerBase to immediately backup a session if the maxIdleBackup parameter is zero. The reason I want this behavior is that, in a load-balanced architecture, the next request might go to another machine which would need to pull it out of store. Does this sound like a good thing to do? If so, what's the right way to do this. I was thinking that overriding add(Session) in PersistentManagerBase would to the trick. On further thought this would only save the session before the application was able to use it which wouldn't be very useful. I guess I don't know how to do this right now. ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
multiple session backups
I've been noticing that sessions are backed up over and over until they are swapped out, even if they haven't been accessed since the first backup. This could put a pretty heavy burden on the database (or file) store, especially if the checkInterval is low. I put a "hack" into my version to check if "timeIdle < maxIdleBackup + 2*checkInterval" in addition to the "timeIdle > maxIdleBackup" check. Now I realize this isn't a very good solution (what if the processing time exceeds the checkInterval?), but it is better in my mind to not back it up at all then to back it up over and over. I suppose the ideal solution would involve saving off the last persisted time (getLastPersistedTime()?) so that you could compare that to getLastAccessedTime (assuming this method actually returned the time of the last access which it doesn't). What do you think? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
StandardSession.getLastAccessedTime()
Can anyone explain to me why the getLastAccessedTime() method returns the PREVIOUS accessed time? The Java docs for HttpSession suggest this should be the "last time the client sent a request associated with this session". To me, that means it should be updated to the current time when access() is called. So why does the access() method set it to the previous time? The reason I ask is that I noticed some unexpected behavior with respect to PersistenceManagerBase and decided it was because of the return from getLastAccessedTime(). Specifically, if you let a session sit past maxIdleSwap then hit that session one time, it still swaps out right away because the previous time was longer than maxIdleSwap. ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
immediate Session backup
I would like the PersistentManagerBase to immediately backup a session if the maxIdleBackup parameter is zero. The reason I want this behavior is that, in a load-balanced architecture, the next request might go to another machine which would need to pull it out of store. Does this sound like a good thing to do? If so, what's the right way to do this. I was thinking that overriding add(Session) in PersistentManagerBase would to the trick. ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [TOMCAT 4.1.18] StandardSession.setId() found incohesive
I'm not quite sure why it does but I am finding the implementation of
this class in particular to be somewhat flawed. For example, the
recycle() method doesn't remove a session from memory (via the Manager)
but expire() does. So, when something is recycled, it is actually
left on the list of sessions too.
What does this have to do with setId(). Well, I only found all of the
above problems because I am trying to fix my copy of JDBCStore to
retain the session ID when it calls setId() (the problem I'm trying to
fix is that I discovered a Session "leak" related to the fact that a
loaded session has a different ID than what's in the database). But
the JDBCStore.load() method calls ManagerBase.createSession() which
ALSO calls Session.setId()... my head is spinning. So if I fix
JDBCStore.load() then setId() gets fired twice with the undesired
side-effects.
I'm not exactly answering your question but hopefully re-affirming that
there is a problem here. My opinion is that the StandardSession class
should have no knowledge of the Manager that created it. Then maybe
these circular calls would be eliminated. But what do I know?
~Tom
On Wednesday, February 5, 2003, at 04:33 PM, Daniel Rall wrote:
Developers, I'd like your input on why setId() has the side effects
that it does, and your suggestions on what solution I should implement
which would be acceptable to both Tomcat 4 and 5, and cleanly usable
by myself. The issue I filed includes the Valve implementation which
excercises the setId() method, and hacks around its side effects.
TRACKING ISSUE: http://issues.apache.org/bugzilla/show_bug.cgi?id=16822
OBSERVED BEHAVIOR: ManagerBase.createSession() calls
StandardSession.setId() as the last thing it does before returning a
newly created session. StandardSession.setId() exhibits the
undocumented side effect of firing a session creation event, iterating
over all HttpSessionListeners registered with the current web context
and notifying them of its "creation":
public void setId(String id) {
if ((this.id != null) && (manager != null))
manager.remove(this);
this.id = id;
if (manager != null)
manager.add(this);
// Notify interested session event listeners
fireSessionEvent(Session.SESSION_CREATED_EVENT, null);
// Notify interested application event listeners
Context context = (Context) manager.getContainer();
Object listeners[] = context.getApplicationListeners();
if (listeners != null) {
HttpSessionEvent event =
new HttpSessionEvent(getSession());
for (int i = 0; i < listeners.length; i++) {
if (!(listeners[i] instanceof HttpSessionListener))
continue;
HttpSessionListener listener =
(HttpSessionListener) listeners[i];
try {
fireContainerEvent(context,
"beforeSessionCreated",
listener);
listener.sessionCreated(event);
fireContainerEvent(context,
"afterSessionCreated",
listener);
} catch (Throwable t) {
try {
fireContainerEvent(context,
"afterSessionCreated",
listener);
} catch (Exception e) {
;
}
// FIXME - should we do anything besides log these?
log(sm.getString("standardSession.sessionEvent"),
t);
}
}
}
}
A second call to setId() will again fire a session creation event for
the existing session. Here's some trace from making a single request
when calling setId() explicitly:
2003-02-03 17:46:43,113 [Ajp13Processor[17025][4]] INFO default -
Adding
session org.apache.catalina.session.StandardSessionFacade@4977e2 with
id of
2EA4840C7D4D6C7F3FF76F1F95C575D5
2003-02-03 17:46:43,175 [Ajp13Processor[17025][4]] INFO default -
Adding
session org.apache.catalina.session.StandardSessionFacade@4977e2 with
id of
C6C908E127E6230CC81AE70E10D914A4
My web application's list of active sessions (stored as a Map of
StandardSessionFacade objects keyed by session ID) will look as
follows:
Session creation event fired as ManagerBase sets the initial ID by
calling StandardSession.setId() during its createSession() method
|
|
v
.__.
| Key | Value |
`--'
| ID A | Facade A --> Session A (ID A) |
`--'
|
|
Second session creation event fired as SessionIdValve resets the ID
[PATCH] JDBCStore.keys() doesn't scale well
As currently written, JDBCStore pulls session IDs from the database
using a query of the form:
SELECT COUNT(s.id), c.id FROM tomcat_sessions s, tomcat_sessions c
GROUP BY c.id;
While this query is a clever way to get the count back as part of the
query, it does not scale well to larger numbers of sessions. I had
about 5000 sessions in a moderately loaded system with a MySQL database
and this query would take about 80 seconds to run by hand because of
the N-squared nature of joining the table with itself.
This much simpler query is not as clever but gets the job done much
more efficiently (it only takes 0.00 seconds according to mysql):
SELECT id FROM tomcat_sessions;
It means a little more work determining the number of IDs after the
query runs but it's more than worth it.
Attached is the patch file.
~Tom
Index: JDBCStore.java
===
RCS file:
/home/cvspublic/jakarta-tomcat-4.0/catalina/src/share/org/apache/catalina/session/JDBCStore.java,v
retrieving revision 1.8
diff -u -r1.8 JDBCStore.java
--- JDBCStore.java 9 Dec 2002 15:05:55 - 1.8
+++ JDBCStore.java 4 Feb 2003 21:37:19 -
@@ -8,7 +8,7 @@
*
* The Apache Software License, Version 1.1
*
- * Copyright (c) 1999 The Apache Software Foundation. All rights
+ * Copyright (c) 1999,2003 The Apache Software Foundation. All rights
* reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -80,6 +80,7 @@
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
+import java.util.ArrayList;
import org.apache.catalina.Container;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.Loader;
@@ -388,9 +389,7 @@
*/
public String[] keys() throws IOException {
String keysSql =
-"SELECT COUNT(s."+sessionIdCol+"), c."+sessionIdCol+
-" FROM "+sessionTable+" s, "+sessionTable+" c"+
-" GROUP BY c."+sessionIdCol;
+"SELECT "+sessionIdCol+" FROM "+sessionTable;
Connection _conn = getConnection();
ResultSet rst = null;
@@ -405,13 +404,13 @@
preparedKeysSql = _conn.prepareStatement(keysSql);
rst = preparedKeysSql.executeQuery();
-if (rst != null && rst.next()) {
-keys = new String[rst.getInt(1)];
-keys[0] = rst.getString(2);
-i=1;
-
-while(rst.next())
-keys[i++] = rst.getString(2);
+if (rst != null) {
+ArrayList ids = new ArrayList();
+while(rst.next()) {
+String id = rst.getString(1);
+ids.add(id);
+}
+keys = (String[])ids.toArray(new String[ids.size()]);
} else {
keys = new String[0];
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: MacOS X - mod_jk.so
Looks like you may need the dlcompat stuff. Check out this URL: http://www.geektimes.com/macintosh/os/x/10_0/and/tomcat_install.html On Sunday, February 2, 2003, at 09:01 PM, Ro wrote: Hi, i am doing Java JSP coding and I installed TOMCAT 4.1.18. Now I would like to link it with apache. I've been try all day to build the mod_jk.so on OSX but with no results: When I try to build it with the dlopen.c and dlfcn.h... I get an error from the dlopen.c file: dlopen.c:84: return type is an incomplete type dlopen.c: In function `NSMakePrivateModulePublic': dlopen.c:95: warning: `return' with a value, in function returning void dlopen.c:97: invalid use of undefined type `enum bool' dlopen.c:97: warning: `return' with a value, in function returning void dlopen.c: In function `dlopen': dlopen.c:255: void value not ignored as it ought to be dlopen.c:337: void value not ignored as it ought to be Please help Rolley ___ Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français ! Yahoo! Mail : http://fr.mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBCStore Connection Pooling
} catch (SQLException e) {
-;
-}
-
this.preparedSizeSql = null;
this.preparedKeysSql = null;
this.preparedSaveSql = null;
this.preparedClearSql = null;
this.preparedRemoveSql = null;
this.preparedLoadSql = null;
-this.conn = null;
+this.connPool.clear();
+this.connPool = null;
}
}
}
On Thursday, January 30, 2003, at 02:31 PM, Tom Anderson wrote:
True, that might be a better design but my motivation was for a quick
fix with minimal impact to the design.
On Thursday, January 30, 2003, at 01:25 PM, Glenn Nielsen wrote:
A better solution might be to have a Store which can use a JNDI named
DataSource. Then let the DataSource worry about connection pooling,
etc.
Glenn
Tom Anderson wrote:
I have been trying to use the JDBCStore stuff and found it to be a
little unstable especially when the database connections have
problems (or when the database is bounced). So, I decided to make
some modifications that would allow it to drop connections on
SQLExceptions and maintain a small connection pool (LinkedList of
Connections).
Should I post the patch to this forum?
~Tom
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: JDBCStore Connection Pooling
True, that might be a better design but my motivation was for a quick fix with minimal impact to the design. On Thursday, January 30, 2003, at 01:25 PM, Glenn Nielsen wrote: A better solution might be to have a Store which can use a JNDI named DataSource. Then let the DataSource worry about connection pooling, etc. Glenn Tom Anderson wrote: I have been trying to use the JDBCStore stuff and found it to be a little unstable especially when the database connections have problems (or when the database is bounced). So, I decided to make some modifications that would allow it to drop connections on SQLExceptions and maintain a small connection pool (LinkedList of Connections). Should I post the patch to this forum? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
JDBCStore Connection Pooling
I have been trying to use the JDBCStore stuff and found it to be a little unstable especially when the database connections have problems (or when the database is bounced). So, I decided to make some modifications that would allow it to drop connections on SQLExceptions and maintain a small connection pool (LinkedList of Connections). Should I post the patch to this forum? ~Tom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
