Re: [VOTE] JK 1.2.14.1
Please vote: [X] Stable -- good build [ ] Alpha -- something serious is wrong: what is it? Functional and stress tested against 2.0.54, and 2.1.6 on NetWare 6.5 SP3 and SP4 on MP server (4 CPUS). Only stress test for apache 1.3. Using Tomcat 4.1.30. Thank you, JJ
mod_jk 1.2.14-dev testing status report
Tested mod_jk 1.2.14-dev with nb_connect() and setsockopt() enabled for NetWare in jk_connect.c: (all testing on NetWare 6.5 SP3 and SP4) 2.0.52 and Tomcat 4.1.30: build correctly and stress tested, 2.0.54 and Tomcat 4.1.31: build correctly, stress and functional testing done, 2.1.x: build correctly, 1.3.33: build with minor tweaking to environment and makefile, stress tested with UP version of Apache. Crash with MP version of Apache (unsupported, so not a problem), 1.3.head: build with same minor tweaking as with 1.3.33. The tweaking to build with 1.3.x might need to be fixed in the future, but it is a low priority item for me. Next week, I will try to push 1.2.14-dev in our official build in order to get it exposed to more testers. The ap_rflush() changes speed up our download rate by a factor of three on large files (700MB), this is cool. Thanks, Jean-Jacques
cvs commit: jakarta-tomcat-connectors/jni/native/common/jk_connect.c
clar2005/06/10 13:31:49
Modified:jk/native/common jk_connect.c
Log:
Enabling nb_connect() and setsockopt for apache build with LibC (2.0.x) on
NetWare.
Revision ChangesPath
1.61 +4 -4 jakarta-tomcat-connectors/jk/native/common/jk_connect.c
Index: jk_connect.c
===
RCS file: /home/cvs/jakarta-tomcat-connectors/jk/native/common/jk_connect.c,v
retrieving revision 1.60
retrieving revision 1.61
diff -u -r1.60 -r1.61
--- jk_connect.c26 May 2005 14:39:37 -1.60
+++ jk_connect.c10 Jun 2005 20:31:49 -1.61
@@ -109,7 +109,7 @@
return 0;
}
-#if defined (WIN32)
+#if defined (WIN32) || (defined(NETWARE) defined(__NOVELL_LIBC__))
/* WIN32 implementation */
static int nb_connect(int sock, struct sockaddr *addr, int timeout)
{
@@ -372,7 +372,7 @@
}
if (timeout 0) {
-#if defined(WIN32)
+#if defined(WIN32) || (defined(NETWARE) defined(__NOVELL_LIBC__))
int tmout = timeout * 1000;
setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,
(const char *) tmout, sizeof(int));
@@ -503,7 +503,7 @@
if (shutdown(s, SHUT_WR)) {
return jk_close_socket(s);
}
-#if defined(WIN32)
+#if defined(WIN32) || (defined(NETWARE) defined(__NOVELL_LIBC__))
setsockopt(s, SOL_SOCKET, SO_RCVTIMEO,
(const char *) tmout, sizeof(int));
#elif defined(SO_RCVTIMEO) defined(USE_SO_RCVTIMEO)
cvs commit: jakarta-tomcat-connectors/jk/native/apache-1.3 Makefile.netware
clar2005/06/10 09:24:35 Modified:jk/native/apache-1.3 Makefile.netware Log: mod_jk is not MT safe for NetWare on Apache 1.3.x Revision ChangesPath 1.7 +1 -1 jakarta-tomcat-connectors/jk/native/apache-1.3/Makefile.netware Index: Makefile.netware === RCS file: /home/cvs/jakarta-tomcat-connectors/jk/native/apache-1.3/Makefile.netware,v retrieving revision 1.6 retrieving revision 1.7 diff -u -r1.6 -r1.7 --- Makefile.netware13 Feb 2005 22:28:07 -1.6 +++ Makefile.netware10 Jun 2005 16:24:35 -1.7 @@ -22,7 +22,7 @@ VERSION= $(JK_VERSION) COPYR= Copyright (c) 2000-2004 The Apache Software Foundation. All rights reserved. DESCR= Apache $(AP_VERSION_STR) plugin for Jakarta/Tomcat $(JK_VERSION_STR) -MTSAFE= YES +MTSAFE= NO STACK= 49152 #SCREEN= NONE EXPORTS= jk_module
Re: JK shutdown problem
Thanks for the reply Mladen.
I was wrong, this is not a lingering time problem, but instead related
to the value that is being used for MAX_SECS_TO_LINGER, which is 16.
I still have data waiting to be transferred from Tomcat after 16 receives.
Each call to jk_tcp_socket_recvfull() returns data, but it takes more than
16 loops to get a JK_SOCKET_EOF.
Apache is using 30 for MAX_SECS_TO_LINGER. Any objection why
mod_jk cannot use the same value?
Thanks,
--JJ
[EMAIL PROTECTED] 5/20/2005 12:21 PM
Jean-Jacques Clar wrote:
Hi,
file: jk_connect.c
in jk_shutdown_socket(); when reading data from tomcat in the while loop,
the variable ttl is incremented every time by one, until breaking out of the
loop:
snippet:
line 505 *-
/* Read all data from the peer until we reach end-of-file (FIN
* from peer) or we've exceeded our overall timeout. If the client does
* not send us bytes within12 second, close the connection.
*/
while (1) {
nbytes = jk_tcp_socket_recvfull(s, dummy, sizeof(dummy));
if (nbytes = 0)
break;
ttl += SECONDS_TO_LINGER;
if (ttl MAX_SECS_TO_LINGER)
break;
}
The problem is because I do not have a Netware box.
You guys can try to enable the following:
#if defined(WIN32)
setsockopt(s, SOL_SOCKET, SO_RCVTIMEO,
(const char *) tmout, sizeof(int));
#endif
for Netware too...
I didn't try to enable that because so many times the
JK has been broken because of Netware build bugs ;)
Also the code in nb_connect:
#if defined(WIN32)
int tmout = timeout * 1000;
setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO,
(const char *) tmout, sizeof(int));
setsockopt(sock, SOL_SOCKET, SO_SNDTIMEO,
(const char *) tmout, sizeof(int));
#else ...
Shuld be checked for: (probably but who knows ;)
for :
#if defined(WIN32) || (defined(NETWARE) defined(__NOVELL_LIBC__))
Regards,
Mladen
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
JK shutdown problem
Hi,
file: jk_connect.c
in jk_shutdown_socket(); when reading data from tomcat in the while loop,
the variable ttl is incremented every time by one, until breaking out of the
loop:
snippet:
line 505 *-
/* Read all data from the peer until we reach end-of-file (FIN
* from peer) or we've exceeded our overall timeout. If the client does
* not send us bytes within12 second, close the connection.
*/
while (1) {
nbytes = jk_tcp_socket_recvfull(s, dummy, sizeof(dummy));
if (nbytes = 0)
break;
ttl += SECONDS_TO_LINGER;
if (ttl MAX_SECS_TO_LINGER)
break;
}
end *--
I don't see any linger time happening when calling recv(), in fact it returns
immediately,
therefore the while will loop 16 times maximum instead of 16 seconds. This is
causing
a problem on NetWare, where I have a socket still trying to send data for few
more loops.
It will then cause an IO exception in Java.
Removing the break based on the number of loop fixes my problem.
Is the problem present for all platforms or this is only because of our
implementation of winsock?
The same code in apache ap_lingering_close() does not seem to cause problems,
at least reported ones.
Thank you very much,
Jean-Jacques
Re: JK 1.2.13 TAGGED - ws_write call to ap_rflush in mod_jk.c
I really like being able to do: write(8K)*n + flush(). +1 on adding a FlushPackets config option set to off by default. Thank you Mladen and Jean-Frederic. Jean-Jacques [EMAIL PROTECTED] 5/18/2005 7:41 AM Henri Gomez wrote: good idea. Which state by default ? off (-FlushPackets). 2005/5/18, jean-frederic clere [EMAIL PROTECTED]: Mladen Turk wrote: jean-frederic clere wrote: Mladen Turk wrote: I think on/off is enough - Think of the questions nnn will bring in user list - So how about to just add the flag to JkOptions like +FlushPackets? +1. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[Q/PATCH] ws_write call to ap_rflush
Callingap_rflush() at the end of ws_write() in mod_jk.c is causing me
problems when doing downloads from the server toa client.
Performance degradation and, less importantly, memory usage, are the problems.
During calls to ap_rwrite() in ws_write, a brigade is created and
used to move data.When done, a call to apr_brigade_destroy()
is made.A call to apr_pool_cleanup_kill() ensures that the registered
cleanupentry is removed from the cleanups linked list associated
with the request pool. Thebrigade is then cleaned up.
After that, a callisdone to ap_rflush() within ws_writeto do what seems to me
the same operations that were just made.ap_rflush is alsocreating a brigade
on every call and adds its cleanup function to the cleanups list.
The brigades created by ap_rflush will not be destroyed before the
request is finished, if I understand correctly.
ap_rflush job is to destroy the brigade created by the ap_rwrite
operation, whichwas already done and removed from the cleanups
list previously. In my test, at the beginning of a download, mod_jk is doing
about 250 calls to ap_rflush() every seconds. This is when the cleanups list
is of reasonable size,gets much slower quickly because of the time spent
scanning the cleanups list.
I am downloading large files ( 1GB), it takes time and
by the time I get to the end of the download, my cleanups list has grow
to over 20 members and it has to be scanned unsuccessfully from beginning to
end in search of a brigade that was previously removed.
Removing the call to ap_rflush() is working great for me:
I ran stress and functional testing on my server w/ no apparent problems.
My questionare:
Whyis ws_write() calling ap_rflush()? I am sure other users are suffering from it.
Could it be just removed?
If it is essential, any suggestions fora workaround for my problem?
This is a link to a bug that is related to the extra brigade created by
ap_rflush():
http://issues.apache.org/bugzilla/show_bug.cgi?id=34589
Thank you very much,
Jean-Jacques
Index: jk/native/apache-2.0/mod_jk.c
===
RCS file:
/home/cvspublic/jakarta-tomcat-connectors/jk/native/apache-2.0/mod_jk.c,v
retrieving revision 1.143
diff -u -r1.143 mod_jk.c
--- jk/native/apache-2.0/mod_jk.c 7 May 2005 08:15:47 - 1.143
+++ jk/native/apache-2.0/mod_jk.c 12 May 2005 21:07:16 -
@@ -388,17 +388,6 @@
}
-/*
- * To allow server push. After writing full buffers
- */
-#ifndef AS400
-if (ap_rflush(p-r) != APR_SUCCESS) {
-ap_log_error(APLOG_MARK, APLOG_STARTUP | APLOG_CRIT, 0,
- NULL, mod_jk: Error flushing);
-return JK_FALSE;
-}
-#endif
-
}
return JK_TRUE;
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tagging 1.2.12
Build and run on NetWare with apache 2.0.54. JJ [EMAIL PROTECTED] 5/6/2005 6:31 AM Hi, It's been a week since 1.2.11 has been tagged and released. Because of bug in wc_close, and sice no other bugs have been reported for a week, I plan to tag the 1.2.12 tomorrow morning, 10:00 GMT. Any objections? Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tagging 1.2.12
Bill, I have been stress-testing JK for four days in my lab, since the wc_close() bug has been fixed.I have 58 clients, each using 10 threads, doing JSPs and servlets requests to 2 different servers (from the examples set) : emt64, dual proc with ht enable, and Compaq proliant 8 CPUS.The new mod_jk was twice tested by our set of automations in addition to spent 2 days being tortured by one of our system test engineer. I built and tried mod_jk against 2.0.52 and 2.1-dev. However, in these two cases it was not tested as for 2.0.54. Stress tests were done against Tomcat 4.1.30, validation against 4.1.31. For 1.3.33, I am sorry to admit that I did not build JK and is not currently planning to do it.I currently have access to these resources. What else do you want me to do? I will not and could never be able to cover this incredible set of combinations.Next time I will put more details in my report, but I was reporting a fact not more, not less.But from what I could do, 1.2.12 is good. --JJ [EMAIL PROTECTED] 5/9/2005 2:28 PM At 02:30 PM 5/9/2005, Jean-Jacques Clar wrote: Build and run on NetWare with apache 2.0.54. Woot! And thank you for the example, Jean-Jacques, of exactly why we need more than a few days before we declare 'stable' releases ;-) But I do think it's wonderful that the NetWare issues are gone. What about 1.3.33? What about 2.1-dev? What about other servers? And you mention it runs, but against which Tomcat, 5(.5)? I hope you see my point, there is an incredible set of combinations which are quite useful and legitimate in tomcat-connectors. I keep hearing the refrain Lets deprecate all this stuff I don't test. Rather than deprecate, what about simply allowing enough time for dust to settle, and reports to roll in. 3 +1's that all looks like it's good, and then let the community (with all their goofy combinations of OS's, connectors, and backends) chime in with complaints. [Yes - I mean complaints, few ever give us the good news when all is well :-] Bill - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [ANN/VOTE] JK 1.2.11 Released
Thank you Mladen, the patch is working for me. I cannot put 1.2.11 in our NetWare build because of that problem. That patch is critical for us. I will run stress test over the week-end and will have functionality testing done in our lab on Monday, both on HEAD. Thank you, Jean-Jacques Mladen Turk [EMAIL PROTECTED] 4/30/2005 12:09 AM Jean-Jacques Clar wrote: It looks like wc_close() is called twice when JK is unloading and destroying its memory pools resulting in trying to free the same mutex twice. Anyway, I have to run to the airport, but will look at it tomorrow sometime. I've committed a patch that should resolve the issue. Can you check against the HEAD? Sorry about the delay, If you think it's a critical bug, we'll retag and release 1.2.12 next week. Regards, Mladen.
Re: [ANN/VOTE] JK 1.2.11 Released
I just put 1.2.11 on my server and is having a problem with an invalid parameter passed from pthread_mutex_destroy() to kMutexFree(). pthread_mutex_destroy() is called within wc_close() when unloading jk. I am just getting started on trying to find the source of the problem. I don't see that problem with 1.2.10 on the same version of NetWare. Which make sense because the worker_lock is new in 1.2.11. Any hints? JJ [EMAIL PROTECTED] 4/29/2005 11:43 AM Henri Gomez wrote: Binaries for iSeries v5R3 and Suse 8.0 SLES PPC uploaded Cool :) Seems we'll have a best jk release ever. Where are the Netware guys? Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [ANN/VOTE] JK 1.2.11 Released
It looks like wc_close() is called twice when JK is unloading and destroying its memory pools resulting in trying to free the same mutex twice. Anyway, I have to run to the airport, but will look at it tomorrow sometime. Sorry about the delay, JJ [EMAIL PROTECTED] 4/29/2005 2:57:11 PM I just put 1.2.11 on my server and is having a problem with an invalid parameter passed from pthread_mutex_destroy() to kMutexFree(). pthread_mutex_destroy() is called within wc_close() when unloading jk. I am just getting started on trying to find the source of the problem. I don't see that problem with 1.2.10 on the same version of NetWare. Which make sense because the worker_lock is new in 1.2.11. Any hints? JJ [EMAIL PROTECTED] 4/29/2005 11:43 AM Henri Gomez wrote: Binaries for iSeries v5R3 and Suse 8.0 SLES PPC uploaded Cool :) Seems we'll have a best jk release ever. Where are the Netware guys? Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tagging JK 1.2.9 on 3/17/2005
Greetings, As Norm did, I was able to build and test against 2.1.4 beta and 2.0.52 for NetWare. I ran load testing against 1.2.9dev today with 400 virtual clients requesting JSPs and Servlets on a dual proc p4 with HT enabled. It is looking solid. The test will run overnight. I will find time tomorrow to look at the 1.3 issues, I did not build for 1.3 since jk1.2.8. Regards, JJ [EMAIL PROTECTED] 3/17/2005 1:26:49 PM Greetings All, Still get a clean build for NetWare against Apache 2.1 and 2.0.53. The XML pages work fine, although the HTML status pages might put the numbers under headings better if 'border=1' was enabled. AFAIK builds for Apache 1.1.3 for NetWare still has issues, since I've not seen mention of updates for that version. Perhaps JJC can comment? Regards, Norm - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[PATCH] NetWare threading using pthreads
OK.I'm also waiting Jean-Jaques Clar for a Netware threading patch. Here you go. NetWare has now a pthreads library. Thank you for your help and patience Mladen. It has been running on a 4 CPUS box for now over an hour. As far as performance, I will benchmark it against 1.2.6 after the holidays break. Attached is the jk_mt.h patched file and the patch. Gunter and Norm, If you have a chance would you please build and run the new mod_jk. Thanks, Index: jk/native/common/jk_mt.h===RCS file: /home/cvspublic/jakarta-tomcat-connectors/jk/native/common/jk_mt.h,vretrieving revision 1.11diff -u -r1.11 jk_mt.h--- jk/native/common/jk_mt.h2 Dec 2004 06:50:01 -1.11+++ jk/native/common/jk_mt.h23 Dec 2004 18:57:35 -@@ -28,11 +28,6 @@#if defined(WIN32)#define jk_gettid() ((int)GetCurrentThreadId())-#elif defined(NETWARE)-#define jk_gettid() ((int)GetThreadID())-#if !defined(__NOVELL_LIBC__)-#define getpid() ((int)GetThreadGroupID())-#endif /* __NOVELL_LIBC__ */#endif@@ -40,7 +35,7 @@ * All WIN32 code is MT, UNIX code that uses pthreads is marked by the POSIX * _REENTRANT define. */-#if defined (WIN32) || defined(_REENTRANT)+#if defined (WIN32) || defined(_REENTRANT) || defined (NETWARE) /* * Marks execution under MT compilation - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
JK 1.2.7 Beta3 Page Fault
Greetings, I did a clean co of jtc w/ JK_1_2_7_BETA_3 tag. I am building for NetWare with ap2.0.52. Running stress test on a 2 procs machine. Doing request to simple JSPs and servlets using WebBench with up to 250 virtual clients. It takes between few minutes and an hour to reproduce, this is the stack: ajp_reset_endpoint + 4 ajp_close_endpoint + 36 ajp_get_endpoint + E3 ajp_worker_factory%12 + 119B jk_set_options + 649 The ajp_endpoint_t pointer passed in ajp_reset_endpoint is a valid address but the reuse field is NULL. Setting up reuse to JK_FALSE seg fault the server. Since I inherited responsibility of mod_jk few weeks ago, and had my first look at the code 2 days ago, help would probably be usefull to resolve this problem. Any explanation of why this will happen? Any more information that I could provide to help narrow down that problem? Next Monday, I will try to reproduce that problem with the newly tagged 1.2.8rc1. Thanks, Jean-Jacques
[PATCH] /apache-2.0/NWGNUmakefile
The patch allows mod_jk to build for NetWare with APR existing outside of the httpd/srclib file structure for apache 2.1.x. Could someonecommitthat change tocvs. Thank you, Jean-Jacques --- jk/native/apache-2.0/NWGNUmakefile9 Apr 2004 20:00:18 -1.2+++ jk/native/apache-2.0/NWGNUmakefile1 Dec 2004 22:33:58 -@@ -39,9 +39,9 @@$(AP_WORK)/include \$(NWOS) \$(AP_WORK)/modules/arch/netware \-$(AP_WORK)/srclib/apr/include \-$(AP_WORK)/srclib/apr-util/include \-$(AP_WORK)/srclib/apr \+$(APR)/include \+$(APRUTIL)/include \+$(APR) \$(JAVA_HOME)/include \$(JAVA_HOME)/include/netware \$(EOLIST) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
