Bugs item #840496, was opened at 2003-11-11 23:15
Message generated for change (Comment added) made by dward2
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=376685&aid=840496&group_id=22866
Category: JBossWeb
Group: v3.0 Rabbit Hole
Status: Open
Resolution: None
Priority: 5
Submitted By: David Ward (dward2)
Assigned to: Scott M Stark (starksm)
Summary: Cocoon war dir hot-redeploy causes JBoss Shutdown
Initial Comment:
I've discovered a problem that only happens with jboss
3.0 + tomcat + cocoon 2.0 + expanded war directory,
relating to hot-redeploy. Basically, if you have a
cocoon.war/ *directory* web application, and touch it's
web.xml file, JBoss shuts down! Here are the steps to
re-create:
1) Download a completely fresh
jboss-3.0.7_tomcat-4.1.24 from sourceforge.
2) Download a completely fresh cocoon-2.0.4 (vm14) from
apache.
3) Dropped cocoon.war in jboss' deploy directory.
4) Started up jboss.
5) Hit the cocoon servlet (got a sitemap error but
didn't care)
6) "touch" cocoon.war.
7) NO REDEPLOY PROBLEM
8) Hit the cocoon servlet (got same sitemap error but
still didn't care - at least could still hit it).
9) Stop jboss.
10) Delete the db, tmp and log dirs to be completely
clean distro again.
11) Explode the cocoon.war file into cocoon.war/
*DIRECTORY INSTEAD*.
12) Start up jboss.
13) Hit the cocoon servlet (sitemap error; still don't
care)
14) "touch" cocoon.war/WEB-INF/web.xml
15) *** REDEPLOY PROBLEM: jboss shuts down!!!!!! ***
JBoss/Jetty does not do this. JBoss 3.2.x does not do
this. A JBoss/Tomcat webapp without Cocoon does not do
this. JBoss/Tomcat with a Cocoon war *file* does not
do this. It has to be JBoss 3.0/Tomcat with a Cocoon
2.0 war *directory*, and you touch it's web.xml file to
cause a redeploy. I have tested it on many different
combinations, and here is the breakdown:
1) jboss-3.0.7_tomcat-4.1.24 + cocoon-2.0.4 - FAILED
2) jboss-3.0.8_tomcat-4.1.24 + cocoon-2.0.4 - FAILED
3) jboss-3.0.8_tomcat-4.1.27 + cocoon-2.0.4 - FAILED
4) jboss-3.0.8_tomcat-4.1.29 + cocoon-2.0.4 - FAILED
5) jboss-3.2.2 (tomcat_4.1.27) + cocoon-2.0.4 - PASSED
6) jboss-3.0.8_tomcat-4.1.24 + cocoon-2.1.2 - PASSED,
though 500 errors
Something was fixed in JBoss 3.2.2 / Tomcat that fixed
this problem. Does anyone have any idea what this was?
Unfortunately, we cannot upgrade to JBoss 3.2.2 right
now. If the fix is known, can it be backported back to
3.0? A jboss-3.0.9_tomcat-4.1.29 with the backported
fix would be awesome and greatly appreciated!
BTW, with JBoss 3.2.2, it looks like the shutdown was
intercepted; here's a line from the console:
22:43:58,980 INFO [STDOUT] Mon Nov 03 22:43:58 EST
2003 SHUTDOWN : System.exit() was not called
When I see that line with 3.2.2, the redeploy finishes
and JBoss does not shutdown. Maybe find where that log
is done and see what can be similarly done in 3.0.x to
intercept the shutdown?
Last bit of info: This is using Sun JDK 1.4.1 and
1.4.2. It is recreateable on Windows; not sure about
Linux and MacOSX.
Thanks!
David
----------------------------------------------------------------------
>Comment By: David Ward (dward2)
Date: 2003-11-30 15:11
Message:
Logged In: YES
user_id=526282
Okay. Gotten around the problem, and I also found I was
wrong about something. Even though cocoondb.properties says
version=1.6, the hsqldb jar file included in cocoon 2.0.4 is
hsqldb-1.7.1.jar. There's no version on the one included
with jboss; I'm guessing that one's 1.6.x? Anyway, our
cocoon webapp doesn't need hsqldb, so I removed it and the
shutdown problem goes away. Specifically, I searched for
all occurrences for hsql in these files:
deploy/cocoon.war/WEB-INF/web.xml (1 place)
deploy/cocoon.war/WEB-INF/cocoon.xconf (2 places)
and commented out anything to do with hsqldb.
I then removed this file:
deploy/cocoon.war/WEB-INF/lib/hsqldb-1.7.1.jar
I turns out that the jar being there doesn't *seem* to hurt
anything, but I still removed it because 1) will make the
war smaller, and 2) I just wanted to be safe concerning
classloading if indeed jboss' version of hsqldb is different.
So, it looks like we can get around this problem by getting
rid of hsqldb in cocoon (good thing we didn't need it
there). So, I guess this bug isn't as high priority now as
it used to be. However, I'm still curious about the
cocoon.war file vs. dir thing...
Thanks,
David
----------------------------------------------------------------------
Comment By: David Ward (dward2)
Date: 2003-11-29 16:37
Message:
Logged In: YES
user_id=526282
Another note - I see in jboss-3.2.2, in
server/default/deploy/hsqldb-ds.xml, that there is a
*commented out* mbean block for
org.jboss.jdbc.HypersonicDatabase with <attribute
name="No_system_exit">true</attribute>. I wonder how this
is working, even though it's commented out... The fact that
it's there sounds to me that jboss-3.2.2 uses hsqldb 1.7.x.
A-ha! Looking at the jboss 3.2.2 source,
org.jboss.jdbc.HypersonicDatabase has a no_system_exit
property that *defaults* to true! It has a javadoc comment
over it that says "New embedded support in 1.7".
Soooo... I'm guessing the only way to backport this fix to
jboss-3.0.x is to also upgrade hsqldb to 1.7.x? Is that
feasible?
Still, still - I don't understand why the shutdown on
redeploy only happens when cocoon.war is in expanded form.
Scott, now that you know who's calling the exit, can you
make a guess at what the difference might be (expanded dir
vs. not)? Maybe to patch this for jboss 3.0.x we don't need
the newer hsqldb, as long as we can figure why jboss is
handling the expanded cocoon deployment differently...
Could it be some classloading difference, maybe compounded
with the confusion that cocoon.war also contains it's own
copy of hsqldb?
Thanks,
David
----------------------------------------------------------------------
Comment By: David Ward (dward2)
Date: 2003-11-29 16:17
Message:
Logged In: YES
user_id=526282
Okay - found the problem. ServerConnection is an hsqldb
class, org.hsqldb.ServerConnection. In hsqldb 1.61, there
is a call to System.exit(0); in ServerConnection, just under
a System.out.println("The database is shutdown"). If you
look at the server.log I had previously attached, you will
see these lines:
2003-11-29 12:07:29,799 INFO
[org.jboss.deployment.MainDeployer] Starting deployment of
package:
file:/C:/Temp/jboss-3.0.8_tomcat-4.1.24/server/default/deploy/
cocoon.war/^M
2003-11-29 12:07:29,940 INFO [STDOUT] The database is
shutdown^M
2003-11-29 12:07:29,940 INFO [STDOUT] The database is
shutdown^M
2003-11-29 12:07:29,950 INFO
[org.jboss.system.server.Server] Undeploying all packages^M
2003-11-29 12:07:29,950 INFO
[org.jboss.deployment.MainDeployer] Undeploying
file:/C:/Temp/jboss-3.0.8_tomcat-4.1.24/server/default/deploy/jmx-console.war/^M
I'm pretty sure it's hsqldb that's shutting down the server.
It looks like there's a fix in hsqldb 1.7.0 that fixes this:
//[EMAIL PROTECTED] 20020424 - patch 1.7.0 by fredt - shutdown
without exit
Looking at the source for 1.7.1, there is no more
System.exit(0) called in ServerConnection.java. There is in
Server.java (same package), but only if it is supposed to.
Here's a noe from the class javadoc:
"If the server is embedded in an application server, such as
when
DataSource or HsqlServerFactory classes are used, it is
necessary
to avoid calling System.exit() when the HSQLDB is shutdown with
an SQL command.<br>
For this, the server.no_system_exit property can be
set either on the command line or in server.properties file.
This ensures that System.exit() is not called at the end.
All that is left for the embedder application server is to
release
the "empty" Server object and create another one to reopen the
database ([EMAIL PROTECTED])."
It looks like there are two hsqldb properties files setup,
one for jboss 3.0.8 and one for cocoon 2.0.4, each saying
they use hsqldb "version=1.6":
server/default/db/hypersonic/default.properties
server/default/deploy/cocoon.war/WEB-INF/db/cocoondb.properties
I bet jboss 3.2.2 uses 1.7?
What confuses me still is why does this system shutdown only
happen when cocoon.war is deployed as a directory, and not
when it is deployed as a file?!?!
----------------------------------------------------------------------
Comment By: David Ward (dward2)
Date: 2003-11-29 13:19
Message:
Logged In: YES
user_id=526282
Hi again Scott,
I am now running the same jboss/java/os version as described
in my last comment (where I referenced server.log), except I
launched it from Ecplipse 2.1.1 using JBoss-IDE 1.2.2. I
set a breakpoint in java.lang.System.exit(int) as you
suggested, halting the VM when it reached the only line in
that method, Runtime.getRuntime().exit(int). On the Debug
view tab, I see this:
Thread [Thread-29] (Suspended (breakpoint at line 715 in
System))
System.exit(int) line: 715
ServerConnection.run() line: 146
I'm assuming "ServerConnection.run() line: 146" is JBoss
source, as Eclipse can't find the source. Does this help
you debug the problem? Is it enough, or should I download
the jboss source for the version I'm using and get a more
detailed trace?
Thanks yet again,
David
----------------------------------------------------------------------
Comment By: David Ward (dward2)
Date: 2003-11-29 12:22
Message:
Logged In: YES
user_id=526282
Hi Scott,
I will try that next. I do have more helpful information,
though. Using the "jboss-3.0.8_tomcat-4.1.24 +
cocoon-2.0.4" configuration on WinXP-Pro with Sun JDK
1.4.2_02, I changed cocoon.war/WEB-INF/web.xml to use the
sevlet 2.3 spec instead of 2.2. I could then add a custom
TestServletContextListener with the listener-class element.
I put a printStackTrace() in the contextDestroyed method.
When I touch web.xml, I see it (contextDestroyed) being
called twice. The first time was triggered by
AbstractDeploymentScanner$ScannerTrhead.run() - which I
would expect because I "touch"ed web.xml. However the
second time is the one I didn't want to happen, and that was
triggered by ServerImpl$ShutdownHook.run(). I am attaching
a server.log file. Search for the text "contextDestroyed
called" to find the 2 stack traces. Again, the first is
expected, the second shouldn't have happened.
Thanks again,
David
----------------------------------------------------------------------
Comment By: Scott M Stark (starksm)
Date: 2003-11-12 19:13
Message:
Logged In: YES
user_id=175228
Put the server in a debugger and set a break point on the
System.exit call and show the stack trace to see who is
calling it.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=376685&aid=840496&group_id=22866
-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive? Does it
help you create better code? SHARE THE LOVE, and help us help
YOU! Click Here: http://sourceforge.net/donate/
_______________________________________________
JBoss-Development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
