Hi,
Firstly, this is actually a continuation of a previous thread
(specifically the email from Make Thomas, dated 21/02/2022 08:10)
discussing the specification for enhancing the reloading of webapps
after changes to the deployed files. This thread is titled ‘<Context …
reloadable = "true"> is too quick to respond’ and remains active with
outstanding discussion points. However, in order to try and implement
the suggested changes, I needed to create a development environment
and so the thread quickly shifted into a bunch of configuration issues
which took the thread off topic.
Secondly, this email is quite long as I wanted to describe the steps I
took to get to where I am now (briefly, nearly there, just can't fully
test, unless all of them!). I have also included some suggestions for
alterations to the NetBeans configuration files that I am happy to
commit back to git if my understanding is correct./ deductions are
correct although one of them appears to be an issue with the ant build
and I am not confident enough to research this.
There are a couple of attachments that, as before, I have placed in my
shared Dropbox folder, accessed from
https://www.dropbox.com/sh/2ewipogzr48qcxi/AAAf3Rqv6WoRO9hyMC0W7P2za?dl=0
Mark, your insightful email has proved invaluable and having spent a
couple of days working through this, it has provided the catalyst for
me to be able to start joining the dots to achieving a full
development environment.
So, responding to each of your suggestions has taken me on a massive
learning curve, but....
Firstly, I started back with a blank piece of paper.
If your eventual aim is to provide a pull request to address this issue then
I'd recommend:
- Fork the Tomcat project in GitHub
- Checkout your clone
As a life-long supporter of Subversion, this was my first foray into
the world of git, but I believe, after a quick crash course, I have
managed to have forked and cloned Tomcat onto my laptop!
This is where using a clone would speed things up as you'd be able to fetch the
update from Git.
Therefore, you can now assume that I can speed things up :)
I have built Tomcat using Ant.
Ran the amended 'ant ide-netbeans' script (checked GitHub to see what
changes had been made - the details of this change log request turned
out to be very useful later!)
Hmm. You shouldn't see any warnings about modules. A handful of warnings about
deprecation are expected. If you still see these warnings then yes, please
provide a copy so we can take a look.
Launched NetBeans and did a clean > deploy. The results are in my
shared DropBox folder (along with everything else) called "Ant Clean
and Build from Netbeans.txt" and it did actually look to have done a
successful build if you believe the summary line! While I did get
warnings about deprecated features as expected, I also had quite a
number of other warnings that looked a bit concerning (which you asked
me to submit to you)
e.g. - WARN aQute.bnd.plugin.jpms.JPMSModuleInfoPlugin - Can't find a
module name for imported package: jakarta.websocket
At this point, I still had a lot of red [!] against the Tests folder,
just as before and so the version amendment for jUnit didn't appear to
work, although I verified that it is the latest jUnit v4 build from
Mavens repository. As an aside, as I am sure you are aware, NetBeans
didn't support any later versions of jUnit using the Ant framework as
they believe that no-one should be using Ant anymore and should
migrate to Maven / Gradle!
So, rather than push back immediately, I thought I would spend some
time yesterday trying to figure it out, given that you had (by your
change) hinted which of the files might be related to my issue. Also,
your reference to tomcat\res\ide-support\netbeans\README.txt was
enlightening!
Anyway, to cut a long investigation short, I found two environments
that were out of sync.
As far as I can tell, in nbproject\project.xml, the classpath just
underneath the <unit-tests/> element
<classpath
mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest/hamcrest-2.2.jar</classpath>
should look more like
<classpath
mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest-2.2/hamcrest-2.2.jar:${base.path}/unboundid-ldapsdk-6.0.3/unboundid-ldapsdk-6.0.3.jar</classpath>
There was a typo in the hamcrest folder name (missing version) and
some of the classes the test sources referred to were missing. I used
the Maven repository to identify the latest versions and downloaded
them.
The reason that the jUnit classes weren't found is that they weren't
being downloaded into the {base.path} folder as part of the 'ant'
script, used to build Tomcat, but looks like only the jars needed to
build the project were (but that's an assumption of mine). I do not
know enough about Ant to try and determine why this may be.
Also, while looking here, should the <source-level> be at least
updated to '11' as '1.7' gives a bucketload of exceptions when loaded
into NetBeans?
So, my folder structure now looks like below, with jUnit, easyMock,
hamcrest, odjenesis, unboundid having being downloaded manually. I may
have missed one out of that list as I forgot to take a directory
listing prior to adding the additional folders to compare with.
C:\Community\DownloadedTomcatLibraries\bnd-6.1.0
C:\Community\DownloadedTomcatLibraries\cglib-3.3.0
C:\Community\DownloadedTomcatLibraries\commons-daemon-1.2.4
C:\Community\DownloadedTomcatLibraries\derby-10.15.2.0
C:\Community\DownloadedTomcatLibraries\easymock-4.3
C:\Community\DownloadedTomcatLibraries\ecj-4.22
C:\Community\DownloadedTomcatLibraries\hamcrest-2.2
C:\Community\DownloadedTomcatLibraries\jacoco-0.8.7
C:\Community\DownloadedTomcatLibraries\jaxrpc-1.1-rc4
C:\Community\DownloadedTomcatLibraries\junit-4.13.2
C:\Community\DownloadedTomcatLibraries\migration-1.0.0
C:\Community\DownloadedTomcatLibraries\objenesis-3.2
C:\Community\DownloadedTomcatLibraries\osgi-annotations-1.1.1
C:\Community\DownloadedTomcatLibraries\unboundid-6.0.3
C:\Community\DownloadedTomcatLibraries\unboundid-ldapsdk-6.0.3
C:\Community\DownloadedTomcatLibraries\wsdl4j-1.6.3
C:\Community\DownloadedTomcatLibraries\bnd-6.1.0\biz.aQute.bnd-6.1.0.jar
C:\Community\DownloadedTomcatLibraries\cglib-3.3.0\cglib-nodep-3.3.0.jar
I still have a handful of red ! against some of the test files.
org.apache.catalina.ant.TestDeployTask
- I have a red ! flag against this file, but the source file looks
to be fine (no markers against any of the lines to backup the red !
against the file). I will ask the Netbeans community if I can find out
what that means!
org.apache.coyote.http2.TestStream
- package trailers does not exist - import trailers.ResponseTrailers;
util.TestCookieFilter
- cannot find symbol CookieFilter in (e.g., multiple instances)
Assert.assertEquals("a=b", CookieFilter.filter("a=b", null));
(e.g.) webapp/bug36923.jsp, but there are quite a few similar.
- This may be Netbeans validating against specific HTML versions. I
have taken 2 screenshots (prefixed JSP Exception) of the exception and
proposed fix but not done anything as not sure whether these are
historic or used as part of the test framework. It would be nice to
start with a clean sheet though, more in case others later come on
board and get distracted by exceptions that aren't important.
Apart from those remaining exceptions, the tests folder looks to be
acceptable to NetBeans.
I then ran 'ant test' and encouragingly, all the tests appeared to pass!!!!
Following instructions in points 7 & 8 of the
tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test)
From NetBeans, I then went to see if I could run an individual test
and came unstuck. OK, I picked a test at random that looked
interesting so I tried
org.apache.catalina.servlets.ServletOptionsBaseTest. The log (Test
output for org.apache.catalina.servlets.ServletOptionsBaseTest.txt)
reported an Error and so FAILED the test. I have uploaded the log to
DropBox.
I also tried adding a breakpoint on the first line (as described in
point 8 of the document) and got the same response and the breakpoint
wasn't triggered.
Using details in the BUILDING.txt, I tried implementing the
instructions for a running a single test from the CMD using 'ant
test', but first, amending the build.properties file and adding the
line
test.entry=org.apache.catalina.util.TestServerInfo
which is the one documented in BUILDING.txt.
That failed as well, with the following (similar to NetBeans) error
Testsuite: org.apache.catalina.servlets.ServletOptionsBaseTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.128 sec
------------- Standard Error -----------------
java.util.logging.ErrorManager: 0: Unable to delete log files older
than [90] days
java.util.logging.ErrorManager: 0: Unable to delete log files older
than [90] days
java.util.logging.ErrorManager: 0: Unable to delete log files older
than [90] days
java.util.logging.ErrorManager: 0: Unable to delete log files older
than [90] days
------------- ---------------- ---------------
Testcase: testOptions took 0.002 sec
Caused an ERROR
null
java.lang.InstantiationException
at
java.base/jdk.internal.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
So it may be the specific test I picked (at random) that only makes
sense when run as part of a suite of tests so any help would be
appreciated.
I also tried the simple test case detailed in BUILDING.txt
(test.entry=org.apache.catalina.util.TestServerInfo) and this worked
both in NetBeans and Ant standalone, as well as adding a breakpoint
and so that adds weight to the thought that I picked a test that is
not meant to be run in isolation.
The simplest way to test the build is to try the following:
cd output\build\bin
catalina.bat run
I then tried to launch Tomcat from a CMD line (again, thanks for the
'obvious' starting script but these hints really help so that I know
that taking the right path)
The first time I attempted it, it appeared to work beautifully, until
I realised that it had actually launched my production copy of Tomcat
as I could see the listeners that I have setup reporting that they had
been triggered successfully! So I amended the environment variable
CATALINA_HOME = C:\Community\tomcat (the location of the webapps
directory) and tried again and came to an abrupt halt.
The output was as follows
C:\Community\tomcat\output\build\bin>catalina.bat run
Using CATALINA_BASE: "C:\Community\tomcat"
Using CATALINA_HOME: "C:\Community\tomcat"
Using CATALINA_TMPDIR: "C:\Community\tomcat\temp"
Using JRE_HOME: "C:\Development\JDK\jdk-17"
Using CLASSPATH:
"C:\Community\tomcat\bin\bootstrap.jar;C:\Community\tomcat\bin\tomcat-juli.jar"
Using CATALINA_OPTS: ""
NOTE: Picked up JDK_JAVA_OPTIONS:
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
Error: Could not find or load main class org.apache.catalina.startup.Bootstrap
Caused by: java.lang.ClassNotFoundException:
org.apache.catalina.startup.Bootstrap
This brings me to a point I noticed earlier - that there are two 'bin'
folders - C:\Community\tomcat\bin &
C:\Community\tomcat\output\build\bin and the first didn't have
bootstrap.jar within it, and from the date stamps are 21/02/2022 at
11:23 and no jars, and so this may be an outline folder, also, while
the dates are yesterday, that may be around the time that I cloned
Tomcat onto my laptop although I would have thought that the date/time
would represent when they were last amended rather than when they were
copied to me (as I would expect from Subversion).
So I amended the CATALINA_HOME = C:\Community\tomcat\output\build
C:\Community\tomcat\output\build\bin>catalina.bat run
Using CATALINA_BASE: "C:\Community\tomcat\output\build"
Using CATALINA_HOME: "C:\Community\tomcat\output\build"
Using CATALINA_TMPDIR: "C:\Community\tomcat\output\build\temp"
Using JRE_HOME: "C:\Development\JDK\jdk-17"
Using CLASSPATH:
"C:\Community\tomcat\output\build\bin\bootstrap.jar;C:\Community\tomcat\output\build\bin\tomcat-juli.jar"
Using CATALINA_OPTS: ""
and it launched!!!
I checked http://localhost:8080/examples and that all looks OK
So, out of interest, why are there two bin folders?
I believe the way most Tomcat committers work is running Tomcat on the command
line and remote debugging from their IDE of choice.
Following instructions in points 9 of the
tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test)
In short, I was successful in this. Again, I had never done remote
debugging before! I needed to change the port to 8080 everywhere but
that was about it.
I fudged a few parameters in catalina.bat and tweaked the source of
org.apache.catalina.util.ServerInfo (added a console keyboard read to
pause the routine) in the first instance as I could test the remote
debugging on a simple example. Netbeans initially claimed that the
source with the breakpoint in wasn't from tomcat\java (it was) but
mysteriously, that message went away and got the "lineBreakpoint
ServerInfo:117 successfully submitted" in the status bar and all was
well.
Finally (you will be glad to know), I repeated the exercise, setting
the debugger to break at WebappLoaded.backgroundProcess(), as advised
and that worked a treat. This Java Remote Debugging is pretty
impressive!
Tomcat 10.1.x (the current main branch) requires Java 11 as a minimum. You
should be fine setting the source level to anything 11 or above. Personally,
I'd set it to 11 as that way you avoid the (very low) risk of proposing a fix
that doesn't work on Java 11.
If I set the build version to 11, do I need to install JDK 11 as well
and update my 'env' JAVA_HOME, or is the Java compiler clever enough
to limit the byte code generation to that which will run on Java 11
while still allowing me to have JDK 17 as my environment. I think the
answer is Yes, but would be nice to have confirmation.
Tell NetBeans to ignore that folder. It should not be part of the source you
are trying to compile. It is part of a (rather old) tutorial on building a web
application.
To do this, I amended the project.xml in nbproject to remove the three
references to sample. I also removed the single reference to
'tomcat-lite'. The folder wasn't cloned from git and I could't find
any reference to a 'lite' version of Tomcat and so I was wondering if
this is an historic / abandoned project. Should the master
\tomcat\res\ide-support\netbeans\project.xml file be edited to remove
these permanently (again to provide a clean experience for others).
In summary, I am assuming from the lack of response from my request
about NetBeans, that this is not a favourite environment for Tomcat
development. It may just have been a long time since anyone deployed
the development framework into NetBeans and so my efforts might be
worthwhile as a refresher check on the current documentation. I am
keeping notes of the process I have been through so far as well as how
to resolve issues I have encountered and so will look to update the
README for NetBeans to help future developers.
John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
