On 22/02/2022 17:59, John Barrow wrote:
John,
<snip/>
Thanks for separating this out into a new thread.
<snip/>
As a life-long supporter of Subversion, this was my first foray into
the world of git, but I believe, after a quick crash course, I have
managed to have forked and cloned Tomcat onto my laptop!
This is where using a clone would speed things up as you'd be able to fetch the
update from Git.
Therefore, you can now assume that I can speed things up :)
I have built Tomcat using Ant.
Great. If you can do this, you are at least 90% of the way there.
<snip/>
While I did get
warnings about deprecated features as expected, I also had quite a
number of other warnings that looked a bit concerning (which you asked
me to submit to you)
e.g. - WARN aQute.bnd.plugin.jpms.JPMSModuleInfoPlugin - Can't find a
module name for imported package: jakarta.websocket
Ah. Yes, those. You can ignore those too. Although thanks for checking
as looking at your logs I spotted a typo which I've now fixed.
Anyway, to cut a long investigation short, I found two environments
that were out of sync.
As far as I can tell, in nbproject\project.xml, the classpath just
underneath the <unit-tests/> element
<classpath
mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest/hamcrest-2.2.jar</classpath>
should look more like
<classpath
mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest-2.2/hamcrest-2.2.jar:${base.path}/unboundid-ldapsdk-6.0.3/unboundid-ldapsdk-6.0.3.jar</classpath>
Agreed.
There was a typo in the hamcrest folder name (missing version) and
some of the classes the test sources referred to were missing. I used
the Maven repository to identify the latest versions and downloaded
them.
The reason that the jUnit classes weren't found is that they weren't
being downloaded into the {base.path} folder as part of the 'ant'
script, used to build Tomcat, but looks like only the jars needed to
build the project were (but that's an assumption of mine). I do not
know enough about Ant to try and determine why this may be.
Ant only downloads those files if you try and run the tests via Ant. You
can trigger those downloads using:
ant download-test-compile
This might also be of interest
ant -projecthelp
Only documented targets are listed and I am fairly sure there are a few
targets that need to be documented so they appear in that list.
Also, while looking here, should the <source-level> be at least
updated to '11' as '1.7' gives a bucketload of exceptions when loaded
into NetBeans?
Agreed.
So, my folder structure now looks like below, with jUnit, easyMock,
hamcrest, odjenesis, unboundid having being downloaded manually. I may
have missed one out of that list as I forgot to take a directory
listing prior to adding the additional folders to compare with.
C:\Community\DownloadedTomcatLibraries\bnd-6.1.0
C:\Community\DownloadedTomcatLibraries\cglib-3.3.0
C:\Community\DownloadedTomcatLibraries\commons-daemon-1.2.4
C:\Community\DownloadedTomcatLibraries\derby-10.15.2.0
C:\Community\DownloadedTomcatLibraries\easymock-4.3
C:\Community\DownloadedTomcatLibraries\ecj-4.22
C:\Community\DownloadedTomcatLibraries\hamcrest-2.2
C:\Community\DownloadedTomcatLibraries\jacoco-0.8.7
C:\Community\DownloadedTomcatLibraries\jaxrpc-1.1-rc4
C:\Community\DownloadedTomcatLibraries\junit-4.13.2
C:\Community\DownloadedTomcatLibraries\migration-1.0.0
C:\Community\DownloadedTomcatLibraries\objenesis-3.2
C:\Community\DownloadedTomcatLibraries\osgi-annotations-1.1.1
C:\Community\DownloadedTomcatLibraries\unboundid-6.0.3
C:\Community\DownloadedTomcatLibraries\unboundid-ldapsdk-6.0.3
C:\Community\DownloadedTomcatLibraries\wsdl4j-1.6.3
C:\Community\DownloadedTomcatLibraries\bnd-6.1.0\biz.aQute.bnd-6.1.0.jar
C:\Community\DownloadedTomcatLibraries\cglib-3.3.0\cglib-nodep-3.3.0.jar
I noticed Checkstyle is missing.
ant download-validate
If you submit a patch it is helpful if you configure and run Checkstyle
(see section 8 of BUILDING.txt for details)
If you do a release build you'll end up downloading at least one
additional library - the NSIS installer but unless you want to tweak the
Windows installer I doubt you'll need a release build.
I still have a handful of red ! against some of the test files.
org.apache.catalina.ant.TestDeployTask
- I have a red ! flag against this file, but the source file looks
to be fine (no markers against any of the lines to backup the red !
against the file). I will ask the Netbeans community if I can find out
what that means!
Odd. Eclipse does that sometimes. Sometimes opening and closing the file
fixes it. Sometimes I just delete the warning and it doesn't come back.
Don't know if that is an option in NetBeans.
org.apache.coyote.http2.TestStream
- package trailers does not exist - import trailers.ResponseTrailers;
util.TestCookieFilter
- cannot find symbol CookieFilter in (e.g., multiple instances)
Assert.assertEquals("a=b", CookieFilter.filter("a=b", null));
Ah. You need to add webapps/examples/WEB-INF/classes as a source folder.
That should fix the two issues above.
(e.g.) webapp/bug36923.jsp, but there are quite a few similar.
- This may be Netbeans validating against specific HTML versions. I
have taken 2 screenshots (prefixed JSP Exception) of the exception and
proposed fix but not done anything as not sure whether these are
historic or used as part of the test framework.
That is a NetBeans bug. The JSP is valid but NetBeans is not taking into
account the isELIgnored="true" page directive.
To be fair to NetBeans, this is a can of worms. JSP Page directives can
be set in both the current page and any included page. To add to the
'fun' you can also specify includes in web.xml. Tomcat parses the page
(and all its includes) twice. Once to find all the directives and once
for everything else, once it knows what directives have been set.
It would be nice to
start with a clean sheet though, more in case others later come on
board and get distracted by exceptions that aren't important.
I agree completely. IDE warnings have often provided me with a hint that
something isn't quite right with a change I have just made. That only
works because the project is warning free to start with.
I'd see if you can disable the JSP validation. If it makes you feel
better, Eclipse's JSP validation has similar issues.
Apart from those remaining exceptions, the tests folder looks to be
acceptable to NetBeans.
I then ran 'ant test' and encouragingly, all the tests appeared to pass!!!!
Excellent.
Following instructions in points 7 & 8 of the
tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test)
From NetBeans, I then went to see if I could run an individual test
and came unstuck. OK, I picked a test at random that looked
interesting so I tried
org.apache.catalina.servlets.ServletOptionsBaseTest. The log (Test
output for org.apache.catalina.servlets.ServletOptionsBaseTest.txt)
reported an Error and so FAILED the test. I have uploaded the log to
DropBox.
That is an abstract base class. You won't be able to run it. You should
be able to run one of the concrete sub-classes such as
org.apache.catalina.servlets.TestDefaultServletOptions
The naming convention is:
- Test* - tests you can run
- Tester* - support class - no tests to run
- Anything else - something you can't run as as test
<snip/>
So I amended the CATALINA_HOME = C:\Community\tomcat\output\build
C:\Community\tomcat\output\build\bin>catalina.bat run
Using CATALINA_BASE: "C:\Community\tomcat\output\build"
Using CATALINA_HOME: "C:\Community\tomcat\output\build"
Using CATALINA_TMPDIR: "C:\Community\tomcat\output\build\temp"
Using JRE_HOME: "C:\Development\JDK\jdk-17"
Using CLASSPATH:
"C:\Community\tomcat\output\build\bin\bootstrap.jar;C:\Community\tomcat\output\build\bin\tomcat-juli.jar"
Using CATALINA_OPTS: ""
and it launched!!!
I checked http://localhost:8080/examples and that all looks OK
So, out of interest, why are there two bin folders?
If we use GIT_CLONE to represent wherever you cloned the Tomcat source
%GIT_CLONE%\bin is the source folder.
When you run
ant
It creates a combined CATALINA_HOME/CATALINA_BASE in
%GIT_CLONE%\output\build
%GIT_CLONE% doesn't have a lib or logs directory either.
In Maven terms, output\build == target
I believe the way most Tomcat committers work is running Tomcat on the command
line and remote debugging from their IDE of choice.
Following instructions in points 9 of the
tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test)
In short, I was successful in this. Again, I had never done remote
debugging before! I needed to change the port to 8080 everywhere but
that was about it.
I think you mean 8000 for remote debugging but otherwise great. If you
can get this working, you are doing really well.
I fudged a few parameters in catalina.bat and tweaked the source of
org.apache.catalina.util.ServerInfo (added a console keyboard read to
pause the routine) in the first instance as I could test the remote
debugging on a simple example. Netbeans initially claimed that the
source with the breakpoint in wasn't from tomcat\java (it was) but
mysteriously, that message went away and got the "lineBreakpoint
ServerInfo:117 successfully submitted" in the status bar and all was
well.
Finally (you will be glad to know), I repeated the exercise, setting
the debugger to break at WebappLoaded.backgroundProcess(), as advised
and that worked a treat. This Java Remote Debugging is pretty
impressive!
It is incredibly useful. I use it all the time.
Tomcat 10.1.x (the current main branch) requires Java 11 as a minimum. You
should be fine setting the source level to anything 11 or above. Personally,
I'd set it to 11 as that way you avoid the (very low) risk of proposing a fix
that doesn't work on Java 11.
If I set the build version to 11, do I need to install JDK 11 as well
and update my 'env' JAVA_HOME, or is the Java compiler clever enough
to limit the byte code generation to that which will run on Java 11
while still allowing me to have JDK 17 as my environment. I think the
answer is Yes, but would be nice to have confirmation.
Yes, the Java compiler is smart enough to generate the byte code as if
it was generated with Java 11 so you are fine to stick with Java 17 as
long as the build version is 11.
Tell NetBeans to ignore that folder. It should not be part of the source you
are trying to compile. It is part of a (rather old) tutorial on building a web
application.
To do this, I amended the project.xml in nbproject to remove the three
references to sample. I also removed the single reference to
'tomcat-lite'. The folder wasn't cloned from git and I could't find
any reference to a 'lite' version of Tomcat and so I was wondering if
this is an historic / abandoned project. Should the master
\tomcat\res\ide-support\netbeans\project.xml file be edited to remove
these permanently (again to provide a clean experience for others).
Yes. tomcat-lite was an early experiment that evolved into the embedded
JARs. It was remove years ago.
In summary, I am assuming from the lack of response from my request
about NetBeans, that this is not a favourite environment for Tomcat
development. It may just have been a long time since anyone deployed
the development framework into NetBeans and so my efforts might be
worthwhile as a refresher check on the current documentation. I am
keeping notes of the process I have been through so far as well as how
to resolve issues I have encountered and so will look to update the
README for NetBeans to help future developers.
Thanks for persevering with this.
It is hard to know exactly why the issues with the NetBeans support
haven't been spotted sooner. It doesn't mean no-one is using NetBeans.
For example, I use Eclipse but never use the equivalent Eclipse scripts.
It looks like you have a bunch of changes that need to be made to the
NetBeans support files. Time to contribute those changes back :)
There are a couple of ways to contribute changes. The main ways are:
- Patch file in diff -u format attached to a BugZilla issue
- GitHub pull request
Happy to provide pointers for either approve if needed.
Looking forward to adding your name to the change log.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
