I was asked in a private email how I got an enterprise application to run on the system class loader as WAR/EAR-packaging (currently) defeats the module system at runtime. Therefore, I wanted to quickly add the following advice if anybody wants to try out their own application:
I had to tweak the application a bit; it is typically bundled in a "war" file but all we do this for is to deploy upon the servlet API. What I therefore did was to skip this last bundling step of our build. Instead, I wrote a small starter module which runs Jetty programmatically while explicitly registering the application's servlets. This way, I could use modules on the system class loader rather than including any dependencies in the war file. 2016-09-30 14:35 GMT+02:00 Rafael Winterhalter <rafael....@gmail.com>: > I have tested the latest EA builds, both on my open-source projects and in > several production applications, one of them quite large with thousands of > development hours invested. I wanted to share my experiences. > > After making files with the .class extension visible, my agent/code > generation library Byte Buddy works more or less as it did before. > Previously, I had to apply a work-around where I mapped package names of > all modules to their Module instance what allowed me read class files via > this instance. I discovered all modules via Layer.boot() but was of course > missing support for modules of other layers. This approach also hurt > runtime performance. With the newest EA build, out of 7000 unit tests, only > 42 tests still fail and those are all because of using non-exported API of > the JCL. None of those usages are essential to using the library such that > I can live with this minor limitation. (I did not yet look to adding a file > in the META-INF folder which I read somewhere was possible.) I can also > confirm that using the latest EA build, cglib and Javassist work again. > This is great news as this breaking change really concerned me with regards > to Java 9 compatibility. Code generation libraries are used everywhere and > breaking them would have had an impact on any user of the JVM. As I > currently support Java 6, I do not plan to apply modules on my OSS in the > near future. This is of course not great as plain jars cannot be packaged > into jdeps but I hope to include a generated module-info.class at some > point even in a Java 6 jar in order to allow doing this. > > I also tried to migrate a small and a large production application. The > larger applications is currently running on a Java 8 VM and code level but > started out in the days of Java 4. As the project is already organized in > Maven modules, approaching the migration was straight-forward. Creating the > first module already helped to uncover a minor breach of the intended > module boundaries that was now discovered by the Java 9 compiler. The > modularization did however not pay off too much due to the previous > modularization where the Maven modularization already offered a good > approximation. Alltogether, my experience was however quite positive. One > minor problem was that many times, several Maven modules declared the same > packages to communicate "cross-module shared classes" via package private > types. This required some refactoring of classes and splitting up of some > modules. > > Getting the application to run was however not as easy as the adoption of > compile-time modules. As mentioned, the application has grown quite big > over the years and has many dependencies without anybody having a complete > overview of what is used and for what. In this sense, the application is > not polished but runs quite robust in production. (In the end, I think the > application is quite representative of many commercial applications out in > the wild.) Using Java 9 but without applying modules, the application did > start but sometimes fails at runtime due to libraries and code accessing > non-exported classes of the JCL. Many of these breaches happen high up the > stack in library and framework code. For now, I have therefore stripped > down the application to avoid using those application parts which I do not > know intimately enough to fix right now. > > After deploying the application with some first modules on the module > path, I encountered similar problems. Adding correct exports to cover the > use of reflection is unfortunately non-trivial. My IDE lists over 420 > usages of Method::invoke in the project and its libraries what makes it > practically impossible to trace all possible reflective access. Generally, > I do neither want to depend too much on such implementation details as it > makes updating dependencies quite tedious as I have to revalidate the > access patterns. My major problem with adding additional modules onto the > module path is that each breaks code that is still on the class path. > Step-by-step migration is unfortunately impossible as I always need to > consider all of our code and the library code. There is (obviously) no way > to check at compile time if a module is accessed refletively from the class > path what only leaves runtime. This is also true for non-reflective access > if I do not recompile all code on the class path when transforming a > module. This turned out to be quite tedious, especially, since some > problems only occur when triggering a particular subroutine of the > application. Unit tests do not offer a good alternative either as I cannot > really separate what tests are supposed to have access to the module > internals and which should not. The application has thousands of tests, > some of them not making much sense to me, going through them would be > economically impractical. Integration tests sometimes uncover those access > problems but unfortunately often disguise them in the error reports. > > To be fair, I think that I would get the application going if I invested > more time to trial and error. However, I still find the migration process > too difficult as it requires a lot of manual runs to validate. I think it > would be helpful if code on the class path would behave exactly as before > to allow for a better step-by-step migration, i.e. could access modularized > code reflectively without any constraints. Right now, it feels easiest to > export about everything before I add something to the module path. > Otherwise, it does not feel safe to apply the migration when I think about > putting the application into production: even when the application appears > to be running, errors can always occur later when a specific unit of code > is triggered. I am mostly scared when thinking of rarely triggered routines > like security failovers or error reporting mechanisms which are often > implemented quite generically (reads: use a lot of reflection). > > I currently experiment with a Java agent to force Java 9 to adopt this > desired behavior by manipulating the boot layer's module graph or by > stubbing AccessibleObject::checkCanSetAccessible. I hope to trial this a > bit broader soon but it seems promising to allow us a "safe migration" to > Java 9. I would of course prefer to remove this agent at some point in time > but I would not know when or if all of our dependencies are "Java 9 ready". > (Some of them have not been updated for years.) I am also doing this as a > means of delegating migration to developers with less Java experience as > migrating to Java 9 turns out to be quite a demanding task. To be honest, > many junior developers do not even know what reflection is and it would be > difficult to explain to them how they need to modularize the code they work > with. As many libraries (or our own modules) do not document their > implementation but only their API, the only way to figure out the module > requirements is by reading its source code. For framework code, this often > outchallanges less experienced developers. > > Finally, I already identified some Maven modules which would be difficult > to Java-9-modularize as they basically only work with reflection and are > used throughout many parts of the application. It feels a bit strange to > export packages from the users of this module as those "reflection modules" > are only used via several indirections. I would prefer it if I could > declare the module to be a form of "transitive module" which is simply > allowed to reflect as it was allowed previously. I understand that security > might be an issue here but why not simply ask a potentially present > security manager if such a transitive module is allowed to be used? > > My conclusion is that I wished that migration would be easier to do > step-by-step. This way I can better spread the costs of adoption. > Currently, I think it would be easiest to not migrate to modules as the > benefits are limited in the context of existing Maven modularization. Right > now, any introduced Java module requires a lot of additional runtime > testing and I doubt that my customers would be willing to pay for it. With > a greenfield project, I would always try to adopt modules as they push > people into the right direction but as long as the library landscape is not > modularized, this again bears a high risk of runtime errors if those > libraries are not module aware. As many libraries are careful about > updating to not alienate legacy VM users, I imagine that this might take a > while. As for "hard updates", I still think a "force-exporting" Java agent > is a solution that I consider, simply to run on a Java 9 VM without having > to change or validate any code. There are a lot of great features that I > hope to take advantage of, besides Jigsaw. > > Even though I know this idea is not popular, my recommendation is to have > the class path to behave just as it did before, i.e. reflection can cross > module boundaries without preparing those modules. This especially in the > context of the impossibility of putting the JCL onto the class path. > > In my trial runs, I did not identify any problems with > #ResourceEncapsulation as all of our custom "cross-module files" were put > into the base folder, i.e. the unnamed module where they remain available. > I do however not know how this would affect other applications that put > resources into packages. > > So much for my experience report. Altogether, I really think Java 9 has a > lot to offer and I really appreciate Oracle is not rushing this but takes > the time it takes to getting this right. Great work, also beyond Jigsaw > there is much I am looking forward too. > > Best regards, Rafael > > PS: I noticed that when running java (without commands), it still lists > the non-GNU-style options that are no longer supported. >
