Think I found the solution, but first want to run all tests in OpenJPA. I did use the standard ClassWriter in ASMHelper. This does use Class.forName. Changed ASMHelper to use the BCClassWriter you wrote some time ago. That should do the trick I hope.
Gimme 10 minutes plz. LieGrue, strub > Am 03.07.2023 um 17:34 schrieb Romain Manni-Bucau <rmannibu...@gmail.com>: > > @Francesco: can you *javap -v -cp $yourclasspath the.class.which.Fail* when > it works vs when it fails please? > > Romain Manni-Bucau > @rmannibucau <https://twitter.com/rmannibucau> | Blog > <https://rmannibucau.metawerx.net/> | Old Blog > <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > <https://www.packtpub.com/application-development/java-ee-8-high-performance> > > > Le lun. 3 juil. 2023 à 17:23, Mark Struberg <strub...@yahoo.de.invalid> a > écrit : > >> Does not yet ring any bells, sorry :( >> >> Right now trying to fire up the enhancer manually in the debugger. >> Difference really is that we now do the ASM part way earlier, but still >> class per class. >> Before that it was done as part of the ASMAdapter. >> >> Still digging. >> >> LieGrue, >> strub >> >> >>> Am 03.07.2023 um 17:00 schrieb Francesco Chicchiriccò < >> ilgro...@apache.org>: >>> >>> Adding some info. >>> >>> First of all, the class to which the error message is referring is not >> always the same. >>> >>> Secondly, at least about AccountPolicy as reported below, the original >> error message is stating >>> >>> Execution enhancer of goal >> org.apache.openjpa:openjpa-maven-plugin:4.0.0-SNAPSHOT:enhance failed: An >> error occurred while enhancing >> org.apache.syncope.core.persistence.jpa.entity.JPARealm. Exception message: >> Type org/apache/syncope/core/persistence/api/entity/policy/AccountPolicy >> not present: >> org.apache.syncope.core.persistence.api.entity.policy.AccountPolicy >>> >>> If you look at JPARealm: >>> >>> >> https://github.com/apache/syncope/blob/master/core/persistence-jpa/src/main/java/org/apache/syncope/core/persistence/jpa/entity/JPARealm.java#L82 >>> >>> it declares >>> >>> @ManyToOne(fetch = FetchType.EAGER) >>> private JPAAccountPolicy accountPolicy; >>> >>> e.g. a reference to the JPA entity JPAAccountPolicy, not to the >> interface that this class implements, e.g. AccountPolicy >>> >>> It looks to me that somewhere the OpenJPA enhancement process is trying >> to resolve more than it should do. >>> >>> Does this make ringing any bell? >>> >>> Regards. >>> >>> On 03/07/23 16:32, Francesco Chicchiriccò wrote: >>>> On 03/07/23 16:27, Mark Struberg wrote: >>>>> Went forward and compiled the Syncope main branch. >>>>> >>>>> Getting this error when running with mvn -X >>>>> >>>>> Caused by: java.lang.ClassNotFoundException: >> org.apache.syncope.core.persistence.api.entity.policy.AccountPolicy >>>>> at >> org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass >> (SelfFirstStrategy.java:50) >>>>> at >> org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass >> (ClassRealm.java:271) >>>>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass >> (ClassRealm.java:247) >>>>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass >> (ClassRealm.java:239) >>>>> at java.lang.Class.forName0 (Native Method) >>>>> at java.lang.Class.forName (Class.java:467) >>>>> at org.apache.xbean.asm9.ClassWriter.getCommonSuperClass >> (ClassWriter.java:1043) >>>>> at org.apache.xbean.asm9.SymbolTable.addMergedType >> (SymbolTable.java:1202) >>>>> at org.apache.xbean.asm9.Frame.merge (Frame.java:1299) >>>>> at org.apache.xbean.asm9.Frame.merge (Frame.java:1244) >>>>> at org.apache.xbean.asm9.MethodWriter.computeAllFrames >> (MethodWriter.java:1611) >>>>> at org.apache.xbean.asm9.MethodWriter.visitMaxs >> (MethodWriter.java:1547) >>>>> at org.apache.xbean.asm9.tree.MethodNode.accept >> (MethodNode.java:767) >>>>> at org.apache.xbean.asm9.tree.MethodNode.accept >> (MethodNode.java:647) >>>>> at org.apache.xbean.asm9.tree.ClassNode.accept >> (ClassNode.java:468) >>>>> at org.apache.openjpa.util.asm.AsmHelper.readIntoBCClass >> (AsmHelper.java:146) >>>>> at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:606) >>>>> at org.apache.openjpa.enhance.PCEnhancer.run >> (PCEnhancer.java:5889) >>>>> at org.apache.openjpa.enhance.PCEnhancer.run >> (PCEnhancer.java:5829) >>>>> at org.apache.openjpa.enhance.PCEnhancer$1.run >> (PCEnhancer.java:5799) >>>>> at org.apache.openjpa.lib.conf.Configurations.launchRunnable >> (Configurations.java:760) >>>>> at >> org.apache.openjpa.lib.conf.Configurations.runAgainstAllAnchors >> (Configurations.java:745) >>>>> >>>>> >>>>> Is this the one you experience too? >>>> >>>> Yes, exactly. >>>> >>>> As you might have found out, AccountPolicy is an interface implemented >> by JPAAccountPolicy, which is the corresponding JPA entity. >>>> We take this approach for all JPA entities, actually. >>>> >>>> I thought the reason why AccountPolicy or JPAAccountPolicy are not >> found is that enhancement does not actually finds them. >>>> >>>>> Will take a look at it now. >>>> >>>> Thank you very much! >>>> >>>> Regards. >>>>>> Am 03.07.2023 um 15:49 schrieb Mark Struberg >> <strub...@yahoo.de.INVALID>: >>>>>> >>>>>> Hi Francesco! >>>>>> >>>>>> Can you see which one of the classes fail to enhance? >>>>>> >>>>>> In which case you might try to run the enhancement of that very class >> manually in your IDE. >>>>>> >>>>>> Main class: org.apache.openjpa.enhance.PCEnhancer >>>>>> VMOptions:: -ea >> -Dopenjpa.ConnectionURL=jdbc:derby:target/database/openjpa-derby-database;create=true >> -Dopenjpa.ConnectionDriverName=org.apache.derby.jdbc.EmbeddedDriver >>>>>> or something similar for your connection >>>>>> >>>>>> Then you also need the parameter pointing to the class file which you >> want to enhance, e.g. >>>>>> >> target/test-classes/org/apache/openjpa/persistence/proxy/entities/Annuity.class >>>>>> >>>>>> Please keep me updated! >>>>>> >>>>>> >>>>>> LieGrue, >>>>>> strub >>>>>> >>>>>> PS: does it add value if we'd write all this up? You will likely know >> most of that stuff, but others who dig into OpenJPA do not. >>>>>> >>>>>> >>>>>> >>>>>>> Am 03.07.2023 um 15:28 schrieb Francesco Chicchiriccò < >> ilgro...@apache.org>: >>>>>>> >>>>>>> Hi Mark, >>>>>>> thanks for your reply. >>>>>>> >>>>>>> So I am proceeding as follows. >>>>>>> >>>>>>> Took the executions at >>>>>>> >>>>>>> https://ci-builds.apache.org/job/OpenJPA/job/OpenJPA-master-deploy/ >>>>>>> >>>>>>> (the Jenkins job which is publishing SNAPSHOT artifacts from master >> to ASF Maven repo) and saw that the artifacts with which the Syncope build >> succeeded last Friday are the one from May 16th, so the corresponding >> commit id is >>>>>>> >>>>>>> b238417dabeac935846fe1ce2fef28aafaeef205 >>>>>>> >>>>>>> I confirm that building locally from that commit id everything is >> working. >>>>>>> >>>>>>> As soon as I move to next commit (dated May 22nd, pushed and >> deployed today) with id >>>>>>> >>>>>>> b51d003ed98aa7663465aad7c9a1b554b006cc7d >>>>>>> >>>>>>> I see the same behavior as with HEAD, e.g. >>>>>>> >>>>>>> 2b9b024f273d63e479a02cad751c28b8ef974ace >>>>>>> >>>>>>> I will try to take a closer look at the code. >>>>>>> Regards. >>>>>>> >>>>>>> On 03/07/23 14:44, Mark Struberg wrote: >>>>>>>> Hi Francesco! >>>>>>>> >>>>>>>> That's very valuable information! >>>>>>>> Could you please do a git-bisect to find out which change did break >> it? >>>>>>>> >>>>>>>> Actually the algorithm to select which classes are to be enhanced >> should not have changed. Thus I'm really curious what I did break. >>>>>>>> >>>>>>>> Most times the amount of enhanced classes are the same but slightly >> differ in the generated code. That might be due to now we really generate >> Java11++ code whereas before we only generated Java 1.1 code (yes, not even >> 1.5!) >>>>>>>> >>>>>>>> What I usually do is to have 2 checkouts. You might want to change >> the pom GAV version of the working checkout to something something >> different so you might have both in your ~/.m2/repository in parallel. >>>>>>>> >>>>>>>> The first thing I do is to compare the class files in >> target/classes of both versions with 'compare with clipboard'. >>>>>>>> I just open the class file in target/classes and let >> Intellij/Netbeans do the decompilation for me. >>>>>>>> >>>>>>>> You might find some differences here in which case this needs >> further investigation. Note that the order of the methods might have >> changed. >>>>>>>> Sometimes you will also see some 'cannot decompile' in the code >> block. This happens when something in the bytecode is messed up. You can >> also watch out for ClassNotFound or VerifyError, etc. >>>>>>>> >>>>>>>> If this is the case I then use >>>>>>>> $> javap -c -p -constants >> target/classes/com/foo/mycorp/MyBlaEntity.class > MyBlaEntity.decompile >>>>>>>> on both versions and check whether something is different. >>>>>>>> Often it's a wrong offset in the stack calculation (e.g. ALOAD 1 vs >> ALOAD 2) >>>>>>>> >>>>>>>> Note that the 'old' Java 1.1 bytecode doesn't understand LDC of >> Class constants (LoaD Constant). This was only introduced in Java 1.4 or >> 1.5 bytecode afair. Before that a dynamic Class.forName was called and >> stored in a static variable. Ugly stuff, and happy to finally get rid of >> all those hacks... >>>>>>>> >>>>>>>> Hope that helps to get you started. Feel free to ping me again over >> here or at the #apachee ASF slack channel. >>>>>>>> >>>>>>>> txs and LieGrue, >>>>>>>> strub >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Am 03.07.2023 um 14:13 schrieb Francesco Chicchiriccò < >> ilgro...@apache.org>: >>>>>>>>> >>>>>>>>> From a very preliminary analysis it seems that just a subset of >> the persistence-capable classes are effectively found and enhanced: now >> [2], it used to be [3]. >>>>>>>>> >>>>>>>>> I'll continue investigating. >>>>>>>>> >>>>>>>>> Regards. >>>>>>>>> >>>>>>>>> On 03/07/23 13:56, Francesco Chicchiriccò wrote: >>>>>>>>>> Hi Mark, >>>>>>>>>> the build on Syncope master branch, using OpenJPA 4.0.0-SNAPSHOT >> is now failing - it used to work at least until last June 30th. >>>>>>>>>> >>>>>>>>>> The issues seem to be related to entity enhancement, which we >> perform via openjpa-maven-plugin - see [1]. >>>>>>>>>> >>>>>>>>>> Is there anything obvious we should be changing? >>>>>>>>>> Meanwhile I'll try to dig to see if I am able to understand >> something more. >>>>>>>>>> >>>>>>>>>> Regards. >>>>>>>>>> >>>>>>>>>> [1] >> https://github.com/apache/syncope/blob/master/core/persistence-jpa/pom.xml#L143-L174 >>>>>>>>> [2] https://paste.apache.org/gxcm6 >>>>>>>>> [3] https://paste.apache.org/1g9k5 >>>>>>>>> >>>>>>>>>> On 03/07/23 12:15, Mark Struberg wrote: >>>>>>>>>>> Update: I've now merged the wip to our master and pushed it to >> the ASF gitox repo. >>>>>>>>>>> >>>>>>>>>>> I think we already came pretty far and it looks really >> promising. So there is a high chance that we succeed, although there is >> still plenty of work in front of us. >>>>>>>>>>> >>>>>>>>>>> How it works: >>>>>>>>>>> >>>>>>>>>>> Basically there are 2 sets of information in the PCEnhancer >> right now: >>>>>>>>>>> >>>>>>>>>>> _managed: the BCClass version of the original class >>>>>>>>>>> _pc: the BCClass of the generated/modified class. Might be the >> same as _managed but in case of subclassing or interface case it is >> different. In those cases _managed represents the original class and _pc >> the subclass/concrete implementation. >>>>>>>>>>> >>>>>>>>>>> managed and pc are the ASM variants of _managed and _pc. >>>>>>>>>>> >>>>>>>>>>> The main entry point for debugging into it is PCEnhancer#run() >>>>>>>>>>> >>>>>>>>>>> There are a few methods in ASMHelper to update from BCClass -> >> ASM and vice versa: >>>>>>>>>>> >>>>>>>>>>> AsmHelper.readIntoBCClass(pc, _pc); >>>>>>>>>>> takes all the information from the ASM pc, creates a class >> byte[] and reads that back into the existing _pc instance. >>>>>>>>>>> >>>>>>>>>>> pc = AsmHelper.toClassNode(_pc); >>>>>>>>>>> is the corresponding method in the other direction. from BCClass >> to ASM. >>>>>>>>>>> >>>>>>>>>>> Right now we have the following methods left to migrate in >> PCEnhancer >>>>>>>>>>> addAttachDetachCode(); >>>>>>>>>>> addSerializationCode(); >>>>>>>>>>> addCloningCode(); >>>>>>>>>>> runAuxiliaryEnhancers(); >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I'll be off for a week on holidays in Venice. >>>>>>>>>>> If someone want's to join in with the effort then please go on! >> You might want to compare the current PCEnhancer code with an older version >> to get an idea how it can be done and to see some patterns I've used. >>>>>>>>>>> >>>>>>>>>>> As a quick hint: Java is basically a stack based virtual CPU. >> The 'this' pointer is on position 0 of the stack, so ALOAD 0. After that >> comes the parameters. The size of the param on the stack depends on the >> type. Mostly 1, but for long eg 2 positions. So ALOAD 1 is the 1st method >> parameter, ALOAD 2 the 2nd (except 1st param was a 2-pos type). >>>>>>>>>>> In case of a static method there is ofc no 'this' on the stack, >> so the parameters start with zero offset. >>>>>>>>>>> After all the parameters there are the unnamed 'local' >> variables. I tried to consistently name them xxxVarPos. >>>>>>>>>>> See also the various helper methods in ASMHelper. For example >> for the various LOAD, RETURN, STORE operations which depends based on the >> type (ALOAD vs ILOAD vs IALOAD etc) >>>>>>>>>>> Also read the guide to ASM PDF from objectweb. >>>>>>>>>>> >>>>>>>>>>> I'll be mostly offline till Sunday, but will try to answer mails >> if there are some questions. >>>>>>>>>>> >>>>>>>>>>> txs and LieGrue, >>>>>>>>>>> strub >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Am 19.06.2023 um 17:11 schrieb Matt Pavlovich < >> mattr...@gmail.com>: >>>>>>>>>>>> >>>>>>>>>>>> +1 thanks for tackling this, Mark! >>>>>>>>>>>> >>>>>>>>>>>> ASM definitely more widely used going forward. >>>>>>>>>>>> >>>>>>>>>>>> -Matt >>>>>>>>>>>> >>>>>>>>>>>>> On Jun 17, 2023, at 3:01 PM, Mark Struberg >> <strub...@yahoo.de.INVALID> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Small update: >>>>>>>>>>>>> >>>>>>>>>>>>> I've worked on it over the last few weeks, and I'm getting >> closer >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>>>>>>>> >>>>>>>>>>>>> contains the latest work on the PCEnhancer. Right in the >> middle of replacing Serp with native ASM code. >>>>>>>>>>>>> >>>>>>>>>>>>> Note that I also had to modify a few interfaces with a few >> more to follow from BCClass to ASM ClassNode. >>>>>>>>>>>>> >>>>>>>>>>>>> Happy to get some feedback! >>>>>>>>>>>>> >>>>>>>>>>>>> If anybody wants to help with this effort I'm happy to also >> push this feature branch to our ASF repo. It looks reasonably promising >> already. >>>>>>>>>>>>> >>>>>>>>>>>>> LieGrue, >>>>>>>>>>>>> strub >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Am 25.05.2023 um 18:36 schrieb Mark Struberg >> <strub...@yahoo.de.INVALID>: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Right now I'm trying to get rid of serp step by step. >>>>>>>>>>>>>> The code is right now at my own github repo in the >> fb_asmEnhance branch: >>>>>>>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> The approach I took for now is to create a few methods in >> AsmHelper to be able to move from BCClass -> ASM ClassWriter and the other >> way around. That way we should be able to replace functionality part by >> part but still keep all things afloat. >>>>>>>>>>>>>> >>>>>>>>>>>>>> For now I've started with the PCSubclassValidator. >>>>>>>>>>>>>> Right now this evaluates the attributs using Serp plus ASM >> and then compare the results. >>>>>>>>>>>>>> If something is fishy, you'll see the following >>>>>>>>>>>>>> throw new IllegalStateException("MSX ASMTODO " + bcField + " >> " + field); >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> It would be extremely helpful, if you could go through your >> projects and let it run and report how it's going. >>>>>>>>>>>>>> If you see that "MSX ASMTODO" somewhere then we know I messed >> something up. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'll gonna finally remove the BCClass handling from those >> parts in a few days. Current commit is >>>>>>>>>>>>>> >> https://github.com/struberg/openjpa/commit/3ea2412003028d37f2a69971a47bb20abf589f8b >>>>>>>>>>>>>> >>>>>>>>>>>>>> txs and LieGrue, >>>>>>>>>>>>>> strub >>>> >>> >>> -- >>> Francesco Chicchiriccò >>> >>> Tirasa - Open Source Excellence >>> http://www.tirasa.net/ >>> >>> Member at The Apache Software Foundation >>> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail >>> http://home.apache.org/~ilgrosso/ >>> >> >>
