Does not yet ring any bells, sorry :( Right now trying to fire up the enhancer manually in the debugger. Difference really is that we now do the ASM part way earlier, but still class per class. Before that it was done as part of the ASMAdapter.
Still digging. LieGrue, strub > Am 03.07.2023 um 17:00 schrieb Francesco Chicchiriccò <ilgro...@apache.org>: > > Adding some info. > > First of all, the class to which the error message is referring is not always > the same. > > Secondly, at least about AccountPolicy as reported below, the original error > message is stating > > Execution enhancer of goal > org.apache.openjpa:openjpa-maven-plugin:4.0.0-SNAPSHOT:enhance failed: An > error occurred while enhancing > org.apache.syncope.core.persistence.jpa.entity.JPARealm. Exception message: > Type org/apache/syncope/core/persistence/api/entity/policy/AccountPolicy not > present: org.apache.syncope.core.persistence.api.entity.policy.AccountPolicy > > If you look at JPARealm: > > https://github.com/apache/syncope/blob/master/core/persistence-jpa/src/main/java/org/apache/syncope/core/persistence/jpa/entity/JPARealm.java#L82 > > it declares > > @ManyToOne(fetch = FetchType.EAGER) > private JPAAccountPolicy accountPolicy; > > e.g. a reference to the JPA entity JPAAccountPolicy, not to the interface > that this class implements, e.g. AccountPolicy > > It looks to me that somewhere the OpenJPA enhancement process is trying to > resolve more than it should do. > > Does this make ringing any bell? > > Regards. > > On 03/07/23 16:32, Francesco Chicchiriccò wrote: >> On 03/07/23 16:27, Mark Struberg wrote: >>> Went forward and compiled the Syncope main branch. >>> >>> Getting this error when running with mvn -X >>> >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.syncope.core.persistence.api.entity.policy.AccountPolicy >>> at >>> org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass >>> (SelfFirstStrategy.java:50) >>> at >>> org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass >>> (ClassRealm.java:271) >>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass >>> (ClassRealm.java:247) >>> at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass >>> (ClassRealm.java:239) >>> at java.lang.Class.forName0 (Native Method) >>> at java.lang.Class.forName (Class.java:467) >>> at org.apache.xbean.asm9.ClassWriter.getCommonSuperClass >>> (ClassWriter.java:1043) >>> at org.apache.xbean.asm9.SymbolTable.addMergedType >>> (SymbolTable.java:1202) >>> at org.apache.xbean.asm9.Frame.merge (Frame.java:1299) >>> at org.apache.xbean.asm9.Frame.merge (Frame.java:1244) >>> at org.apache.xbean.asm9.MethodWriter.computeAllFrames >>> (MethodWriter.java:1611) >>> at org.apache.xbean.asm9.MethodWriter.visitMaxs >>> (MethodWriter.java:1547) >>> at org.apache.xbean.asm9.tree.MethodNode.accept (MethodNode.java:767) >>> at org.apache.xbean.asm9.tree.MethodNode.accept (MethodNode.java:647) >>> at org.apache.xbean.asm9.tree.ClassNode.accept (ClassNode.java:468) >>> at org.apache.openjpa.util.asm.AsmHelper.readIntoBCClass >>> (AsmHelper.java:146) >>> at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:606) >>> at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:5889) >>> at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:5829) >>> at org.apache.openjpa.enhance.PCEnhancer$1.run (PCEnhancer.java:5799) >>> at org.apache.openjpa.lib.conf.Configurations.launchRunnable >>> (Configurations.java:760) >>> at org.apache.openjpa.lib.conf.Configurations.runAgainstAllAnchors >>> (Configurations.java:745) >>> >>> >>> Is this the one you experience too? >> >> Yes, exactly. >> >> As you might have found out, AccountPolicy is an interface implemented by >> JPAAccountPolicy, which is the corresponding JPA entity. >> We take this approach for all JPA entities, actually. >> >> I thought the reason why AccountPolicy or JPAAccountPolicy are not found is >> that enhancement does not actually finds them. >> >>> Will take a look at it now. >> >> Thank you very much! >> >> Regards. >>>> Am 03.07.2023 um 15:49 schrieb Mark Struberg <strub...@yahoo.de.INVALID>: >>>> >>>> Hi Francesco! >>>> >>>> Can you see which one of the classes fail to enhance? >>>> >>>> In which case you might try to run the enhancement of that very class >>>> manually in your IDE. >>>> >>>> Main class: org.apache.openjpa.enhance.PCEnhancer >>>> VMOptions:: -ea >>>> -Dopenjpa.ConnectionURL=jdbc:derby:target/database/openjpa-derby-database;create=true >>>> -Dopenjpa.ConnectionDriverName=org.apache.derby.jdbc.EmbeddedDriver >>>> or something similar for your connection >>>> >>>> Then you also need the parameter pointing to the class file which you want >>>> to enhance, e.g. >>>> target/test-classes/org/apache/openjpa/persistence/proxy/entities/Annuity.class >>>> >>>> Please keep me updated! >>>> >>>> >>>> LieGrue, >>>> strub >>>> >>>> PS: does it add value if we'd write all this up? You will likely know most >>>> of that stuff, but others who dig into OpenJPA do not. >>>> >>>> >>>> >>>>> Am 03.07.2023 um 15:28 schrieb Francesco Chicchiriccò >>>>> <ilgro...@apache.org>: >>>>> >>>>> Hi Mark, >>>>> thanks for your reply. >>>>> >>>>> So I am proceeding as follows. >>>>> >>>>> Took the executions at >>>>> >>>>> https://ci-builds.apache.org/job/OpenJPA/job/OpenJPA-master-deploy/ >>>>> >>>>> (the Jenkins job which is publishing SNAPSHOT artifacts from master to >>>>> ASF Maven repo) and saw that the artifacts with which the Syncope build >>>>> succeeded last Friday are the one from May 16th, so the corresponding >>>>> commit id is >>>>> >>>>> b238417dabeac935846fe1ce2fef28aafaeef205 >>>>> >>>>> I confirm that building locally from that commit id everything is working. >>>>> >>>>> As soon as I move to next commit (dated May 22nd, pushed and deployed >>>>> today) with id >>>>> >>>>> b51d003ed98aa7663465aad7c9a1b554b006cc7d >>>>> >>>>> I see the same behavior as with HEAD, e.g. >>>>> >>>>> 2b9b024f273d63e479a02cad751c28b8ef974ace >>>>> >>>>> I will try to take a closer look at the code. >>>>> Regards. >>>>> >>>>> On 03/07/23 14:44, Mark Struberg wrote: >>>>>> Hi Francesco! >>>>>> >>>>>> That's very valuable information! >>>>>> Could you please do a git-bisect to find out which change did break it? >>>>>> >>>>>> Actually the algorithm to select which classes are to be enhanced should >>>>>> not have changed. Thus I'm really curious what I did break. >>>>>> >>>>>> Most times the amount of enhanced classes are the same but slightly >>>>>> differ in the generated code. That might be due to now we really >>>>>> generate Java11++ code whereas before we only generated Java 1.1 code >>>>>> (yes, not even 1.5!) >>>>>> >>>>>> What I usually do is to have 2 checkouts. You might want to change the >>>>>> pom GAV version of the working checkout to something something different >>>>>> so you might have both in your ~/.m2/repository in parallel. >>>>>> >>>>>> The first thing I do is to compare the class files in target/classes of >>>>>> both versions with 'compare with clipboard'. >>>>>> I just open the class file in target/classes and let Intellij/Netbeans >>>>>> do the decompilation for me. >>>>>> >>>>>> You might find some differences here in which case this needs further >>>>>> investigation. Note that the order of the methods might have changed. >>>>>> Sometimes you will also see some 'cannot decompile' in the code block. >>>>>> This happens when something in the bytecode is messed up. You can also >>>>>> watch out for ClassNotFound or VerifyError, etc. >>>>>> >>>>>> If this is the case I then use >>>>>> $> javap -c -p -constants >>>>>> target/classes/com/foo/mycorp/MyBlaEntity.class > MyBlaEntity.decompile >>>>>> on both versions and check whether something is different. >>>>>> Often it's a wrong offset in the stack calculation (e.g. ALOAD 1 vs >>>>>> ALOAD 2) >>>>>> >>>>>> Note that the 'old' Java 1.1 bytecode doesn't understand LDC of Class >>>>>> constants (LoaD Constant). This was only introduced in Java 1.4 or 1.5 >>>>>> bytecode afair. Before that a dynamic Class.forName was called and >>>>>> stored in a static variable. Ugly stuff, and happy to finally get rid of >>>>>> all those hacks... >>>>>> >>>>>> Hope that helps to get you started. Feel free to ping me again over here >>>>>> or at the #apachee ASF slack channel. >>>>>> >>>>>> txs and LieGrue, >>>>>> strub >>>>>> >>>>>> >>>>>> >>>>>>> Am 03.07.2023 um 14:13 schrieb Francesco Chicchiriccò >>>>>>> <ilgro...@apache.org>: >>>>>>> >>>>>>> From a very preliminary analysis it seems that just a subset of the >>>>>>> persistence-capable classes are effectively found and enhanced: now >>>>>>> [2], it used to be [3]. >>>>>>> >>>>>>> I'll continue investigating. >>>>>>> >>>>>>> Regards. >>>>>>> >>>>>>> On 03/07/23 13:56, Francesco Chicchiriccò wrote: >>>>>>>> Hi Mark, >>>>>>>> the build on Syncope master branch, using OpenJPA 4.0.0-SNAPSHOT is >>>>>>>> now failing - it used to work at least until last June 30th. >>>>>>>> >>>>>>>> The issues seem to be related to entity enhancement, which we perform >>>>>>>> via openjpa-maven-plugin - see [1]. >>>>>>>> >>>>>>>> Is there anything obvious we should be changing? >>>>>>>> Meanwhile I'll try to dig to see if I am able to understand something >>>>>>>> more. >>>>>>>> >>>>>>>> Regards. >>>>>>>> >>>>>>>> [1] >>>>>>>> https://github.com/apache/syncope/blob/master/core/persistence-jpa/pom.xml#L143-L174 >>>>>>> [2] https://paste.apache.org/gxcm6 >>>>>>> [3] https://paste.apache.org/1g9k5 >>>>>>> >>>>>>>> On 03/07/23 12:15, Mark Struberg wrote: >>>>>>>>> Update: I've now merged the wip to our master and pushed it to the >>>>>>>>> ASF gitox repo. >>>>>>>>> >>>>>>>>> I think we already came pretty far and it looks really promising. So >>>>>>>>> there is a high chance that we succeed, although there is still >>>>>>>>> plenty of work in front of us. >>>>>>>>> >>>>>>>>> How it works: >>>>>>>>> >>>>>>>>> Basically there are 2 sets of information in the PCEnhancer right now: >>>>>>>>> >>>>>>>>> _managed: the BCClass version of the original class >>>>>>>>> _pc: the BCClass of the generated/modified class. Might be the same >>>>>>>>> as _managed but in case of subclassing or interface case it is >>>>>>>>> different. In those cases _managed represents the original class and >>>>>>>>> _pc the subclass/concrete implementation. >>>>>>>>> >>>>>>>>> managed and pc are the ASM variants of _managed and _pc. >>>>>>>>> >>>>>>>>> The main entry point for debugging into it is PCEnhancer#run() >>>>>>>>> >>>>>>>>> There are a few methods in ASMHelper to update from BCClass -> ASM >>>>>>>>> and vice versa: >>>>>>>>> >>>>>>>>> AsmHelper.readIntoBCClass(pc, _pc); >>>>>>>>> takes all the information from the ASM pc, creates a class byte[] and >>>>>>>>> reads that back into the existing _pc instance. >>>>>>>>> >>>>>>>>> pc = AsmHelper.toClassNode(_pc); >>>>>>>>> is the corresponding method in the other direction. from BCClass to >>>>>>>>> ASM. >>>>>>>>> >>>>>>>>> Right now we have the following methods left to migrate in PCEnhancer >>>>>>>>> addAttachDetachCode(); >>>>>>>>> addSerializationCode(); >>>>>>>>> addCloningCode(); >>>>>>>>> runAuxiliaryEnhancers(); >>>>>>>>> >>>>>>>>> >>>>>>>>> I'll be off for a week on holidays in Venice. >>>>>>>>> If someone want's to join in with the effort then please go on! You >>>>>>>>> might want to compare the current PCEnhancer code with an older >>>>>>>>> version to get an idea how it can be done and to see some patterns >>>>>>>>> I've used. >>>>>>>>> >>>>>>>>> As a quick hint: Java is basically a stack based virtual CPU. The >>>>>>>>> 'this' pointer is on position 0 of the stack, so ALOAD 0. After that >>>>>>>>> comes the parameters. The size of the param on the stack depends on >>>>>>>>> the type. Mostly 1, but for long eg 2 positions. So ALOAD 1 is the >>>>>>>>> 1st method parameter, ALOAD 2 the 2nd (except 1st param was a 2-pos >>>>>>>>> type). >>>>>>>>> In case of a static method there is ofc no 'this' on the stack, so >>>>>>>>> the parameters start with zero offset. >>>>>>>>> After all the parameters there are the unnamed 'local' variables. I >>>>>>>>> tried to consistently name them xxxVarPos. >>>>>>>>> See also the various helper methods in ASMHelper. For example for the >>>>>>>>> various LOAD, RETURN, STORE operations which depends based on the >>>>>>>>> type (ALOAD vs ILOAD vs IALOAD etc) >>>>>>>>> Also read the guide to ASM PDF from objectweb. >>>>>>>>> >>>>>>>>> I'll be mostly offline till Sunday, but will try to answer mails if >>>>>>>>> there are some questions. >>>>>>>>> >>>>>>>>> txs and LieGrue, >>>>>>>>> strub >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Am 19.06.2023 um 17:11 schrieb Matt Pavlovich <mattr...@gmail.com>: >>>>>>>>>> >>>>>>>>>> +1 thanks for tackling this, Mark! >>>>>>>>>> >>>>>>>>>> ASM definitely more widely used going forward. >>>>>>>>>> >>>>>>>>>> -Matt >>>>>>>>>> >>>>>>>>>>> On Jun 17, 2023, at 3:01 PM, Mark Struberg >>>>>>>>>>> <strub...@yahoo.de.INVALID> wrote: >>>>>>>>>>> >>>>>>>>>>> Small update: >>>>>>>>>>> >>>>>>>>>>> I've worked on it over the last few weeks, and I'm getting closer >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>>>>>> >>>>>>>>>>> contains the latest work on the PCEnhancer. Right in the middle of >>>>>>>>>>> replacing Serp with native ASM code. >>>>>>>>>>> >>>>>>>>>>> Note that I also had to modify a few interfaces with a few more to >>>>>>>>>>> follow from BCClass to ASM ClassNode. >>>>>>>>>>> >>>>>>>>>>> Happy to get some feedback! >>>>>>>>>>> >>>>>>>>>>> If anybody wants to help with this effort I'm happy to also push >>>>>>>>>>> this feature branch to our ASF repo. It looks reasonably promising >>>>>>>>>>> already. >>>>>>>>>>> >>>>>>>>>>> LieGrue, >>>>>>>>>>> strub >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Am 25.05.2023 um 18:36 schrieb Mark Struberg >>>>>>>>>>>> <strub...@yahoo.de.INVALID>: >>>>>>>>>>>> >>>>>>>>>>>> Hi! >>>>>>>>>>>> >>>>>>>>>>>> Right now I'm trying to get rid of serp step by step. >>>>>>>>>>>> The code is right now at my own github repo in the fb_asmEnhance >>>>>>>>>>>> branch: >>>>>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> The approach I took for now is to create a few methods in >>>>>>>>>>>> AsmHelper to be able to move from BCClass -> ASM ClassWriter and >>>>>>>>>>>> the other way around. That way we should be able to replace >>>>>>>>>>>> functionality part by part but still keep all things afloat. >>>>>>>>>>>> >>>>>>>>>>>> For now I've started with the PCSubclassValidator. >>>>>>>>>>>> Right now this evaluates the attributs using Serp plus ASM and >>>>>>>>>>>> then compare the results. >>>>>>>>>>>> If something is fishy, you'll see the following >>>>>>>>>>>> throw new IllegalStateException("MSX ASMTODO " + bcField + " " + >>>>>>>>>>>> field); >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It would be extremely helpful, if you could go through your >>>>>>>>>>>> projects and let it run and report how it's going. >>>>>>>>>>>> If you see that "MSX ASMTODO" somewhere then we know I messed >>>>>>>>>>>> something up. >>>>>>>>>>>> >>>>>>>>>>>> I'll gonna finally remove the BCClass handling from those parts in >>>>>>>>>>>> a few days. Current commit is >>>>>>>>>>>> https://github.com/struberg/openjpa/commit/3ea2412003028d37f2a69971a47bb20abf589f8b >>>>>>>>>>>> >>>>>>>>>>>> txs and LieGrue, >>>>>>>>>>>> strub >> > > -- > Francesco Chicchiriccò > > Tirasa - Open Source Excellence > http://www.tirasa.net/ > > Member at The Apache Software Foundation > Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail > http://home.apache.org/~ilgrosso/ >
