Hi Francesco! Can you see which one of the classes fail to enhance?
In which case you might try to run the enhancement of that very class manually in your IDE. Main class: org.apache.openjpa.enhance.PCEnhancer VMOptions:: -ea -Dopenjpa.ConnectionURL=jdbc:derby:target/database/openjpa-derby-database;create=true -Dopenjpa.ConnectionDriverName=org.apache.derby.jdbc.EmbeddedDriver or something similar for your connection Then you also need the parameter pointing to the class file which you want to enhance, e.g. target/test-classes/org/apache/openjpa/persistence/proxy/entities/Annuity.class Please keep me updated! LieGrue, strub PS: does it add value if we'd write all this up? You will likely know most of that stuff, but others who dig into OpenJPA do not. > Am 03.07.2023 um 15:28 schrieb Francesco Chicchiriccò <ilgro...@apache.org>: > > Hi Mark, > thanks for your reply. > > So I am proceeding as follows. > > Took the executions at > > https://ci-builds.apache.org/job/OpenJPA/job/OpenJPA-master-deploy/ > > (the Jenkins job which is publishing SNAPSHOT artifacts from master to ASF > Maven repo) and saw that the artifacts with which the Syncope build succeeded > last Friday are the one from May 16th, so the corresponding commit id is > > b238417dabeac935846fe1ce2fef28aafaeef205 > > I confirm that building locally from that commit id everything is working. > > As soon as I move to next commit (dated May 22nd, pushed and deployed today) > with id > > b51d003ed98aa7663465aad7c9a1b554b006cc7d > > I see the same behavior as with HEAD, e.g. > > 2b9b024f273d63e479a02cad751c28b8ef974ace > > I will try to take a closer look at the code. > Regards. > > On 03/07/23 14:44, Mark Struberg wrote: >> Hi Francesco! >> >> That's very valuable information! >> Could you please do a git-bisect to find out which change did break it? >> >> Actually the algorithm to select which classes are to be enhanced should not >> have changed. Thus I'm really curious what I did break. >> >> Most times the amount of enhanced classes are the same but slightly differ >> in the generated code. That might be due to now we really generate Java11++ >> code whereas before we only generated Java 1.1 code (yes, not even 1.5!) >> >> What I usually do is to have 2 checkouts. You might want to change the pom >> GAV version of the working checkout to something something different so you >> might have both in your ~/.m2/repository in parallel. >> >> The first thing I do is to compare the class files in target/classes of both >> versions with 'compare with clipboard'. >> I just open the class file in target/classes and let Intellij/Netbeans do >> the decompilation for me. >> >> You might find some differences here in which case this needs further >> investigation. Note that the order of the methods might have changed. >> Sometimes you will also see some 'cannot decompile' in the code block. This >> happens when something in the bytecode is messed up. You can also watch out >> for ClassNotFound or VerifyError, etc. >> >> If this is the case I then use >> $> javap -c -p -constants target/classes/com/foo/mycorp/MyBlaEntity.class > >> MyBlaEntity.decompile >> on both versions and check whether something is different. >> Often it's a wrong offset in the stack calculation (e.g. ALOAD 1 vs ALOAD 2) >> >> Note that the 'old' Java 1.1 bytecode doesn't understand LDC of Class >> constants (LoaD Constant). This was only introduced in Java 1.4 or 1.5 >> bytecode afair. Before that a dynamic Class.forName was called and stored in >> a static variable. Ugly stuff, and happy to finally get rid of all those >> hacks... >> >> Hope that helps to get you started. Feel free to ping me again over here or >> at the #apachee ASF slack channel. >> >> txs and LieGrue, >> strub >> >> >> >>> Am 03.07.2023 um 14:13 schrieb Francesco Chicchiriccò <ilgro...@apache.org>: >>> >>> From a very preliminary analysis it seems that just a subset of the >>> persistence-capable classes are effectively found and enhanced: now [2], it >>> used to be [3]. >>> >>> I'll continue investigating. >>> >>> Regards. >>> >>> On 03/07/23 13:56, Francesco Chicchiriccò wrote: >>>> Hi Mark, >>>> the build on Syncope master branch, using OpenJPA 4.0.0-SNAPSHOT is now >>>> failing - it used to work at least until last June 30th. >>>> >>>> The issues seem to be related to entity enhancement, which we perform via >>>> openjpa-maven-plugin - see [1]. >>>> >>>> Is there anything obvious we should be changing? >>>> Meanwhile I'll try to dig to see if I am able to understand something more. >>>> >>>> Regards. >>>> >>>> [1] >>>> https://github.com/apache/syncope/blob/master/core/persistence-jpa/pom.xml#L143-L174 >>> [2] https://paste.apache.org/gxcm6 >>> [3] https://paste.apache.org/1g9k5 >>> >>>> On 03/07/23 12:15, Mark Struberg wrote: >>>>> Update: I've now merged the wip to our master and pushed it to the ASF >>>>> gitox repo. >>>>> >>>>> I think we already came pretty far and it looks really promising. So >>>>> there is a high chance that we succeed, although there is still plenty of >>>>> work in front of us. >>>>> >>>>> How it works: >>>>> >>>>> Basically there are 2 sets of information in the PCEnhancer right now: >>>>> >>>>> _managed: the BCClass version of the original class >>>>> _pc: the BCClass of the generated/modified class. Might be the same as >>>>> _managed but in case of subclassing or interface case it is different. In >>>>> those cases _managed represents the original class and _pc the >>>>> subclass/concrete implementation. >>>>> >>>>> managed and pc are the ASM variants of _managed and _pc. >>>>> >>>>> The main entry point for debugging into it is PCEnhancer#run() >>>>> >>>>> There are a few methods in ASMHelper to update from BCClass -> ASM and >>>>> vice versa: >>>>> >>>>> AsmHelper.readIntoBCClass(pc, _pc); >>>>> takes all the information from the ASM pc, creates a class byte[] and >>>>> reads that back into the existing _pc instance. >>>>> >>>>> pc = AsmHelper.toClassNode(_pc); >>>>> is the corresponding method in the other direction. from BCClass to ASM. >>>>> >>>>> Right now we have the following methods left to migrate in PCEnhancer >>>>> addAttachDetachCode(); >>>>> addSerializationCode(); >>>>> addCloningCode(); >>>>> runAuxiliaryEnhancers(); >>>>> >>>>> >>>>> I'll be off for a week on holidays in Venice. >>>>> If someone want's to join in with the effort then please go on! You might >>>>> want to compare the current PCEnhancer code with an older version to get >>>>> an idea how it can be done and to see some patterns I've used. >>>>> >>>>> As a quick hint: Java is basically a stack based virtual CPU. The 'this' >>>>> pointer is on position 0 of the stack, so ALOAD 0. After that comes the >>>>> parameters. The size of the param on the stack depends on the type. >>>>> Mostly 1, but for long eg 2 positions. So ALOAD 1 is the 1st method >>>>> parameter, ALOAD 2 the 2nd (except 1st param was a 2-pos type). >>>>> In case of a static method there is ofc no 'this' on the stack, so the >>>>> parameters start with zero offset. >>>>> After all the parameters there are the unnamed 'local' variables. I tried >>>>> to consistently name them xxxVarPos. >>>>> See also the various helper methods in ASMHelper. For example for the >>>>> various LOAD, RETURN, STORE operations which depends based on the type >>>>> (ALOAD vs ILOAD vs IALOAD etc) >>>>> Also read the guide to ASM PDF from objectweb. >>>>> >>>>> I'll be mostly offline till Sunday, but will try to answer mails if there >>>>> are some questions. >>>>> >>>>> txs and LieGrue, >>>>> strub >>>>> >>>>> >>>>> >>>>> >>>>>> Am 19.06.2023 um 17:11 schrieb Matt Pavlovich <mattr...@gmail.com>: >>>>>> >>>>>> +1 thanks for tackling this, Mark! >>>>>> >>>>>> ASM definitely more widely used going forward. >>>>>> >>>>>> -Matt >>>>>> >>>>>>> On Jun 17, 2023, at 3:01 PM, Mark Struberg <strub...@yahoo.de.INVALID> >>>>>>> wrote: >>>>>>> >>>>>>> Small update: >>>>>>> >>>>>>> I've worked on it over the last few weeks, and I'm getting closer >>>>>>> >>>>>>> >>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>> >>>>>>> contains the latest work on the PCEnhancer. Right in the middle of >>>>>>> replacing Serp with native ASM code. >>>>>>> >>>>>>> Note that I also had to modify a few interfaces with a few more to >>>>>>> follow from BCClass to ASM ClassNode. >>>>>>> >>>>>>> Happy to get some feedback! >>>>>>> >>>>>>> If anybody wants to help with this effort I'm happy to also push this >>>>>>> feature branch to our ASF repo. It looks reasonably promising already. >>>>>>> >>>>>>> LieGrue, >>>>>>> strub >>>>>>> >>>>>>> >>>>>>>> Am 25.05.2023 um 18:36 schrieb Mark Struberg >>>>>>>> <strub...@yahoo.de.INVALID>: >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> Right now I'm trying to get rid of serp step by step. >>>>>>>> The code is right now at my own github repo in the fb_asmEnhance >>>>>>>> branch: >>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The approach I took for now is to create a few methods in AsmHelper to >>>>>>>> be able to move from BCClass -> ASM ClassWriter and the other way >>>>>>>> around. That way we should be able to replace functionality part by >>>>>>>> part but still keep all things afloat. >>>>>>>> >>>>>>>> For now I've started with the PCSubclassValidator. >>>>>>>> Right now this evaluates the attributs using Serp plus ASM and then >>>>>>>> compare the results. >>>>>>>> If something is fishy, you'll see the following >>>>>>>> throw new IllegalStateException("MSX ASMTODO " + bcField + " " + >>>>>>>> field); >>>>>>>> >>>>>>>> >>>>>>>> It would be extremely helpful, if you could go through your projects >>>>>>>> and let it run and report how it's going. >>>>>>>> If you see that "MSX ASMTODO" somewhere then we know I messed >>>>>>>> something up. >>>>>>>> >>>>>>>> I'll gonna finally remove the BCClass handling from those parts in a >>>>>>>> few days. Current commit is >>>>>>>> https://github.com/struberg/openjpa/commit/3ea2412003028d37f2a69971a47bb20abf589f8b >>>>>>>> >>>>>>>> txs and LieGrue, >>>>>>>> strub > > -- > Francesco Chicchiriccò > > Tirasa - Open Source Excellence > http://www.tirasa.net/ > > Member at The Apache Software Foundation > Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail > http://home.apache.org/~ilgrosso/ >
