Hi Francesco! That's very valuable information! Could you please do a git-bisect to find out which change did break it?
Actually the algorithm to select which classes are to be enhanced should not have changed. Thus I'm really curious what I did break. Most times the amount of enhanced classes are the same but slightly differ in the generated code. That might be due to now we really generate Java11++ code whereas before we only generated Java 1.1 code (yes, not even 1.5!) What I usually do is to have 2 checkouts. You might want to change the pom GAV version of the working checkout to something something different so you might have both in your ~/.m2/repository in parallel. The first thing I do is to compare the class files in target/classes of both versions with 'compare with clipboard'. I just open the class file in target/classes and let Intellij/Netbeans do the decompilation for me. You might find some differences here in which case this needs further investigation. Note that the order of the methods might have changed. Sometimes you will also see some 'cannot decompile' in the code block. This happens when something in the bytecode is messed up. You can also watch out for ClassNotFound or VerifyError, etc. If this is the case I then use $> javap -c -p -constants target/classes/com/foo/mycorp/MyBlaEntity.class > MyBlaEntity.decompile on both versions and check whether something is different. Often it's a wrong offset in the stack calculation (e.g. ALOAD 1 vs ALOAD 2) Note that the 'old' Java 1.1 bytecode doesn't understand LDC of Class constants (LoaD Constant). This was only introduced in Java 1.4 or 1.5 bytecode afair. Before that a dynamic Class.forName was called and stored in a static variable. Ugly stuff, and happy to finally get rid of all those hacks... Hope that helps to get you started. Feel free to ping me again over here or at the #apachee ASF slack channel. txs and LieGrue, strub > Am 03.07.2023 um 14:13 schrieb Francesco Chicchiriccò <ilgro...@apache.org>: > > From a very preliminary analysis it seems that just a subset of the > persistence-capable classes are effectively found and enhanced: now [2], it > used to be [3]. > > I'll continue investigating. > > Regards. > > On 03/07/23 13:56, Francesco Chicchiriccò wrote: >> Hi Mark, >> the build on Syncope master branch, using OpenJPA 4.0.0-SNAPSHOT is now >> failing - it used to work at least until last June 30th. >> >> The issues seem to be related to entity enhancement, which we perform via >> openjpa-maven-plugin - see [1]. >> >> Is there anything obvious we should be changing? >> Meanwhile I'll try to dig to see if I am able to understand something more. >> >> Regards. >> >> [1] >> https://github.com/apache/syncope/blob/master/core/persistence-jpa/pom.xml#L143-L174 > > [2] https://paste.apache.org/gxcm6 > [3] https://paste.apache.org/1g9k5 > >> On 03/07/23 12:15, Mark Struberg wrote: >>> Update: I've now merged the wip to our master and pushed it to the ASF >>> gitox repo. >>> >>> I think we already came pretty far and it looks really promising. So there >>> is a high chance that we succeed, although there is still plenty of work in >>> front of us. >>> >>> How it works: >>> >>> Basically there are 2 sets of information in the PCEnhancer right now: >>> >>> _managed: the BCClass version of the original class >>> _pc: the BCClass of the generated/modified class. Might be the same as >>> _managed but in case of subclassing or interface case it is different. In >>> those cases _managed represents the original class and _pc the >>> subclass/concrete implementation. >>> >>> managed and pc are the ASM variants of _managed and _pc. >>> >>> The main entry point for debugging into it is PCEnhancer#run() >>> >>> There are a few methods in ASMHelper to update from BCClass -> ASM and vice >>> versa: >>> >>> AsmHelper.readIntoBCClass(pc, _pc); >>> takes all the information from the ASM pc, creates a class byte[] and reads >>> that back into the existing _pc instance. >>> >>> pc = AsmHelper.toClassNode(_pc); >>> is the corresponding method in the other direction. from BCClass to ASM. >>> >>> Right now we have the following methods left to migrate in PCEnhancer >>> addAttachDetachCode(); >>> addSerializationCode(); >>> addCloningCode(); >>> runAuxiliaryEnhancers(); >>> >>> >>> I'll be off for a week on holidays in Venice. >>> If someone want's to join in with the effort then please go on! You might >>> want to compare the current PCEnhancer code with an older version to get an >>> idea how it can be done and to see some patterns I've used. >>> >>> As a quick hint: Java is basically a stack based virtual CPU. The 'this' >>> pointer is on position 0 of the stack, so ALOAD 0. After that comes the >>> parameters. The size of the param on the stack depends on the type. Mostly >>> 1, but for long eg 2 positions. So ALOAD 1 is the 1st method parameter, >>> ALOAD 2 the 2nd (except 1st param was a 2-pos type). >>> In case of a static method there is ofc no 'this' on the stack, so the >>> parameters start with zero offset. >>> After all the parameters there are the unnamed 'local' variables. I tried >>> to consistently name them xxxVarPos. >>> See also the various helper methods in ASMHelper. For example for the >>> various LOAD, RETURN, STORE operations which depends based on the type >>> (ALOAD vs ILOAD vs IALOAD etc) >>> Also read the guide to ASM PDF from objectweb. >>> >>> I'll be mostly offline till Sunday, but will try to answer mails if there >>> are some questions. >>> >>> txs and LieGrue, >>> strub >>> >>> >>> >>> >>>> Am 19.06.2023 um 17:11 schrieb Matt Pavlovich <mattr...@gmail.com>: >>>> >>>> +1 thanks for tackling this, Mark! >>>> >>>> ASM definitely more widely used going forward. >>>> >>>> -Matt >>>> >>>>> On Jun 17, 2023, at 3:01 PM, Mark Struberg <strub...@yahoo.de.INVALID> >>>>> wrote: >>>>> >>>>> Small update: >>>>> >>>>> I've worked on it over the last few weeks, and I'm getting closer >>>>> >>>>> >>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>> >>>>> contains the latest work on the PCEnhancer. Right in the middle of >>>>> replacing Serp with native ASM code. >>>>> >>>>> Note that I also had to modify a few interfaces with a few more to follow >>>>> from BCClass to ASM ClassNode. >>>>> >>>>> Happy to get some feedback! >>>>> >>>>> If anybody wants to help with this effort I'm happy to also push this >>>>> feature branch to our ASF repo. It looks reasonably promising already. >>>>> >>>>> LieGrue, >>>>> strub >>>>> >>>>> >>>>>> Am 25.05.2023 um 18:36 schrieb Mark Struberg <strub...@yahoo.de.INVALID>: >>>>>> >>>>>> Hi! >>>>>> >>>>>> Right now I'm trying to get rid of serp step by step. >>>>>> The code is right now at my own github repo in the fb_asmEnhance branch: >>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance >>>>>> >>>>>> >>>>>> >>>>>> The approach I took for now is to create a few methods in AsmHelper to >>>>>> be able to move from BCClass -> ASM ClassWriter and the other way >>>>>> around. That way we should be able to replace functionality part by part >>>>>> but still keep all things afloat. >>>>>> >>>>>> For now I've started with the PCSubclassValidator. >>>>>> Right now this evaluates the attributs using Serp plus ASM and then >>>>>> compare the results. >>>>>> If something is fishy, you'll see the following >>>>>> throw new IllegalStateException("MSX ASMTODO " + bcField + " " + field); >>>>>> >>>>>> >>>>>> It would be extremely helpful, if you could go through your projects and >>>>>> let it run and report how it's going. >>>>>> If you see that "MSX ASMTODO" somewhere then we know I messed something >>>>>> up. >>>>>> >>>>>> I'll gonna finally remove the BCClass handling from those parts in a few >>>>>> days. Current commit is >>>>>> https://github.com/struberg/openjpa/commit/3ea2412003028d37f2a69971a47bb20abf589f8b >>>>>> >>>>>> txs and LieGrue, >>>>>> strub > > -- > Francesco Chicchiriccò > > Tirasa - Open Source Excellence > http://www.tirasa.net/ > > Member at The Apache Software Foundation > Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail > http://home.apache.org/~ilgrosso/
