Went forward and compiled the Syncope main branch.
Getting this error when running with mvn -X
Caused by: java.lang.ClassNotFoundException:
org.apache.syncope.core.persistence.api.entity.policy.AccountPolicy
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass
(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass
(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass
(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass
(ClassRealm.java:239)
at java.lang.Class.forName0 (Native Method)
at java.lang.Class.forName (Class.java:467)
at org.apache.xbean.asm9.ClassWriter.getCommonSuperClass
(ClassWriter.java:1043)
at org.apache.xbean.asm9.SymbolTable.addMergedType (SymbolTable.java:1202)
at org.apache.xbean.asm9.Frame.merge (Frame.java:1299)
at org.apache.xbean.asm9.Frame.merge (Frame.java:1244)
at org.apache.xbean.asm9.MethodWriter.computeAllFrames
(MethodWriter.java:1611)
at org.apache.xbean.asm9.MethodWriter.visitMaxs (MethodWriter.java:1547)
at org.apache.xbean.asm9.tree.MethodNode.accept (MethodNode.java:767)
at org.apache.xbean.asm9.tree.MethodNode.accept (MethodNode.java:647)
at org.apache.xbean.asm9.tree.ClassNode.accept (ClassNode.java:468)
at org.apache.openjpa.util.asm.AsmHelper.readIntoBCClass
(AsmHelper.java:146)
at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:606)
at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:5889)
at org.apache.openjpa.enhance.PCEnhancer.run (PCEnhancer.java:5829)
at org.apache.openjpa.enhance.PCEnhancer$1.run (PCEnhancer.java:5799)
at org.apache.openjpa.lib.conf.Configurations.launchRunnable
(Configurations.java:760)
at org.apache.openjpa.lib.conf.Configurations.runAgainstAllAnchors
(Configurations.java:745)
Is this the one you experience too?
Will take a look at it now.
LieGrue
strub
> Am 03.07.2023 um 15:49 schrieb Mark Struberg <strub...@yahoo.de.INVALID>:
>
> Hi Francesco!
>
> Can you see which one of the classes fail to enhance?
>
> In which case you might try to run the enhancement of that very class
> manually in your IDE.
>
> Main class: org.apache.openjpa.enhance.PCEnhancer
> VMOptions:: -ea
> -Dopenjpa.ConnectionURL=jdbc:derby:target/database/openjpa-derby-database;create=true
> -Dopenjpa.ConnectionDriverName=org.apache.derby.jdbc.EmbeddedDriver
> or something similar for your connection
>
> Then you also need the parameter pointing to the class file which you want to
> enhance, e.g.
> target/test-classes/org/apache/openjpa/persistence/proxy/entities/Annuity.class
>
> Please keep me updated!
>
>
> LieGrue,
> strub
>
> PS: does it add value if we'd write all this up? You will likely know most of
> that stuff, but others who dig into OpenJPA do not.
>
>
>
>> Am 03.07.2023 um 15:28 schrieb Francesco Chicchiriccò <ilgro...@apache.org>:
>>
>> Hi Mark,
>> thanks for your reply.
>>
>> So I am proceeding as follows.
>>
>> Took the executions at
>>
>> https://ci-builds.apache.org/job/OpenJPA/job/OpenJPA-master-deploy/
>>
>> (the Jenkins job which is publishing SNAPSHOT artifacts from master to ASF
>> Maven repo) and saw that the artifacts with which the Syncope build
>> succeeded last Friday are the one from May 16th, so the corresponding commit
>> id is
>>
>> b238417dabeac935846fe1ce2fef28aafaeef205
>>
>> I confirm that building locally from that commit id everything is working.
>>
>> As soon as I move to next commit (dated May 22nd, pushed and deployed today)
>> with id
>>
>> b51d003ed98aa7663465aad7c9a1b554b006cc7d
>>
>> I see the same behavior as with HEAD, e.g.
>>
>> 2b9b024f273d63e479a02cad751c28b8ef974ace
>>
>> I will try to take a closer look at the code.
>> Regards.
>>
>> On 03/07/23 14:44, Mark Struberg wrote:
>>> Hi Francesco!
>>>
>>> That's very valuable information!
>>> Could you please do a git-bisect to find out which change did break it?
>>>
>>> Actually the algorithm to select which classes are to be enhanced should
>>> not have changed. Thus I'm really curious what I did break.
>>>
>>> Most times the amount of enhanced classes are the same but slightly differ
>>> in the generated code. That might be due to now we really generate Java11++
>>> code whereas before we only generated Java 1.1 code (yes, not even 1.5!)
>>>
>>> What I usually do is to have 2 checkouts. You might want to change the pom
>>> GAV version of the working checkout to something something different so you
>>> might have both in your ~/.m2/repository in parallel.
>>>
>>> The first thing I do is to compare the class files in target/classes of
>>> both versions with 'compare with clipboard'.
>>> I just open the class file in target/classes and let Intellij/Netbeans do
>>> the decompilation for me.
>>>
>>> You might find some differences here in which case this needs further
>>> investigation. Note that the order of the methods might have changed.
>>> Sometimes you will also see some 'cannot decompile' in the code block. This
>>> happens when something in the bytecode is messed up. You can also watch out
>>> for ClassNotFound or VerifyError, etc.
>>>
>>> If this is the case I then use
>>> $> javap -c -p -constants target/classes/com/foo/mycorp/MyBlaEntity.class >
>>> MyBlaEntity.decompile
>>> on both versions and check whether something is different.
>>> Often it's a wrong offset in the stack calculation (e.g. ALOAD 1 vs ALOAD 2)
>>>
>>> Note that the 'old' Java 1.1 bytecode doesn't understand LDC of Class
>>> constants (LoaD Constant). This was only introduced in Java 1.4 or 1.5
>>> bytecode afair. Before that a dynamic Class.forName was called and stored
>>> in a static variable. Ugly stuff, and happy to finally get rid of all those
>>> hacks...
>>>
>>> Hope that helps to get you started. Feel free to ping me again over here or
>>> at the #apachee ASF slack channel.
>>>
>>> txs and LieGrue,
>>> strub
>>>
>>>
>>>
>>>> Am 03.07.2023 um 14:13 schrieb Francesco Chicchiriccò
>>>> <ilgro...@apache.org>:
>>>>
>>>> From a very preliminary analysis it seems that just a subset of the
>>>> persistence-capable classes are effectively found and enhanced: now [2],
>>>> it used to be [3].
>>>>
>>>> I'll continue investigating.
>>>>
>>>> Regards.
>>>>
>>>> On 03/07/23 13:56, Francesco Chicchiriccò wrote:
>>>>> Hi Mark,
>>>>> the build on Syncope master branch, using OpenJPA 4.0.0-SNAPSHOT is now
>>>>> failing - it used to work at least until last June 30th.
>>>>>
>>>>> The issues seem to be related to entity enhancement, which we perform via
>>>>> openjpa-maven-plugin - see [1].
>>>>>
>>>>> Is there anything obvious we should be changing?
>>>>> Meanwhile I'll try to dig to see if I am able to understand something
>>>>> more.
>>>>>
>>>>> Regards.
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/syncope/blob/master/core/persistence-jpa/pom.xml#L143-L174
>>>> [2] https://paste.apache.org/gxcm6
>>>> [3] https://paste.apache.org/1g9k5
>>>>
>>>>> On 03/07/23 12:15, Mark Struberg wrote:
>>>>>> Update: I've now merged the wip to our master and pushed it to the ASF
>>>>>> gitox repo.
>>>>>>
>>>>>> I think we already came pretty far and it looks really promising. So
>>>>>> there is a high chance that we succeed, although there is still plenty
>>>>>> of work in front of us.
>>>>>>
>>>>>> How it works:
>>>>>>
>>>>>> Basically there are 2 sets of information in the PCEnhancer right now:
>>>>>>
>>>>>> _managed: the BCClass version of the original class
>>>>>> _pc: the BCClass of the generated/modified class. Might be the same as
>>>>>> _managed but in case of subclassing or interface case it is different.
>>>>>> In those cases _managed represents the original class and _pc the
>>>>>> subclass/concrete implementation.
>>>>>>
>>>>>> managed and pc are the ASM variants of _managed and _pc.
>>>>>>
>>>>>> The main entry point for debugging into it is PCEnhancer#run()
>>>>>>
>>>>>> There are a few methods in ASMHelper to update from BCClass -> ASM and
>>>>>> vice versa:
>>>>>>
>>>>>> AsmHelper.readIntoBCClass(pc, _pc);
>>>>>> takes all the information from the ASM pc, creates a class byte[] and
>>>>>> reads that back into the existing _pc instance.
>>>>>>
>>>>>> pc = AsmHelper.toClassNode(_pc);
>>>>>> is the corresponding method in the other direction. from BCClass to ASM.
>>>>>>
>>>>>> Right now we have the following methods left to migrate in PCEnhancer
>>>>>> addAttachDetachCode();
>>>>>> addSerializationCode();
>>>>>> addCloningCode();
>>>>>> runAuxiliaryEnhancers();
>>>>>>
>>>>>>
>>>>>> I'll be off for a week on holidays in Venice.
>>>>>> If someone want's to join in with the effort then please go on! You
>>>>>> might want to compare the current PCEnhancer code with an older version
>>>>>> to get an idea how it can be done and to see some patterns I've used.
>>>>>>
>>>>>> As a quick hint: Java is basically a stack based virtual CPU. The 'this'
>>>>>> pointer is on position 0 of the stack, so ALOAD 0. After that comes the
>>>>>> parameters. The size of the param on the stack depends on the type.
>>>>>> Mostly 1, but for long eg 2 positions. So ALOAD 1 is the 1st method
>>>>>> parameter, ALOAD 2 the 2nd (except 1st param was a 2-pos type).
>>>>>> In case of a static method there is ofc no 'this' on the stack, so the
>>>>>> parameters start with zero offset.
>>>>>> After all the parameters there are the unnamed 'local' variables. I
>>>>>> tried to consistently name them xxxVarPos.
>>>>>> See also the various helper methods in ASMHelper. For example for the
>>>>>> various LOAD, RETURN, STORE operations which depends based on the type
>>>>>> (ALOAD vs ILOAD vs IALOAD etc)
>>>>>> Also read the guide to ASM PDF from objectweb.
>>>>>>
>>>>>> I'll be mostly offline till Sunday, but will try to answer mails if
>>>>>> there are some questions.
>>>>>>
>>>>>> txs and LieGrue,
>>>>>> strub
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Am 19.06.2023 um 17:11 schrieb Matt Pavlovich <mattr...@gmail.com>:
>>>>>>>
>>>>>>> +1 thanks for tackling this, Mark!
>>>>>>>
>>>>>>> ASM definitely more widely used going forward.
>>>>>>>
>>>>>>> -Matt
>>>>>>>
>>>>>>>> On Jun 17, 2023, at 3:01 PM, Mark Struberg <strub...@yahoo.de.INVALID>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Small update:
>>>>>>>>
>>>>>>>> I've worked on it over the last few weeks, and I'm getting closer
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance
>>>>>>>>
>>>>>>>> contains the latest work on the PCEnhancer. Right in the middle of
>>>>>>>> replacing Serp with native ASM code.
>>>>>>>>
>>>>>>>> Note that I also had to modify a few interfaces with a few more to
>>>>>>>> follow from BCClass to ASM ClassNode.
>>>>>>>>
>>>>>>>> Happy to get some feedback!
>>>>>>>>
>>>>>>>> If anybody wants to help with this effort I'm happy to also push this
>>>>>>>> feature branch to our ASF repo. It looks reasonably promising already.
>>>>>>>>
>>>>>>>> LieGrue,
>>>>>>>> strub
>>>>>>>>
>>>>>>>>
>>>>>>>>> Am 25.05.2023 um 18:36 schrieb Mark Struberg
>>>>>>>>> <strub...@yahoo.de.INVALID>:
>>>>>>>>>
>>>>>>>>> Hi!
>>>>>>>>>
>>>>>>>>> Right now I'm trying to get rid of serp step by step.
>>>>>>>>> The code is right now at my own github repo in the fb_asmEnhance
>>>>>>>>> branch:
>>>>>>>>> https://github.com/struberg/openjpa/tree/fb_asmEnhance
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The approach I took for now is to create a few methods in AsmHelper
>>>>>>>>> to be able to move from BCClass -> ASM ClassWriter and the other way
>>>>>>>>> around. That way we should be able to replace functionality part by
>>>>>>>>> part but still keep all things afloat.
>>>>>>>>>
>>>>>>>>> For now I've started with the PCSubclassValidator.
>>>>>>>>> Right now this evaluates the attributs using Serp plus ASM and then
>>>>>>>>> compare the results.
>>>>>>>>> If something is fishy, you'll see the following
>>>>>>>>> throw new IllegalStateException("MSX ASMTODO " + bcField + " " +
>>>>>>>>> field);
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It would be extremely helpful, if you could go through your projects
>>>>>>>>> and let it run and report how it's going.
>>>>>>>>> If you see that "MSX ASMTODO" somewhere then we know I messed
>>>>>>>>> something up.
>>>>>>>>>
>>>>>>>>> I'll gonna finally remove the BCClass handling from those parts in a
>>>>>>>>> few days. Current commit is
>>>>>>>>> https://github.com/struberg/openjpa/commit/3ea2412003028d37f2a69971a47bb20abf589f8b
>>>>>>>>>
>>>>>>>>> txs and LieGrue,
>>>>>>>>> strub
>>
>> --
>> Francesco Chicchiriccò
>>
>> Tirasa - Open Source Excellence
>> http://www.tirasa.net/
>>
>> Member at The Apache Software Foundation
>> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
>> http://home.apache.org/~ilgrosso/
>>
>
