Re: Issue with enhancement of annotations

[Craig Russell]

Hi Andy,

> On Jul 21, 2017, at 8:05 AM, Andy Jefferson  wrote:
> 
>> I updated the pom.xml to 5.1.1-SNAPSHOT and the enhancer step worked but
>> many tck tests fail. [1]
> 
> I'd suggest that is down to whatever is different in your TCK from the SVN 
> trunk TCK ...

Clearly this is the case. The class/interface/annotation that is being 
complained about is the new composite annotation.

The failure is in a subclass of a class that has the new annotation. 
FCDSMedicalInsurance extends FCDSInsurance.

I've prepared a patch that shows the error. Can you please try it and let me 
know what you think the problem is.

https://issues.apache.org/jira/browse/JDO-764

Thanks,

Craig

[1]
Insert of object 
"org.apache.jdo.tck.pc.companyAnnotatedFC.FCDSMedicalInsurance@5553d0f5" using 
statement "INSERT INTO DATASTOREIDENTITY0.FCDSINSURANCE 
(CARRIER,EMPLOYEE,INSID,PLANTYPE,DISCRIMINATOR) VALUES (?,?,?,?,?)" failed : 
Table/View 'DATASTOREIDENTITY0.FCDSINSURANCE' does not exist.
org.datanucleus.exceptions.NucleusDataStoreException: Insert of object 
"org.apache.jdo.tck.pc.companyAnnotatedFC.FCDSMedicalInsurance@5553d0f5" using 
statement "INSERT INTO DATASTOREIDENTITY0.FCDSINSURANCE 
(CARRIER,EMPLOYEE,INSID,PLANTYPE,DISCRIMINATOR) VALUES (?,?,?,?,?)" failed : 
Table/View 'DATASTOREIDENTITY0.FCDSINSURANCE' does not exist.


> because SVN trunk works fine with current DN code, and the only 
> change in my fix is ignoring classes that are annotations, hence no 
> persistable classes will be ignored due to this check.
> 
> 
> Regards
> -- 
> Andy
> DataNucleus (Web: http://www.datanucleus.org   Twitter: @datanucleus)

Craig L Russell
c...@apache.org

Re: Issue with enhancement of annotations

[Andy Jefferson]

> I updated the pom.xml to 5.1.1-SNAPSHOT and the enhancer step worked but
> many tck tests fail. [1]

I'd suggest that is down to whatever is different in your TCK from the SVN 
trunk TCK ... because SVN trunk works fine with current DN code, and the only 
change in my fix is ignoring classes that are annotations, hence no 
persistable classes will be ignored due to this check.


Regards
-- 
Andy
DataNucleus (Web: http://www.datanucleus.org   Twitter: @datanucleus)

Re: Issue with enhancement of annotations

[Craig Russell]

Hi Andy,

> On Jul 20, 2017, at 11:40 PM, Andy Jefferson  wrote:
> 
> Hi Craig,
> 
>> It looks like the DN enhancer recognizes the annotation as a
>> persistence-capable class because it is annotated with PersistenceCapable
>> but doesn't recognize it as an annotation that should not be enhanced.
> 
> Quite probably. I've only been enhancing through a persistence.xml file 
> recently, which explicitly specifies the classes to be enhanced so never hit 
> that one.
> 
>> I looked at the code and think that perhaps a patch like this would fix it.
>> org.datanucleus.enhancer.ClassEnhancerImpl around line 298 after the cls
>> object is obtained:
>> 
>>if (cls.isAnnotation())
>>{
>>return false;
>>}
>> 
>> Other possible solutions: move the composite annotations out of the package
>> into a sibling package. This probably works but is awkward.
> 
> No, let's just fix the code rather than imposing the user has to do something.

I agree. That didn't work anyway (see subsequent message for details).
> 
> Instead of doing your proposed fix, I put a couple of checks in further 
> upstream, where it reads annotations for a class to check if it is has 
> metadata. 
> 
> See https://github.com/datanucleus/datanucleus-core/commit/
> f048a37494066b46545a1849d536104dc149950d
> 
> A simple test with that fix, explicitly adding the meta-annotation class to 
> persistence.xml results in a log message that the class has no metadata and 
> is 
> ignored.
> 
> That should be in current DN nightly builds, so give it a try and see if it 
> catches your situation. 

I updated the pom.xml to 5.1.1-SNAPSHOT and the enhancer step worked but many 
tck tests fail. [1]

Thanks,

Craig

09:58:35,295 (main) INFO  [DataNucleus.JDO] - Exception thrown
Class 
"org.apache.jdo.tck.pc.companyAnnotatedFC.DatastoreIdDiscriminatorClassNameInheritanceSuperclass"
 has been specified with an inheritance strategy of "superclass-table", yet no 
superclass exists or none exists with its own table!
org.datanucleus.metadata.InvalidClassMetaDataException: Class 
"org.apache.jdo.tck.pc.companyAnnotatedFC.DatastoreIdDiscriminatorClassNameInheritanceSuperclass"
 has been specified with an inheritance strategy of "superclass-table", yet no 
superclass exists or none exists with its own table!
at 
org.datanucleus.metadata.AbstractClassMetaData.validateUserInputForInheritanceMetaData(AbstractClassMetaData.java:874)
at 
org.datanucleus.metadata.InterfaceMetaData.populate(InterfaceMetaData.java:120)
at 
org.datanucleus.metadata.MetaDataManagerImpl$1.run(MetaDataManagerImpl.java:2926)
at java.security.AccessController.doPrivileged(Native Method)
at 
org.datanucleus.metadata.MetaDataManagerImpl.populateAbstractClassMetaData(MetaDataManagerImpl.java:2920)
at 
org.datanucleus.metadata.MetaDataManagerImpl.populateFileMetaData(MetaDataManagerImpl.java:2724)
at 
org.datanucleus.metadata.MetaDataManagerImpl.initialiseFileMetaDataForUse(MetaDataManagerImpl.java:1365)
at 
org.datanucleus.metadata.MetaDataManagerImpl.loadClasses(MetaDataManagerImpl.java:645)
at 
org.datanucleus.enhancer.DataNucleusEnhancer.getFileMetadataForInput(DataNucleusEnhancer.java:672)
at 
org.datanucleus.enhancer.DataNucleusEnhancer.enhance(DataNucleusEnhancer.java:500)
at org.datanucleus.api.jdo.JDOEnhancer.enhance(JDOEnhancer.java:125)
at org.apache.jdo.exectck.Enhance.execute(Enhance.java:288)
at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at 
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at 
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 

Re: Issue with enhancement of annotations

[Andy Jefferson]

Hi Craig,

> It looks like the DN enhancer recognizes the annotation as a
> persistence-capable class because it is annotated with PersistenceCapable
> but doesn't recognize it as an annotation that should not be enhanced.

Quite probably. I've only been enhancing through a persistence.xml file 
recently, which explicitly specifies the classes to be enhanced so never hit 
that one.

> I looked at the code and think that perhaps a patch like this would fix it.
> org.datanucleus.enhancer.ClassEnhancerImpl around line 298 after the cls
> object is obtained:
> 
> if (cls.isAnnotation())
> {
> return false;
> }
> 
> Other possible solutions: move the composite annotations out of the package
> into a sibling package. This probably works but is awkward.

No, let's just fix the code rather than imposing the user has to do something.

Instead of doing your proposed fix, I put a couple of checks in further 
upstream, where it reads annotations for a class to check if it is has 
metadata. 

See https://github.com/datanucleus/datanucleus-core/commit/
f048a37494066b46545a1849d536104dc149950d

A simple test with that fix, explicitly adding the meta-annotation class to 
persistence.xml results in a log message that the class has no metadata and is 
ignored.

That should be in current DN nightly builds, so give it a try and see if it 
catches your situation. 




Regards
-- 
Andy
DataNucleus (Web: http://www.datanucleus.org   Twitter: @datanucleus)

Re: Issue with enhancement of annotations

[Craig Russell]

I experimented with putting the annotations into a different directory that has 
nothing but annotations.

Even though the annotations are only referenced by their use in persistent 
classes, the DataNucleus enhancer still wants to enhance them.

[ERROR] Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance 
(default) on project jdo-tck: Execution default of goal 
org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance failed: Class 
"org.apache.jdo.tck.pc.compositeAnnotation.DatastoreIdDiscriminatorClassNameInheritanceSuperclass"
 has been specified with an inheritance strategy of "superclass-table", yet no 
superclass exists or none exists with its own table! -> [Help 1]

So it looks like we will need to change DataNucleus enhancer in order for these 
tests to work.

Craig

> On Jul 20, 2017, at 12:03 PM, Craig Russell  wrote:
> 
> Hi Andy,
> 
> It looks like the DN enhancer recognizes the annotation as a 
> persistence-capable class because it is annotated with PersistenceCapable but 
> doesn't recognize it as an annotation that should not be enhanced.
> 
> I looked at the code and think that perhaps a patch like this would fix it.
> org.datanucleus.enhancer.ClassEnhancerImpl around line 298 after the cls 
> object is obtained:
> 
>if (cls.isAnnotation())
>{
>return false;
>}
> 
> Other possible solutions: move the composite annotations out of the package 
> into a sibling package. This probably works but is awkward.
> 
> We could try to remove the annotation classes from the list of classes to 
> enhance, but this would really slow things down by needing to get the class 
> object from the class names even though it isn't really needed for anything 
> else.
> 
> Thanks,
> 
> Craig
> 
>> On Jul 19, 2017, at 5:46 PM, Craig Russell  wrote:
>> 
>> Hi,
>> 
>> I'm trying to run the tck including this composite annotation:
>> 
>> @Target(ElementType.TYPE)
>> @Retention(RetentionPolicy.RUNTIME)
>> @javax.jdo.annotations.PersistenceCapable(detachable="true", 
>> identityType=IdentityType.DATASTORE)
>> @javax.jdo.annotations.DatastoreIdentity(strategy=IdGeneratorStrategy.IDENTITY,
>>  column="DATASTORE_IDENTITY")
>> @javax.jdo.annotations.Discriminator(strategy=DiscriminatorStrategy.CLASS_NAME,
>>  column="DISCRIMINATOR", indexed="true")
>> @javax.jdo.annotations.Inheritance(strategy=InheritanceStrategy.SUPERCLASS_TABLE)
>> 
>> public @interface DatastoreIdDiscriminatorClassNameInheritanceSuperclass { }
>> 
>> When I run the enhancement step, it complains:
>> 
>> Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance 
>> (default) on project jdo-tck: Execution default of goal 
>> org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance failed: Class 
>> "org.apache.jdo.tck.pc.companyAnnotatedFC.DatastoreIdDiscriminatorClassNameInheritanceSuperclass"
>>  has been specified with an inheritance strategy of "superclass-table", yet 
>> no superclass exists or none exists with its own table! -> [Help 1]
>> 
>> It looks like the enhancer is trying to enhance this class, but it should 
>> not. An annotation cannot be persistent so it should not be enhanced.
>> 
>> Any ideas?
>> 
>> Thanks,
>> 
>> Craig
>> 
>> Craig L Russell
>> c...@apache.org
>> 
> 
> Craig L Russell
> Secretary, Apache Software Foundation
> c...@apache.org http://db.apache.org/jdo
> 

Craig L Russell
Secretary, Apache Software Foundation
c...@apache.org http://db.apache.org/jdo

Re: Issue with enhancement of annotations

[Craig Russell]

Hi Andy,

It looks like the DN enhancer recognizes the annotation as a 
persistence-capable class because it is annotated with PersistenceCapable but 
doesn't recognize it as an annotation that should not be enhanced.

I looked at the code and think that perhaps a patch like this would fix it.
org.datanucleus.enhancer.ClassEnhancerImpl around line 298 after the cls object 
is obtained:

if (cls.isAnnotation())
{
return false;
}

Other possible solutions: move the composite annotations out of the package 
into a sibling package. This probably works but is awkward.

We could try to remove the annotation classes from the list of classes to 
enhance, but this would really slow things down by needing to get the class 
object from the class names even though it isn't really needed for anything 
else.

Thanks,

Craig

> On Jul 19, 2017, at 5:46 PM, Craig Russell  wrote:
> 
> Hi,
> 
> I'm trying to run the tck including this composite annotation:
> 
> @Target(ElementType.TYPE)
> @Retention(RetentionPolicy.RUNTIME)
> @javax.jdo.annotations.PersistenceCapable(detachable="true", 
> identityType=IdentityType.DATASTORE)
> @javax.jdo.annotations.DatastoreIdentity(strategy=IdGeneratorStrategy.IDENTITY,
>  column="DATASTORE_IDENTITY")
> @javax.jdo.annotations.Discriminator(strategy=DiscriminatorStrategy.CLASS_NAME,
>  column="DISCRIMINATOR", indexed="true")
> @javax.jdo.annotations.Inheritance(strategy=InheritanceStrategy.SUPERCLASS_TABLE)
> 
> public @interface DatastoreIdDiscriminatorClassNameInheritanceSuperclass { }
> 
> When I run the enhancement step, it complains:
> 
> Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance 
> (default) on project jdo-tck: Execution default of goal 
> org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:enhance failed: Class 
> "org.apache.jdo.tck.pc.companyAnnotatedFC.DatastoreIdDiscriminatorClassNameInheritanceSuperclass"
>  has been specified with an inheritance strategy of "superclass-table", yet 
> no superclass exists or none exists with its own table! -> [Help 1]
> 
> It looks like the enhancer is trying to enhance this class, but it should 
> not. An annotation cannot be persistent so it should not be enhanced.
> 
> Any ideas?
> 
> Thanks,
> 
> Craig
> 
> Craig L Russell
> c...@apache.org
> 

Craig L Russell
Secretary, Apache Software Foundation
c...@apache.org http://db.apache.org/jdo