Hi John, 1. The @InverseLogical annotation shouldn't affect your scenario. I was just curious as to why you were using it. My ignorance of the usefulness of this property would shy me away from its usage, but that's just me.
2. Thanks for the explanation of your update() method. This sounds like the proper usage of the merge() processing. 3. Thanks. Just curious on your persistence model. 4. Your persistence.xml looks fine. Nothing else jumps out at me. This leaves us at figuring out a simple testcase to reproduce the problem. Since we have many JUnits with caching and collections already, I'm surprised that we haven't discovered the issue prior to this. I'm not sure if I'll have time today to create or modify or JUnit to reproduce the issue (taking some vacation). If you have any cycles to simplify your scenario, it would help. Otherwise, maybe early next week we can get to the bottom of this... I did look into your comments concerning the openjpa.RefreshFromDataCache property... Unfortunately, this property doesn't seem to be working as you might expect. First, it's not documented in our OpenJPA manual. And, when I look at the code, it doesn't look like the value of this property is being used properly to bypass the cache. So, I wouldn't count on this. I then looked at the proper means of specifying a datacache bypass with the javax.persistence.cache.RetrieveMode properties as defined by JPA 2.0. You can specify a value of USE or BYPASS that can be used for finds, queries, and refresh operations. But, then there's this statement in the spec: *"The retrieveMode property is ignored for the refresh method,* *which always causes data to be retrieved from the database, not the cache." * You have mentioned that you are performing a refresh. Are you calling em.refresh(object)? If so, this sounds like there might be a bug in that processing... You could try passing in the RetrieveMode property of BYPASS on the refresh just to see if it makes a difference. That's all I got right now... Have a good weekend! Kevin On Fri, Jun 8, 2012 at 1:57 AM, Boblitz John <john.bobl...@bertschi.com>wrote: > Hi Kevin, > > Thanks for the response. Here some answers: > > 1. "Why is the @InverseLogical annotation required?" > I included this quite early in the development of as the manual suggested > that > using the annotation would cause openJpa to manage the relations for me. > Are you implying that mappedBy in uneeded if I use @InverseLogical? > I could remove the annotation and check - but this is prevalent > throuhout the model, and I have not noticed any problems elsewhere (yet). > > 2. "Can you explain the use of the update() method?" > update receives the detached entities which are then merged and committed. > The application is on three tiers in a Java SE Environment- DB -> App -> > Gui. The GUI > gets the data in one step and the enitities are detached. At some point > later, the data is > passed back to the app. The process is, basically: > > Get a manager > Read the data from a DTO Object > Validate the data > Create entity object and fill with DTO Data > Merge into the context > Commit > > I am stuck with using separate DTO Objects which unfortunately also break > the references as only the > uid is passed - but, other than the overhead, I have noticed no problems > in doing so > > 3. "How is the lifecycle of the EM being managed, ...?" > As I mentioned in 2, since I'm in SE, each request essentially gets it's > own manager. > > 4. "Can you post your whole persistence.xml? " > Below my persistence.xml. Yes, the QueryCache & QuerySQLCache are active. > > I just reran the scenario and produced a trace (some very nasty sqls in > there as the apps are > all still using default fetch plans and the Equipment entity is complex :D > ) > > Essentially, when the merge() is executed - several selects are executed > and since > nothing changed yet, all info is pulled from cache. > > Then, several updates / inserts are executed as expected. Just after > these statements I get another > list of hits on the cache ... > > Then I get this: (413938 is the Equipment Entity and 502801 is the new > Axle] > > - Performing a commit on the cache. Adding [502801], updating [] and > [414690, 414571, 414570, 414082, 414688, 414689, 414626, 414627, 414624, > 413938, 414572, 414625, 414681, > 414621, 414680, 414623, 414683, 414622, 414682, 414685, 414684, 414687, > 414686, 414679], and removing []. > > Just after that, I do a refresh and this shows: > - Cache hit while looking up key "413938". > > Here I expected that the cache would not be used and that the DB would be > read to resynch the cache since > I have: openjpa.RefreshFromDataCache: false > > > > I appreciate the help!!! > >  > > John > > ---- > > Who is General Failure, and why is he reading my hard disk? > > > > <?xml version="1.0" encoding="UTF-8"?> > <persistence > version="2.0" > xmlns="http://java.sun.com/xml/ns/persistence"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://java.sun.com/xml/ns/persistence > > http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"; > > > > <persistence-unit > name="g11.persistence" > transaction-type="RESOURCE_LOCAL" > > > > > <provider>org.apache.openjpa.persistence.PersistenceProviderImpl > </provider> > > > <!-- JPQL Named Queries --> > <mapping-file>META-INF/dbNamedQueries.xml</mapping-file> > <!-- JPQL Named Queries generated from the database schema.xml --> > <mapping-file>META-INF/guiNamedQueries.xml</mapping-file> > <!-- JPQL Named Queries generated for gui selection screens --> > <mapping-file>META-INF/acNamedQueries.xml</mapping-file> > <!-- JPQL Named Queries generated for gui selection screens --> > <mapping-file>META-INF/bdisNamedQueries.xml</mapping-file> > <!-- JPQL Named Queries generated for general bdis queries --> > > > > <!-- Class Definitions --> > <class> base.BaseEntity</class> > <class> ams.locations.BusinessPartner</class> > <class> ams.locations.Country</class> > <class> ams.locations.CountryGroup</class> > <class> ams.locations.Location</class> > <class> ams.locations.Subdivision</class> > <class> ams.locations.ValueAddedTax</class> > <class> com.codes.Currency</class> > <class> com.codes.ExchangeRate</class> > <class> com.codes.ModeOfTransportation</class> > <class> com.codes.Notes</class> > <class> com.codes.Period</class> > <class> com.codes.Text</class> > <class> com.codes.TranslatedText</class> > <class> com.codes.UnitOfMeasurement</class> > <class> com.codes.WritingSystem</class> > <class> com.company.BusinessUnit</class> > <class> com.company.Company</class> > <class> com.company.Department</class> > <class> com.company.OrganisationalUnit</class> > <class> com.company.ServiceCenter</class> > <class> com.system.Printer</class> > <class> com.system.UserGroup</class> > <class> com.system.Users</class> > <class> com.system.Permission</class> > <class> com.system.EntityExport</class> > <class> fms.equipment.Equipment</class> > <class> fms.equipment.EquipmentAttachment</class> > <class> fms.equipment.EquipmentNote</class> > <class> fms.equipment.HireOut</class> > <class> fms.equipment.RentalAgreement</class> > <class> fms.equipment.TransferAgreement</class> > <class> fms.equipment.certifications.Certification</class> > <class> > fms.equipment.certifications.CertificationType</class> > <class> > fms.equipment.certifications.DriverCertification</class> > <class> > fms.equipment.certifications.DriverCertificationDocument</class> > <class> > fms.equipment.certifications.DriverRequiredCertifications</class> > <class> > fms.equipment.certifications.EquipmentCertification</class> > <class> > fms.equipment.certifications.EquipmentCertificationDocument</class> > <class> > fms.equipment.certifications.EquipmentRequiredCertifications</class> > <class> fms.equipment.certifications.Issuer</class> > <class> fms.equipment.components.Axle</class> > <class> fms.equipment.components.Chamber</class> > <class> fms.equipment.components.Chassis</class> > <class> fms.equipment.components.TankBoxUsageHistory</class> > <class> fms.equipment.components.Hose</class> > <class> fms.equipment.components.Motor</class> > <class> fms.equipment.components.MotorUsage</class> > <class> fms.equipment.components.TankBox</class> > <class> fms.equipment.damage.DamageHow</class> > <class> fms.equipment.damage.DamageWhat</class> > <class> fms.equipment.damage.DamageWhere</class> > <class> fms.equipment.damage.EquipmentDamage</class> > <class> fms.equipment.properties.Characteristic</class> > <class> fms.equipment.properties.CharacteristicType</class> > <class> fms.equipment.properties.EquipmentType</class> > <class> fms.equipment.properties.Manufacturer</class> > <class> fms.equipment.properties.Model</class> > <class> fms.equipment.properties.PropertyType</class> > <class> fms.equipment.properties.Specification</class> > <class> fms.equipment.properties.SpecificationType</class> > <class> > fms.equipment.properties.TechnicalCharacteristic</class> > <class> fms.equipment.service.Service</class> > <class> fms.equipment.service.ServiceHistory</class> > <class> fms.equipment.service.ServiceType</class> > <class> fms.equipment.service.ServicesOffered</class> > <class> hrs.Driver</class> > <class> hrs.DriverRestriction</class> > <class> hrs.DriverVacationModel</class> > <class> hrs.Employee</class> > <class> hrs.RestrictionType</class> > > > <properties> > > <!-- Logging --> > <property > name="openjpa.Log" > value="log4j" /> > > <!-- Connection --> > <property > name="openjpa.ConnectionDriverName" > value="org.postgresql.Driver" /> > <property > name="openjpa.ConnectionProperties" > value="MaxActive=100, MaxIdle=5, MinIdle=2, > MaxWait=60000" /> > <property > name="openjpa.ConnectionFactoryProperties" > value="QueryTimeOut=5000, PrettyPrint=true, > PrettyPrintLineLength=80, PrintParameters=true" /> > > <property > name="openjpa.ConnectionURL" > value=**** /> > <property > name="openjpa.ConnectionUserName" > value=**** /> > <property > name="openjpa.ConnectionPassword" > value=**** /> > <property > name="openjpa.jdbc.DBDictionary" > > value="postgres(supportsNullTableForGetImportedKeys=false)" /> > > <!-- Caching --> > <property > name="openjpa.DataCache" > value="true(CacheSize=5000, > EnableStatistics=true)" /> > <property > name="openjpa.RemoteCommitProvider" > value="sjvm" /> > <property > name="openjpa.QueryCache" > value="true(CacheSize=1000, > SoftReferenceSize=100, EvictPolicy='timestamp')" /> > <property > name="openjpa.QueryCompilationCache" > value="all" /> > <property > name="openjpa.jdbc.QuerySQLCache" > value="true(EnableStatistics=true)" /> > > > <!-- Misc --> > <property > name="openjpa.InverseManager" > value="true" /> > <property > name="openjpa.DetachState" > > value="fetch-groups(DetachedStateField=true)" /> > <property > name="openjpa.jdbc.SchemaFactory" > value="native(ForeignKeys=true)" /> > > </properties> > </persistence-unit> > </persistence> > > > > > > > -----Ursprüngliche Nachricht----- > > Von: Kevin Sutter [mailto:kwsut...@gmail.com] > > Gesendet: Donnerstag, 7. Juni 2012 20:48 > > An: users@openjpa.apache.org > > Betreff: Re: Bug? DataCache after update contains only the > > updated information from a Collection > > > > Hi John, > > Good background information. Thanks for the detail. > > Unfortunately, nothing is jumping out at me as an "easy fix". > > I do have a couple of observations and/or questions... > > > > o Why is the @InverseLogical annotation required? It would > > seem that you have already defined the bidirectional > > relationship via the mappedBy attribute on the @OneToMany. > > Is there something else you were trying to model by using the > > @InverseLogical? And, does the removal of that annotation > > change the processing in any way? > > > > o Can you explain the use of the update() method? Are the > > entities being passed into this method detached from a > > persistence context? That's where the merge() method comes > > into play -- merging detached entities into a persistence > > context. The merge() method is not used just because there > > are updates to an Entity. > > > > o Your code example doesn't show the actual interaction with > > the EM. How is the lifecycle of the EM being managed, and is > > the same EM being used for all of the interactions with > > transactions, persisting, merging, etc? > > > > o Can you post your whole persistence.xml? For example, you > > didn't mention the use of the QueryCache property, but it > > looks like this is enabled via the dump of the properties. > > The QueryCache is no longer automatically enabled when the > > DataCache is enabled, so it must be explicitly enabled. Not > > that it should affect this particular scenario, but I'm just > > curious if there are other property "mismatches" that I didn't catch. > > > > o If none of these questions provide any fruit, then you > > might have found a bug and a JIRA will be required. But, > > let's do a bit more Q&A first... > > > > Thanks, > > Kevin > > > > On Thu, Jun 7, 2012 at 2:05 AM, Boblitz John > > <john.bobl...@bertschi.com>wrote: > > > > > Hello, > > > > > > I'm running openJpa 2.2.0 and have a problem with the > > DataCache function. > > > I spent the better part of two days localizing the problem > > and looking > > > for solutions, but alas ... > > > > > > So, hoping some guru out there can spot my error, here the details: > > > > > > Environment JavaSE > > > > > > Persistence XML: > > > <property name="openjpa.DataCache" value="true(CacheSize=5000, > > > EnableStatistics=true)" /> <property > > > name="openjpa.RemoteCommitProvider" value="sjvm" /> <property > > > name="openjpa.DetachState" > > > value="fetch-groups(DetachedStateField=true)" /> > > > > > > Entities: > > > > > > @MappedSuperclass > > > @EntityListeners({ EntityManipulationLogger.class, > > EntityLogger.class > > > }) public abstract class BaseEntity { @Version > > > @Column(columnDefinition = "int8") private long versionId; > > > > > > @Id > > > @GeneratedValue(strategy = GenerationType.SEQUENCE) @Column(name = > > > "uniqueid", columnDefinition = "int8") protected long uniqueId; > > > > > > ... > > > > > > @Entity > > > @Table(name = "Equipment") > > > public class Equipment extends BaseEntity { > > > > > > @JsonManagedReference > > > @OneToMany(cascade = CascadeType.ALL, mappedBy = "equipmentId") > > > @InverseLogical("equipmentId") > > > private Set<Axle> axles = new HashSet<Axle>(); > > > > > > ... > > > > > > @Entity > > > @Table(name = "Axle") > > > public class Axle extends BaseEntity { > > > > > > @JsonBackReference > > > @ManyToOne(fetch = FetchType.EAGER) > > > @JoinColumn(name = "equipmentId", columnDefinition = > > "int8", nullable > > > = > > > false) > > > private Equipment equipmentId; > > > > > > > > > Code used to Update the DataBase: > > > > > > @Override > > > public <T extends BaseEntity> T update(T pEntity) { > > > this.lock.lock(); > > > T object = null; > > > try { > > > try { > > > getTransaction().begin(); > > > object = merge(pEntity); > > > } catch (Exception e) { > > > mTrc.error("Error in update(): ", e); > > > getTransaction().setRollbackOnly(); > > > handleException(e); > > > } finally { > > > if (getTransaction().isActive()) { > > > if (getTransaction().getRollbackOnly()) { > > > getTransaction().rollback(); > > > } else { > > > getTransaction().commit(); > > > } > > > } > > > } > > > } catch (Exception e) { > > > mTrc.error("Error in update(): ", e); > > > handleException(e); > > > } finally { > > > this.lock.unlock(); > > > } > > > return object; > > > } > > > > > > Symptom: > > > When updating the Equipment entity, if an additional Axle > > is added to > > > the Set and the other Axles remain unchanged: > > > 1. Upon entry into update(), pEntity contains all the Axels (new & > > > old) including UID & Version. > > > 2. object = merge(pEntity) - performs as expected, object contains > > > all the Axels and the new Axel has been assigned a UID 3. After > > > commit(), the new Axle is added to the Database (good) 4. > > The entity > > > is updated in the DataCache (I would assume this is good as > > > well) > > > 5. "object" however now only contains one element in the > > Set<Axle> - > > > the one we added, the others are no longer there. > > > 6. A subsequent refresh of the returns the same results as > > in #5! It > > > hits in the DataCache, so I assume that the Update of the Cache > > > contains only the "new" data from object. > > > > > > Turning off the DataCache (or excluding the Equipment > > Entity) solves > > > the problem. > > > > > > Here the complete config: > > > > > > openjpa.AutoClear: 0 > > > openjpa.AutoDetach: [Ljava.lang.String;@60cf710e > > > openjpa.BrokerFactory: jdbc > > > openjpa.BrokerImpl: default > > > openjpa.CacheDistributionPolicy: default > > > openjpa.Callbacks: default > > > openjpa.ClassResolver: default > > > openjpa.Compatibility: default > > > openjpa.ConnectionDriverName: org.postgresql.Driver > > > openjpa.ConnectionFactoryMode: false > > > openjpa.ConnectionFactoryProperties: QueryTimeOut=5000, > > > PrettyPrint=true, PrettyPrintLineLength=80, PrintParameters=true > > > openjpa.ConnectionPassword: ****** > > > openjpa.ConnectionProperties: MaxActive=100, MaxIdle=5, MinIdle=2, > > > MaxWait=60000 > > > openjpa.ConnectionRetainMode: 0 > > > openjpa.ConnectionURL: ****** > > > openjpa.ConnectionUserName: ***** > > > openjpa.DataCache: true(CacheSize=5000, EnableStatistics=true) > > > openjpa.DataCacheManager: default > > > openjpa.DataCacheTimeout: -1 > > > openjpa.DetachState: fgs(DetachedStateField=true) > > > openjpa.DynamicDataStructs: false > > > openjpa.DynamicEnhancementAgent: true > > > openjpa.EntityManagerFactory: default > > > openjpa.FetchBatchSize: -1 > > > openjpa.FetchGroups: [Ljava.lang.String;@53077fc9 > > > openjpa.FlushBeforeQueries: 0 > > > openjpa.Id: g11.persistence > > > openjpa.IgnoreChanges: false > > > openjpa.InitializeEagerly: false > > > openjpa.InstrumentationManager: default > > > openjpa.InverseManager: true > > > openjpa.LifecycleEventManager: validating > > > openjpa.LockManager: mixed > > > openjpa.Log: log4j > > > openjpa.ManagedRuntime: auto > > > openjpa.MaxFetchDepth: -1 > > > openjpa.MetaDataFactory: *** truncated!! > > > openjpa.MetaDataRepository: default > > > openjpa.Multithreaded: false > > > openjpa.NontransactionalRead: true > > > openjpa.NontransactionalWrite: true > > > openjpa.Optimistic: true > > > openjpa.OrphanedKeyAction: log > > > openjpa.ProxyManager: default > > > openjpa.QueryCache: true(CacheSize=1000, SoftReferenceSize=100, > > > EvictPolicy='timestamp') > > > openjpa.QueryCompilationCache: all > > > openjpa.ReadLockLevel: 10 > > > openjpa.RefreshFromDataCache: false > > > openjpa.RemoteCommitProvider: sjvm > > > openjpa.RestoreState: 1 > > > openjpa.RetainState: true > > > openjpa.RetryClassRegistration: false > > > openjpa.RuntimeUnenhancedClasses: 1 > > > openjpa.SavepointManager: in-mem > > > openjpa.Sequence: table > > > openjpa.TransactionMode: false > > > openjpa.WriteLockLevel: 20 > > > openjpa.jdbc.DBDictionary: > > > postgres(supportsNullTableForGetImportedKeys=false) > > > openjpa.jdbc.DriverDataSource: auto > > > openjpa.jdbc.EagerFetchMode: 2 > > > openjpa.jdbc.FetchDirection: 1000 > > > openjpa.jdbc.FinderCache: true > > > openjpa.jdbc.IdentifierUtil: default > > > openjpa.jdbc.LRSSize: 2 > > > openjpa.jdbc.MappingDefaults: jpa > > > openjpa.jdbc.QuerySQLCache: true(EnableStatistics=true) > > > openjpa.jdbc.ResultSetType: 1003 > > > openjpa.jdbc.SQLFactory: default > > > openjpa.jdbc.SchemaFactory: native(ForeignKeys=true) > > > openjpa.jdbc.Schemas: [Ljava.lang.String;@60cf710e > > > openjpa.jdbc.SubclassFetchMode: 1 > > > openjpa.jdbc.SynchronizeMappings: null > > > openjpa.jdbc.TransactionIsolation: -1 > > > openjpa.jdbc.UpdateManager: default > > > > > > > > > > > > > > > > > >  > > > > > > John > > > > > > ---- > > > > > > Who is General Failure, and why is he reading my hard disk? > > > > > > > > >
