Re: Audit log with OpenJPA
Pinaki, I've been on vacation again for a week which is why I'm late replying. I think there must be a way to get hold of the original object since that's the object I need to audit log. An unmanaged POJO is fine for my purposes since I just want to serialize it. I will then persist it to a separate audit log table using JPA as I showed before. Just providing the dirty fields is not enough for most people's audit logging requirements (in my opinon...). It seems to me there are two main routes to go here: a) Provide the plumbing necessary for developers to roll their own audit log. Then the following is needed: - Access to the current and previous versions of the POJO (since I want to log the whole previous version) - Information about what fields are dirty (if you only want to log what has been changed) - Support for creating new persistent entities in the life cycle callback. Ideally using the same entity manager. At least it must be possible to use the same transaction otherwise the audit log will not be reliable. b) Provide audit logging capabilities out-of-the-box. I think this is what Envers does. The implementation details are hidden but the developer is given an audit logging API to use. I would of course like to configure the audit logging more than what is possible in Envers. E g I would perhaps like the possibility to serialize my entities instead. Also, more options regarding what fields to log would be nice. These two routes need not be mutually exclusive. Like I said, if OpenJPA supported something like Envers, that would probably be good enough for me. If not, then I just need the necessary plumbing support to roll my own audit logging with OpenJPA. But I need to know that I'm not using implementation details that might change in the next version of OpenJPA. /Bengt 2011/8/2 Pinaki Poddar ppod...@apache.org Hi Bengt, It is possible (with some effort :) to offer a different POJO y that holds the original state of a managed POJO x *when* x entered a persistence context. But y can not be managed by the same context, because x is already present in that context. So y will be essentially an unmanaged (not even detached) instance. Because of that conceptual dilemma, I went for the option of offering only the dirty field values. It is also possible to offer all field values. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6645998.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi Bengt, It is possible (with some effort :) to offer a different POJO y that holds the original state of a managed POJO x *when* x entered a persistence context. But y can not be managed by the same context, because x is already present in that context. So y will be essentially an unmanaged (not even detached) instance. Because of that conceptual dilemma, I went for the option of offering only the dirty field values. It is also possible to offer all field values. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6645998.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
perhaps http://jpasecurity.sourceforge.net/ can help you to control access as JPA xxtension, but it's more for security auditing than just logging regards Frank -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6632956.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Interesting - will definitely take a look. /Bengt Den 29 jul 2011 19:24 skrev pif frank.pien...@googlemail.com: perhaps http://jpasecurity.sourceforge.net/ can help you to control access as JPA xxtension, but it's more for security auditing than just logging regards Frank -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6632956.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi! I've now looked at your test code and I'm not sure if it really is expected to work. The JPA spec imo isn't very clear about exactly _when_ the @PreUpdate gets called. From the JPA-2.0 spec: The PreUpdate and PostUpdate callbacks occur before and after the database update operations to entity data respectively. These database operations may occur at the time the entity state is updated or they may occur at the time state is flushed to the database (which may be at the end of the transaction). Note that it is implementation-dependent as to whether PreUpdate and PostUpdate call- backs occur when an entity is persisted and subsequently modified in a single transaction or when an entity is modified and subsequently removed within a single transaction. Portable applications should not rely on such behavior. It also doesn't say whether it will get called if the flush is performed for getting reproducible selects, etc. Also the term 'entity data' caused a lot discussions already. What happens with a EllementCollection or a CascadeType.PERSIST 1:n which is stored in your class? This obviously will render data in a different table. So is this covered by 'entity data' or not? There are lots of details in the JPA spec and not everything works as expected once you leave the simple samples and touch more complicated areas like lazy loading, fetch groups, detached entities, etc… You should for example try to close your EntityManager in your test (forces a detach) and then create a new one afterwards. Of course you are relying on OpenJPA specific code, so you can argue that you don't need portability to other JPA providers at all. But still: it's not even guaranteed within OpenJPA that such subtle behavior will not change. I'm also not sure if it's allowed to create a new entity within the same EntityManager while it is getting committed. I remember that I tried that and got an Exception because you cannot modify the EM while it is in the process of getting committed. But not 100% sure yet (grey cells refuse to find the paragraph in the docs) - Pinaki, you know what I mean and can shed a light on it? txs! Of course it imo IS doable with OpenJPA, but it's definitely not a straight way ;) The StateManager code needs a bit of tweaking anyway. You could take a look at OPENJPA-1873 to get an idea about it ;) I've already debugged into the StateManager stuff a big time and am ready to help out once we know how we'd like to solve this. LieGrue, strub --- On Tue, 7/26/11, Bengt Rodehav be...@rodehav.com wrote: From: Bengt Rodehav be...@rodehav.com Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Tuesday, July 26, 2011, 11:35 PM Yes, you're right Pinaki. I've attached a test case to the JIRA now. I'm very interested in your verdict... /Bengt 2011/7/26 Pinaki Poddar ppod...@apache.org Hi Bengt, This thread has grown too lengthy for me to find where was the @PreUpdate code. For better convergence, please create a JUnit test and post it to OPENJPA-2030 [1] [1] https://issues.apache.org/jira/browse/OPENJPA-2030 - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6623086.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hello Pinaki, Thanks for looking into this. What you are suggesting seems useful and it would be really nice if this functionality is supported in the future. However, I can't see how it directly solves my problem. What I'm trying to do in my audit log is to save a serialized form of my entity - the whole entity and not just the dirty fields. My intention was to use JSON (with Gson) for this. I'm sure there are many different approaches to an audit log (as can be seen from this long conversation) but my rationale is: I want an easy way to show the user exactly what a previous version looked like. Either I save the complete previous version or I have to reconstruct it when showing it to the user. The former seems easier. Furthermore, audit logging is used for rather static (as opposed to transactional data) which means that there is no problem using up a few more bytes. Clearly indicating what fields were changed is a bonus. If I could save both a serialization of the previous version and some metadata indicating what fields were changed that would be perfect. I would probably have to drop JSON and go for XML instead then. The API extension you're proposing will give me information about the dirty fields but I can't clearly see how I can create a serialization of the previous version for the entity. /Bengt 2011/7/27 Pinaki Poddar ppod...@apache.org Hi Bengt, I find that accessing the old state is possible using internal mechanics -- but could not find a way to do it from the current API unless I expose some internals. After a bit of deliberation, I am suggesting that we add the following method to SaveFieldManager. /** * Gets the old values for the given list of fields provided they are dirty. * @param fields a list of field names. null implies all dirty fields. * @return a map of old values for each given field, if they are dirty. * The map values are a pair of objects in an array -- the 0-th element * is the old value and the 1-st element is the current value. */ public Maplt;String, Object[]gt; getOldValues(String...fields) I have tested your supplied classes with the classes you supplied. If you do not mind being in the bleeding edge, i.e. if you can consume OpenJPA nightly builds, then I will commit the changes. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6624609.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi Bengt, This thread has grown too lengthy for me to find where was the @PreUpdate code. For better convergence, please create a JUnit test and post it to OPENJPA-2030 [1] [1] https://issues.apache.org/jira/browse/OPENJPA-2030 - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6623086.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Yes, you're right Pinaki. I've attached a test case to the JIRA now. I'm very interested in your verdict... /Bengt 2011/7/26 Pinaki Poddar ppod...@apache.org Hi Bengt, This thread has grown too lengthy for me to find where was the @PreUpdate code. For better convergence, please create a JUnit test and post it to OPENJPA-2030 [1] [1] https://issues.apache.org/jira/browse/OPENJPA-2030 - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6623086.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi Bengt, I find that accessing the old state is possible using internal mechanics -- but could not find a way to do it from the current API unless I expose some internals. After a bit of deliberation, I am suggesting that we add the following method to SaveFieldManager. /** * Gets the old values for the given list of fields provided they are dirty. * @param fields a list of field names. null implies all dirty fields. * @return a map of old values for each given field, if they are dirty. * The map values are a pair of objects in an array -- the 0-th element * is the old value and the 1-st element is the current value. */ public Maplt;String, Object[]gt; getOldValues(String...fields) I have tested your supplied classes with the classes you supplied. If you do not mind being in the bleeding edge, i.e. if you can consume OpenJPA nightly builds, then I will commit the changes. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6624609.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hello again Mark and Pinaki, I've tried with the properties you mentioned Mark (and the one you mentioned in your blog Pinaki). B ut I can see no difference. I also switched from OpenJPA 2.0.1 to 2.1.1 but still no difference. My persistence.xml looks like this: *?xml version=1.0 encoding=UTF-8?* *persistence xmlns=http://java.sun.com/xml/ns/persistence; xmlns:xsi= http://www.w3.org/2001/XMLSchema-instance* * xsi:schemaLocation=http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd* * version=2.0* * persistence-unit name=maiaTestPU transaction-type=RESOURCE_LOCAL* * providerorg.apache.openjpa.persistence.PersistenceProviderImpl/provider* *classse.digia.maia.core.bu.domain.BusinessUnit/class* *classse.digia.maia.common.persistence.EntityBase/class* *classse.digia.maia.common.security.domain.User/class* *classse.digia.maia.common.auditlog.domain.AuditLogEntry/class* *exclude-unlisted-classestrue/exclude-unlisted-classes* *validation-modeNONE/validation-mode* *properties* * property name=openjpa.RestoreState value=all /* * property name=openjpa.DetachState value=loaded(DetachedStateField=true) /* * property name=openjpa.Compatibility value=IgnoreDetachedStateFieldForProxySerialization=true /* * property name=javax.persistence.jdbc.password value=root /* * property name=javax.persistence.jdbc.user value=root /* * property name=javax.persistence.jdbc.driver value=com.mysql.jdbc.Driver /* * property name=javax.persistence.jdbc.url value=jdbc:mysql://localhost:3306/maia?createDatabaseIfNotExist=true /* * property name=openjpa.jdbc.SynchronizeMappings value=buildSchema(ForeignKeys=true) /* * property name=openjpa.jdbc.DBDictionary value=org.apache.openjpa.jdbc.sql.MySQLDictionary /* * property name=openjpa.jdbc.UpdateManager value=operation-order /* * property name=openjpa.Log value=DefaultLevel=WARN, Tool=INFO /* */properties* * /persistence-unit* */persistence* My JUnit test case look like this: *...* *// Create the business unit* *cEntityManager.getTransaction().begin();* *BusinessUnit bu = createPersistentBusinessUnit(Created, Created department);* *cEntityManager.getTransaction().commit();* * * *// Update the business unit* *cEntityManager.getTransaction().begin();* *BusinessUnit bu2 = cEntityManager.find(BusinessUnit.class, bu.getId()); * *bu2.setShortName(Updated);* *bu2.setName(Updated too);* *cEntityManager.merge(bu2);* *cEntityManager.getTransaction().commit(); // Throws an exception* * * *// Verify business unit* *BusinessUnit bu3 = cEntityManager.find(BusinessUnit.class, bu.getId()); * *assertEquals(bu.getId(), bu3.getId());* *assertEquals(bu.getShortName(), bu3.getShortName());* *assertEquals(bu.getName(), bu3.getName());* The @PreUpdate callback is as I mentioned in a previous post. The marked line (with a comment) throws an exception. The reason is that mandatory values are missing. Values that exist in the persisted entity (variable bu) but are not updated. If I remove the audit logging from my @PreUpdate method, the JUnit test case succeeds. Any clues? /Bengt 2011/7/21 Bengt Rodehav be...@rodehav.com I'm currently in Turkey on vacation but will try when I get back to Sweden. /Bengt Den 19 jul 2011 17:48 skrev Pinaki Poddar ppod...@apache.org: Hi Bengt, It is not obvious to me how DetachState related options will impact in-transaction entities. But I am interested to know the result of your experiment with the options Mark has suggested. I will recommend you make sure the following 1. Enhance at build time 2. Set openjpa.RestoreState=true - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6599036.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
I'm currently in Turkey on vacation but will try when I get back to Sweden. /Bengt Den 19 jul 2011 17:48 skrev Pinaki Poddar ppod...@apache.org: Hi Bengt, It is not obvious to me how DetachState related options will impact in-transaction entities. But I am interested to know the result of your experiment with the options Mark has suggested. I will recommend you make sure the following 1. Enhance at build time 2. Set openjpa.RestoreState=true - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6599036.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Thanks Mark, I'll give it a try. However, it seems a bit shaky if this is not officially supported by OpenJPA. Do you know if this will clearly be supported in the future? /Bengt Den 17 jul 2011 02:45 skrev Mark Struberg strub...@yahoo.de: It does keep track, but you might need to experiment a bit with your persistence.xml. In my case (doing lots of detached work), I needed the following settings: property name=openjpa.DetachState value=loaded(DetachedStateField=true)/ property name=openjpa.Compatibility value=IgnoreDetachedStateFieldForProxySerialization=true/ Be aware that some of those settings radically change the way OpenJPA works internally. But otherwise detached, merging, etc does not really work as this area is pretty under-specced. I also remember that I needed to fix a few things to get the correct state in OPENJPA-1873. Not sure how much of this patch got already committed (got my time eaten up by a few other Apache projects lately). LieGrue, stru --- On Sat, 7/16/11, Bengt Rodehav be...@rodehav.com wrote: From: Bengt Rodehav be...@rodehav.com Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Saturday, July 16, 2011, 8:40 PM Did you have a suggestion regarding this Pinaki - or does OpenJPA only keep track of changed values - not the whole original object? /Bengt 2011/7/15 Bengt Rodehav be...@rodehav.com I don't understand Pinaki. The this pointer is the current version of the object (the one that is about to be persisted) right? I want the previous version of the object. I have used your code from the blog (and added some of course) like this: * PersistenceCapable currentState = (PersistenceCapable) this; StateManagerImpl sm = (StateManagerImpl) currentState.pcGetStateManager(); SaveFieldManager sfm = sm.getSaveFieldManager(); PersistenceCapable oldState = sfm.getState(); EntityBase old = (EntityBase) oldState; old = (EntityBase) oldState; AuditLogEntry entry = new AuditLogEntry(); entry.setSerializedState(cAuditLog.serialize(old)); entry.setEntityId(old.getId()); entry.setCreatedWhen(old.getCreatedWhen()); entry.setCreatedBy(old.getCreatedBy()); entry.setUpdatedWhen(old.getUpdatedWhen()); entry.setUpdatedBy(old.getUpdatedBy()); entry.setVersion(old.getVersion()); EntityManager em = OpenJPAPersistence.getEntityManager(this); em.persist(entry); * cAuditLog is an object which uses Gson to serialize the object to a String. The problem is that only those fields that I have actually updated are initialized in the variable old. The rest of the fields are uninitialized. In my case I do not only want to save the changed fields but the whole object. How can I make sure that old is fully initialized? /Bengt 2011/7/15 Pinaki Poddar ppod...@apache.org The whole object is *this* pointer on which you received the callback. You can safely cast it into PersistenceCapable and get to know all about its state. Please see the blog post -- it has some toy code. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6585154.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi Bengt, It is not obvious to me how DetachState related options will impact in-transaction entities. But I am interested to know the result of your experiment with the options Mark has suggested. I will recommend you make sure the following 1. Enhance at build time 2. Set openjpa.RestoreState=true - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6599036.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Yes, I just mentioned it because in my case it made a _huge_ difference when working with previously detached entities which got em.merged(). I don't know the exact scenario, but since EntityManagers are not Serializable, there is detachment needed in a lot cases. At least if you need it also working in a cluster or load balancer scenario. LieGrue, strub --- On Tue, 7/19/11, Pinaki Poddar ppod...@apache.org wrote: From: Pinaki Poddar ppod...@apache.org Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Tuesday, July 19, 2011, 2:47 PM Hi Bengt, It is not obvious to me how DetachState related options will impact in-transaction entities. But I am interested to know the result of your experiment with the options Mark has suggested. I will recommend you make sure the following 1. Enhance at build time 2. Set openjpa.RestoreState=true - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6599036.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Yes, I just mentioned it because in my case it made a _huge_ difference when working with previously detached entities which got em.merged(). I don't know the exact scenario, but since EntityManagers are not Serializable, there is detachment needed in a lot cases. At least if you need it also working in a cluster or load balancer scenario. LieGrue, strub --- On Tue, 7/19/11, Pinaki Poddar ppod...@apache.org wrote: From: Pinaki Poddar ppod...@apache.org Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Tuesday, July 19, 2011, 2:47 PM Hi Bengt, It is not obvious to me how DetachState related options will impact in-transaction entities. But I am interested to know the result of your experiment with the options Mark has suggested. I will recommend you make sure the following 1. Enhance at build time 2. Set openjpa.RestoreState=true - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6599036.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Bengt, I had been busy with other things not to address your question. Did you read the very two last lines of the blog? They have the right configuration options required for the restorable state. Let me know if things do not work even after you have used those two directives. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6594830.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Did you have a suggestion regarding this Pinaki - or does OpenJPA only keep track of changed values - not the whole original object? /Bengt 2011/7/15 Bengt Rodehav be...@rodehav.com I don't understand Pinaki. The this pointer is the current version of the object (the one that is about to be persisted) right? I want the previous version of the object. I have used your code from the blog (and added some of course) like this: * PersistenceCapable currentState = (PersistenceCapable) this; StateManagerImpl sm = (StateManagerImpl) currentState.pcGetStateManager(); SaveFieldManager sfm = sm.getSaveFieldManager(); PersistenceCapable oldState = sfm.getState(); EntityBase old = (EntityBase) oldState; old = (EntityBase) oldState; AuditLogEntry entry = new AuditLogEntry(); entry.setSerializedState(cAuditLog.serialize(old)); entry.setEntityId(old.getId()); entry.setCreatedWhen(old.getCreatedWhen()); entry.setCreatedBy(old.getCreatedBy()); entry.setUpdatedWhen(old.getUpdatedWhen()); entry.setUpdatedBy(old.getUpdatedBy()); entry.setVersion(old.getVersion()); EntityManager em = OpenJPAPersistence.getEntityManager(this); em.persist(entry); * cAuditLog is an object which uses Gson to serialize the object to a String. The problem is that only those fields that I have actually updated are initialized in the variable old. The rest of the fields are uninitialized. In my case I do not only want to save the changed fields but the whole object. How can I make sure that old is fully initialized? /Bengt 2011/7/15 Pinaki Poddar ppod...@apache.org The whole object is *this* pointer on which you received the callback. You can safely cast it into PersistenceCapable and get to know all about its state. Please see the blog post -- it has some toy code. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6585154.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
It does keep track, but you might need to experiment a bit with your persistence.xml. In my case (doing lots of detached work), I needed the following settings: property name=openjpa.DetachState value=loaded(DetachedStateField=true)/ property name=openjpa.Compatibility value=IgnoreDetachedStateFieldForProxySerialization=true/ Be aware that some of those settings radically change the way OpenJPA works internally. But otherwise detached, merging, etc does not really work as this area is pretty under-specced. I also remember that I needed to fix a few things to get the correct state in OPENJPA-1873. Not sure how much of this patch got already committed (got my time eaten up by a few other Apache projects lately). LieGrue, stru --- On Sat, 7/16/11, Bengt Rodehav be...@rodehav.com wrote: From: Bengt Rodehav be...@rodehav.com Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Saturday, July 16, 2011, 8:40 PM Did you have a suggestion regarding this Pinaki - or does OpenJPA only keep track of changed values - not the whole original object? /Bengt 2011/7/15 Bengt Rodehav be...@rodehav.com I don't understand Pinaki. The this pointer is the current version of the object (the one that is about to be persisted) right? I want the previous version of the object. I have used your code from the blog (and added some of course) like this: * PersistenceCapable currentState = (PersistenceCapable) this; StateManagerImpl sm = (StateManagerImpl) currentState.pcGetStateManager(); SaveFieldManager sfm = sm.getSaveFieldManager(); PersistenceCapable oldState = sfm.getState(); EntityBase old = (EntityBase) oldState; old = (EntityBase) oldState; AuditLogEntry entry = new AuditLogEntry(); entry.setSerializedState(cAuditLog.serialize(old)); entry.setEntityId(old.getId()); entry.setCreatedWhen(old.getCreatedWhen()); entry.setCreatedBy(old.getCreatedBy()); entry.setUpdatedWhen(old.getUpdatedWhen()); entry.setUpdatedBy(old.getUpdatedBy()); entry.setVersion(old.getVersion()); EntityManager em = OpenJPAPersistence.getEntityManager(this); em.persist(entry); * cAuditLog is an object which uses Gson to serialize the object to a String. The problem is that only those fields that I have actually updated are initialized in the variable old. The rest of the fields are uninitialized. In my case I do not only want to save the changed fields but the whole object. How can I make sure that old is fully initialized? /Bengt 2011/7/15 Pinaki Poddar ppod...@apache.org The whole object is *this* pointer on which you received the callback. You can safely cast it into PersistenceCapable and get to know all about its state. Please see the blog post -- it has some toy code. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6585154.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
I don't understand Pinaki. The this pointer is the current version of the object (the one that is about to be persisted) right? I want the previous version of the object. I have used your code from the blog (and added some of course) like this: * PersistenceCapable currentState = (PersistenceCapable) this; StateManagerImpl sm = (StateManagerImpl) currentState.pcGetStateManager(); SaveFieldManager sfm = sm.getSaveFieldManager(); PersistenceCapable oldState = sfm.getState(); EntityBase old = (EntityBase) oldState; old = (EntityBase) oldState; AuditLogEntry entry = new AuditLogEntry(); entry.setSerializedState(cAuditLog.serialize(old)); entry.setEntityId(old.getId()); entry.setCreatedWhen(old.getCreatedWhen()); entry.setCreatedBy(old.getCreatedBy()); entry.setUpdatedWhen(old.getUpdatedWhen()); entry.setUpdatedBy(old.getUpdatedBy()); entry.setVersion(old.getVersion()); EntityManager em = OpenJPAPersistence.getEntityManager(this); em.persist(entry); * cAuditLog is an object which uses Gson to serialize the object to a String. The problem is that only those fields that I have actually updated are initialized in the variable old. The rest of the fields are uninitialized. In my case I do not only want to save the changed fields but the whole object. How can I make sure that old is fully initialized? /Bengt 2011/7/15 Pinaki Poddar ppod...@apache.org The whole object is *this* pointer on which you received the callback. You can safely cast it into PersistenceCapable and get to know all about its state. Please see the blog post -- it has some toy code. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6585154.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi again Jim,
2011/7/14 Jim Talbut jtal...@spudsoft.co.uk
On 13/07/2011 21:14, Pinaki Poddar wrote:
Yes. Any audit facility needs to have a snapshot of the entity when it
entered a persistence context, so at @PreUpdate or at any other time
points,
it can figure out what has essentially been changed about that entity in a
transaction. Now either one can build their own mechanics to store the
original state of the entity or can use OpenJPA's own facility to access
the
original state. The blog article showed the later approach.
I also went with using the OpenJPA stored version of the original object.
There are a few niggles to getting that data that one has to work with, but
I still think it's considerably less effort than building your own
mechanism.
Secondly, in my view, an audit facility should be orthogonal. The actual
domain entity need not know that it is being audited. Thereby, the domain
entity need not have an association or knowledge of an Audit object.
I'd agree with that for a general purpose audit facility.
In my case I don't even call it audit it's change tracking, and the
change information is displayed alongside the entities in the most common
cases.
Thirdly, the audit facility should allow the audit information be stored
in
a separate database, in the same database or may even be logged in a file.
That is to say that persistence of audit information should be decoupled
from persistence of the domain objects.
Again, true for a general purpose audit facility.
If you want to be really secure the audit log should be sent directly to a
printer :)
If you intend to store audit information as a persistent entity in the
same
database as the domain entity, then the simple solution is something like
this in a domain class:
@PreUpdate
public void audit() {
Audit audit = new Audit();
// now populate audit information
// some serious delta computation
// Now get the entity manager that is managing this current
domain
object
OpenJPAEntityManager em =
OpenJPAPersistence.**getEntityManager(this);
// And persist the audit information in the same transaction
em.persist(audit);
}
I wonder if that bends the rules any more than my approach?
Exactly what I was thinking. Since you seem to be bending the rules in your
approach as well (creating new entities in the @PreUpdate), why not bend it
a little more? The benefit is that you can get the audit logging out of the
domain model (if you want to). Also, in my case, I have no use for the join
table between my entities so it feel nice to get rid of it...
Since neither of these approaches are supported by JPA but depends on the
implementation of OpenJPA, one might as well choose the most convenient one.
It'd be nice though if at least OpenJPA could officially support this kind
of mechanism since I think it's needed in real life anyway.
/Bengt
Jim
Re: Audit log with OpenJPA
Jim, I was making generic statements about audit and was not meant to address the specific approach you might have taken to suit your needs. I will take a closer look at the approach you have come up with. About 'delta computation', however, I find OpenJPA provides strong feature via its SavePointManager and restore facility. The advantage is that leaves the callback listeners stateless. Also, the managed entity knows which fields have been modified/dirtied -- which is essential to any delta computation. Bengt, Good to know you have made it working. Now the question/concern about spec-compliance what is permissible in @PreUpdate a) JPA spec so far have discouraged entity modification in callbacks. Specifically, JPA 2.0 spec does not allow access to EntityManager and modification of existing relations. These restrictions stem from the concern to maintain the referential and transactional integrity of the unit of work. But as the footnote in page 93 (Section 3.5) mentions, the idea is to make this area of the spec more standardized in future. As the spec moves forward, we expect that new entity creation in @Pre- callbacks are likely to be supported. Changing existing relations may not. You are welcome to contribute your views to the JSR-318 mailing group. b) Access to entity manager is quite likely to be standardized in future version of the spec. In expert group for version 2.1, we have seen several requests on this aspect. So the broad approach for audit/change tracking via callback listeners, that we have been discussing in this thread, is aligned with the evolution of JPA specification. You are not taking a non-kosher approach. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6583813.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Pinaki, Thanks for your very insightful post. It feels like this is the way to go then. I'll stick with this approach for now and try to follow the development of JPA and OpenJPA. Thanks for your help, /Bengt 2011/7/14 Pinaki Poddar ppod...@apache.org Jim, I was making generic statements about audit and was not meant to address the specific approach you might have taken to suit your needs. I will take a closer look at the approach you have come up with. About 'delta computation', however, I find OpenJPA provides strong feature via its SavePointManager and restore facility. The advantage is that leaves the callback listeners stateless. Also, the managed entity knows which fields have been modified/dirtied -- which is essential to any delta computation. Bengt, Good to know you have made it working. Now the question/concern about spec-compliance what is permissible in @PreUpdate a) JPA spec so far have discouraged entity modification in callbacks. Specifically, JPA 2.0 spec does not allow access to EntityManager and modification of existing relations. These restrictions stem from the concern to maintain the referential and transactional integrity of the unit of work. But as the footnote in page 93 (Section 3.5) mentions, the idea is to make this area of the spec more standardized in future. As the spec moves forward, we expect that new entity creation in @Pre- callbacks are likely to be supported. Changing existing relations may not. You are welcome to contribute your views to the JSR-318 mailing group. b) Access to entity manager is quite likely to be standardized in future version of the spec. In expert group for version 2.1, we have seen several requests on this aspect. So the broad approach for audit/change tracking via callback listeners, that we have been discussing in this thread, is aligned with the evolution of JPA specification. You are not taking a non-kosher approach. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6583813.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Sorry Pinaki - not done with me yet My intention was to audit log the whole object (serialized though) but the method: *SaveFieldManager.getState()* returns an object where only the changed fields are initialized (seems like it anyway). Is there a way I can get hold of the whole object (the previous version of the object that is)? /Bengt 2011/7/14 Bengt Rodehav be...@rodehav.com Pinaki, Thanks for your very insightful post. It feels like this is the way to go then. I'll stick with this approach for now and try to follow the development of JPA and OpenJPA. Thanks for your help, /Bengt 2011/7/14 Pinaki Poddar ppod...@apache.org Jim, I was making generic statements about audit and was not meant to address the specific approach you might have taken to suit your needs. I will take a closer look at the approach you have come up with. About 'delta computation', however, I find OpenJPA provides strong feature via its SavePointManager and restore facility. The advantage is that leaves the callback listeners stateless. Also, the managed entity knows which fields have been modified/dirtied -- which is essential to any delta computation. Bengt, Good to know you have made it working. Now the question/concern about spec-compliance what is permissible in @PreUpdate a) JPA spec so far have discouraged entity modification in callbacks. Specifically, JPA 2.0 spec does not allow access to EntityManager and modification of existing relations. These restrictions stem from the concern to maintain the referential and transactional integrity of the unit of work. But as the footnote in page 93 (Section 3.5) mentions, the idea is to make this area of the spec more standardized in future. As the spec moves forward, we expect that new entity creation in @Pre- callbacks are likely to be supported. Changing existing relations may not. You are welcome to contribute your views to the JSR-318 mailing group. b) Access to entity manager is quite likely to be standardized in future version of the spec. In expert group for version 2.1, we have seen several requests on this aspect. So the broad approach for audit/change tracking via callback listeners, that we have been discussing in this thread, is aligned with the evolution of JPA specification. You are not taking a non-kosher approach. - Pinaki Poddar Chair, Apache OpenJPA Project -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6583813.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi,
On PreUpdate I save an audit log entry describing the current entity.
However, I guess I should
audit log the object in its previous state not its current state -
otherwise I need to audit log in the PreCreate as well - right? I think
I'll have to use your solution, Pinaki, to get the previous state of the
object since
that's what I have to log in the PreUpdate.
Yes. Any audit facility needs to have a snapshot of the entity when it
entered a persistence context, so at @PreUpdate or at any other time points,
it can figure out what has essentially been changed about that entity in a
transaction. Now either one can build their own mechanics to store the
original state of the entity or can use OpenJPA's own facility to access the
original state. The blog article showed the later approach.
Secondly, in my view, an audit facility should be orthogonal. The actual
domain entity need not know that it is being audited. Thereby, the domain
entity need not have an association or knowledge of an Audit object.
Thirdly, the audit facility should allow the audit information be stored in
a separate database, in the same database or may even be logged in a file.
That is to say that persistence of audit information should be decoupled
from persistence of the domain objects.
If you intend to store audit information as a persistent entity in the same
database as the domain entity, then the simple solution is something like
this in a domain class:
@PreUpdate
public void audit() {
Audit audit = new Audit();
// now populate audit information
// some serious delta computation
// Now get the entity manager that is managing this current domain
object
OpenJPAEntityManager em =
OpenJPAPersistence.getEntityManager(this);
// And persist the audit information in the same transaction
em.persist(audit);
}
-
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context:
http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6580549.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Pinaki,
I agree with all your statements. I would like to keep the audit logging
out of my business domain (separating of concerns). I just hadn't found a
good way of doing this with JPA yet. Jim's approach gave me a way of being
able to create new entities in @PreUpdate however with the drawbacks I
mentioned (where one is that the domain model is affected).
Your solution avoided the relationship but didn't give a way to store the
audit log in a separate table. I fully agree that the persistence of the
audit logging should be independent of the domain objects and could even be
in log files.
I have been thinking of the approach you suggest but I didn't think it was
supported. I'm sure it's not supported by JPA since it seems you're not
allowed to create new entities in a @PreUpdate. Is it supported by OpenJPA?
Will it stay supported?
Thanks for your advice - I appreciate it,
/Bengt
2011/7/13 Pinaki Poddar ppod...@apache.org
Hi,
On PreUpdate I save an audit log entry describing the current entity.
However, I guess I should
audit log the object in its previous state not its current state -
otherwise I need to audit log in the PreCreate as well - right? I think
I'll have to use your solution, Pinaki, to get the previous state of the
object since
that's what I have to log in the PreUpdate.
Yes. Any audit facility needs to have a snapshot of the entity when it
entered a persistence context, so at @PreUpdate or at any other time
points,
it can figure out what has essentially been changed about that entity in a
transaction. Now either one can build their own mechanics to store the
original state of the entity or can use OpenJPA's own facility to access
the
original state. The blog article showed the later approach.
Secondly, in my view, an audit facility should be orthogonal. The actual
domain entity need not know that it is being audited. Thereby, the domain
entity need not have an association or knowledge of an Audit object.
Thirdly, the audit facility should allow the audit information be stored in
a separate database, in the same database or may even be logged in a file.
That is to say that persistence of audit information should be decoupled
from persistence of the domain objects.
If you intend to store audit information as a persistent entity in the same
database as the domain entity, then the simple solution is something like
this in a domain class:
@PreUpdate
public void audit() {
Audit audit = new Audit();
// now populate audit information
// some serious delta computation
// Now get the entity manager that is managing this current domain
object
OpenJPAEntityManager em =
OpenJPAPersistence.getEntityManager(this);
// And persist the audit information in the same transaction
em.persist(audit);
}
-
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context:
http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6580549.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Just tried your suggestion Pinaki - it works fine and I can keep the audit
logging completely separate from my domain model. I even get rid of the join
table.
Now if this were clearly supported by OpenJPA (and by JPA in the future) I
think we have a clear winner...
/Bengt
2011/7/14 Bengt Rodehav be...@rodehav.com
Pinaki,
I agree with all your statements. I would like to keep the audit logging
out of my business domain (separating of concerns). I just hadn't found a
good way of doing this with JPA yet. Jim's approach gave me a way of being
able to create new entities in @PreUpdate however with the drawbacks I
mentioned (where one is that the domain model is affected).
Your solution avoided the relationship but didn't give a way to store the
audit log in a separate table. I fully agree that the persistence of the
audit logging should be independent of the domain objects and could even be
in log files.
I have been thinking of the approach you suggest but I didn't think it was
supported. I'm sure it's not supported by JPA since it seems you're not
allowed to create new entities in a @PreUpdate. Is it supported by OpenJPA?
Will it stay supported?
Thanks for your advice - I appreciate it,
/Bengt
2011/7/13 Pinaki Poddar ppod...@apache.org
Hi,
On PreUpdate I save an audit log entry describing the current entity.
However, I guess I should
audit log the object in its previous state not its current state -
otherwise I need to audit log in the PreCreate as well - right? I think
I'll have to use your solution, Pinaki, to get the previous state of the
object since
that's what I have to log in the PreUpdate.
Yes. Any audit facility needs to have a snapshot of the entity when it
entered a persistence context, so at @PreUpdate or at any other time
points,
it can figure out what has essentially been changed about that entity in a
transaction. Now either one can build their own mechanics to store the
original state of the entity or can use OpenJPA's own facility to access
the
original state. The blog article showed the later approach.
Secondly, in my view, an audit facility should be orthogonal. The actual
domain entity need not know that it is being audited. Thereby, the domain
entity need not have an association or knowledge of an Audit object.
Thirdly, the audit facility should allow the audit information be stored
in
a separate database, in the same database or may even be logged in a file.
That is to say that persistence of audit information should be decoupled
from persistence of the domain objects.
If you intend to store audit information as a persistent entity in the
same
database as the domain entity, then the simple solution is something like
this in a domain class:
@PreUpdate
public void audit() {
Audit audit = new Audit();
// now populate audit information
// some serious delta computation
// Now get the entity manager that is managing this current
domain
object
OpenJPAEntityManager em =
OpenJPAPersistence.getEntityManager(this);
// And persist the audit information in the same transaction
em.persist(audit);
}
-
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context:
http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6580549.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
On 13/07/2011 21:14, Pinaki Poddar wrote:
Yes. Any audit facility needs to have a snapshot of the entity when it
entered a persistence context, so at @PreUpdate or at any other time points,
it can figure out what has essentially been changed about that entity in a
transaction. Now either one can build their own mechanics to store the
original state of the entity or can use OpenJPA's own facility to access the
original state. The blog article showed the later approach.
I also went with using the OpenJPA stored version of the original object.
There are a few niggles to getting that data that one has to work with,
but I still think it's considerably less effort than building your own
mechanism.
Secondly, in my view, an audit facility should be orthogonal. The actual
domain entity need not know that it is being audited. Thereby, the domain
entity need not have an association or knowledge of an Audit object.
I'd agree with that for a general purpose audit facility.
In my case I don't even call it audit it's change tracking, and the
change information is displayed alongside the entities in the most
common cases.
Thirdly, the audit facility should allow the audit information be stored in
a separate database, in the same database or may even be logged in a file.
That is to say that persistence of audit information should be decoupled
from persistence of the domain objects.
Again, true for a general purpose audit facility.
If you want to be really secure the audit log should be sent directly to
a printer :)
If you intend to store audit information as a persistent entity in the same
database as the domain entity, then the simple solution is something like
this in a domain class:
@PreUpdate
public void audit() {
Audit audit = new Audit();
// now populate audit information
// some serious delta computation
// Now get the entity manager that is managing this current domain
object
OpenJPAEntityManager em =
OpenJPAPersistence.getEntityManager(this);
// And persist the audit information in the same transaction
em.persist(audit);
}
I wonder if that bends the rules any more than my approach?
Jim
Re: Audit log with OpenJPA
On 07/07/2011 12:05, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. A bit late to the party, but this is what I've got working: Change the entity ID column so it's not allocated by the database, then it becomes valid from the time the entity is persisted, rather than the time it's flushed. By using a table type sequence generator the disadvantages relative to a database generated ID are few (and IME are only seen if you are abusing the IDs :) ) I only carry out an audit in @PreUpdate, because all I track is changes and there aren't any changes for a new record - so actually the ID generation should be irrelevant. But by working in PreUpdate I've found that I am able to work with the entity manager and create new entities for the audit. The entity being audited has a OneToMany join to the audit entries, so I don't have to call persist on the audit log object, I just add it to the collection in the entity being audited. Jim
Re: Audit log with OpenJPA
Thanks for your post Jim, In the past (when we rolled our own O/R mapping) we also generated the id's ourselves via sequences stored in a dedicated table. Come to think of it, one of the reasons was exactly what you are talking about - one can reference the id even before the object is persisted. This is of course a big benefit when it comes to audit logging (and actually other types of logging as well). I will definitely revisit that. Your other design decision is also interesting. By having a relationship to the audit log entries JPA will persist the audit log entry for you - very convenient indeed. A little similar to the way Pinaki described previously. However you store the audit log in a separate table while Pinaki stores the audit log together with the entity (in a separate column) that is being persisted. I kind of like your approach better. I think an audit log can be viewed from two different angles: the entity view and the user view. You need to be able to see what changes have been made to an entity and possibly even implment undo (like Pinaki). However, you also need to be able to track all changes done by a specific user during a specific time period. The latter is much easier with a centralized place for all audit log entries. Can't really see how that can easily be done with Pinaki's approach. The only thing I might miss from your solution then is the ability to configure whether audit logging is enabled for an entity or not. Our customers have different requirements here. But then again, this should be possible in your approach as well. In the @PreUpdate method I could check (somehow) if audit logging is enabled for this entity. If yes, then create the audit log entry and add it to the collection. If not, then don't create the audit log entry and nothing will be logged - right? Have you had the need to configure OpenJPA in some specific way for performance reasons? I guess you really want to avoid OpenJPA fetching all your audit log entries every time an entity is fetched... Also, I need to know whether this is a supported JPA/OpenJPA way of doing things. My guess is yes since you don't actually use the entity manager yourself. Very interesting approach. Thanks, /Bengt 2011/7/12 Jim Talbut jtal...@spudsoft.co.uk On 07/07/2011 12:05, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. A bit late to the party, but this is what I've got working: Change the entity ID column so it's not allocated by the database, then it becomes valid from the time the entity is persisted, rather than the time it's flushed. By using a table type sequence generator the disadvantages relative to a database generated ID are few (and IME are only seen if you are abusing the IDs :) ) I only carry out an audit in @PreUpdate, because all I track is changes and there aren't any changes for a new record - so actually the ID generation should be irrelevant. But by working in PreUpdate I've found that I am able to work with the entity manager and create new entities for the audit. The entity being audited has a OneToMany join to the audit entries, so I don't have to call persist on the audit log object, I just add it to the collection in the entity being audited. Jim
Re: Audit log with OpenJPA
Jim, Why do you think that your approach is not supported by JPA? From other posts on this list I've seen that you are allowed to add/enrich your entity in the PreUpdate/PreCreate callbacks. That's all you are doing right. Then JPA persists your enriched entities. What do you violate? /Bengt 2011/7/12 Jim Talbut jtal...@spudsoft.co.uk On 12/07/2011 08:49, Bengt Rodehav wrote: Your other design decision is also interesting. By having a relationship to the audit log entries JPA will persist the audit log entry for you - very convenient indeed. A little similar to the way Pinaki described previously. However you store the audit log in a separate table while Pinaki stores the audit log together with the entity (in a separate column) that is being persisted. I kind of like your approach better. I think an audit log can be viewed from two different angles: the entity view and the user view. You need to be able to see what changes have been made to an entity and possibly even implment undo (like Pinaki). However, you also need to be able to track all changes done by a specific user during a specific time period. The latter is much easier with a centralized place for all audit log entries. Can't really see how that can easily be done with Pinaki's approach. With the specifics of my implementation undo would be very awkward to implement, but that's because I'm trying to produce a textual change tracking log rather than an undo per-se. I could modify my layout to support undo, but it would still be more complex than a model that put the undo records in the original table. The only thing I might miss from your solution then is the ability to configure whether audit logging is enabled for an entity or not. Our customers have different requirements here. But then again, this should be possible in your approach as well. In the @PreUpdate method I could check (somehow) if audit logging is enabled for this entity. If yes, then create the audit log entry and add it to the collection. If not, then don't create the audit log entry and nothing will be logged - right? That's right. For my purposes I don't want to exclude any entities of a given type - so three different (unrelated) entity classes perform auditing but there is no filtering of entities of an audited type. Have you had the need to configure OpenJPA in some specific way for performance reasons? I guess you really want to avoid OpenJPA fetching all your audit log entries every time an entity is fetched... All I've done is make the audit entries lazy-fetch and I believe that works (though honestly I haven't actually tested it). Also, I need to know whether this is a supported JPA/OpenJPA way of doing things. My guess is yes since you don't actually use the entity manager yourself. It's not, that's the big downside. The only way you're going to get an official approach is with a JDBC call in the same transaction (which I ditched because of the difficulties of getting hold of the transactional context and other things from within the listener). If I find that the approach fails in a future version of OpenJPA I'll switch to JDBC, by having the entities in a separate table this is quite a clean swap and the only difficulty is getting the necessary context available at the right place. Jim
Re: Audit log with OpenJPA
On 12/07/2011 09:33, Bengt Rodehav wrote: Why do you think that your approach is not supported by JPA? From other posts on this list I've seen that you are allowed to add/enrich your entity in the PreUpdate/PreCreate callbacks. That's all you are doing right. Then JPA persists your enriched entities. I asked on this list a few months ago: http://web.archiveorange.com/archive/v/tXrrxw9bXkttUykS7B6K You can change the state of entities, but creating new entities is not permitted by the JPA spec. At the moment it does work, and I now what I'm going to do if it stops working, so I'm happy. I'd be happier if OpenJPA explicitly supported it, but it doesn't keep me awake at night. Jim
Re: Audit log with OpenJPA
Jim,
I tried your approach (I think) but got the following exception:
*openjpa-2.0.1-r422266:989424 nonfatal user error
org.apache.openjpa.persistence.InvalidStateException: Encountered unmanaged
object in persistent field
se.digia.maia.common.persistence.EntityBase.auditLogelement:class
se.digia.maia.common.auditlog.domain.AuditLog during flush. However, this
field does not allow cascade persist. Set the cascade attribute for this
field to CascadeType.PERSIST or CascadeType.ALL (JPA annotations) or
persist or all (JPA orm.xml), or enable cascade-persist globally, or
manually persist the related field value prior to flushing. You cannot flush
unmanaged objects or graphs that have persistent associations to unmanaged
objects.
**FailedObject: se.digia.maia.common.auditlog.domain.AuditLog@39bc82
** at
org.apache.openjpa.kernel.SingleFieldManager.preFlushPC(SingleFieldManager.java:767)
** at
org.apache.openjpa.kernel.SingleFieldManager.preFlushPCs(SingleFieldManager.java:748)
** at
org.apache.openjpa.kernel.SingleFieldManager.preFlush(SingleFieldManager.java:642)
** at
org.apache.openjpa.kernel.SingleFieldManager.preFlush(SingleFieldManager.java:575)
** at
org.apache.openjpa.kernel.SingleFieldManager.preFlush(SingleFieldManager.java:491)
** at
org.apache.openjpa.kernel.StateManagerImpl.preFlush(StateManagerImpl.java:2957)
** at
org.apache.openjpa.kernel.PDirtyState.beforeFlush(PDirtyState.java:38)
** at
org.apache.openjpa.kernel.StateManagerImpl.beforeFlush(StateManagerImpl.java:1047)
** at org.apache.openjpa.kernel.BrokerImpl.flush(BrokerImpl.java:2077)
** at org.apache.openjpa.kernel.BrokerImpl.flushSafe(BrokerImpl.java:2037)
** at
org.apache.openjpa.kernel.BrokerImpl.beforeCompletion(BrokerImpl.java:1955)
** at
org.apache.openjpa.kernel.LocalManagedRuntime.commit(LocalManagedRuntime.java:81)
** at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1479)
** at
org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:925)
** at
org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:560)
** at
se.digia.maia.core.bu.test.CRUDTest.updateBusinessUnitTest(CRUDTest.java:141)
*
I use OpenJPA 2.0.1. This is my relationship to the audit log table:
@OneToMany
@JoinTable(
name = ENTITY_ALOG,
joinColumns = @JoinColumn(name = ENTITY_ID),
inverseJoinColumns = @JoinColumn(name = ALOG_ID))
private CollectionAuditLog auditLog;
This is my @PreUpdate method:
@PreUpdate
public void beforeUpdate() {
setUpdatedWhen(Calendar.getInstance());
setUpdatedBy(Session.getCurrentUser());
if (this.getClass().getAnnotation(Audit.class) != null) {
AuditLog al = new AuditLog();
al.setLogEntry(serialization of object);
auditLog.add(al);
}
}
OpenJPA doesn't seem to like this. What did you do differently?
/Bengt
2011/7/12 Bengt Rodehav be...@rodehav.com
OK - I didn't know that.
I agree that it would be nice if the OpenJPA guys would support this
behaviour. My experience of standards (like JPA) is that the standard itself
is rarely enough for production purposes. Things outside of the standard
need to be in place as well. Then, after a while, the standard will
sometimes adapt to reality and incorporate what has become the de facto
standard.
In this case, it is clear that the behaviour of the life-cycle callback
methods is not defined enough in the standard. The implementations should
therefore create a de facto standard. Let's hope they do...
/Bengt
2011/7/12 Jim Talbut jtal...@spudsoft.co.uk
On 12/07/2011 09:33, Bengt Rodehav wrote:
Why do you think that your approach is not supported by JPA? From other
posts on this list I've seen that you are allowed to add/enrich your
entity
in the PreUpdate/PreCreate callbacks. That's all you are doing right.
Then
JPA persists your enriched entities.
I asked on this list a few months ago:
http://web.archiveorange.com/**archive/v/tXrrxw9bXkttUykS7B6Khttp://web.archiveorange.com/archive/v/tXrrxw9bXkttUykS7B6K
You can change the state of entities, but creating new entities is not
permitted by the JPA spec.
At the moment it does work, and I now what I'm going to do if it stops
working, so I'm happy.
I'd be happier if OpenJPA explicitly supported it, but it doesn't keep me
awake at night.
Jim
Re: Audit log with OpenJPA
Indeed very interesting stuff (JEST). I'm not sure whether I want straight lines from the client to the persistence layer though. Normally you would want a layer in between for adding business logic, security etc. What is interesting is to send general queries from the client to the server and also the handling of whole object graphs both to and from the server. I would like to be able to use those facilities in my internal implementation of my server side services. I would, however, hesitate to use it as my server side services straight off. /Bengt 2011/7/8 Pinaki Poddar ppod...@apache.org However, I store the JSON in a database and I was hoping to use JPA for this Actually, storing JSON in relational database would not permit query based on property values. But OpenJPA can provide JSON formatted-data for any web UI. To more about t, following references can help [1] http://www.ibm.com/developerworks/java/library/j-jest/?ca=drs- [2] http://openjpa.apache.org/jest [3] https://www.ibm.com/developerworks/mydeveloperworks/blogs/pinaki/tags/jest?lang=en - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6560732.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Pinaki, Can you point me to where I can find documentation about acessing the entity manager from the life cycle callbacks? Will it be supported in OpenJPA in the future? Can I rely on it? /Bengt 2011/7/8 Pinaki Poddar ppod...@apache.org you actually store it in the object itself. That's right. This allows in-memory rollback. Given that the entire 'clean' or 'original' state is available in the technique I described, the application can make several decisions on a) what should be logged in an audit trail -- the entire object in a separate database/schema, a serialized blob or a JSON stream or whatever. b) because the technique is based on OpenJPA internal, if an application decides to take the 'blue pill', then it can simply ask the managed object which fields have been 'dirtied' if it requires to compute delta efficiently. Another related but separate point: I have noticed lot of requests/annoyance from the users on the limitation of callback listener methods having no access to the persistence context. Again, in OpenJPA runtime, the managed entity does know its persistence context. If the application is ready for a OpenJPA-specific cast, then the handle to the persistence context is right there in the entity. - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6560548.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
7.7.2011 14:05, Bengt Rodehav kirjoitti: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? I have not done anything like this, but some kind of queue and a separate processor for that queue might be a solution. Maybe some message queue? Or a singleton Hashtable containing entries, and a separate thread for persisting these to database. Something like that. -- AWAKE! FEAR! FIRE! FOES! AWAKE! FEAR! FIRE! FOES! AWAKE! AWAKE! -- J. R. R. Tolkien signature.asc Description: OpenPGP digital signature
Re: Audit log with OpenJPA
On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David
Re: Audit log with OpenJPA
You might take a look at how hades (http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David
Re: Audit log with OpenJPA
Jari, Yes an asynchronous queue is definitely an option. I've actually used that approach before. It makes a lot of sense when trying to achieve high throughput since the audit logging can then be done on lower priority. I was however hoping to be able to use JPA for this since a queue increases the complexity significantly (also regarding testing). Thanks, /Bengt 2011/7/7 Jari Fredriksson ja...@iki.fi 7.7.2011 14:05, Bengt Rodehav kirjoitti: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? I have not done anything like this, but some kind of queue and a separate processor for that queue might be a solution. Maybe some message queue? Or a singleton Hashtable containing entries, and a separate thread for persisting these to database. Something like that. -- AWAKE! FEAR! FIRE! FOES! AWAKE! FEAR! FIRE! FOES! AWAKE! AWAKE! -- J. R. R. Tolkien
Re: Audit log with OpenJPA
Thanks for your reply David, I wouldn't rule out cheating although I'd rather not. I have no experience in mixing JDBC and JPA. What would happen transaction wise? Can they both be part of the same transaction? /Bengt 2011/7/7 David Goodenough david.goodeno...@btconnect.com On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David
Re: Audit log with OpenJPA
Perhaps you could use a separate PU for your audit logging? That should be safe to use inside of a lifecycle callback. On Thu, Jul 7, 2011 at 11:36 AM, Bengt Rodehav be...@rodehav.com wrote: Jari, Yes an asynchronous queue is definitely an option. I've actually used that approach before. It makes a lot of sense when trying to achieve high throughput since the audit logging can then be done on lower priority. I was however hoping to be able to use JPA for this since a queue increases the complexity significantly (also regarding testing). Thanks, /Bengt 2011/7/7 Jari Fredriksson ja...@iki.fi 7.7.2011 14:05, Bengt Rodehav kirjoitti: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? I have not done anything like this, but some kind of queue and a separate processor for that queue might be a solution. Maybe some message queue? Or a singleton Hashtable containing entries, and a separate thread for persisting these to database. Something like that. -- AWAKE! FEAR! FIRE! FOES! AWAKE! FEAR! FIRE! FOES! AWAKE! AWAKE! -- J. R. R. Tolkien -- *Rick Curtis*
Re: Audit log with OpenJPA
I actually use the same approach as Hades for createdBy, updatedBy, createdWhen and updatedWhen. In addition to this basic audit logging I also want to log all historical versions together with information about who updated the object. I've read a little bit about Envers. I didn't want to bring it up since this is an OpenJPA mailing list. It does look interesting but I think it requires Hibernate which I do not intend to go back to. (I'm actually moving away from Hibernate). Also, I'm a bit hesitant to store copies of my rows in special audit tables since it also means database migration of those tables. I think serializing the audit log entries and putting them in one column is a better approach from a maintenance perspective. But it would be nice if OpenJPA would provide a callback or some mechanism (even if it's not standard JPA) that would allow further updates/inserts in a callback. Are there any such plans? /Bengt 2011/7/7 Andrew Thompson at2...@columbia.edu You might take a look at how hades ( http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David
Re: Audit log with OpenJPA
Interesting suggestion. Can I still share the same transaction between the original update and my audit log insert? /Bengt 2011/7/7 Rick Curtis curti...@gmail.com Perhaps you could use a separate PU for your audit logging? That should be safe to use inside of a lifecycle callback. On Thu, Jul 7, 2011 at 11:36 AM, Bengt Rodehav be...@rodehav.com wrote: Jari, Yes an asynchronous queue is definitely an option. I've actually used that approach before. It makes a lot of sense when trying to achieve high throughput since the audit logging can then be done on lower priority. I was however hoping to be able to use JPA for this since a queue increases the complexity significantly (also regarding testing). Thanks, /Bengt 2011/7/7 Jari Fredriksson ja...@iki.fi 7.7.2011 14:05, Bengt Rodehav kirjoitti: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? I have not done anything like this, but some kind of queue and a separate processor for that queue might be a solution. Maybe some message queue? Or a singleton Hashtable containing entries, and a separate thread for persisting these to database. Something like that. -- AWAKE! FEAR! FIRE! FOES! AWAKE! FEAR! FIRE! FOES! AWAKE! AWAKE! -- J. R. R. Tolkien -- *Rick Curtis*
Re: Audit log with OpenJPA
How is your JPA accessed? I have done JPA, and Audit logging on top of it, but it was into XML files, and performed in a EJB3 Session Bean. The JPA objects were plain simple, but they were all accessed via a Session Bean Facade. I did it with XML because boss wanted so, but could have been database as well. 7.7.2011 19:49, Bengt Rodehav kirjoitti: I actually use the same approach as Hades for createdBy, updatedBy, createdWhen and updatedWhen. In addition to this basic audit logging I also want to log all historical versions together with information about who updated the object. I've read a little bit about Envers. I didn't want to bring it up since this is an OpenJPA mailing list. It does look interesting but I think it requires Hibernate which I do not intend to go back to. (I'm actually moving away from Hibernate). Also, I'm a bit hesitant to store copies of my rows in special audit tables since it also means database migration of those tables. I think serializing the audit log entries and putting them in one column is a better approach from a maintenance perspective. But it would be nice if OpenJPA would provide a callback or some mechanism (even if it's not standard JPA) that would allow further updates/inserts in a callback. Are there any such plans? /Bengt 2011/7/7 Andrew Thompson at2...@columbia.edu You might take a look at how hades ( http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David -- Q: What's the difference betweeen USL and the Graf Zeppelin? A: The Graf Zeppelin represented cutting edge technology for its time. signature.asc Description: OpenPGP digital signature
Re: Audit log with OpenJPA
Bengt: I'm enjoying this discussion because of the gymnastics that were required for my solutions. There is no easy way that I am aware of -- but I would LOVE to be wrong. Here's what I did (and, no, I'm not particularly proud of the solution): (1) For the primary entities, add tracking data (timestamp, user, etc.) plus an entityID. (2) On an update, * clone the data in the entity and update the tracking data. Copy the old entityID. * mark the previous entity as out-of-date (to speed up searches) * persist both entities (3) On an insert, * persist the new entity and increment the entityID (4) Update the queries to extract only those records where the 'out-of-date' flag was not set. This results in both current and historical entities being preserved in the same table. This will reduce performance as the indices are larger than are necessary for daily operation. This solution can be combined with a nightly script which moves out-of-date records from the main table to archived tables. The entity copy is a real pain but seems to be necessary. See the 'How to persist duplicate of an entity?' thread from January 2011. As I wrote earlier: I would _love_ to have function that would 'reset' a detached entity. Perhaps something like em.detach(myObj); // ensure that the entity has been detached. OpenJPAEntityManager kem = OpenJPAPersistence.cast(em); kem.reset(myObj); with the result of the 'reset' operation being a class which is again virgin, i.e. myObj.id == 0 myObj.version = 0 myObj.pcDetachedState == null myObj.pcStateManager == null and so forth for any children. This would greatly simplify the cloning process. -=- Jerry On Jul 7, 2011, at 12:50 PM, Bengt Rodehav [via OpenJPA] wrote: I actually use the same approach as Hades for createdBy, updatedBy, createdWhen and updatedWhen. In addition to this basic audit logging I also want to log all historical versions together with information about who updated the object. I've read a little bit about Envers. I didn't want to bring it up since this is an OpenJPA mailing list. It does look interesting but I think it requires Hibernate which I do not intend to go back to. (I'm actually moving away from Hibernate). Also, I'm a bit hesitant to store copies of my rows in special audit tables since it also means database migration of those tables. I think serializing the audit log entries and putting them in one column is a better approach from a maintenance perspective. But it would be nice if OpenJPA would provide a callback or some mechanism (even if it's not standard JPA) that would allow further updates/inserts in a callback. Are there any such plans? /Bengt 2011/7/7 Andrew Thompson [hidden email] You might take a look at how hades ( http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David If you reply to this email, your message will be added to the
Re: Audit log with OpenJPA
Hi, Good to see lot of new ideas about auditing. Here is one more [1] that uses OpenJPA specific internals of accessing the original state of an entity. The comparison of the current state and original state to decide upon the audit message is provided trivially in this example -- but at the level at which OpenJPA API is being accessed here, much more facilities are available to compute the 'delta'. [1] http://webspherepersistence.blogspot.com/2009/01/auditing-with-openjpa.html - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6559730.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Hi Folks! I also use auditing with OpenJPA: http://struberg.wordpress.com/2010/07/31/howto-changelog-with-jpa/ but it currently only works with a patched OpenJPA 2.2.x version. https://issues.apache.org/jira/browse/OPENJPA-1873 We fixed a few parts already, but the POST_LOAD_ON_MERGE isn't yet submitted since we didn't quite agree if it is spec conform. LieGrue, strub --- On Thu, 7/7/11, Pinaki Poddar ppod...@apache.org wrote: From: Pinaki Poddar ppod...@apache.org Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Thursday, July 7, 2011, 7:58 PM Hi, Good to see lot of new ideas about auditing. Here is one more [1] that uses OpenJPA specific internals of accessing the original state of an entity. The comparison of the current state and original state to decide upon the audit message is provided trivially in this example -- but at the level at which OpenJPA API is being accessed here, much more facilities are available to compute the 'delta'. [1] http://webspherepersistence.blogspot.com/2009/01/auditing-with-openjpa.html - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6559730.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Jarl, In production I run my application in OSGi (Apache Karaf). I use Apache Aries JPA support to publish entity manager factories and to enlist into JTA transactions. I've abandoned JEE since a while now - am even moving away from Spring I've also used XML before but I've recently moved to JSON. JSON is more compact and also very easy to visualize in a web GUI. However, I store the JSON in a database and I was hoping to use JPA for this - and also to have the audit logging take place in the original transaction. If a roll back occurs I do not want a false audit logging to take place. /Bengt 2011/7/7 Jari Fredriksson ja...@iki.fi How is your JPA accessed? I have done JPA, and Audit logging on top of it, but it was into XML files, and performed in a EJB3 Session Bean. The JPA objects were plain simple, but they were all accessed via a Session Bean Facade. I did it with XML because boss wanted so, but could have been database as well. 7.7.2011 19:49, Bengt Rodehav kirjoitti: I actually use the same approach as Hades for createdBy, updatedBy, createdWhen and updatedWhen. In addition to this basic audit logging I also want to log all historical versions together with information about who updated the object. I've read a little bit about Envers. I didn't want to bring it up since this is an OpenJPA mailing list. It does look interesting but I think it requires Hibernate which I do not intend to go back to. (I'm actually moving away from Hibernate). Also, I'm a bit hesitant to store copies of my rows in special audit tables since it also means database migration of those tables. I think serializing the audit log entries and putting them in one column is a better approach from a maintenance perspective. But it would be nice if OpenJPA would provide a callback or some mechanism (even if it's not standard JPA) that would allow further updates/inserts in a callback. Are there any such plans? /Bengt 2011/7/7 Andrew Thompson at2...@columbia.edu You might take a look at how hades ( http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html ) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in JPA/OpenJPA where I automatically can trigger the storing of an audit log entry as described above. Of course I can move this logic up from the persistence layer to a place where I can first have the entity manager persist my entity and then explicitly call another service to do the audit log. However, this is a pretty general mechanism that I would like to have automatic support for in my framework which is why I would like to have it pushed down into the persistence layer. Any ideas? /Bengt You could of course cheat. While you can not access the entiry manager, there is nothing to stop you using JDBC. It would probably not be a good idea to access a table that JPA is using, but if this audit trail is write only for this app and only read elsewhere that would solve the problem. David -- Q: What's the difference betweeen USL and the Graf Zeppelin? A: The Graf Zeppelin represented cutting edge technology for its time.
Re: Audit log with OpenJPA
Wow - lots of discussion here. I must have hit a good topic for once... I don't see why you shouldn't be proud of your solution. Storing all versions is a well known idiom for preserving your history. Basically you don't allow updates but create new versions all the time instead. I have considered it but I'm a bit afraid for the performance impact since performance is critical. I've never done this with JPA though. I guess from your mail that it's not that straight forward. Ideally what you would like is of course to have a composit key consisting of the id and version together. On every update a new instance would be persisted with the same id but a higher version. I don't think the existing version should be updated at all. I would prefer just creating a new version and leaving the previous (all the previous versions) intact. Doesn't JPA allow you to do this? /Bengt 2011/7/7 No1UNo je...@jerrycarter.org Bengt: I'm enjoying this discussion because of the gymnastics that were required for my solutions. There is no easy way that I am aware of -- but I would LOVE to be wrong. Here's what I did (and, no, I'm not particularly proud of the solution): (1) For the primary entities, add tracking data (timestamp, user, etc.) plus an entityID. (2) On an update, * clone the data in the entity and update the tracking data. Copy the old entityID. * mark the previous entity as out-of-date (to speed up searches) * persist both entities (3) On an insert, * persist the new entity and increment the entityID (4) Update the queries to extract only those records where the 'out-of-date' flag was not set. This results in both current and historical entities being preserved in the same table. This will reduce performance as the indices are larger than are necessary for daily operation. This solution can be combined with a nightly script which moves out-of-date records from the main table to archived tables. The entity copy is a real pain but seems to be necessary. See the 'How to persist duplicate of an entity?' thread from January 2011. As I wrote earlier: I would _love_ to have function that would 'reset' a detached entity. Perhaps something like em.detach(myObj); // ensure that the entity has been detached. OpenJPAEntityManager kem = OpenJPAPersistence.cast(em); kem.reset(myObj); with the result of the 'reset' operation being a class which is again virgin, i.e. myObj.id == 0 myObj.version = 0 myObj.pcDetachedState == null myObj.pcStateManager == null and so forth for any children. This would greatly simplify the cloning process. -=- Jerry On Jul 7, 2011, at 12:50 PM, Bengt Rodehav [via OpenJPA] wrote: I actually use the same approach as Hades for createdBy, updatedBy, createdWhen and updatedWhen. In addition to this basic audit logging I also want to log all historical versions together with information about who updated the object. I've read a little bit about Envers. I didn't want to bring it up since this is an OpenJPA mailing list. It does look interesting but I think it requires Hibernate which I do not intend to go back to. (I'm actually moving away from Hibernate). Also, I'm a bit hesitant to store copies of my rows in special audit tables since it also means database migration of those tables. I think serializing the audit log entries and putting them in one column is a better approach from a maintenance perspective. But it would be nice if OpenJPA would provide a callback or some mechanism (even if it's not standard JPA) that would allow further updates/inserts in a callback. Are there any such plans? /Bengt 2011/7/7 Andrew Thompson [hidden email] You might take a look at how hades ( http://hades.synyx.org/static/2.x/site/org.synyx.hades/reference/html/auditing.html ) does something close to what you're describing. Or http://www.jboss.org/envers -Andy On Thu, 2011-07-07 at 15:35 +0100, David Goodenough wrote: On Thursday 07 Jul 2011, Bengt Rodehav wrote: I'm using OpenJPA for persistence and would like to audit log any changes made to my entities. I serialize the objects to JSON (with Gson) and store them in a separate table in the database. Since the audit log needs to have the correct id's, the audit logging must take place after the entity has been persisted. I was hoping I could use the @PostPersist and @PostUpdate life cycle callbacks for this. I do seem to have the right information available and the serialization works fine but I don't know how I can persist my audit log entries at this point. From what I've read, I'm not allowed to use the entity manager in a Post lifecycle callback which of course makes this hard. What do you recommend? Is there a good place in
Re: Audit log with OpenJPA
Again an interesting approach. It's very similar to the way we did it in the old days (before O/R mappers were abundant). We rolled our own O/R mapping and also compared the previous state with the new state and saved the difference (XML in the database). I guess the same approach can be taken if OpenJPA internals are utilized. This time I was not going to bother determining the difference but store the whole object in a serialized form. Thus, my problem is just how to being able to store the serialized form in the database. Thanks, /Bengt 2011/7/7 Pinaki Poddar ppod...@apache.org Hi, Good to see lot of new ideas about auditing. Here is one more [1] that uses OpenJPA specific internals of accessing the original state of an entity. The comparison of the current state and original state to decide upon the audit message is provided trivially in this example -- but at the level at which OpenJPA API is being accessed here, much more facilities are available to compute the 'delta'. [1] http://webspherepersistence.blogspot.com/2009/01/auditing-with-openjpa.html - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6559730.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
Again an interesting solution. Although, as I wrote previously, I intend to store the whole object and therefore don't need to keep track of what has changed. I intend to do a JSON diff when viewing the audit log in the GUI instead. Your approach does, however, very elegantly solve the problem of how to store the audit log in the database with JPA since you actually store it in the object itself. /Bengt 2011/7/7 Mark Struberg strub...@yahoo.de Hi Folks! I also use auditing with OpenJPA: http://struberg.wordpress.com/2010/07/31/howto-changelog-with-jpa/ but it currently only works with a patched OpenJPA 2.2.x version. https://issues.apache.org/jira/browse/OPENJPA-1873 We fixed a few parts already, but the POST_LOAD_ON_MERGE isn't yet submitted since we didn't quite agree if it is spec conform. LieGrue, strub --- On Thu, 7/7/11, Pinaki Poddar ppod...@apache.org wrote: From: Pinaki Poddar ppod...@apache.org Subject: Re: Audit log with OpenJPA To: users@openjpa.apache.org Date: Thursday, July 7, 2011, 7:58 PM Hi, Good to see lot of new ideas about auditing. Here is one more [1] that uses OpenJPA specific internals of accessing the original state of an entity. The comparison of the current state and original state to decide upon the audit message is provided trivially in this example -- but at the level at which OpenJPA API is being accessed here, much more facilities are available to compute the 'delta'. [1] http://webspherepersistence.blogspot.com/2009/01/auditing-with-openjpa.html - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6559730.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
you actually store it in the object itself. That's right. This allows in-memory rollback. Given that the entire 'clean' or 'original' state is available in the technique I described, the application can make several decisions on a) what should be logged in an audit trail -- the entire object in a separate database/schema, a serialized blob or a JSON stream or whatever. b) because the technique is based on OpenJPA internal, if an application decides to take the 'blue pill', then it can simply ask the managed object which fields have been 'dirtied' if it requires to compute delta efficiently. Another related but separate point: I have noticed lot of requests/annoyance from the users on the limitation of callback listener methods having no access to the persistence context. Again, in OpenJPA runtime, the managed entity does know its persistence context. If the application is ready for a OpenJPA-specific cast, then the handle to the persistence context is right there in the entity. - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6560548.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: Audit log with OpenJPA
However, I store the JSON in a database and I was hoping to use JPA for this Actually, storing JSON in relational database would not permit query based on property values. But OpenJPA can provide JSON formatted-data for any web UI. To more about t, following references can help [1] http://www.ibm.com/developerworks/java/library/j-jest/?ca=drs- [2] http://openjpa.apache.org/jest [3] https://www.ibm.com/developerworks/mydeveloperworks/blogs/pinaki/tags/jest?lang=en - Pinaki -- View this message in context: http://openjpa.208410.n2.nabble.com/Audit-log-with-OpenJPA-tp6557932p6560732.html Sent from the OpenJPA Users mailing list archive at Nabble.com.
