Hi, object versioning looks interesting - even if that might require to use Spring Data JPA - but sounds like quite some work, to me.
One aspect that leaves me not completely convinced is consistency of Users' / Groups' / Any Objects' attributes (from internal storage, e.g. the one subjected to versioning) with data spread across all connected External Resources. Example: 1. user 'ilgrosso' with attribute 'salary=1000', mapped to same attribute on LDAP 2. an administrator raises that value to 'salary=2000' on Syncope, propagation sends such update to LDAP 3. another administrator sees the raised value, thinks 'it's a mistake!' and reverts it to 'salary=1000' by using the object versioning 4. LDAP has still 2000 How would you handle such cases? Regards. On 17/09/19 09:50, Andrea Patricelli wrote: > Hi Misagh, > > thanks for you interest in the proposal :) > > Il 17/09/19 09:41, Misagh Moayyed ha scritto: >> I think it's a very good idea, and should fit quite nicely with the auditing >> capabilities of the system to make auditors happy :) I have done some brief >> work with JaVers myself while I was evaluating it for very similar >> requirements; it's a an easy-to-use library and also has decent and flexible >> persistent mechanisms for record keeping; also somewhat Boot friendly. Maybe >> the two can work with each other in fact; you'd have JaVers as the >> high-level abstraction that keeps track of history and audits, while the >> persistence mechanisms are pushed out to different systems, one of which >> could be handled by Spring Data JPA, while also allowing for MongoDb and etc. > Yes I had the chance to work with it several times, too and can confirm the > high usability and easy configurability through very minimal code changes. >> >> A few things about history and audits in general that I think would be >> useful: >> >> 1. This should be an optional thing; one should have the ability to turn it >> off and I'd argue that it perhaps should be kept off by default until at >> least a release passes to avoid surprises. >> 2. Audit data can grow quite large depending on system activity and may blow >> out the storage system or cause performance issues, etc. There should be >> some sort of "automatic clean up" process that purges records after a >> reasonable time period, or sufficient warnings that adopters are to do the >> cleanup themselves. > > I agree about this two points, data growth is my biggest concern. Very nice > idea the cleanup process! I would add them to the requirements list. > >> --Misagh > > Best regards, > Andrea > >> >> ----- Original Message ----- >>> From: "Andrea Patricelli" <[email protected]> >>> To: "dev" <[email protected]> >>> Sent: Tuesday, September 17, 2019 12:02:25 PM >>> Subject: [DISCUSS] Object versioning >>> Hi all, >>> >>> in several of the last Syncope projects we recently worked on, we faced >>> a recurring issue: data inconsistency (at business level) and the need >>> to know who did what on some object. >>> >>> For "data inconsistency" I do not mean any issue related to database >>> records or provisioning, but the values of the user attributes at the >>> business level (identity info, groups or resources assigned, etc.). >>> Often a sysadmin wonders about "why this attribute has changed?" or "who >>> did this (wrong) change, how can I recover the previous value?" >>> Changes I'm referring to can come from automatic tasks (pull, scheduled) >>> or manual tasks performed by admin or manager users. >>> Syncope already provides info about last change date, change password >>> date or last modifier user, but in my opinion are not enough to >>> understand the history of the record. >>> >>> This is why I'm proposing to add to Syncope the possibility of >>> versioning objects. Something similar has been done for connectors and >>> resources configurations [1]. It is not a simple work I know :) >>> >>> Here are some requirements that I was thinking of: >>> >>> * Versioning should be done on all objects, USER, GROUP and ANY. >>> * Versioning could be heavy for systems with thousands or million of >>> users, so it should be a feature to enable/disable at each moment of >>> the project lifecycle. It should also be tunable, e.g. store only >>> attributes and not resources, groups, etc. >>> * In a first release I would leave apart all issues related to >>> provisioning. I would simply version data stored on Syncope database >>> * User should have the possibility to see a diff of what has changed >>> like per connectors and resources configurations. >>> * User should have the possibility to restore a certain value (or the >>> whole object?) >>> * User should be able to see, at least (in a first release), an >>> history of what has changed on data (e.g. name: foo -> bar) and who >>> did the change and when. >>> * User should be able to query by changed data like users who are >>> subject to "name" change in a certain period (not so useful, maybe). >>> >>> There are some usefu l open source libraries that support objects >>> versioning like JaVers [2] or [3], but I'm not sure if this last one >>> fits all the needs above. If you have any other proposal please don't >>> hesitate to attach it in this thread. >>> >>> WDYT? >>> >>> Have a nice day, >>> Andrea >>> >>> [1] https://issues.apache.org/jira/browse/SYNCOPE-1145 >>> >>> [2] https://javers.org/ >>> >>> [3] >>> https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.auditing >>> -- Francesco Chicchiriccò Tirasa - Open Source Excellence http://www.tirasa.net/ Member at The Apache Software Foundation Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail http://home.apache.org/~ilgrosso/
