[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #18 from Marcel de Rooy --- (In reply to Martin Renvoize from comment #17) > I'm interested in the discussions here and want to help this one move > forward.. I've had it added to the next dev meeting at a topic. I'm feeling > out of date and knowledge at this point so don't want to push either way > until we have that discussion. Great, thanks. Confident that we will decide the right thing, lol -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Martin Renvoize changed: What|Removed |Added CC||martin.renvoize@ptfs-europe ||.com --- Comment #17 from Martin Renvoize --- I'm interested in the discussions here and want to help this one move forward.. I've had it added to the next dev meeting at a topic. I'm feeling out of date and knowledge at this point so don't want to push either way until we have that discussion. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #16 from Victor Grousset/tuxayo --- (In reply to Marcel de Rooy from comment #14) > Please explain why a separate table causes bad bugs. Some issues that we > experienced in the past were related to autoincrement. Hm, I though bug 19532 comment 5 was about more stuff than autoincrement. Because these are indeed reliably avoidable now that it's known. > > Introducing a flag will cause bugs only on that moment. (found and fixed in > > the next months) > > Hopefully :) One could argue that having one table requires more careful > coding and is therefore more error prone? When doing delete flags, there should be something in the data layer to by default query only the non deleted data. Otherwise it will be hard to never forget to filter. And code everywhere will have some additional complexity to filter the data. It's a bit of code duplication in many places. Do we have all/almost all the interactions with the authorities table in the data layer? Otherwise it's not sure it's possible to deal with these issues :/ > Some related questions: > Why enforce something here that we have not been able to do in the last 6 > years for a few similar tables? Here, there is no tombstone table so introducing the flag is less work than doing that + removing the tombstone table. Still more work than for a totally new table like recalls. But before that, the question is, should we even avoid introducing a tombstone table? That's suddenly not as clear. > Will add yet another comment about the two scenarios from a more general > perspective more or less proving that both ways have pro and cons. > [...] > Please read this on dba.stackexchange.com: > https://dba.stackexchange.com/questions/14402/tombstone-table-vs-deleted- > flag-in-database-syncronization-soft-delete-scenari/14419 From the discussions and attempts to switch to delete flags in the last years, it seemed a given that a tombstone table was an anti-pattern but it turns out totally not Back to trying to find the contention point: Now that it's known how to avoid autoincrement issues with tombstone tables, do they have a significant drawback for our use case? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #15 from Marcel de Rooy --- Please read this on dba.stackexchange.com: https://dba.stackexchange.com/questions/14402/tombstone-table-vs-deleted-flag-in-database-syncronization-soft-delete-scenari/14419 It contains questions like: How fast do deletes need to be? How fast do un-deletes need to be? How often will deleted data be queried and will it be queried with data that has not been deleted? How fast do queries of deleted data need to be? Do you need to preserve only deleted items or changes as well? Do you need to keep the table/indexes on the primary table small? What partitioning and/or change tracking technologies are available on the database platform? How much disk space is available? Will the deleting occur on the fly or in batch operations? In this authorities case, deletes are not required to be very fast. And we are not undeleting right now. The deleted data is not queried often. We do not preserve all changes. There is no need to keep all indexes on deleted data (I removed one here as well). So I would tend (as objective as I can..) to the tombstone table here for performance reasons. (And less code as argued already.) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #14 from Marcel de Rooy --- (In reply to Victor Grousset/tuxayo from comment #13) Thanks for taking a look. > So IIUC the contention is that authorities is an existing table and adding a > deleted flag will cause it to be forgotten to be checked for in places => so > bad bugs. > VS having a separate table, which causes bad bugs. Please explain why a separate table causes bad bugs. Some issues that we experienced in the past were related to autoincrement. And note that I deliberately removed the autoincrement here from the deleted table. > Introducing a flag will cause bugs only on that moment. (found and fixed in > the next months) Hopefully :) One could argue that having one table requires more careful coding and is therefore more error prone? > Does having a separate table causes bugs over time or will that also be on > the moment of introduction? In other words, were the bugs due to the other > deleted* tables always there? Or were they more recent due to other changes > and it's easier to introduce bugs than with a deleted flag? See above too. I am having the strong imnpression that the option with a flag might introduce more bugs than the current development. Touching more code etc. Some related questions: Why enforce something here that we have not been able to do in the last 6 years for a few similar tables? And what about: I opened this bug close to 2 years ago. And already mentioned the 'merging tables' discussion in my description. When I finally submit code, the whole discussion is revived. Is that fair? This small patch set is imo ready to be used but now the whole things must be designed differently? Will add yet another comment about the two scenarios from a more general perspective more or less proving that both ways have pro and cons. We cannot say here that the deleted flag-option should be better. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Victor Grousset/tuxayo changed: What|Removed |Added CC||vic...@tuxayo.net --- Comment #13 from Victor Grousset/tuxayo --- So IIUC the contention is that authorities is an existing table and adding a deleted flag will cause it to be forgotten to be checked for in places => so bad bugs. VS having a separate table, which causes bad bugs. Introducing a flag will cause bugs only on that moment. (found and fixed in the next months) Does having a separate table causes bugs over time or will that also be on the moment of introduction? In other words, were the bugs due to the other deleted* tables always there? Or were they more recent due to other changes and it's easier to introduce bugs than with a deleted flag? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #12 from Marcel de Rooy --- (In reply to Jonathan Druart from comment #11) > We had the discussion for "recalls" and decided to not stick to the current > (wrong) pattern. > > So I don't think this is acceptable, you need to add a new column to the > existing table instead. > > See recalls.completed. > Also bug 19532 comment 5. I just read: Regarding the number of critical issues we had the last few months, I do not think it is a good idea to reuse this pattern. We should not move data from one table to another but add a deleted/old flag instead. Is that a discussion? :) There is a lot of discussion on 20271. And it is open for 6 years already. That is imo not a good argument to block a simple and needed development like this one. I could understand that if you add a NEW table that you should do it as you said. But adding this burden to an EXISTING table is really something else. Adding this 'requirement' effectively kills this development. Can imagine that we could raise serious issues too when we add a flag and forget a few occurrences throughout the whole codebase where we read records from the table. If if would be a requirement, we should have community consensus about it and make it a coding guideline. Which I do not recommend. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Jonathan Druart changed: What|Removed |Added CC||jonathan.dru...@gmail.com --- Comment #11 from Jonathan Druart --- We had the discussion for "recalls" and decided to not stick to the current (wrong) pattern. So I don't think this is acceptable, you need to add a new column to the existing table instead. See recalls.completed. Also bug 19532 comment 5. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added Blocks||30887 See Also|https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=30887 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30887 [Bug 30887] Gravestone page for deleted authorities or biblios -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added CC||janus...@gmail.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #10 from Marcel de Rooy --- (In reply to Katrin Fischer from comment #4) > (In reply to Marcel de Rooy from comment #1) > > Should we add 'fake' records for the missing identifiers in the authority > > record number range at upgrade time? These should most probably be the > > records that have been deleted once (ignoring other technical problems, > > perhaps with the autoincrement of the table). > > I would not do that. There could be huge ranges missing due to not resetting > PK during migration. Still thinking about filling it (optionally) with data from a table in a older backup or so? With some maint script perhaps, but not on this report. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added Assignee|koha-b...@lists.koha-commun |m.de.r...@rijksmuseum.nl |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=20271 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=36636 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #9 from Marcel de Rooy --- Created attachment 165043 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165043=edit Bug 30888: Unit tests Extending DelAuthority test in AuthoritiesMarc.t. Adding Koha/Authority.t for new method move_to_deleted. Test plan: Run both tests. Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #8 from Marcel de Rooy --- Created attachment 165042 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165042=edit Bug 30888: Copy info to deleted table Test plan: Delete an authority from the interface. Check presence in the table deletedauth_header, verify that the correct authid and authtype have been inserted. Bonus: Check leader position 05 in deleted table. Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #7 from Marcel de Rooy --- Created attachment 165041 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165041=edit Bug 30888: [DBIX] Schema file for deletedauth_header Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #6 from Marcel de Rooy --- Created attachment 165040 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165040=edit Bug 30888: Db rev for new table deletedauth_header In line with the other deleted tables. (See also bug 20271.) Note: Removed the auto_increment for this deleted table. There is no need for it and it could only mess up the table. Additionally, removed the index on origin. Test plan: Run updatedatabase. Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added Patch complexity|--- |Small patch Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Katrin Fischer changed: What|Removed |Added CC||neng...@gmail.com --- Comment #5 from Katrin Fischer --- *** Bug 10417 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #4 from Katrin Fischer --- (In reply to Marcel de Rooy from comment #1) > Should we add 'fake' records for the missing identifiers in the authority > record number range at upgrade time? These should most probably be the > records that have been deleted once (ignoring other technical problems, > perhaps with the autoincrement of the table). I would not do that. There could be huge ranges missing due to not resetting PK during migration. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #3 from Marcel de Rooy --- (In reply to Katrin Fischer from comment #2) > Should this be a separate table like we have now or should we use a deleted > flag on this table as it has been discussed for others? Separate table: see description. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #2 from Katrin Fischer --- Should this be a separate table like we have now or should we use a deleted flag on this table as it has been discussed for others? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 --- Comment #1 from Marcel de Rooy --- Should we add 'fake' records for the missing identifiers in the authority record number range at upgrade time? These should most probably be the records that have been deleted once (ignoring other technical problems, perhaps with the autoincrement of the table). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=30887 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 30888] Add a table for deletedauthorities
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30888 Marcel de Rooy changed: What|Removed |Added CC||julian.maur...@biblibre.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
