[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Jonathan Druart  changed:

   What|Removed |Added

 Blocks||19317

--- Comment #35 from Jonathan Druart  
---
See bug 19317 for the removal of this line.


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19317
[Bug 19317] Move of checkouts - Remove leftover
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #34 from Colin Campbell  ---
(In reply to Jonathan Druart from comment #32)
> Looks like the "update the fines" part should be removed (it should not
> hurt, no side-effects expected);
> 
> 2177 # Update the fines
> 2178 $dbh->do(q|UPDATE accountlines SET issue_id = ? WHERE issue_id
> = ?|, undef, $old_checkout->issue_id, $issue->issue_id);

Yes its a hangover from the stage when the code was changing issue_id in
issues, its now at best a waste of cpu cycles

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #33 from Katrin Fischer  ---
(In reply to Jonathan Druart from comment #32)
> Looks like the "update the fines" part should be removed (it should not
> hurt, no side-effects expected);
> 
> 2177 # Update the fines
> 2178 $dbh->do(q|UPDATE accountlines SET issue_id = ? WHERE issue_id
> = ?|, undef, $old_checkout->issue_id, $issue->issue_id);

hm?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #32 from Jonathan Druart  
---
Looks like the "update the fines" part should be removed (it should not hurt,
no side-effects expected);

2177 # Update the fines
2178 $dbh->do(q|UPDATE accountlines SET issue_id = ? WHERE issue_id =
?|, undef, $old_checkout->issue_id, $issue->issue_id);

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Jonathan Druart  changed:

   What|Removed |Added

 Blocks||19016


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19016
[Bug 19016] Add a script to fix corrupted data
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #31 from Mason James  ---
Pushed to 16.05.x, for 16.05.15 release

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #30 from Marcel de Rooy  ---
(In reply to Marcel de Rooy from comment #29)
> Opened bug 18996 for this *issue*.

The solution on 18996 is tied in with the fix on 17680.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Marcel de Rooy  changed:

   What|Removed |Added

 CC||colin.campbell@ptfs-europe.
   ||com
 Blocks||18996

--- Comment #29 from Marcel de Rooy  ---
(In reply to Jonathan Druart from comment #24)
> (In reply to Fridolin SOMERS from comment #23)
> > Just, we may use this error message in SIP serveur :
> > C4/SIP/ILS/Transaction/Checkin.pm ?
> 
> What do you mean?
> I do not think we should display a specific message in this case.

It seems that we still have a problem here.
If I check in an item where the issue id is already in old issues (I inserted
this id for testing purposes), AddReturn correctly gives me back me a zero
status and DataCorrupted message.
However, SIP somehow ignores it. It returns a 10 with ok=1 while it should
return a 10 with ok=0.
The problem is in ILS.pm line 219
$circ->ok( ( $checked_in_ok && $item ) || ( $item && $item->{patron} ) );
In this specific case $circ->ok was already set to false in do_checkin just
before that line. But now this line sets ok back to true.
I guess this is a more fundamental problem. Because if AddReturn did set the ok
flag to false for another reason, the same could happen. So I would propose to
solve it on another report.
I am not fully sure if we can just test if $circ->ok is still true before
running this line 219, and leave it false when it is false. 
@Colin: Could you have a look too ?

Opened bug 18996 for this *issue*.


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18996
[Bug 18996] SIP sets ok flag to true for refused checkin for data corruption
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #28 from David Cook  ---
Quick question... are there other bug reports for the other history tables?

I typically see this issue with the reserves/old_reserves table more than any
other.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966
Bug 18966 depends on bug 18651, which changed state.

Bug 18651 Summary: Move of checkouts is still not correctly handled
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18651

   What|Removed |Added

 Status|Pushed to Stable|RESOLVED
 Resolution|--- |FIXED

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #27 from Katrin Fischer  ---
These patches have been pushed to 16.11.x and will be in 16.11.10.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #26 from Mason James  ---
Created attachment 65294
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65294=edit
[16.11.x] Bug 18966: fix unneeded INSERT in AddReturn()

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #25 from Mason James  ---
Created attachment 65293
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65293=edit
[16.05.x] Bug 18966: fix unneeded INSERT in AddReturn()

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #24 from Jonathan Druart  
---
(In reply to Fridolin SOMERS from comment #23)
> Just, we may use this error message in SIP serveur :
> C4/SIP/ILS/Transaction/Checkin.pm ?

What do you mean?
I do not think we should display a specific message in this case.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Fridolin SOMERS  changed:

   What|Removed |Added

 Status|Pushed to Master|Pushed to Stable

--- Comment #23 from Fridolin SOMERS  ---
Pushed to 17.05.x, will be in 17.05.02.

Tested with mysql restart, it's ok.

Just, we may use this error message in SIP serveur :
C4/SIP/ILS/Transaction/Checkin.pm ?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #22 from Fridolin SOMERS  ---
(In reply to Fridolin SOMERS from comment #21)
> Checking for 17.05.x :
> 
> UT passes but there is a new warn "Violation of unique constraint in
> OldIssue".
Only when old_issues already contains datas, so no problem.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Fridolin SOMERS  changed:

   What|Removed |Added

 CC||fridolin.som...@biblibre.co
   ||m

--- Comment #21 from Fridolin SOMERS  ---
Checking for 17.05.x :

UT passes but there is a new warn "Violation of unique constraint in OldIssue".

$ prove t/db_dependent/Circulation/Returns.t
t/db_dependent/Circulation/Returns.t .. 1/4 Violation of unique constraint in
OldIssue at /home/koha/src/t/lib/TestBuilder.pm line 233.
t/db_dependent/Circulation/Returns.t .. ok   
All tests successful.
Files=1, Tests=4,  1 wallclock secs ( 0.01 usr  0.00 sys +  1.22 cusr  0.06
csys =  1.29 CPU)
Result: PASS

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Jonathan Druart  changed:

   What|Removed |Added

 Status|Passed QA   |Pushed to Master

--- Comment #20 from Jonathan Druart  
---
Pushed to master for 17.11, thanks to everybody involved!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Julian Maurice  changed:

   What|Removed |Added

 CC||julian.maur...@biblibre.com
 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Julian Maurice  changed:

   What|Removed |Added

  Attachment #65203|0   |1
is obsolete||

--- Comment #19 from Julian Maurice  ---
Created attachment 65272
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65272=edit
Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they
are moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved
to the old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at
code level on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the
move fails.
Before bug 18242 the data were lost, we inserted the value into
old_issues, which fails silently (because of RaiseError set to 0 in
Koha::Database), then delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment
0, the patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race
   conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit
905572910b3a - Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to
fix this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the
   checkin if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to
   deal with race conditions,
2- Display a warning on the checkin page and link to a
   solution/explanation
3- Communicate as much as we can on the proper fix: Update auto
   increment values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug
   18931)
5- Write and make available a maintenance script to fix corrupted data
   (TODO LATER)

Signed-off-by: Tomas Cohen Arazi 
Signed-off-by: Julian Maurice 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #18 from David Cook  ---
With a new database, I did the following:

1. Add new biblio named 'Test'
2. Add new item with barcode 'test'
3. Checkout item 'test' to borrower 'test'
4. Checkin item 'test'

5a. Can't restart the mysqld process since I'm currently running other things
on the same db server
5b. Run "OPTIMIZE TABLE issues".
#NOTE: This didn't update the auto_increment on issues. We're on MariaDB
10.0.28
5c. Run "truncate issues"
#NOTE: This did reset the auto_increment to 1.

6. Checkout item 'test' to borrower 'test'
#NOTE: It has a issue_id of 1
7. Checkin item 'test' from the checkout tab and observe message: 'Unable to
check in'. 
8. Try to checkin from header/banner and observe message: 'The item has not
been checked in due to a configuration issue in your system. You must ask an
administrator to take a look at the about page and search for the "data
problems" section'.

--

Functionally, I think this does the trick. 

In terms of messages, I think they could be improved. Maybe something like
"Unable to check in due to duplicate ID. Ask a system administrator for
assistance."

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #17 from David Cook  ---
(In reply to Jonathan Druart from comment #15)
> For these patches we need people strongly involved in the testing process.
> We (and especially I) lost a lot of time because the previous tries have not
> been tested widely enough.
> If you follow a test plan that I have written it will be nearly useless to
> validate it (since I already tested it when I wrote the patch). You will
> need to think about something I forgot.

By that token, every test plan is useless. 

Typically, I'll follow a test plan, and during the process I may try something
else and see a problem with the patch. 

I'm really invested in fixing this bug. It constantly affects us, and while we
have local workarounds, it drives me crazy. It's embarrassing constantly
telling people that there is a problem with Koha that we can't immediately fix. 

But I suppose if it bothers me that much... maybe I should involve myself more
in the testing process. 

I have to go buy some new headphones right now, but when I return, I will look
at this again, as this is perhaps my biggest pet peeve about Koha at the
moment.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #16 from Marcel de Rooy  ---
Will spend more time on this Friday if still needed..

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Tomás Cohen Arazi  changed:

   What|Removed |Added

 Status|Needs Signoff   |Signed Off
 CC||tomasco...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #15 from Jonathan Druart  
---
(In reply to David Cook from comment #14)
> (In reply to Jonathan Druart from comment #8)
> > (In reply to David Cook from comment #5)
> > > Do you have a test plan for testers to follow for this one?
> > 
> > Nope, you should only test this if you understand the problem and followed
> > the discussions on the previous bug reports.
> > 
> 
> I understand the problem, but super time poor, so was hoping for some easy
> copy/pasting. Just me being lazy I guess :p.

For these patches we need people strongly involved in the testing process. We
(and especially I) lost a lot of time because the previous tries have not been
tested widely enough.
If you follow a test plan that I have written it will be nearly useless to
validate it (since I already tested it when I wrote the patch). You will need
to think about something I forgot.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #14 from David Cook  ---
(In reply to Jonathan Druart from comment #8)
> (In reply to David Cook from comment #5)
> > Do you have a test plan for testers to follow for this one?
> 
> Nope, you should only test this if you understand the problem and followed
> the discussions on the previous bug reports.
> 

I understand the problem, but super time poor, so was hoping for some easy
copy/pasting. Just me being lazy I guess :p.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Mason James  changed:

   What|Removed |Added

 CC||m...@kohaaloha.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #65156|0   |1
is obsolete||

--- Comment #13 from Tomás Cohen Arazi  ---
Created attachment 65207
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65207=edit
(16.05.x) Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #65205|0   |1
is obsolete||

--- Comment #12 from Katrin Fischer  ---
Created attachment 65206
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65206=edit
[16.11.x] Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

Signed-off-by: Tomas Cohen Arazi 
Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #65155|0   |1
is obsolete||

--- Comment #11 from Tomás Cohen Arazi  ---
Created attachment 65205
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65205=edit
(16.11.x) Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #65154|0   |1
is obsolete||

--- Comment #10 from Tomás Cohen Arazi  ---
Created attachment 65204
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65204=edit
(17.05.x) Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #65151|0   |1
is obsolete||

--- Comment #9 from Tomás Cohen Arazi  ---
Created attachment 65203
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65203=edit
Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #8 from Jonathan Druart  
---
(In reply to David Cook from comment #5)
> Do you have a test plan for testers to follow for this one?

Nope, you should only test this if you understand the problem and followed the
discussions on the previous bug reports.

(In reply to Katrin Fischer from comment #6)
> > => It will only reject 1 checkin (max is 1 * MySQL restart), no need to 
> > deal with race conditions,
> 
> Hm, I think it can be more than one, depending on how many items you check
> out and return immediately before the restart (check-out 3, check-in 3,
> restart mysql, PK is behind by 3)

Yes indeed!

(In reply to Marcel de Rooy from comment #7)
> For another approach, a few unpolished patches on 18970 based on reverting
> 18242 and not keeping the renumbering (taking some possible data loss on
> older issues for granted).

The goal of this patch is to have a simple and safe fix for stable releases. I
would like to avoid black magic and make sure we are going to propose something
riskless.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #7 from Marcel de Rooy  ---
For another approach, a few unpolished patches on 18970 based on reverting
18242 and not keeping the renumbering (taking some possible data loss on older
issues for granted).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Marcel de Rooy  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=18970

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Katrin Fischer  changed:

   What|Removed |Added

 CC||katrin.fisc...@bsz-bw.de

--- Comment #6 from Katrin Fischer  ---
(In reply to David Cook from comment #5)
> Do you have a test plan for testers to follow for this one?

Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.

For starters, instead of BOOM we should see the new warning ;) Please note the
dependencies.

> => It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal 
> with race conditions,

Hm, I think it can be more than one, depending on how many items you check out
and return immediately before the restart (check-out 3, check-in 3, restart
mysql, PK is behind by 3)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

David Cook  changed:

   What|Removed |Added

 CC||dc...@prosentient.com.au

--- Comment #5 from David Cook  ---
Do you have a test plan for testers to follow for this one?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Marcel de Rooy  changed:

   What|Removed |Added

 CC||m.de.r...@rijksmuseum.nl

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #4 from Jonathan Druart  
---
Created attachment 65156
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65156=edit
[16.05.x] Bug 18966: Do not deal with duplicate issue_id on checkin

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #3 from Jonathan Druart  
---
Created attachment 65155
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65155=edit
[16.11.x] Bug 18966: Do not deal with duplicate issue_id on checkin

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #2 from Jonathan Druart  
---
Created attachment 65154
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65154=edit
[17.05.x] Bug 18966: Do not deal with duplicate issue_id on checkin

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

Jonathan Druart  changed:

   What|Removed |Added

 Status|ASSIGNED|Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18966] Move of checkouts - Deal with duplicate IDs at DBMS level

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18966

--- Comment #1 from Jonathan Druart  
---
Created attachment 65151
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65151=edit
Bug 18966: Do not deal with duplicate issue_id on checkin

Koha suffers of big bugs due to its history: When data are deleted, they are
moved to another tables.
For instance issues and old_issues: when a checkin is done, it is moved to the
old_issues table.
That leads to a main problem that is described on
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix

However we tried first to fix the problem (for issues/old_issues) at code level
on bug 18242.
The goal was to prevent data lost.
Data lost may happens in this case:
Check an item out (issue_id = 1)
Check an item in (issue_id = 1)
Restart MySQL (reset auto increment for issue_id to 1)
Check an item out (issue_id = 1)
Check an item in => BOOM, the issue_id is a PK in old_issues and the move
fails.
Before bug 18242 the data were lost, we inserted the value into old_issues,
which fails silently (because of RaiseError set to 0 in Koha::Database), then
delete the row from issues.
That has been fixed using a transaction.

This patch introduced a regression we tried to fix on bug 18651 comment 0, the
patron was charged even if the checkin was rejected.
A good way to fix that would have been to LOCK the tables:
1- Start a transaction
2- LOCK the table to make sure nobody will read id and avoid race conditions
3- Move the content from one table to the other, dealing with ids
4- UNLOCK the table
5- Commit the transaction
But there were problems using LOCK and DBIx::Class (See commit 905572910b3a -
Do no LOCK/UNLOCK the table).

Finally the solution implemented is not acceptable for several reasons:
- batch checkins may fail
- issue_id will always stay out of sync (between issues and old_issues)
See 18651 comment 66.

Since the next stable releases are very soon, and we absolutely need to fix
this problem, I am suggesting to:
1- Execute the move in a transaction to avoid data lost and reject the checkin
if we face IDs dup
=> It will only reject 1 checkin (max is 1 * MySQL restart), no need to deal
with race conditions,
2- Display a warning on the checkin page and link to a solution/explanation
3- Communicate as much as we can on the proper fix: Update auto increment
values when the DBMS is restarted -
https://wiki.koha-community.org/wiki/DBMS_auto_increment_fix
4- Display a warning on the about page for corrupted data (see bug 18931)
5- Write and make available a maintenance script to fix corrupted data (TODO
LATER)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/