[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #55 from Tomás Cohen Arazi --- Created attachment 113530 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113530&action=edit Bug 26963: (QA follow-up) Fix cases where we expected a list Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #56 from Tomás Cohen Arazi --- Created attachment 113531 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113531&action=edit Bug 26963: (QA follow-up) Fix up tests and cover case of undefined ccode While this technically shouldn't happen, if a library creates itemtype limits, then flips the pref, those rules are still in the db until a ccode rule is saved. To be extra safe, when checking for rules of one type, we should make sure the other type is undef - i.e. if looking for ccode rules, we should confirm the itype is undef for those rules Additionally, we shouldn't set the barcode now that we are not deleting all items, so we use copynumber for our item identification field as it doesn't need to be unique in the DB Signed-off-by: Nick Clemens Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #51 from Tomás Cohen Arazi --- Created attachment 113526 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113526&action=edit Bug 26963: (QA follow-up) Convert to ResultSets This patch removes the previously introduced private method by converting the arrayref returns to ResultSets appropriately and inlining the filter search queries. Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #52 from Tomás Cohen Arazi --- Created attachment 113527 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113527&action=edit Bug 26963: (QA follow-up) Migrate unit tests into pickup_location We wrote unit tests for _can_pickup_locations as part of this patchset, but then I inlined the method whilst golfing. This patch moves those tests into the existing pickup_locations test so we more thoroughly cover the case where branch transfer limits are in play. NOTE: The tests all assume that all items have an effective_itemtype and ccode. I'm pretty sure items all have a type at this point, but I'm less sure we enforce collection codes? Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #54 from Tomás Cohen Arazi --- Created attachment 113529 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113529&action=edit Bug 26963: (QA follow-up) Update mocked return of pickup_locations Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #50 from Tomás Cohen Arazi --- Created attachment 113525 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113525&action=edit Bug 26963: (follow-up) Change subroutine name for QA tools It didn't like the ending _at Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #53 from Tomás Cohen Arazi --- Created attachment 113528 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113528&action=edit Bug 26963: (QA follow-up) Don't delete existing data before tests Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963
--- Comment #49 from Tomás Cohen Arazi ---
Created attachment 113524
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113524&action=edit
Bug 26963: Don't call 'can_be_transferred' for each possible library for each
item
Currently When calling Koha::Template::Plugin::Branches::pickup_locations
we call pickup_location for each item of the bib, and for each item we get a
list of possible
branches, we then check those branches against the transfer limits, this is
inefficent
Given a system with 100 branches, and each branch having an item attached to
one bib (100 items)
we will call can_be_transferred ~1 times - and that will happen for each
hold placed on the bib
For me this patch reduced load time from 77 seconds to 18 seconds
To test:
1 - Find a bib
2 - Place 4 title level holds
3 - Add some branches and items for this bib to your system:
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"A"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"B"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"C"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"D"),pickup_location FROM branches;
INSERT INTO items
(biblionumber,biblioitemnumber,barcode,itype,homebranch,holdingbranch) SELECT
1,1,CONCAT("test-",branchcode),'BKS',branchcode,branchcode FROM branches;
4 - Set systempreferences:
UseBranchTransferLimits = 'enforce'
BranchTransferLimitsType = 'item type'
5 - Find the bib and click the holds tab
6 - Wait for a long time, it shoudl eventually load
7 - Apply patch and restart al the things
8 - Load the page again, it should be much faster
Signed-off-by: Andrew Fuerste-Henry
Signed-off-by: Bob Bennhoff
Signed-off-by: Martin Renvoize
Signed-off-by: Tomas Cohen Arazi
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113519|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113520|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #48 from Tomás Cohen Arazi --- Created attachment 113523 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113523&action=edit Bug 26963: Unit tests Signed-off-by: Martin Renvoize Signed-off-by: Tomas Cohen Arazi -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113521|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113516|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113515|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113517|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113518|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113514|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Status|Signed Off |Passed QA CC||[email protected] -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Tomás Cohen Arazi changed: What|Removed |Added Attachment #113513|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113512|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113510|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113509|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113508|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113507|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #47 from Martin Renvoize --- Created attachment 113521 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113521&action=edit Bug 26963: (QA follow-up) Fix up tests and cover case of undefined ccode While this technically shouldn't happen, if a library creates itemtype limits, then flips the pref, those rules are still in the db until a ccode rule is saved. To be extra safe, when checking for rules of one type, we should make sure the other type is undef - i.e. if looking for ccode rules, we should confirm the itype is undef for those rules Additionally, we shouldn't set the barcode now that we are not deleting all items, so we use copynumber for our item identification field as it doesn't need to be unique in the DB Signed-off-by: Nick Clemens Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113506|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #46 from Martin Renvoize --- Created attachment 113520 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113520&action=edit Bug 26963: (QA follow-up) Fix cases where we expected a list Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #45 from Martin Renvoize --- Created attachment 113519 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113519&action=edit Bug 26963: (QA follow-up) Update mocked return of pickup_locations Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #42 from Martin Renvoize --- Created attachment 113516 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113516&action=edit Bug 26963: (QA follow-up) Convert to ResultSets This patch removes the previously introduced private method by converting the arrayref returns to ResultSets appropriately and inlining the filter search queries. Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #43 from Martin Renvoize --- Created attachment 113517 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113517&action=edit Bug 26963: (QA follow-up) Migrate unit tests into pickup_location We wrote unit tests for _can_pickup_locations as part of this patchset, but then I inlined the method whilst golfing. This patch moves those tests into the existing pickup_locations test so we more thoroughly cover the case where branch transfer limits are in play. NOTE: The tests all assume that all items have an effective_itemtype and ccode. I'm pretty sure items all have a type at this point, but I'm less sure we enforce collection codes? Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963
Martin Renvoize changed:
What|Removed |Added
Attachment #113504|0 |1
is obsolete||
--- Comment #40 from Martin Renvoize ---
Created attachment 113514
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113514&action=edit
Bug 26963: Don't call 'can_be_transferred' for each possible library for each
item
Currently When calling Koha::Template::Plugin::Branches::pickup_locations
we call pickup_location for each item of the bib, and for each item we get a
list of possible
branches, we then check those branches against the transfer limits, this is
inefficent
Given a system with 100 branches, and each branch having an item attached to
one bib (100 items)
we will call can_be_transferred ~1 times - and that will happen for each
hold placed on the bib
For me this patch reduced load time from 77 seconds to 18 seconds
To test:
1 - Find a bib
2 - Place 4 title level holds
3 - Add some branches and items for this bib to your system:
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"A"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"B"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"C"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"D"),pickup_location FROM branches;
INSERT INTO items
(biblionumber,biblioitemnumber,barcode,itype,homebranch,holdingbranch) SELECT
1,1,CONCAT("test-",branchcode),'BKS',branchcode,branchcode FROM branches;
4 - Set systempreferences:
UseBranchTransferLimits = 'enforce'
BranchTransferLimitsType = 'item type'
5 - Find the bib and click the holds tab
6 - Wait for a long time, it shoudl eventually load
7 - Apply patch and restart al the things
8 - Load the page again, it should be much faster
Signed-off-by: Andrew Fuerste-Henry
Signed-off-by: Bob Bennhoff
Signed-off-by: Martin Renvoize
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #44 from Martin Renvoize --- Created attachment 113518 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113518&action=edit Bug 26963: (QA follow-up) Don't delete existing data before tests Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113505|0 |1 is obsolete|| --- Comment #41 from Martin Renvoize --- Created attachment 113515 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113515&action=edit Bug 26963: (follow-up) Change subroutine name for QA tools It didn't like the ending _at Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113503|0 |1 is obsolete|| --- Comment #39 from Martin Renvoize --- Created attachment 113513 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113513&action=edit Bug 26963: Unit tests Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113511|0 |1 is obsolete|| --- Comment #38 from Martin Renvoize --- Created attachment 113512 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113512&action=edit Bug 26963: [GOLF] Fix up tests and cover case of undefined ccode While this technically shouldn't happen, if a library creates itemtype limits, then flips the pref, those rules are still in the db until a ccode rule is saved. To be extra safe, when checking for rules of one type, we should make sure the other type is undef - i.e. if looking for ccode rules, we should confirm the itype is undef for those rules Additionally, we shouldn't set the barcode now that we are not deleting all items, so we use copynumber for our item identification field as it doesn't need to be unique in the DB Signed-off-by: Nick Clemens Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #37 from Martin Renvoize --- (In reply to Jonathan Druart from comment #22) > (In reply to Martin Renvoize from comment #15) > > I'd love to see the methods that are currently returning arrayrefs of > > Koha::Objects actually return proper Koha::Object sets which could then be > > chained properly. > > I don't understand. Are you suggesting that Koha::Item->pickup_locations > should return the Koha::Item object? > > If it's a "getter" method I see no reason not to return what you had asked > for :) It is indeed a getter.. but to me it clearly sounds like a relation getter.. i.e that it should return a related set.. i.e. a Koha::Libraries object as apposed to an ARRAREF of Koha::Library objects... with a proper Koha::Libraries object you can further refine the results by chaining additional ->search calls for example.. which is the exact case we're running into here. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #36 from Nick Clemens --- Created attachment 113511 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113511&action=edit Bug 26963: [GOLF] Fix up tests and cover case of undefined ccode While this technically shouldn't happen, if a library creates itemtype limits, then flips the pref, those rules are still in the db until a ccode rule is saved. To be extra safe, when checking for rules of one type, we should make sure the other type is undef - i.e. if looking for ccode rules, we should confirm the itype is undef for those rules Additionally, we shouldn't set the barcode now that we are not deleting all items, so we use copynumber for our item identification field as it doesn't need to be unique in the DB Signed-off-by: Nick Clemens -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113498|0 |1 is obsolete|| --- Comment #35 from Martin Renvoize --- Created attachment 113510 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113510&action=edit Bug 26963: [GOLF] Fix cases where we expected a list Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #34 from Martin Renvoize --- Created attachment 113509 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113509&action=edit Bug 26963: [GOLF] Update mocked return of pickup_locations Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113483|0 |1 is obsolete|| --- Comment #33 from Martin Renvoize --- Created attachment 113508 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113508&action=edit Bug 26963: [GOLF] Don't delete existing data before tests Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113481|0 |1 is obsolete|| --- Comment #32 from Martin Renvoize --- Created attachment 113507 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113507&action=edit Bug 26963: [GOLF] Migrate unit tests into pickup_location We wrote unit tests for _can_pickup_locations as part of this patchset, but then I inlined the method whilst golfing. This patch moves those tests into the existing pickup_locations test so we more thoroughly cover the case where branch transfer limits are in play. NOTE: The tests all assume that all items have an effective_itemtype and ccode. I'm pretty sure items all have a type at this point, but I'm less sure we enforce collection codes? Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113455|0 |1 is obsolete|| --- Comment #30 from Martin Renvoize --- Created attachment 113505 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113505&action=edit Bug 26963: (follow-up) Change subroutine name for QA tools It didn't like the ending _at Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963
Martin Renvoize changed:
What|Removed |Added
Attachment #113454|0 |1
is obsolete||
--- Comment #29 from Martin Renvoize ---
Created attachment 113504
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113504&action=edit
Bug 26963: Don't call 'can_be_transferred' for each possible library for each
item
Currently When calling Koha::Template::Plugin::Branches::pickup_locations
we call pickup_location for each item of the bib, and for each item we get a
list of possible
branches, we then check those branches against the transfer limits, this is
inefficent
Given a system with 100 branches, and each branch having an item attached to
one bib (100 items)
we will call can_be_transferred ~1 times - and that will happen for each
hold placed on the bib
For me this patch reduced load time from 77 seconds to 18 seconds
To test:
1 - Find a bib
2 - Place 4 title level holds
3 - Add some branches and items for this bib to your system:
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"A"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"B"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"C"),pickup_location FROM branches;
INSERT INTO branches (branchcode,branchname,pickup_location) SELECT
CONCAT(branchcode,"D"),CONCAT(branchname,"D"),pickup_location FROM branches;
INSERT INTO items
(biblionumber,biblioitemnumber,barcode,itype,homebranch,holdingbranch) SELECT
1,1,CONCAT("test-",branchcode),'BKS',branchcode,branchcode FROM branches;
4 - Set systempreferences:
UseBranchTransferLimits = 'enforce'
BranchTransferLimitsType = 'item type'
5 - Find the bib and click the holds tab
6 - Wait for a long time, it shoudl eventually load
7 - Apply patch and restart al the things
8 - Load the page again, it should be much faster
Signed-off-by: Andrew Fuerste-Henry
Signed-off-by: Bob Bennhoff
Signed-off-by: Martin Renvoize
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113480|0 |1 is obsolete|| --- Comment #31 from Martin Renvoize --- Created attachment 113506 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113506&action=edit Bug 26963: [GOLF] Convert to ResultSets This patch removes the previously introduced private method by converting the arrayref returns to ResultSets appropriately and inlining the filter search queries. Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Attachment #113453|0 |1 is obsolete|| --- Comment #28 from Martin Renvoize --- Created attachment 113503 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113503&action=edit Bug 26963: Unit tests Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #27 from Nick Clemens --- Created attachment 113498 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113498&action=edit Bug 26963: [GOLF] Fix cases where we expected a list -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 --- Comment #26 from Nick Clemens --- Created attachment 113483 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113483&action=edit Bug 26963: [GOLF] Don't delete existing data before tests -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 26963] Koha::Item::pickup_locations is very inefficient, causing timeouts on records with large numbers of holds/items
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26963 Martin Renvoize changed: What|Removed |Added Summary|Koha::Template::Plugin::Bra |Koha::Item::pickup_location |nches::pickup_locations is |s is very inefficient, |very inefficient, causing |causing timeouts on records |timeouts on records with|with large numbers of |large numbers of|holds/items |holds/items | -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
