[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Keywords|release-notes-needed| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #257 from Nick Clemens --- *** Bug 21146 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Status|Pushed to Master|RESOLVED Resolution|--- |FIXED --- Comment #256 from Martin Renvoize --- Enhancement, will not be backported to 18.05.x series. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #255 from Katrin Fischer --- (In reply to Ere Maijala from comment #254) > I've now updated https://wiki.koha-community.org/wiki/Elasticsearch with > more up to date information about the configuration. +1 Thx Ere! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #254 from Ere Maijala --- I've now updated https://wiki.koha-community.org/wiki/Elasticsearch with more up to date information about the configuration. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #253 from David Gustafsson --- Late to the party, but nice! Great work everyone! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Text to go in the||This patch significantly release notes||improves the performance of ||the ElasticSearch indexing ||process and also improves ||the maintainability of this ||area of the codebase. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #252 from Nicolas Legrand --- Bravo! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #251 from Tomás Cohen Arazi --- Congrats to everyone involved! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Nick Clemens changed: What|Removed |Added Status|Passed QA |Pushed to Master --- Comment #250 from Nick Clemens --- Awesome work all! Pushed to master for 18.11 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #249 from Ere Maijala --- Actually, we don't have anything about Elasticsearch in the current manual, but when this has been committed, I'd be happy to revise the wiki page at https://wiki.koha-community.org/wiki/Elasticsearch/Implementation. It would be nice to revise the manual too, but that's out of scope here. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #248 from Ere Maijala --- I'll try to also post a pull request for an update to the manual. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #247 from Ere Maijala --- Nick, I'm strongly supporting including these, and I'd like to nominate also bug 19365 for consideration. My test plan has been briefly put this: 1. Index records using the master version (and time the process) 2. Index records to another index after applying the patches here (and time the process) 3. Check that the new method is indeed faster after taking into account any caching-related differences. 4. Search for the same records in both indexes directly from Elasticsearch and compare the records to verify that the fields have been correctly indexed. 5. Change the title field rule from 245a to 245(abnp) and reindex. 6. Verify that the title field in the index now contains correctly concatenated subfields. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Nick Clemens changed: What|Removed |Added Keywords||rel_18_11_candidate --- Comment #246 from Nick Clemens --- After discussion with several people I am considering this one and bug 20244 still as ES is not widely used and needs these improvements to be more usable. I tested what I can see, but can I get a detailed test plan to ensure we are covered here? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #245 from David Gustafsson --- (In reply to Martin Renvoize from comment #243) > Fantastic effort everyone.. thank you all for the perseverance and continual > refinement of these patches and responses to feedback! > > I have added the Sponsored-by lines to each of the followup patches and > added two very very minor QA followups. > > Passing QA, again, well done and thank you to everyone involved! Great! I replied regarding the missing documentation for concatenation in bug 21331 (https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331#c7). Have not yet had time to look into it as perhaps need to create a new section for mappings.yaml in the Online manual. But agree this should be addressed. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #244 from Martin Renvoize --- *** Bug 21331 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Keywords||Manual -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Keywords||release-notes-needed -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Status|Signed Off |Passed QA --- Comment #243 from Martin Renvoize --- Fantastic effort everyone.. thank you all for the perseverance and continual refinement of these patches and responses to feedback! I have added the Sponsored-by lines to each of the followup patches and added two very very minor QA followups. Passing QA, again, well done and thank you to everyone involved! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82150|0 |1 is obsolete|| --- Comment #238 from Martin Renvoize --- Created attachment 82263 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82263=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82151|0 |1 is obsolete|| --- Comment #239 from Martin Renvoize --- Created attachment 82264 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82264=edit Bug 19893: Add code review fixes Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82152|0 |1 is obsolete|| --- Comment #240 from Martin Renvoize --- Created attachment 82265 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82265=edit Bug 19893: Increase test coverage Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #242 from Martin Renvoize --- Created attachment 82267 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82267=edit Bug 19893: (QA follow-up) Convert atomicupdate to .perl Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #241 from Martin Renvoize --- Created attachment 82266 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82266=edit Bug 19893: (QA follow-up) Spelling correction in POD Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82143|0 |1 is obsolete|| --- Comment #231 from Martin Renvoize --- Created attachment 82256 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82256=edit Bug 19893: Alternative optimized indexing for Elasticsearch Implement optimized indexing for Elasticsearch How to test: 1) Time a full elasticsearch re-index without this patch by running the rebuild_elastic_search.pl with the -d flag: `koha-shell -c "time rebuild_elastic_search.pl -d"`. 2) Apply this patch. 3) Time a full re-index again, it should be about twice at fast (for a couple of thousand biblios, with fewer biblios results may be more unpredictable). Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82147|0 |1 is obsolete|| --- Comment #235 from Martin Renvoize --- Created attachment 82260 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82260=edit Bug 19893: Restore and fix removed tests Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82145|0 |1 is obsolete|| --- Comment #233 from Martin Renvoize --- Created attachment 82258 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82258=edit Bug 19893: Add index status Add persistent per index "index status" state to provide useful user feedback when update of Elasticsearch server mappings fails Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82144|0 |1 is obsolete|| --- Comment #232 from Martin Renvoize --- Created attachment 82257 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82257=edit Bug 19893: Remove serialization format setting Default to base64 encoded binary MARC with MARCXML fallback if record exceeds maximum size Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82148|0 |1 is obsolete|| --- Comment #236 from Martin Renvoize --- Created attachment 82261 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82261=edit Bug 19893: Support for joined subfields in mappings Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82146|0 |1 is obsolete|| --- Comment #234 from Martin Renvoize --- Created attachment 82259 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82259=edit Bug 19893: Fix Elasticsearch tests. https://bugs.koha-community.org/show_bug.cgi?id=19893 Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Attachment #82149|0 |1 is obsolete|| --- Comment #237 from Martin Renvoize --- Created attachment 82262 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82262=edit Bug 19893: Fix tests and add tests for joined subfields Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #230 from David Gustafsson --- Great! Thanks for the help with rebasing. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #229 from Ere Maijala --- Thanks for your hard work David and Joonas! I've gone through the changes and tested once more, and I believe we're ready for QA. I did a very minor rebase caused by bug 18316 to mappings.tt where just another message had been added. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81965|0 |1 is obsolete|| --- Comment #227 from Ere Maijala --- Created attachment 82151 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82151=edit Bug 19893: Add code review fixes Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81966|0 |1 is obsolete|| --- Comment #228 from Ere Maijala --- Created attachment 82152 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82152=edit Bug 19893: Increase test coverage Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81964|0 |1 is obsolete|| --- Comment #226 from Ere Maijala --- Created attachment 82150 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82150=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81962|0 |1 is obsolete|| --- Comment #224 from Ere Maijala --- Created attachment 82148 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82148=edit Bug 19893: Support for joined subfields in mappings Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81963|0 |1 is obsolete|| --- Comment #225 from Ere Maijala --- Created attachment 82149 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82149=edit Bug 19893: Fix tests and add tests for joined subfields Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81961|0 |1 is obsolete|| --- Comment #223 from Ere Maijala --- Created attachment 82147 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82147=edit Bug 19893: Restore and fix removed tests Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81959|0 |1 is obsolete|| --- Comment #221 from Ere Maijala --- Created attachment 82145 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82145=edit Bug 19893: Add index status Add persistent per index "index status" state to provide useful user feedback when update of Elasticsearch server mappings fails Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81960|0 |1 is obsolete|| --- Comment #222 from Ere Maijala --- Created attachment 82146 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82146=edit Bug 19893: Fix Elasticsearch tests. https://bugs.koha-community.org/show_bug.cgi?id=19893 Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81957|0 |1 is obsolete|| --- Comment #219 from Ere Maijala --- Created attachment 82143 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82143=edit Bug 19893: Alternative optimized indexing for Elasticsearch Implement optimized indexing for Elasticsearch How to test: 1) Time a full elasticsearch re-index without this patch by running the rebuild_elastic_search.pl with the -d flag: `koha-shell -c "time rebuild_elastic_search.pl -d"`. 2) Apply this patch. 3) Time a full re-index again, it should be about twice at fast (for a couple of thousand biblios, with fewer biblios results may be more unpredictable). Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Attachment #81958|0 |1 is obsolete|| --- Comment #220 from Ere Maijala --- Created attachment 82144 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82144=edit Bug 19893: Remove serialization format setting Default to base64 encoded binary MARC with MARCXML fallback if record exceeds maximum size Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Ere Maijala changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81955|0 |1 is obsolete|| --- Comment #218 from David Gustafsson --- Created attachment 81966 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81966=edit Bug 19893: Increase test coverage -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78690|0 |1 is obsolete|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78693|0 |1 is obsolete|| --- Comment #212 from David Gustafsson --- Created attachment 81960 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81960=edit Bug 19893: Fix Elasticsearch tests. https://bugs.koha-community.org/show_bug.cgi?id=19893 Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78696|0 |1 is obsolete|| --- Comment #215 from David Gustafsson --- Created attachment 81963 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81963=edit Bug 19893: Fix tests and add tests for joined subfields Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78694|0 |1 is obsolete|| --- Comment #213 from David Gustafsson --- Created attachment 81961 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81961=edit Bug 19893: Restore and fix removed tests Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81284|0 |1 is obsolete|| --- Comment #216 from David Gustafsson --- Created attachment 81964 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81964=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78695|0 |1 is obsolete|| --- Comment #214 from David Gustafsson --- Created attachment 81962 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81962=edit Bug 19893: Support for joined subfields in mappings Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81903|0 |1 is obsolete|| --- Comment #217 from David Gustafsson --- Created attachment 81965 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81965=edit Bug 19893: Add code review fixes -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #209 from David Gustafsson --- Created attachment 81957 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81957=edit Bug 19893: Alternative optimized indexing for Elasticsearch Implement optimized indexing for Elasticsearch How to test: 1) Time a full elasticsearch re-index without this patch by running the rebuild_elastic_search.pl with the -d flag: `koha-shell -c "time rebuild_elastic_search.pl -d"`. 2) Apply this patch. 3) Time a full re-index again, it should be about twice at fast (for a couple of thousand biblios, with fewer biblios results may be more unpredictable). Sponsored-by: Gothenburg University Library Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78691|0 |1 is obsolete|| --- Comment #210 from David Gustafsson --- Created attachment 81958 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81958=edit Bug 19893: Remove serialization format setting Default to base64 encoded binary MARC with MARCXML fallback if record exceeds maximum size Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #78692|0 |1 is obsolete|| --- Comment #211 from David Gustafsson --- Created attachment 81959 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81959=edit Bug 19893: Add index status Add persistent per index "index status" state to provide useful user feedback when update of Elasticsearch server mappings fails Signed-off-by: Ere Maijala -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #208 from Joonas Kylmälä --- (In reply to David Gustafsson from comment #207) > Created new commit, if should be squashed I can do that if requested. Seems separate enough from the rest of the non-signed-off-by commits to justify its own commit. Will you fix the title "Bug 19893 - Alternative optimized indexing for Elasticsearch" too so I can do the final testing and sign-off the patches if all goes well. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #207 from David Gustafsson --- Created new commit, if should be squashed I can do that if requested. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #206 from David Gustafsson --- Created attachment 81955 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81955=edit Bug 19893: Increase test coverage -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #205 from David Gustafsson --- But to clarify, I will increase test coverage and add test for "ecode_record_from_result" on monday, and we can take it from there. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81901|0 |1 is obsolete|| --- Comment #204 from David Gustafsson --- Created attachment 81903 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81903=edit Bug 19893: Add code review fixes -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #203 from David Gustafsson --- Forgot to add it to sysprefs.sql though, will fix this. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #202 from David Gustafsson --- *"I have _now_ replaced them with exceptions instead." -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #201 from David Gustafsson ---
(In reply to Joonas Kylmälä from comment #197)
> Thanks for the patch!
>
> Instead of croak or die the exceptions need to be used, e.g.
> Koha::Exceptions::Exception->throw(""). Please do a git diff
> origin/master..HEAD to see all the dies and croaks (assuming HEAD contains
> your patches). And the many for loops I mentioned are in the subroutine
> marc_records_to_documents Koha/SearchEngine/Elasticsearch.pm – though take
> this as an optional thing to fix as I know this bug is blocking quite many
> other things.
>
> Continuing still code review... About
> installer/data/mysql/atomicupdate/
> bug_19893_elasticsearch_index_status_sysprefs.sql – a) is it a good idea to
> have these as a syspref (even though the user cannot see them)? b) if it's a
> good idea then the sysprefs should also be in
> installer/data/mysql/sysprefs.sql because otherwise in a new Koha
> installation the syspref will be missing.
>
> In the file Koha/SearchEngine/Elasticsearch.pm: s/string on the
> format/string in the format/
>
> Then we also need a test plan for this code. Like what steps need to be
> taken to index authorities and biblios and what should be the expected
> result.
>
> Unit tests
> (https://wiki.koha-community.org/wiki/Coding_Guidelines#PERL17:
> _Unit_tests_are_required_.28updated_Apr_26.2C_2017.29): at least
> decode_record_from_result is missing one
>
> If all the above mentioned things get fixed then I'm probably ready to
> sign-off but I still need to test this code actually before that.
I for some reason read it as "die" should be replaced with "croak". I have not
replaced them with exceptions instead.
I think it's a bit of an anti-pattern to put a block of code into a function if
only invoked in one location, so would prefer to leave the code as it is.
About the syspref, sadly there is no persistent variable store in Koha that I
know of. In other places where this is needed (like Koha version number), it is
stored in the syspref-table.
Fixed the grammatical error.
I don't know how much time I have to spend on unit tests. I think there is
close to 100% code coverage trough the other tests, but I recently checked this
with cover and there are some minor execution-paths that are never reached.
Which I could fix. If I where to write unit tests for all new helper-methods
added it could take some time.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81822|0 |1 is obsolete|| --- Comment #200 from David Gustafsson --- Created attachment 81901 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81901=edit Bug 19893: Add code review fixes -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #199 from Joonas Kylmälä --- The first patch's title "Bug 19893 - Alternative optimized indexing for Elasticsearch" should be Bug 19893: Alternative optimized indexing for Elasticsearch -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #198 from Joonas Kylmälä --- (In reply to Joonas Kylmälä from comment #197) > Then we also need a test plan for this code. Like what steps need to be > taken to index authorities and biblios and what should be the expected > result. There was one already in the first commit, so forget this. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #197 from Joonas Kylmälä ---
Thanks for the patch!
Instead of croak or die the exceptions need to be used, e.g.
Koha::Exceptions::Exception->throw(""). Please do a git diff
origin/master..HEAD to see all the dies and croaks (assuming HEAD contains your
patches). And the many for loops I mentioned are in the subroutine
marc_records_to_documents Koha/SearchEngine/Elasticsearch.pm – though take this
as an optional thing to fix as I know this bug is blocking quite many other
things.
Continuing still code review... About
installer/data/mysql/atomicupdate/bug_19893_elasticsearch_index_status_sysprefs.sql
– a) is it a good idea to have these as a syspref (even though the user cannot
see them)? b) if it's a good idea then the sysprefs should also be in
installer/data/mysql/sysprefs.sql because otherwise in a new Koha installation
the syspref will be missing.
In the file Koha/SearchEngine/Elasticsearch.pm: s/string on the format/string
in the format/
Then we also need a test plan for this code. Like what steps need to be taken
to index authorities and biblios and what should be the expected result.
Unit tests
(https://wiki.koha-community.org/wiki/Coding_Guidelines#PERL17:_Unit_tests_are_required_.28updated_Apr_26.2C_2017.29):
at least decode_record_from_result is missing one
If all the above mentioned things get fixed then I'm probably ready to sign-off
but I still need to test this code actually before that.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #196 from David Gustafsson --- I also took the liberty to fix commit line length to 80 (perhaps not for all comments though), since some where a bit long. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #195 from David Gustafsson ---
Ok, thanks for the feedback! I think I fixed most things except:
(In reply to Joonas Kylmälä from comment #188)
> Comment on attachment 78690 [details] [review]
> Bug 19893 - Alternative optimized indexing for Elasticsearch
>
> Review of attachment 78690 [details] [review]:
> -
>
> ::: Koha/SearchEngine/Elasticsearch.pm
> @@ +333,5 @@
> > +}
> > +push @{$record_document->{$target}}, $_data;
> > +}
> > +}
> > +foreach my $record (@{$records}) {
>
> Instead of stacking multiple for loops here, we can move this block to its
> own function, maybe called marc_record_to_document. This should make it
> easier to understand the code as the for loop would be then named and code
> doesn't go horizontally off the screen, right?
I don't get the line numbers to match up, but perhaps I'm using a different
base. I don't recognize this code in my current branch.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #194 from David Gustafsson --- Created attachment 81822 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81822=edit Bug 19893: Add code review fixes -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Katrin Fischer changed: What|Removed |Added Priority|P5 - low|P1 - high -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #193 from Joonas Kylmälä ---
Comment on attachment 78692
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78692
Bug 19893: Add index status
Review of attachment 78692:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=78692)
-
::: admin/searchengine/elasticsearch/mappings.pl
@@ +47,4 @@
>
> my $marc_type = lc C4::Context->preference('marcflavour');
>
> +my @index_names = ('biblios', 'authorities');
Should use $Koha::SearchEngine::BIBLIOS_INDEX and
$Koha::SearchEngine::AUTHORITIES_INDEX
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #192 from Joonas Kylmälä ---
Comment on attachment 78690
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78690
Bug 19893 - Alternative optimized indexing for Elasticsearch
Review of attachment 78690:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=78690)
-
::: Koha/SearchEngine/Elasticsearch.pm
@@ +464,5 @@
> +
> +if ($type eq 'sum') {
> +push @{$rules->{sum}}, $name;
> +}
> +elsif($type eq 'boolean') {
missing space after elsif
::: t/Koha/SearchEngine/Elasticsearch.t
@@ +219,5 @@
> +);
> +}
> +});
> +
> +my $see = Koha::SearchEngine::Elasticsearch->new({ index => 'biblios' });
Should use $Koha::SearchEngine::BIBLIOS_INDEX
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #191 from Joonas Kylmälä ---
Comment on attachment 81284
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81284
Bug 19893: Add pods, remove syspref, add tests for serialization format
Review of attachment 81284:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=81284)
-
::: Koha/SearchEngine/Elasticsearch.pm
@@ +592,5 @@
> +rules keyed by MARC field tags holding all the mapping rules for that
> particular tag.
> +
> +We can then iterate through all MARC fields for each record and apply all
> relevant
> +rules once per fields instead of retreiving fields multiple times for each
> mapping rule
> +wich is terribly slow.
wich -> which
@@ +605,5 @@
> +sub _get_marc_mapping_rules {
> +my ($self) = @_;
> +
> +my $marcflavour = lc C4::Context->preference('marcflavour');
> +my @rules;
There are now two definitions for rules variable. Let's remove this first one.
@@ +699,4 @@
> push @{$rules->{leader}}, @mappings;
> }
> else {
> +die("Invalid MARC field: $marc_field");
https://wiki.koha-community.org/wiki/Coding_Guidelines#Perl says "Instead of
die or croak when meeting unfavorable conditions in our code, we should raise
exceptions via Koha::Exceptions. "
There is also "die("Unmatched opening parenthesis for $marc_field");" and maybe
others that need to be fixed
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #190 from Joonas Kylmälä --- Comment on attachment 81284 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81284 Bug 19893: Add pods, remove syspref, add tests for serialization format Review of attachment 81284: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=81284) - ::: Koha/SearchEngine/Elasticsearch.pm @@ +310,5 @@ > return $self->_sort_fields_accessor(); > } > > +=head2 _process_mappings($mappings, $data, $record_document) > + Missing example usage @@ +480,4 @@ > return \@record_documents; > } > > +=head2 _field_mappings($facet, $suggestible, $sort, $target_name, > $target_type, $range) This POD is missing example usage: my @mappings = $self->_field_mappings($facet, $suggestible, $sort, $target_name, $target_type, $range) This will be somewhat useful to know what the function returns. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #189 from Joonas Kylmälä --- Comment on attachment 81284 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81284 Bug 19893: Add pods, remove syspref, add tests for serialization format Review of attachment 81284: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=81284) - ::: Koha/SearchEngine/Elasticsearch.pm @@ +484,5 @@ > > +Get mappings, an internal data structure later used by > L<_process_mappings($mappings, $data, $record_document)> > +to process MARC target data, for a MARC mapping. > + > +The returned C<$mappings> is to to be confused with mappings provided by > C<_foreach_mapping>, rather this Should be: The returned C<$mappings> is not to be confused with mappings provided by C<_foreach_mapping>, rather this (notice also the extra whitespace) @@ +515,5 @@ > +Elasticsearch document target field type. > + > +=item C<$range> > + > +An optinal range as a string on the format "-" or "", Should be: An optional range as a string in the format "-" or "", @@ +517,5 @@ > +=item C<$range> > + > +An optinal range as a string on the format "-" or "", > +where "" and "" are integers specifying a range that will be used > +for extracting a substing from MARC data as Elasticsearch field target value. substing -> substring @@ +522,5 @@ > + > +The first character position is "1", and the range is inclusive, > +so "1-3" means the first three characters of MARC data. > + > +If only "" is provided only one character as position "" will as position -> at position -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #188 from Joonas Kylmälä ---
Comment on attachment 78690
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78690
Bug 19893 - Alternative optimized indexing for Elasticsearch
Review of attachment 78690:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=78690)
-
::: Koha/SearchEngine/Elasticsearch.pm
@@ +333,5 @@
> +}
> +push @{$record_document->{$target}}, $_data;
> +}
> +}
> +foreach my $record (@{$records}) {
Instead of stacking multiple for loops here, we can move this block to its own
function, maybe called marc_record_to_document. This should make it easier to
understand the code as the for loop would be then named and code doesn't go
horizontally off the screen, right?
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #187 from Joonas Kylmälä --- Comment on attachment 81284 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81284 Bug 19893: Add pods, remove syspref, add tests for serialization format Review of attachment 81284: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=81284) - ::: Koha/SearchEngine/Elasticsearch.pm @@ +329,5 @@ > +The source data from a MARC record field. > + > +=item C<$record_document> > + > +Hashref representing the Elasticsearch document on which mappings should be > applied. extra space between "the Elasticsearch" -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #186 from Joonas Kylmälä --- Comment on attachment 78690 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78690 Bug 19893 - Alternative optimized indexing for Elasticsearch Review of attachment 78690: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=78690) - ::: Koha/SearchEngine/Elasticsearch.pm @@ +34,5 @@ > use Try::Tiny; > use YAML::Syck; > > +use List::Util qw( sum0 reduce ); > +use Search::Elasticsearch; use Search::Elasticsearch; is already defined above -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #185 from Ere Maijala --- I'm okay with the big patch. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #184 from Joonas Kylmälä --- If one big commit is fine for everybody else then I can pass it through this time. And if you come across the situation again where you have to change the commit or split them up you can do git reset --soft HEAD~1 and then just write new commit messages and in the end check with git diff whether the code is still identical to the situation before you started. I will do the actual code review hopefully by the end of this week. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 Martin Renvoize changed: What|Removed |Added Blocks||21331 See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=21331 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21331 [Bug 21331] [Elasticsearch] Concatenated subfields are not indexed correctly -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #183 from David Gustafsson --- What you could do is to to click "Show Obsolute" and download the two latest patches, then run "interdiff old_patch.patch new_patch.patch" and you will get the latest changes. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #182 from David Gustafsson --- (In reply to Joonas Kylmälä from comment #181) > Hi, > > could you still split the changes to their own commits? I was considering your previous comment: (In reply to Joonas Kylmälä from comment #166) > (In reply to David Gustafsson from comment #164) > > Created attachment 80983 [details] [review] [review] > > Bug 19893: Fixed some typos > > Please squash this to the previous patch as a new commit created by it is > not necessary > (https://wiki.koha-community.org/wiki/ > Development_IRC_meeting_27_September_2017) which I interpreted as patch was supposed to be squashed in. As I no longer have the specific git commits in my local git repo I would have to rebuild them from the different iterations of patches in this issue and I would rather not do that due time constrains and risk of screwing things up. I also think there might be a benefit to review the patch as a whole since you don't have to account for errors made in previous commits that might have been corrected in a later one. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #181 from Joonas Kylmälä --- Hi, could you still split the changes to their own commits? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81015|0 |1 is obsolete|| --- Comment #180 from David Gustafsson --- Created attachment 81284 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81284=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #179 from Joonas Kylmälä --- (In reply to Jonathan Druart from comment #176) > Or moved to Koha::FieldMappings. > Tests must be provided in either case. Koha::FieldMappings is not really related to the MARC to Elasticsearch mappings that we are talking here, since Koha::FieldMappings doesn't have rules/mappings whether the field or combination of fields is a boolean, sortable, and so on. The Elasticsearch.pm module is already object oriented and the functions we are now talking about are just helper functions that don't necessarily need to be object oriented. So I'm all in for the current solution David is suggesting (but little refactoring is needed). -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #178 from David Gustafsson --- There is also no meaningful way of testing them in isolation, tests cover them through the tests for "marc_records_to_document". -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #177 from David Gustafsson --- I can have a look at this an move the subs out, I confused them with another case where closures where used, so no closures thus they work also outside. I don't really agree this is an improvement since the subs are not supposed to be used outside of the subroutines in which they are defined, but it's no big deal. Don't think Koha::FieldMappings is the right way to put them though sinceis an internal data structure used by _process_mappings, that has no purpose outside of that subroutine. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 e.betemps changed: What|Removed |Added CC||e.bete...@gmail.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #176 from Jonathan Druart --- (In reply to Nick Clemens from comment #175) > The subroutine should be moved to the top level Or moved to Koha::FieldMappings. Tests must be provided in either case. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #175 from Nick Clemens --- (In reply to Joonas Kylmälä from comment #174) > (In reply to David Gustafsson from comment #173) > > I agree It might benefit from documentation, it should (or even can) not be > > moved outside though, since I think there are some closures within. It is > > also a specific and private helper-method, thus belongs inside the method. > > I couldn't see any closures in the two subroutines I mentioned (and with > "closure" I mean they don't refer to any variables in the outer subroutine). > The search functionality and indexing seemed to work fine also when they are > in the top level (I tested). Anyways, I put this topic on the Koha developer > meeting list and maybe we will get a better consensus on how to do this. The general consensus at the meeting was that seemed like ti could be a better function on its own and that the nesting was confusing in this case and didn't seem to be a traditional closure. The subroutine should be moved to the top level -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #174 from Joonas Kylmälä --- (In reply to David Gustafsson from comment #173) > I agree It might benefit from documentation, it should (or even can) not be > moved outside though, since I think there are some closures within. It is > also a specific and private helper-method, thus belongs inside the method. I couldn't see any closures in the two subroutines I mentioned (and with "closure" I mean they don't refer to any variables in the outer subroutine). The search functionality and indexing seemed to work fine also when they are in the top level (I tested). Anyways, I put this topic on the Koha developer meeting list and maybe we will get a better consensus on how to do this. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 --- Comment #173 from David Gustafsson --- I agree It might benefit from documentation, it should (or even can) not be moved outside though, since I think there are some closures within. It is also a specific and private helper-method, thus belongs inside the method. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #172 from Joonas Kylmälä ---
Comment on attachment 78690
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=78690
Bug 19893 - Alternative optimized indexing for Elasticsearch
Review of attachment 78690:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=78690)
-
::: Koha/SearchEngine/Elasticsearch.pm
@@ +309,5 @@
> +my $serialization_format =
> C4::Context->preference('ElasticsearchMARCSerializationFormat');
> +
> +my @record_documents;
> +
> +sub _process_mappings {
This sub could be moved out of the marc_records_to_documents sub so that it
could be then also documented
@@ +397,5 @@
>
> my $marcflavour = lc C4::Context->preference('marcflavour');
> my @rules;
>
> +sub _field_mappings {
Also I think this sub could benefit from documentation and not being inside
get_marc_mapping_rules as there is plenty of other things going already in
get_marc_mapping_rules so it would make it easier to follow the code.
I tried to make a follow-up patch with documentation but it's going pretty
slowly since I have to now gather the intel what each variable actually
contains.
$mappings: a tuple (the name of the field in elasticsearch index, "options" or
rules to modify the field data)
$record_document: document to put to Elasticsearch index
...
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #171 from David Gustafsson ---
(In reply to Joonas Kylmälä from comment #168)
> Comment on attachment 80971 [details] [review]
> Bug 19893: Add pods, remove syspref, add tests for serialization format
>
> Review of attachment 80971 [details] [review]:
> -
>
> ::: Koha/SearchEngine/Elasticsearch/Indexer.pm
> @@ +307,4 @@
> > if ( !$self->store ) {
> > my $params = $self->get_elasticsearch_params();
> > $self->store(
> > +Catmandu::Store::Elasticsearch->new(
>
> This line is changed and it somehow creates an unnoticable character that is
> different from the one being imported so deletion of biblio records doesn't
> work. This change of the line needs to be reverted.
Good catch, that was because I replaced some occurences of "ElasticSearch" with
"Elasticsearch" in comments (since that is how it is spelled), and accidently
searched and replaced that one. Fixed and attached new patch.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #81014|0 |1 is obsolete|| --- Comment #170 from David Gustafsson --- Created attachment 81015 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81015=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893 David Gustafsson changed: What|Removed |Added Attachment #80971|0 |1 is obsolete|| Attachment #80983|0 |1 is obsolete|| --- Comment #169 from David Gustafsson --- Created attachment 81014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=81014=edit Bug 19893: Add pods, remove syspref, add tests for serialization format Add missing pods, remove obsolete syspref and add test for serialization format for records exceeding max record size -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 19893] Alternative optimized indexing for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893
--- Comment #168 from Joonas Kylmälä ---
Comment on attachment 80971
--> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=80971
Bug 19893: Add pods, remove syspref, add tests for serialization format
Review of attachment 80971:
-->
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=19893=80971)
-
::: Koha/SearchEngine/Elasticsearch/Indexer.pm
@@ +307,4 @@
> if ( !$self->store ) {
> my $params = $self->get_elasticsearch_params();
> $self->store(
> +Catmandu::Store::Elasticsearch->new(
This line is changed and it somehow creates an unnoticable character that is
different from the one being imported so deletion of biblio records doesn't
work. This change of the line needs to be reverted.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
