[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Katrin Fischer changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=28884 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Status|Pushed to oldstable |Pushed to oldoldstable -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Version(s)|21.11.00,21.05.05 |21.11.00,21.05.05,20.11.14 released in|| Status|Pushed to stable|Pushed to oldstable --- Comment #119 from Victor Grousset/tuxayo --- Backported: Pushed to 20.11.x branch for 20.11.14 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #118 from Martin Renvoize --- Any chance of having this for 20.11 now that 29284 is in master and 21.11 and 21.05? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Nick Clemens changed: What|Removed |Added Blocks||29418 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29418 [Bug 29418] Zebra clean_search_term does not clean search and so results in error -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #117 from Fridolin Somers --- Yep, I was waiting for Bug 29284. Will be in 20.11.12 release. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #116 from Martin Renvoize --- Can we get a backport for this for 20.11 pretty please? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Blocks||29284 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29284 [Bug 29284] Koha dies when an analytics search fails in Elasticsearch -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Kyle M Hall changed: What|Removed |Added Status|Pushed to master|Pushed to stable Version(s)|21.11.00|21.11.00,21.05.05 released in|| CC||k...@bywatersolutions.com --- Comment #115 from Kyle M Hall --- Pushed to 21.05.x for 21.05.05 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #114 from Jonathan Druart --- Pushed to master for 21.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Jonathan Druart changed: What|Removed |Added Status|Passed QA |Pushed to master Version(s)||21.11.00 released in|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #113 from Martin Renvoize --- (In reply to Andrew Nugged from comment #112) > :huggs: > > yeah, right, Petro, look, we should next time add follow-up (thanks Martin!) No worries; if there's a mistake to be made like this I've probably made it at least a few times already myself ;) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #112 from Andrew Nugged --- :huggs: yeah, right, Petro, look, we should next time add follow-up (thanks Martin!) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Status|Signed Off |Passed QA --- Comment #111 from Martin Renvoize --- Phew.. I had an old branch hanging around I could diff with. Passing QA again, looks like everything has been taken care of. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124566|0 |1 is obsolete|| --- Comment #110 from Martin Renvoize --- Created attachment 124715 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124715=edit Bug 28316: (QA follow-up) Make clean_search_term public With all the work that's gone into improving the internal _clean_search_term method I feel we should expose it publically as it's going to be more widely helpful Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124565|0 |1 is obsolete|| --- Comment #109 from Martin Renvoize --- Created attachment 124714 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124714=edit Bug 28316: escape ES ranges if QueryAutoTruncate is enabled if QueryAutoTruncate enabled we will have any special operators ruined for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]" so no reason to keep ranges when QueryAutoTruncate set to "enabled" 1) enable QueryAutoTruncate at your sysprefs. 2) perform a search using range, for example: "[1999 TO 2020]", it shouldn't work the way it's supposed to. 3) apply the patch. 4) perform the same search with range, ensure that it works correctly. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124564|0 |1 is obsolete|| --- Comment #108 from Martin Renvoize --- Created attachment 124713 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124713=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124563|0 |1 is obsolete|| --- Comment #107 from Martin Renvoize --- Created attachment 124712 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124712=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124562|0 |1 is obsolete|| --- Comment #106 from Martin Renvoize --- Created attachment 124711 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124711=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning. To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124561|0 |1 is obsolete|| --- Comment #105 from Martin Renvoize --- Created attachment 124710 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124710=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #124560|0 |1 is obsolete|| --- Comment #104 from Martin Renvoize --- Created attachment 124709 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124709=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #103 from Martin Renvoize --- For future reference, once a bug has gone through SO and QA please respond to change requests as follow-ups... that way it's easy for a reviewer to see that the requested change has been made and nothing else beyond it.. with the changed squashed in it's much more difficult.. especially if one has cleaned up their previous branches. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #102 from Peter Vashchuk --- (In reply to Jonathan Druart from comment #91) > Questions and suggestions: > 1. Could we use named capture in the regexs? > 2. What means XI? > 3. Why do we need the 'e' modifier in the second one? > 4. Can we split on several lines the regexs to make them more readable? > (/xms) I implemented those changes and pushed the changed version of the patches, please take a look :) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #101 from Peter Vashchuk --- Created attachment 124566 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124566=edit Bug 28316: (QA follow-up) Make clean_search_term public With all the work that's gone into improving the internal _clean_search_term method I feel we should expose it publically as it's going to be more widely helpful Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #100 from Peter Vashchuk --- Created attachment 124565 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124565=edit Bug 28316: escape ES ranges if QueryAutoTruncate is enabled if QueryAutoTruncate enabled we will have any special operators ruined for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]" so no reason to keep ranges when QueryAutoTruncate set to "enabled" 1) enable QueryAutoTruncate at your sysprefs. 2) perform a search using range, for example: "[1999 TO 2020]", it shouldn't work the way it's supposed to. 3) apply the patch. 4) perform the same search with range, ensure that it works correctly. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #99 from Peter Vashchuk --- Created attachment 124564 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124564=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #98 from Peter Vashchuk --- Created attachment 124563 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124563=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #97 from Peter Vashchuk --- Created attachment 124562 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124562=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning. To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #96 from Peter Vashchuk --- Created attachment 124561 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124561=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Peter Vashchuk changed: What|Removed |Added Attachment #124372|0 |1 is obsolete|| Attachment #124373|0 |1 is obsolete|| Attachment #124374|0 |1 is obsolete|| Attachment #124375|0 |1 is obsolete|| Attachment #124376|0 |1 is obsolete|| Attachment #124377|0 |1 is obsolete|| Attachment #124378|0 |1 is obsolete|| --- Comment #95 from Peter Vashchuk --- Created attachment 124560 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124560=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Peter Vashchuk changed: What|Removed |Added Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Blocks|11175 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11175 [Bug 11175] Show the parent record's component parts in the detailed views -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 mathieu saby changed: What|Removed |Added CC||mathsaby...@gmail.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #94 from Jonathan Druart --- (In reply to Andrew Nugged from comment #92) > 1: yes, naming can be added > > 2: this is a "placeholder" which: Yes, I got that, and L for Left and R for Right. But then XI for... what? :) > 3: because it has " .. ? .. : .." depending of what bracket was found No, the second regex does not have it. > , but > otherwise if you speaking security-wise and don't want to have "e"'s at all > this RE probably can be split to few "just with different brackets" each, > should Petro split this regex to two without "e"? We discussed this too, I > also pointed that this is less secure "if something changes", but in this > case, I don't see how this can be hacked (how?) Looks ok for now. But if it can be avoided I should do it. > 4: for sure, this is good point. And some more comments in code. Let's do > this. Thx! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #93 from Andrew Nugged --- > This is for temporary hiding out current brackets from "next RegExs" I agree that this is not the best solution: with 0.00..1% probability that some books can have in the name "~~LC1~~" and this then will make code work unexpectedly for that book ... but then we need to have ES to allow to index "~~..~~" things, ... :). I also pointed this to Petro but we thought together that then the best solution is to scan text for placeholders and change placeholder (i.e. "generate placeholder" to be unique) but oh – that then will make code so complicated... we will create our own MIME then :)... If think from perfectionism POV this not solved yet, but other way – ... )... eh? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Andrew Nugged changed: What|Removed |Added Status|Passed QA |In Discussion --- Comment #92 from Andrew Nugged --- 1: yes, naming can be added 2: this is a "placeholder" which: a) first: part of text removed and replaced by placeholders (~~XI1~~ / ~~LC1~~ for example and others, life "LC for LeftCurly" and so on). This is for temporary hiding out current brackets from "next RegExs" b) then text processed expecting that all others curly/square brackets and regex and what must be kept – removed and hidden, so we can clean up and be brave with all other not-replaced-by-placeholders brackets, c) so we cleaning up text, screening, removing or whatever those special characters, d) and now we replacing BACK those placeholders with their saved in step a) values back to keep unscreened and working special characters we wanted to be kept 3: because it has " .. ? .. : .." depending of what bracket was found, but otherwise if you speaking security-wise and don't want to have "e"'s at all this RE probably can be split to few "just with different brackets" each, should Petro split this regex to two without "e"? We discussed this too, I also pointed that this is less secure "if something changes", but in this case, I don't see how this can be hacked (how?) 4: for sure, this is good point. And some more comments in code. Let's do this. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Jonathan Druart changed: What|Removed |Added CC||jonathan.druart+koha@gmail. ||com --- Comment #91 from Jonathan Druart --- +$term =~ s/(?https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=28682 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Blocks||11175 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11175 [Bug 11175] Show the parent record's component parts in the detailed views -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Blocks||26314 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26314 [Bug 26314] "Volumes: show volumes" showing regardless of whether there are volumes linked to the record -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added CC||ha...@orex.es --- Comment #90 from Martin Renvoize --- *** Bug 28326 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added CC||martin.renvoize@ptfs-europe ||.com Status|Signed Off |Passed QA --- Comment #89 from Martin Renvoize --- Wow, this is a pretty rigorous treatment.. I'm impressed. I've tested to the extent of my knowledge and am really happy with the results. No regressions found, but I'd love to see it pushed and tested with various configurations in the wild with quick fixes to follow. Passing QA. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #88 from Martin Renvoize --- Created attachment 124378 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124378=edit Bug 28316: (QA follow-up) Make clean_search_term public With all the work that's gone into improving the internal _clean_search_term method I feel we should expose it publically as it's going to be more widely helpful Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123606|0 |1 is obsolete|| --- Comment #87 from Martin Renvoize --- Created attachment 124377 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124377=edit Bug 28316: escape ES ranges if QueryAutoTruncate is enabled if QueryAutoTruncate enabled we will have any special operators ruined for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]" so no reason to keep ranges when QueryAutoTruncate set to "enabled" 1) enable QueryAutoTruncate at your sysprefs. 2) perform a search using range, for example: "[1999 TO 2020]", it shouldn't work the way it's supposed to. 3) apply the patch. 4) perform the same search with range, ensure that it works correctly. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123605|0 |1 is obsolete|| --- Comment #86 from Martin Renvoize --- Created attachment 124376 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124376=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123604|0 |1 is obsolete|| --- Comment #85 from Martin Renvoize --- Created attachment 124375 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124375=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123603|0 |1 is obsolete|| --- Comment #84 from Martin Renvoize --- Created attachment 124374 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124374=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning. To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123602|0 |1 is obsolete|| --- Comment #83 from Martin Renvoize --- Created attachment 124373 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124373=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Martin Renvoize changed: What|Removed |Added Attachment #123601|0 |1 is obsolete|| --- Comment #82 from Martin Renvoize --- Created attachment 124372 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124372=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley Signed-off-by: Martin Renvoize -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Andreas Jonsson changed: What|Removed |Added CC||andreas.jons...@kreablo.se -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #81 from Andrew Nugged --- Why this fuzziness and the extra job was done by us? We wanted to limit syntax errors from ES and make it easier for librarians to copy-paste just book names and have results, but NOT TO FORCE THEM backslash or re-edit text "for as long as possible", so for example earlier, they pasted "normal human phrases", like each of these: The Book! The Book [directed by robert b weide] The Book: and that lead earlier for ES to crash and bring "yellow Error box", even worse: before Bug 28736 people were confused that "oh, nothing found!" but for real there not even searched because of syntax error. Now, these above examples working and finding results (except the middle one because no such book in test DB yet :) ). -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #80 from Andrew Nugged --- So, the patch adds more order with "QueryRegexEscapeOptions" than before, same with "QueryAutoTruncate", and keeps ranges working as well. Yet it l+ng read above :), for a strong coffee and murky cat nearby ^_^ -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #79 from Andrew Nugged ---
(In reply to Victor Grousset/tuxayo from comment #41)
> > 4) additionally perform a search using range,
Vicor, this seems working now,
By the way, it should find normally this with and without patches for you:
date-of-publication:[1983 TO 1985]
But before the patch, if you will use "not including border values" ranges:
date-of-publication:{1983 TO 1985}
and then because not-patched code converts it to this:
date-of-publication:"1983 TO 1985"
it even did not fail to "yellow error box", just said "no results", but
there is: with patch now it finds 4 books for me on test DB for {1983 TO 1985}.
Besides you for sure can find when ES crashes, :hugs:
but without the patch, you will have much more possibilities to get that ES
syntax error now.
And I covered come in/outs and experiments above
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #78 from Andrew Nugged --- And the exclamation mark: == this used to be the "negate" command for words in queries, ONLY when '!' sticking to the beginning of the word, so patch fuzziness added: non-sticky and trailing exclamations marks removed, so in the case of the search: Dalek! without the patch, this brought a yellow error box, but with the patch dangling "!" was removed and it just passes "Dalek" to ES. It finds two books in our test DB, btw. And this one: Dalek !War works normally with both, with and without the patch. It searches for "Dalek" but negates "war" word presence. In our test dev-DB by the way, it gives just one book as a result. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #77 from Andrew Nugged ---
Tricks with colons:
==
colons in Koha expected to separate search field and search value,
so "title:good" should search word "good" only in "titles",
but if you will use "some : like this" form, when copypasting from usual
combined
book names we often have, current code removed colons at all from the text, if
colon has the space at any side, but there was forgotten trailing ":" removal,
i.e. ES failed without patch if to search:
Books:
also, some fuzziness added, for example, if you rename any book to have "23:59"
in the name, then will search with an old code for:
title:23:59
it will die ("yellow error box"),
but with the patch, it will find, as this transformed by the patch to:
title:23\:59
i.e. there every next semicolon after first in the term preslashed.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #76 from Andrew Nugged ---
Ranges in ES language
=
let's disable QueryAutoTruncate for not adding those asterisks,
and QueryRegexEscapeOptions doesn't matter here unless we will add
regular expressions. So:
ES allows us to have ranges requests, like:
novel {1947 TO 1950]
this expects to find word "novel" and one of the words in the range between
"1948 to 1950", and 1948 said here because '{' means "not included in range"
value, but starting from it (i.e. not '>=', but '>'),
but code in current master replaces '{' with '"', and then even more,
unbalanced
(unpaired) '"' removed as well, so the final request to ES will be:
novel 1947 TO 1950]
which has 'hanging ]' that ES doesn't understands and throws an exception
(welcome to the 'yellow Error box' instead of 'found' result here again).
Now, with the patch:
novel {1947 TO 1950]
transferred correctly as it is, and ES on test DB finds single:
The vixens. A novel.
Publication details: London, 1948.
for obvious reasons.
Now to play another example "without and with patch", here:
[novel] [1900 TO 1950]
without the patch, it transferred "as is" to ES, and gives a yellow error,
because ES doesn't understands range as in '[novel]'.
With patch it becomes transformed to such request to ES:
\[novel\] [1900 TO 1950]
and because backslashed non-alphanumeric characters are ignored, it
works and brings the result - one book, like in the previous example.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #75 from Andrew Nugged ---
QueryRegexEscapeOptions
=
Now let's turn "QueryAutoTruncate" off and check with QueryRegexEscapeOptions:
It's all about RegEx in ES queries,
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-regexp-query.html
Koha admin settings QueryRegexEscapeOptions has three options:
- escape,
i.e. if the request will have any regular expressions between slashes:
/book/
this will be passed in "disabled, not RegEx anymore" to ES:
\/book\/
so ES will get this just as regular text to search, and in this example
will just search for the word 'book', ignoring other non-alphanum chars.
(*)NOTE: but this means that passing RegEx to ES is impossible, at all.
and this means any book name like in the search field will work:
We are / They are
so if we copy-paste this to the search field, ES gets:
We are \/ They are
and Koha just searches it as it is.
I assume this was added as a feature to disable "/" meaning as a special
RegEx
limiter and allow librarians to search "We are / They are" text directly.
- don't escape,
so:
/go+d/
will be passed as it is to ES, so this will be special "Regular Expressions"
in this example, this means "'g' then 'o' one or more times then 'd'"
so search in ES for /go+d/ example is similar to search:
god OR good
( yet not the same because RegExp 'go+d' is not limited by a few 'o's,
so might match for 'good' too)
BTW it founds 7 books for me in both ways (/go+d/ and god OR good) in
the current master dev-test DB.
(*)NOTE: but this means that now any '/' which appears in the text will
cause ES to throw an exception if the syntax is not understood,
i.e. search for exactly this:
We are / They are
now won't work so the users need to do escaping themself and ask:
We are \/ They are
(by backslashing '/' in the request field) to make ES not "die"
and search to work.
That's why I assume to make 1+2 both working, 3rd one was introduced:
- unescape escaped,
yeah, that's "The Trick" to make 1st way "We are / They are" requests to work
and ES not to fail, but at the same time allowing to bring Regular
Expressions
by users, just in "pre-slashed form", i.e. what pre-slashed WILL BE RegExs,
what isn't -- won't be but won't "throw a syntax error" too, so:
We are / They are
will be tossed to ES as "We are \/ They are" and will search
for "We are They are", bit this (NOW THE TRICK! BEHOLD!):
We are / They are \/go+\/
will be transferred to ES in such form:
We are \/ They are /go+/
and ES will search for 'we', 'they', 'are' words, all to be present, plus
one of the any: go, goo, gooo, g.. words
( this founds, btw,
"Seesaw [sound recording] / Beth Hart, Joe Bonamassa"
in current test DB :-), hey! )
This is how it is expected to work. But the reality in current master a little
worse,
Let's have "unescape escaped", let's invent such request:
\/go{1,2}d\/
which EXPECTED by us to be transferred to ES as RegEx:
/go{1,2}d/
what means we want ES to find for us: 'g', then one or max two 'o', and 'd'
i.e. this is exactly ONLY for finding "good" or "god".
But current master branch code breaks these expectations: replaces "{" and "}"
with double quotes), so for real this transferred to ES as the request:
/go"1,2"d/
and obviously, nothing was found.
But with this patch, the request:
\/go{1,2}d\/
will be passed to ES as:
/go{1,2}d/
and voila,
"7 result(s) found for '\/go{1,2}d\/'." in our test DB.
Huh!
More, because this patch respects {,} in regex but escapes those outside,
it will pass those escaped, so if you will search:
\/go{1,2}d\/ [go]
this will pass to ES request WITHOUT patch:
/go"1,2"d/ [go]
and bring yellow "Error" text box,
and this will be passed to ES request WITH patch:
/go{1,2}d/ \[go\]
and because "\[" and "\]" ignored by ES, it searches just for "good" or "god"
plus the word "go", so the final searched text should contain "good to go"
or "go god" combinations (or others like those).
And yes, on test DB it founds 3 results! With patch.
And gives "error/nothing" without the patch.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #74 from Andrew Nugged ---
QueryAutoTruncate case:
=
This is a feature that BLINDLY adds to alphanumeric-ended words asterisk
in the end before tossing that phrase to ES as a search query.
So, if QueryAutoTruncate is enabled (default), the Perl part just adds
to the end of any alphanumeric-ended word asterisk, disregards ES syntax
(ranges too), so if we will put such request phrase, the current master branch
(Perl part) will transfer the below examples to ES as a query string (here I
provide "what we entered" in the search field as 'IN:' and what transferred
to ES as 'OUT:' pairs):
(before the patch, phrase 1)
IN:The [Book]
OUT: The* [Book]
(will make ES fail with a yellow error text box)
in fact, ES doesn't understand your "special language usage"
[Book] confuses ES because not in [... TO ...] form,
and with ranges:
(before the patch, phrase 2)
IN:The "Book" {2000 TO 2002} [1900 TO 1990]
OUT: The* "Book" "2000 TO 2002" [1900* TO* 1990]
(will make ES fail with a yellow error text box)
[1900* TO* 1990] confuses ES because not in [... TO ...] form,
Here we see that it converted '{' and '}' to '"', and added '*' because of
"QueryAutoTruncate enabled" to all alphanumeric-ended words.
Second even more weird example:
(before the patch, phrase 3)
IN:The "Book" [2000 TO 2002}
OUT: The* Book* [2000* TO* 2002*
(will make ES fail with a yellow error text box)
[2000* confuses ES because [ has no pair and no range there,
And here quotes get lost because one "}" brace was converted to doublequote,
but because it was only one, so was "unpaired", then the logic of the current
code is to remove quotes at all if they are unbalanced (I agree, but it
shouldn't convert braces to quotes).
All requests above lead to a "yellow" Error text box, which means that ES
fired an exception because of the error in query language syntax (and in Koha
server error log there will be the error message ES 'Failed to parse query').
Anyway, as I see, this "QueryAutoTruncate" mode is designed for very simple
usage behavior, i.e. user expected to do the requests without using any special
language at the most, and give as much as possible results to the user assuming
that there was "just beginnings of the words" requested if one enters:
so bi
this will be translated by Perl code to
so* bi*
and passed to ES so ES will match with for examples books named "Something Big"
and "Solved Binary Book"
This QueryAutoTruncate as well ruins the syntax of ES range requests
(that '[1900 TO 1990]' becomes '[1900* TO* 1990]' in ES-language)
so no reason to keep the ranges, and that accounted in the patch:
Now with the patch:
This patch is not intended to make the "full fix", it's not so easy for this,
but at least the patch makes braces in QueryAutoTruncate mode just regular
symbols by pre-slashing them. Same as above but WITH this patch will make such
internal transformations, let's see by example:
(after patch, phrase 1)
IN:The "Book" {2000 TO 2002} [1900 TO 1990]
OUT: The* \[Book\]
ES understands that here we search 'the* book' and actually even will find
6 results in the current dev-test database, instead of "failing to the error"
like in the example above without the patch,
(after patch, phrase 2)
IN:The "Book" {2000 TO 2002} [1900 TO 1990]
OUT: The* "Book" \{2000* TO* 2002\} \[1900* TO* 1990\]
ES understands that here we search 'The* "Book" 2000* TO* 2002 1900* TO* 1990'
but it's a pretty rare phrase to find something, so - zero results,
but no "ES exception",
(after patch, phrase 3)
IN:The "Book" [2000 TO 2002}
OUT: The* "Book" \[2000* TO* 2002\}
Same here as above, zero results but no "exceptions".
I.e. those braces now become not-special-language characters and just passed by
to ES to decide. In real life, those braces ignored by ES as non-alphanumeric,
so querying:
some [word]
with QueryAutoTruncate enabled after the patch will be the same as search:
some* word
and BEFORE the patch it was just the "yellow" "error happened" text box result.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #73 from Andrew Nugged ---
First, if you create" crazy request like (exactly this to the search field):
The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
it's impossible to do such fuzziness to guess what's wrong in such message
in ES query, so we will just tell the user this:
Error: There might be results, but there is ElasticSearch query language
syntax error in your search request: remove or escape with backslash
ES-language reserved characters (those are: + - = && || > < ! ( ) { }
[ ] ^ " ~ * ? : / ) or change it to be the proper ES-language
request and try again."
(as from Bug 28736)
But in the current ticket, we added some more clean fuzziness (let me describe
difference between "current master" and what this current bugfix brings
further
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #72 from Andrew Nugged ---
Now regarding the current ticket - why it's appeared:
There already was some "prevention from dies (exceptions) in ES language" to
make requests "just regular characters" instead of special commands,
We already had some FUZY pre-processing before this current bugfix code in
Koha/SearchEngine/Elasticsearch/QueryBuilder.pm, "sub _clean_search_term":
some extra "precleaning",
for example, curly braces "{}" any of them were replaced by '"' doublequote,
and "=" replaced with ":",
and colons with space before or space after (": " or " :") were just removed so
only the space left,
but trailing colon, trailing exclamation mark, and square range braces which
actually not range (like in "[OHO]" phrase: ES dies because it's not [X TO Y]
format) misled ES and anyway made ES die sometimes,
So, that was incomplete and this ticket adds some more of such fuzziness but in
the proper way (for example, ranges like "{... TO ...]" (i.e. full mix of
],},[,{ now works),
But please don't demand from it to be "fully covering ES language" thing,
in my opinion, this ticket decreases the number of "crashes" and "syntax
errors" but at the same time, fixes range searches and started to recognize
even "ES regular expressions" if they are enabled (yes,
"QueryRegexEscapeOptions" case!), and fixes behavior with "QueryAutoTruncate"
enabled, let me provide some examples below.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #71 from Andrew Nugged --- Generally about these complications and some mess with ES search: we don't have at all ES query language parser and transmitting requests directly to ES and getting its "parse_exception" reactions, so in Perl can't explain in which part of the syntax the issue, if Operator wants to have a direct search result, then he/she must escape special symbols for not to be "commands" for ES or use quoted texts to search exact terms. this is a "fuzzy" search though so sometimes wrong queries might lead to not having results even if there are such characters in the book name, a tradeoff for full-text search. We have only basic analysis which by the way was lost and not worked and fixed in Bug 28736, at least now the user will know that it's not "nothing found", but there are problems in special ES query syntax. How to use that syntax the user should read from ES language docs (or it we have some intro in Koha Doc for that?), but if the operator knows that there is ES syntax used and there are special characters and should use those properly: escape or quote. In other cases, it becomes a "weird" mix of words and special characters which leads to "unpredictable behavior" if you don't know how to use ES syntax. And also error message was mangled before, but now in Bug 28736 user knows that it's not "book not found", but the syntax and special language for ES used wrongly, so "please escape or quote". But this my answer actually to forward that 28736 brought some light, as first part covering "most things", which we not covering in this current ticket. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #70 from Victor Grousset/tuxayo --- (In reply to Andrew Nugged from comment #61) > Victor, please can you again give-this-a-go? > i.e. Re-Sign-off this ticket again according to that so we now might have > proper "error messages" for the wrong syntax in ES requests (bug 28736), if > that comforts you? test result: my signoffs still hold on top of bug 28736 :) Last two patches still not tested. - I don't know how QueryRegexEscapeOptions works - comment 41 about the last patch But thanks to Alex we have these covered. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #69 from Victor Grousset/tuxayo --- (In reply to Peter Vashchuk from comment #58) > I think search query like that should never work as it will always cause ES > syntax error, ElasticSearch is a mini-language after all and has it's own > syntax and it will die on something like that. Yes it makes sense. > Though we do need to notify the user, in cases like that, that the search > query is incorrect and that they need to correct it and search with it > again. I will make a ticket for it a bit later if. Thanks for bug 28736 :) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Alex Buckley changed: What|Removed |Added CC||alexbuck...@catalyst.net.nz --- Comment #68 from Alex Buckley --- Followed patch test plans (and run unit test t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t) which work as described in devbox running Elasticsearch. So have signed off. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #67 from Alex Buckley --- Created attachment 123606 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123606=edit Bug 28316: escape ES ranges if QueryAutoTruncate is enabled if QueryAutoTruncate enabled we will have any special operators ruined for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]" so no reason to keep ranges when QueryAutoTruncate set to "enabled" 1) enable QueryAutoTruncate at your sysprefs. 2) perform a search using range, for example: "[1999 TO 2020]", it shouldn't work the way it's supposed to. 3) apply the patch. 4) perform the same search with range, ensure that it works correctly. Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #66 from Alex Buckley --- Created attachment 123605 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123605=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #65 from Alex Buckley --- Created attachment 123604 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123604=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #64 from Alex Buckley --- Created attachment 123603 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123603=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning. To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #63 from Alex Buckley --- Created attachment 123602 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123602=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Alex Buckley changed: What|Removed |Added Attachment #122801|0 |1 is obsolete|| Attachment #122802|0 |1 is obsolete|| Attachment #122803|0 |1 is obsolete|| Attachment #122804|0 |1 is obsolete|| Attachment #122805|0 |1 is obsolete|| Attachment #122806|0 |1 is obsolete|| --- Comment #62 from Alex Buckley --- Created attachment 123601 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123601=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo Signed-off-by: Alex Buckley -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Alex Buckley changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #61 from Andrew Nugged --- (In reply to Victor Grousset/tuxayo from comment #51) Victor, please can you again give-this-a-go? i.e. Re-Sign-off this ticket again according to that so we now might have proper "error messages" for the wrong syntax in ES requests (bug 28736), if that comforts you? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #60 from Peter Vashchuk ---
Victor, please check this ticket:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28736
it will make sense why search queries like this one "+ - = && || > < ! ( ) { }
[ ] ^ " ~ * ? : \ /" fail :)
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 e.betemps changed: What|Removed |Added CC||e.bete...@gmail.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Peter Vashchuk changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=28736 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Alex Arnaud changed: What|Removed |Added Blocks||28484 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 [Bug 28484] Elasticsearch fails to parse query if colon or exclamation point is in 245$a -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Eric Phetteplace changed: What|Removed |Added CC||ephettepl...@cca.edu -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #59 from Peter Vashchuk --- Pushed the patches with fixed descriptions and removed some unnecessary newlines that were left there. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #58 from Peter Vashchuk ---
(In reply to Victor Grousset/tuxayo from comment #51)
> With QueryAutoTruncate=only if *
> For a search of the following and a record named the following:
> The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
> => Error: Unable to perform your search
>
> With QueryAutoTruncate=auto
> For a search of the following and a record named the following:
> The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
> => Error: Unable to perform your search
Hey Victor, thanks for the response,
I think search query like that should never work as it will always cause ES
syntax error, ElasticSearch is a mini-language after all and has it's own
syntax and it will die on something like that.
Though we do need to notify the user, in cases like that, that the search query
is incorrect and that they need to correct it and search with it again. I will
make a ticket for it a bit later if.
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #57 from Peter Vashchuk --- Created attachment 122806 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122806=edit Bug 28316: escape ES ranges if QueryAutoTruncate is enabled if QueryAutoTruncate enabled we will have any special operators ruined for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]" so no reason to keep ranges when QueryAutoTruncate set to "enabled" 1) enable QueryAutoTruncate at your sysprefs. 2) perform a search using range, for example: "[1999 TO 2020]", it shouldn't work the way it's supposed to. 3) apply the patch. 4) perform the same search with range, ensure that it works correctly. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #56 from Peter Vashchuk --- Created attachment 122805 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122805=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #55 from Peter Vashchuk --- Created attachment 122804 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122804=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #54 from Peter Vashchuk --- Created attachment 122803 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122803=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning. To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #53 from Peter Vashchuk --- Created attachment 122802 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122802=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Peter Vashchuk changed: What|Removed |Added Attachment #122201|0 |1 is obsolete|| Attachment #122202|0 |1 is obsolete|| Attachment #122203|0 |1 is obsolete|| Attachment #122204|0 |1 is obsolete|| Attachment #122205|0 |1 is obsolete|| Attachment #122206|0 |1 is obsolete|| --- Comment #52 from Peter Vashchuk --- Created attachment 122801 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122801=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Peter Vashchuk changed: What|Removed |Added Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Jason Boyer changed: What|Removed |Added CC||jbo...@equinoxoli.org -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316
--- Comment #51 from Victor Grousset/tuxayo ---
I forgot I tested this in the past.
What should be expected to be different from comment 11 results?
- Should I still have to quote stuff (the cases of comment 8)
- Should some stuff return results instead of none?
- No crash I guess? Speaking of that:
With QueryAutoTruncate=only if *
For a search of the following and a record named the following:
The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
=> Error: Unable to perform your search
With QueryAutoTruncate=auto
For a search of the following and a record named the following:
The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
=> Error: Unable to perform your search
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Andrew Nugged changed: What|Removed |Added Status|Needs Signoff |Failed QA --- Comment #50 from Andrew Nugged --- You right, Victor, I see also that description messed up and diverged after lot of reworks, let's Petro fix this, Petro, I assume patch 122205 explanation should be without "QueryRegexEscapeOptions" paragraph, and patch 122206 related to: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316#c27 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316#c45 so it should be about "range", not regex, about hiding ranges from escaping actually, I see most description from 122206 appears in 122203, and 122203 is just about "replacing brackets which not have special meaning" only. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #49 from Victor Grousset/tuxayo --- The test plan yielded good results :) Patches were reordered to make it easier to check the tests (they indeed fail when the implementation is missing) Do these commits need a test plan? Bug 28316: avoid messing up regexes in the search queries Bug 28316: avoid messing up regexes in the search queries (they have the same name and description, suspicious ^^) -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #48 from Victor Grousset/tuxayo --- Created attachment 122206 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122206=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #122106|0 |1 is obsolete|| Attachment #122107|0 |1 is obsolete|| --- Comment #47 from Victor Grousset/tuxayo --- Created attachment 122205 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122205=edit Bug 28316: avoid messing up regexes in the search queries This patch ensures that the behavior with QueryRegexEscapeOptions set to values other than "Escape" still will works as expected. It does so by storing the contents of regexes before escaping special characters and then restores the contents of regexes back to how it was before, ensuring that searching with regex is possible. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #122105|0 |1 is obsolete|| --- Comment #46 from Victor Grousset/tuxayo --- Created attachment 122204 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122204=edit Bug 28316: escape exclamation signs in the query Currently having exclamation sign at the end of the query makes ES search fail, and when you try to search for a book that has exclamation sign in the tittle (something like "Words! words") won't show results correctly as it tries to negate everything that is after exclamation sign, making it impossible to search for books that have in in the title This patch escapes exclamation signs if it's at the end of the query or has a space after it, resolving both of the issues listed above. To reproduce: 1) with ES enabled, search for the book with title that contains exclamation sight at the end, like "book!", this search should result in error. 2) do another search, but this time find/prepare beforehand book with a title that has exclamation sign with a space after it, e.g "exclamation! sign", it shouldn't find it as ES treats everything after that exclamation sign as negation. 2) apply the patch. 3) perform searches from the steep one and two again. Search from step one should no longer fail, while search from the step two should find that book. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #122104|0 |1 is obsolete|| --- Comment #45 from Victor Grousset/tuxayo --- Created attachment 122203 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122203=edit Bug 28316: escape brackets in the search query This patch screens square and curly brackets which have no special language meaning, while keeping ranges as they are, allowing ES to search for biblios with titles that contain square and curly brackets and at the same time allows the use of range searches ([1990 TO 2000]). To reproduce: 1) using ES, search for the book with title that contains square and/or curly brackets, like "book [second edition]", which will result in error. 2) apply the patch. 3) search for that book again, ensure that it works now. 4) additionally perform a search using range, for example: "[1999 TO 2020]", ensure that it works correctly too. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #122103|0 |1 is obsolete|| --- Comment #44 from Victor Grousset/tuxayo --- Created attachment 122202 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122202=edit Bug 28316: screen unquoted semicolons and all followup colons Currently searches like: "book:", ":book" and "host-item:test:n" cause internal server errors. This patch adds additional regexes that remove the colons at the start and end of the query, and another regex that screens all follow-up colons that go after the first colon to avoid errors when searching for "host-item:test:n". To reproduce: 1) using ES, search for the book with title that contains semicolon at the start or at the end of the line, separated with spaces, this should cause internal server error. 2) try doing the same with something like "host-item:test:n", it should result in error as well. 3) apply the patch. 4) repeat steps 1-2, ensure that it works now. Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 Victor Grousset/tuxayo changed: What|Removed |Added Attachment #122108|0 |1 is obsolete|| --- Comment #43 from Victor Grousset/tuxayo --- Created attachment 122201 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122201=edit Bug 28316: add tests Signed-off-by: Victor Grousset/tuxayo -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #42 from Victor Grousset/tuxayo --- errata: «date-of-publication:[1950 TO 2020]» -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #41 from Victor Grousset/tuxayo --- > 4) additionally perform a search using range, for example: "[1999 TO 2020]", ensure that it works correctly too. What values should be in the record for the search to yield it? It doesn't say anymore "You did not specify any search criteria." So it's better. But it says that there are not matches. I'm not sure ranges actually work. (because i don't know how to use them ^^") I tried «date-of-publication:[1950:2020]» Is that a correct query for a record with [1984] or 1984 in 260$c -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 28316] Fix ES crashes related to various punctuation characters
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 --- Comment #40 from Peter Vashchuk --- Created attachment 122108 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122108=edit Bug 28316: add tests -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
