UPDATE Dan,
I've built a new branch, and a bug pointing at it, available here: https://bugs.launchpad.net/evergreen/+bug/1251353 The branch fixes other things as well, so I gave it a separate bug at: https://bugs.launchpad.net/evergreen/+bug/1251359 If you have a chance to test, that would be grand. --miker On Wed, Nov 13, 2013 at 8:50 PM, Mike Rylander <[email protected]> wrote: > Dan, > > I've found and fixed your Case 2 problem. It all came down to > needless (and harmful) localization of a variable. IIRC, it was long > like that, but seems to have become bad after my bool-op pushdown > change back at the beginning of this hear. I'm still lacking tuits > ATM, but I'll bug this tomorrow. The branch containing (most of) the > fix is here*, however, it needs just a smidge more work to reset that > variable at the right time to be safe outside a test environment. > FWIW, I can't break it now. > > Case 1 may need to be a syntax documentation and tpac fixup thing. If > you wrap all top-level-only stuff (most filters, modifiers, etc) in {{ > }} then all's well. We can teach the tpac to do that easily for query > components that come from from UI elements other than a text input ... > I think I've suggested this before on a bug or two, and certainly in > person to some folks. Anyway, tuits just ran out. More tomorrow! > > * > http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/qp-curlys > > --miker > > On Wed, Nov 13, 2013 at 3:06 PM, Mike Rylander <[email protected]> wrote: >> I only have a moment, but I can comment on both cases, at least a little. >> >> * Case 1 is very likely a variant of the problem fixed >> (superficially) in ab37336715cb7d84ed0c30b1cd2e9e6b85933774 and is not >> fixed in QP-proper, yet. The current workaround is "put filters and >> modifiers at the end". >> * Case 2 is something new to me. Could you pull the full query >> generated by the working and non-working versions out of your logs, >> per chance? If you're logging long-running queries longer than a >> second (not a bad idea, IMO), they're likely in there. >> >> --miker >> >> On Wed, Nov 13, 2013 at 2:24 PM, Dan Wells <[email protected]> wrote: >>> Hello all, >>> >>> Having made the leap from 2.3 to 2.5, we have run into a few issues with the >>> updated query parser, and rather than dig into something I know from the >>> surface to be very complex, I am hoping someone might shed some light on >>> what is going on. The issues I am finding *seem* like bugs, but maybe I am >>> missing something fundamental in my queries. >>> >>> You see, we have an external system (SFX) which does automated queries of >>> the catalog to find book content. These queries are largely based on ISBN, >>> and when we upgraded to 2.5, they started to fail in strange ways. After >>> much poking and prodding, I have boiled things down to a couple simple cases >>> with surprising results. In both cases, the order of operands changes the >>> result set despite using what I think is a commutative operator (||). You >>> can test these queries using our current catalog, http://ulysses.calvin.edu/ >>> , but I can also take some time to find similar issues using Concerto >>> records if it comes to that. >>> >>> Case 1a: >>> item_form(s) && identifier|isbn:0830837035 || identifier|isbn:1844743829 (no >>> results) >>> vs. >>> identifier|isbn:1844743829 || identifier|isbn:0830837035 && item_form(s) (1 >>> result) >>> >>> In this case, I would have expected both to return 1 result. I also see the >>> same behavior even if the given ISBN is identical (a contrived example): >>> >>> Case 1b: >>> item_form(s) && identifier|isbn:0830837035 || identifier|isbn:0830837035 (no >>> results) >>> vs. >>> identifier|isbn:0830837035 || identifier|isbn:0830837035 && item_form(s) (1 >>> result) >>> >>> >>> >>> The next case is similar, but with slightly more nuance. I have two of the >>> same title, one print, one electronic. If I OR the ISBNs together, it >>> works: >>> >>> Case 2a: >>> identifier|isbn:074944990X || identifier|isbn:0749452897 -- WORKS >>> >>> identifier|isbn:0749452897 || identifier|isbn:074944990X – WORKS >>> >>> However, if I add a third ISBN to the mix, I now get different results >>> depending on the order of operands: >>> >>> Case2b: >>> identifier|isbn:074944990X || identifier|isbn:0749452897 || >>> identifier|isbn:7313054289 -- DOESN'T WORK (print result) >>> >>> identifier|isbn:074944990X || identifier|isbn:7313054289 || >>> identifier|isbn:0749452897 -- DOESN'T WORK (print result) >>> >>> identifier|isbn:7313054289 || identifier|isbn:074944990X || >>> identifier|isbn:0749452897 -- DOESN'T WORK (no results) >>> >>> identifier|isbn:7313054289 || identifier|isbn:0749452897 || >>> identifier|isbn:074944990X -- DOESN'T WORK (no results) >>> >>> identifier|isbn:0749452897 || identifier|isbn:074944990X || >>> identifier|isbn:7313054289 -- DOESN'T WORK (e result) >>> >>> identifier|isbn:0749452897 || identifier|isbn:7313054289 || >>> identifier|isbn:074944990X -- DOESN'T WORK (e result) >>> >>> I do seem to get the same behavior when using a more compact query notation >>> (which I believe should be identical in effect): >>> >>> Case 2c: >>> identifier|isbn:(074944990X || 0749452897) -- WORKS >>> >>> identifier|isbn:(074944990X || 0749452897 || 7313054289) -- DOESN'T WORK >>> (print result) >>> >>> identifier|isbn:(0749452897 || 074944990X || 7313054289) -- DOESN'T WORK (e >>> result) >>> >>> >>> >>> Based on when development in query parsing was most active, I imagine this >>> behavior has existed since 2.4. Can anyone verify that? Also, is there an >>> explanation for this behavior which I may be missing? If not, can anyone >>> more familiar with this code at least narrow down what is causing these >>> issues? I’m willing to dive in if necessary, but given the complexity of >>> this code, I may not soon have enough free time to effectively troubleshoot >>> this. >>> >>> Finally, I am happy to move the conversation over to LP if that is a better >>> venue, but I was struggling with pinpointing exactly what this bug affects >>> (and therefore how to file it properly), so I thought I would first seek >>> input from the list. >>> >>> Thanks, >>> Dan >>> >>> >>> Daniel Wells >>> Library Programmer/Analyst >>> Hekman Library, Calvin College >>> 616.526.7133 >>> >>> >>> >> >> >> >> -- >> Mike Rylander >> | Director of Research and Development >> | Equinox Software, Inc. / Your Library's Guide to Open Source >> | phone: 1-877-OPEN-ILS (673-6457) >> | email: [email protected] >> | web: http://www.esilibrary.com > > > > -- > Mike Rylander > | Director of Research and Development > | Equinox Software, Inc. / Your Library's Guide to Open Source > | phone: 1-877-OPEN-ILS (673-6457) > | email: [email protected] > | web: http://www.esilibrary.com -- Mike Rylander | Director of Research and Development | Equinox Software, Inc. / Your Library's Guide to Open Source | phone: 1-877-OPEN-ILS (673-6457) | email: [email protected] | web: http://www.esilibrary.com
